Compare commits

..
450 Commits
Author SHA1 Message Date
Ryan Houdek 597d524f9e Docs: Update for release FEX-2111 2021-11-06 21:50:00 -07:00
Ryan Houdek b4a71a2144 Merge pull request #1353 from lioncash/bmi
OpcodeDispatcher: Implement BLSR/BLSMSK
2021-11-06 20:56:23 -07:00
Ryan Houdek 09ee6d3bf4 Merge pull request #1352 from Sonicadvance1/more_symlink
Linux/FM: Follow more symlinks in emulation
2021-11-06 20:53:45 -07:00
lioncash 7d2b3d0846 CPUID: Signify support for BMI1
Now that all of BMI1's instructions are implemented, we can signify that
we support it in CPUID.
2021-11-06 23:37:54 -04:00
lioncash e0973e19fc OpcodeDispatcher: Implement handling for BLSMSK 2021-11-06 23:36:52 -04:00
lioncash ff9190204c OpcodeDispatcher: Implement handling for BLSR 2021-11-06 23:28:23 -04:00
Ryan Houdek decd8bec31 Linux/FM: Follow more symlinks in emulation
Depending on how wine is launching it may do a PATH scan.
So we need to follow symlinks in a few more syscalls
2021-11-06 18:49:52 -07:00
Ryan Houdek a393d6609f Merge pull request #1351 from Sonicadvance1/fix_execve_softlinks
Linux: Fixes execve on softlinks in rootfs
2021-11-06 17:24:53 -07:00
Ryan Houdek 8aebbbd0ca Merge pull request #1350 from Sonicadvance1/FEXConfig_fix_timeout
FEXConfig: Fixes timeout in select causing 100% CPU load
2021-11-06 17:24:47 -07:00
Ryan Houdek 8b64546579 Merge pull request #1349 from Sonicadvance1/fix_paranoid
Arm64: Fixes paranoid TSO mode
2021-11-06 17:24:41 -07:00
Ryan Houdek 95457bc78c Merge pull request #1348 from Sonicadvance1/sigchld_drop
SignalDelegator: No longer do magic on SIGCHLD
2021-11-06 17:24:35 -07:00
Ryan Houdek a9d31227bf Merge pull request #1347 from Sonicadvance1/cpuid_hybrid_flag
CPUID: Adds support for hybrid flag
2021-11-06 17:24:29 -07:00
Ryan Houdek cae2f8cac4 Merge pull request #1346 from Sonicadvance1/hide_48bit_va
Allocator: Reserve upper 128TB of VA on 64-bit process
2021-11-06 17:24:00 -07:00
Ryan Houdek d7764d37db Linux: Fixes execve on softlinks in rootfs
Ubuntu soft links a bunch of binaries in /usr/bin to softlinks that live
in /etc/alternatives/

When hitting any of these alternative softlinks execve would fail if the
host also didn't have the same softlink paths.

Allows us to correctly follow the symlinks on execve as well which fixes
launching wine directly from the wine symlink.
Alternatively you could have launched /usr/bin/wine-stable directly.

Also fixes FEX strace again.
2021-11-06 16:26:23 -07:00
Ryan Houdek bf5042bdf0 FEXConfig: Fixes timeout in select causing 100% CPU load
glibc 2.34 changed the select interface to update the timeout on return
to more closely match the kernel interface.
glibc 2.33 always made a copy instead of updating.
Make sure to set the timeout on each iteration of select otherwise we
will end up having a timeout of zero. Thus burning a CPU core.
2021-11-06 14:33:44 -07:00
Ryan Houdek 61a0508ff6 Arm64: Fixes paranoid TSO mode
Vector loadstores were crashing. Now we emulate on load and backpatch on
store.

Store can't effectively emulate so it's better to backpatch.
2021-11-06 04:03:37 -07:00
Ryan Houdek 4ebbca45be CPUID: Adds support for hybrid flag
CPUID lets the application know if it is running on a CPU with hybrid
CPU clusters.
This matches big.little fairly easily. Walk the affinity mask and
check if we are running on a big.little system and report it to the
guest.

For x86-64 host just pass through the flag.
2021-11-06 03:01:39 -07:00
Ryan Houdek 8244d55276 SignalDelegator: No longer do magic on SIGCHLD
We have been setting the host sa_flags to handle this for a while now.
So just pass the signals to the guest as expected
2021-11-06 03:00:00 -07:00
Ryan Houdek 4b47e66135 FEXCore/Utils: Adds File loading helper
This will be used in multiple locations now.
2021-11-06 02:56:31 -07:00
Ryan Houdek df2f1ad074 Allocator: Reserve upper 128TB of VA on 64-bit process
Only a partial fix for #1330, still needs preemption disabled to work.

On x86-64 hosts the Linux kernel resides in the top bit of VA which
isn't mapped in to userspace.
This means that userspace will never receive pointers living with that
top bit set unless you're running a 57bit VA host.

This results in userspace pointers never needing to do the sign
extending pointer canonicalization. But additionally some applications
actually don't understand the pointer canonicalization.
This results in bugs like: https://github.com/golang/go/issues/49405
Now if you're running on a 57bit VA host, this will end up behaving like
FEX but it seems like no one in golang land has really messed with 57bit
VA yet.

In AArch64, when configured with a 48bit VA, the userspace gets the full
48bit VA space and on EL mode switch has the full address range change
to the kernel's 48bit VA.
This means that we will /very/ likely allocate pointers in the high
48bit space since Linux currently allocates top-down.

So behave more like x86-64, hide the top 128TB of memory space from the
guest before boot.

Testing: Took the M1Max 15ms to 21ms allocate the top 128TB.
2021-11-06 01:10:00 -07:00
Ryan Houdek 8f170d4aa0 Merge pull request #1345 from Sonicadvance1/EnvironmentLoader_Parse
Fixes environment loader not hooked up to ArgumentLoader
2021-11-05 17:31:21 -07:00
Ryan Houdek 285ef38717 Merge pull request #1344 from lioncash/bmi
OpcodeDispatcher: Implement handling for BLSI
2021-11-05 17:31:10 -07:00
Ryan Houdek be12059e8f Fixes environment loader not hooked up to ArgumentLoader
Fixes #1334

Fixes the issue of `FEX_CORE=irjit` not working.
2021-11-04 23:41:22 -07:00
lioncash b47cb20619 OpcodeDispatcher: Implement handling for BLSI
Now all that remains is handling for BLSMSK and BLSR
2021-11-04 18:49:15 -04:00
lioncash 166c96320c Frontend: Handle VEX-encoded destination operands
BLSI, BLSMSK, and BLSR make use of these, for example.
2021-11-04 17:52:57 -04:00
lioncash ff24fe872d X86Tables: Relocate size descriptors at the end of uint64_t
This leaves the remaining bits available for use without needing to work
around the size fields.
2021-11-04 16:10:23 -04:00
lioncash e317424b86 X86Tables: Increase InstFlags to uint64_t
We've run out of the range of 32 bits already and will need to use
another flag in upcoming changes, so we need to expand our flags to be
64-bit.

While we're at it, we can use a dedicated type alias for the instruction
flags to make the interface changeable from one spot in the future.
2021-11-04 16:10:20 -04:00
Ryan Houdek babb81a240 Merge pull request #1343 from Sonicadvance1/sigbus_share
Arm64: Consolidate HandleSIGBUS
2021-11-03 01:46:36 -07:00
Ryan Houdek 235367b67a Merge pull request #1342 from Sonicadvance1/tear_telemetry
Telemetry: Adds telemetry for when an application tears
2021-11-03 01:46:27 -07:00
Ryan Houdek 56e5e78b25 Merge pull request #1341 from Sonicadvance1/store_op_size
IR: Fixes memory ops having a duplicate size field
2021-11-03 01:46:17 -07:00
Ryan Houdek 9e8af23456 Merge pull request #1340 from Sonicadvance1/syscall_nanosleep
Syscall: Fix 32-bit nanosleep always passing valid remainder
2021-11-03 01:46:04 -07:00
Ryan Houdek 5b9da4f2be Arm64: Consolidate HandleSIGBUS
We can share this between the interpreter and the JIT. Necessary to
support the TSO-correct interpreter path.

With this change the interpreter is TSO-correct for GPRs. Just not FPRs
yet.
2021-11-02 23:54:25 -07:00
Ryan Houdek 1f4a10ef1f unittests: Update tests for new IR operand ordering 2021-11-02 22:51:38 -07:00
Ryan Houdek 4c712ca111 Telemetry: Adds telemetry for when an application tears
This can be used as an early indicator of an application doing nefarious
things.
2021-11-02 22:46:50 -07:00
Ryan Houdek 3bcc8ca695 IR: Fixes memory ops having a duplicate size field
There's zero need for these to have an independent size field and it was
just confusing.
For stores it was always set to zero and for loads it was just
duplicated.

In addition this allows introspection of the store op without casting
the op, which can be useful in edge cases
2021-11-02 22:38:59 -07:00
Ryan Houdek 83073a880a Syscall: Fix 32-bit nanosleep always passing valid remainder
This doesn't really change behaviour but makes sure we are consistent
2021-11-02 21:54:40 -07:00
Ryan Houdek 34b2f93ddf Merge pull request #1338 from lioncash/bic
IR: Add handling for ANDN operations
2021-11-02 19:29:29 -07:00
lioncash 49dae08b3d OpcodeDispatcher: Make use of the new Andn IR op where applicable
Now that we have the handling in place, we can make use of it to
simplify some operations and resolve some lingering TODO comments.
2021-11-02 21:59:56 -04:00
lioncash 0b700de7d9 IR: Add handling for ANDN operations
This is a pretty straightforward operation that can be nicely modeled
by the BIC instruction on ARMv8, which is nice since we can get rid of
the need to manually perform the And and Not operations.
2021-11-02 21:59:53 -04:00
Ryan Houdek 43454abc63 Merge pull request #1339 from lioncash/nodiscard
Core: Mark relevant Interpreter/JIT functions as [[nodiscard]]
2021-11-02 18:29:51 -07:00
Ryan Houdek 76538be0e0 Merge pull request #1337 from lioncash/bmi-bextr
OpcodeDispatcher: Handle BMI1 BEXTR
2021-11-02 18:26:52 -07:00
Ryan Houdek aa1c47cd75 Merge pull request #1336 from lioncash/fmt
ALUOps: Fix left-over printf specifier in fmt log
2021-11-02 18:19:26 -07:00
lioncash 002867bc2a Core: Mark relevant Interpreter/JIT functions as [[nodiscard]]
Lets the compiler warn loudly when the result from any of these
functions are left unused (indicating a bug).
2021-11-02 18:31:41 -04:00
lioncash 79d6bf2840 OpcodeDispatcher: Handle BMI BEXTR 2021-11-02 16:07:37 -04:00
Lioncash 31030e6f85 ALUOps: Fix left-over printf specifier in fmt log 2021-11-02 14:26:44 -04:00
lioncash 26d493a66e Frontend: Handle VEX on second source operands 2021-11-01 14:55:32 -04:00
Ryan Houdek e0343647c9 Merge pull request #1333 from Sonicadvance1/virtio_ioctls
Linux: Implements virtio ioctls for 32-bit
2021-10-28 11:09:03 -07:00
Ryan Houdek 33151e16a2 Linux: Implements virtio ioctls for 32-bit
This makes running Steam under parallels more sane
2021-10-27 13:01:28 -07:00
Ryan Houdek 7e9201cf0d Merge pull request #1325 from lioncash/bmi
Frontend: Handle VEX source operands
2021-10-22 08:38:14 -07:00
Lioncash 877db85428 OpcodeDecoder: Handle ANDN 2021-10-22 11:18:46 -04:00
Ryan Houdek f9078f8ded Merge pull request #1329 from Sonicadvance1/fix_fexloader_argument_passing
Linux: Fixes FEXLoader argument passing
2021-10-21 23:42:47 -07:00
Ryan Houdek e547f0cad6 Merge pull request #1328 from Sonicadvance1/static_pie_error
Cmake: Change static-pie message to indicate compiled without it
2021-10-21 23:42:37 -07:00
Ryan Houdek a3b39afef2 Merge pull request #1327 from Sonicadvance1/less_native
Arm64: Don't fall back to native
2021-10-21 23:42:24 -07:00
Ryan Houdek 43431edd45 Linux: Fixes FEXLoader argument passing
In the case of binfmt_misc being installed, but the user was still using
FEXLoader to pass in arguments then we wouldn't pass the arguments
forward to applications passed through execve.

This resolves an issue where Wine would fail to know where the rootfs
is since Wine launches a bunch of processes.

eg: `FEXLoader -R Ubuntu_21_04 wine winecfg` would fail before

Fixes #1323
2021-10-21 21:21:40 -07:00
Ryan Houdek c59efaef7a Cmake: Change static-pie message to indicate compiled without it
If glibc is compiled without static-pie then we can't detect that. We
will just get a compile failure.
Looks like ALARM is compiling glibc without --enable-static-pie for
whatever reason.

Fixes #1326 as much as we can. We need to ask the ALARM maintainers to
change their configuration.
2021-10-21 20:40:53 -07:00
Ryan Houdek 0bfc1bbe70 Arm64: Don't fall back to native
In the case of Arm64, make sure not to fallback to native if we hit an
unsupported CPU.
Can cause issues depending on system configuration.
2021-10-21 20:39:49 -07:00
Ryan Houdek e9937d9a85 Merge pull request #1307 from Sonicadvance1/InterpreterDispatcher
Interpreter: Splits ops in to separate files
2021-10-21 16:22:45 -07:00
Lioncash f088f0a236 Frontend: Handle VEX source operands
This will allow us to begin implementing BMI instructions.
2021-10-21 10:50:13 -04:00
Ryan Houdek a40a0cbb12 Interpreter: Splits ops in to separate files
I need this for something else so I'm doing this now
2021-10-20 00:15:33 -07:00
Ryan Houdek 28d084bf78 Merge pull request #1321 from Sonicadvance1/fix_arm_asserts
JIT: Fixes asserts added to the JIT
2021-10-19 11:13:33 -07:00
Ryan Houdek 435137e1a2 JIT: Fixes asserts added to the JIT
Fixes #1319
2021-10-19 10:42:55 -07:00
Ryan Houdek ff74e0a0ad Merge pull request #1317 from Sonicadvance1/JITSymbols_by_library
JITSymbols: Allow grouping JIT symbols by guest named regions
2021-10-16 22:13:59 -07:00
Ryan Houdek d847f6e1b3 JITSymbols: Allow grouping JIT symbols by guest named regions
This lets us have JITsymbols grouped by library.
Useful for determining where to thunk.

Sadly perf doesn't have an option to deduplicate regions by name, so
some external tooling is necessary to make it look nice.
2021-10-16 21:10:57 -07:00
Ryan Houdek 64aa4f00ca Merge pull request #1316 from neobrain/fix_attribute_warnings
Thunks/vulkan: Suppress compiler warnings about unknown attributes
2021-10-15 20:40:30 -07:00
Tony Wasserka 50c165d291 Thunks/vulkan: Suppress compiler warnings about unknown attributes 2021-10-15 10:47:16 +02:00
Ryan Houdek eb8a8bf929 Merge pull request #1315 from lioncash/test
TestHarnessRunner: Make argument check more strict
2021-10-14 21:16:55 -07:00
Lioncash 17fd5f7f79 TestHarnessRunner: Make argument check more strict
Overlooked that more than one argument was being when replacing the
throw macro.
2021-10-15 00:06:30 -04:00
Ryan Houdek c9c352627f Merge pull request #1314 from lioncash/test
TestHarnessRunner: Convert LOGMAN_THROW_A into error log and exit
2021-10-14 18:58:46 -07:00
Lioncash e670f8f0e6 TestHarnessRunner: Convert logging calls over to fmt
Given we're in the same area, we may as well move things over to the
other logging system.
2021-10-14 21:45:03 -04:00
Lioncash c431cdebcc TestHarnessRunner: Convert LOGMAN_THROW_A into error log and exit
In release builds LOGMAN_THROW_A doesn't do anything, so running the
program without arguments would lead to a segfault.
2021-10-14 21:43:53 -04:00
Ryan Houdek 8b3c46154d Merge pull request #1312 from Sonicadvance1/JITSymbolsConfig
JITSymbols: Change over to runtime enablement of symbols
2021-10-13 18:09:32 -07:00
Ryan Houdek 1d9b66044a JITSymbols: Change over to runtime enablement of symbols
Adds a new option for just describing all JIT state as a single symbol.
Useful for simple profiling of total time spent in the JIT
2021-10-13 17:48:34 -07:00
Ryan Houdek 031fa8a7d6 Merge pull request #1311 from lioncash/op
OpcodeDispatcher: Deduplicate OpToIndex definition
2021-10-13 15:00:46 -07:00
Lioncash c9621da51c OpcodeDispatcher: Deduplicate OpToIndex definition
We can just make the one defined in X86Tables visible instead to keep
everything in one spot.
2021-10-13 16:18:39 -04:00
Ryan Houdek b1ab252c68 Merge pull request #1310 from lioncash/printf
DeadContextStoreElimination: Fix missing printf specifier entry
2021-10-13 10:51:49 -07:00
Ryan Houdek d09706aa1c Merge pull request #1309 from lioncash/tables
X86Tables: Make flag helper functions constexpr
2021-10-13 10:51:33 -07:00
Lioncash eb8ca16402 DeadContextStoreElimination: Fix missing printf specifier entry
Previously the offset mismatch error was expecting two arguments, but
only one was provided.

While we're in the area we can convert the logging type over to the
fmt-capable one which can catch these.
2021-10-13 12:43:03 -04:00
Lioncash 8df16460d1 X86Tables: Mark initialization instruction tables as static constexpr
While the previous change eliminated much of the codegen caused by
constructing everything individually on the stack, it didn't eliminate a
memcpy of all the elements onto the stack.

This eliminates the memcpys by allowing the compiler to place all the
data into RO and just reference that data.
2021-10-13 12:30:23 -04:00
Lioncash 5758c65983 X86Tables: Make flag helper functions constexpr
These only perform bit arithmetic, so we can allow them to be used in
constexpr contexts.

This allows clang to collapse quite a bit of code for the table
initializing functions. For example, in InitializeVEXTables(),
with these as inline (but not constexpr) functions, clang will
individually put all of the table entries onto the stack.

With these as constexpr functions, clang will be able to deduce that it
can construct the tables at compile time and reduces the amount of
generated code quite a bit.
2021-10-13 11:51:14 -04:00
Ryan Houdek fa1648c6d5 Merge pull request #1308 from Sonicadvance1/fix_missing_drm_include_path
Thunks: Fix missing libdrm include path
2021-10-11 23:17:03 -07:00
Ryan Houdek 98ba0bfa82 Merge pull request #1306 from Sonicadvance1/spill_fprs
Arm64: Make sure to spill static FPRs on guest signal
2021-10-11 23:16:51 -07:00
Ryan Houdek 366122338e Merge pull request #1305 from Sonicadvance1/spill_slot_debug
RAPass: Add debug compile option to disable spill slot reuse
2021-10-11 23:16:35 -07:00
Ryan Houdek bdc66a33ef Merge pull request #1304 from Sonicadvance1/explicit_x87_abi
Arm64: Be more explicit about x87 ABI usage
2021-10-11 23:16:07 -07:00
Ryan Houdek 48955da5f3 Thunks: Fix missing libdrm include path 2021-10-11 19:43:36 -07:00
Ryan Houdek 6cd73a6724 Merge pull request #1303 from Sonicadvance1/destdir_thunks
Thunks: Respect DESTDIR environment variable
2021-10-11 06:18:58 -07:00
Ryan Houdek 8dfe305aab Merge pull request #1302 from Sonicadvance1/missing_header_xcb
Thunks: XCB Add missing header file
2021-10-11 06:18:50 -07:00
Ryan Houdek 6fb0b3d85c Arm64: Make sure to spill FPRs on guest signal
This wasn't ever wired up
2021-10-10 22:17:41 -07:00
Ryan Houdek 69b27d7715 RAPass: Add debug compile option to disable spill slot reuse
Useful for debugging if spill slots are bugged
2021-10-10 20:45:21 -07:00
Ryan Houdek cffd10d0f7 Arm64: Be more explicit about x87 ABI usage
Just using zero extending moves to ensure that we don't fill any
register's upper bits with garbage
2021-10-10 20:43:27 -07:00
Ryan Houdek f2ef58630c Thunks: Respect DESTDIR environment variable
This allows local install to actually work
2021-10-08 20:57:23 -07:00
Ryan Houdek de8d8d8751 Thunks: XCB Add missing header file 2021-10-08 20:49:19 -07:00
Ryan Houdek 27072d2853 Docs: Update for release FEX-2110 2021-10-08 16:29:35 -07:00
Ryan Houdek 0dc8e23342 Merge pull request #1301 from Sonicadvance1/thunk_versioning
Thunks: Support versioned libraries
2021-10-08 03:01:04 -07:00
Ryan Houdek b102714d5c Thunks: Support versioned libraries
We can't expect users to have development libraries installed.
Load the versioned libraries if they exist instead

Also load them in global namespace, which is required for getting
symbols.

Behaviour on x86-64 host seems sporatic here, not sure if unintended
feature.
Doesn't quite behave the same on AArch64 host
2021-10-08 02:41:37 -07:00
Ryan Houdek 72125c9ccf Merge pull request #1300 from Sonicadvance1/thunksdb_global_file
Thunks: Install a global thunksDB for our current thunks
2021-10-08 02:02:28 -07:00
Ryan Houdek 9a07b550f3 Merge pull request #1298 from Sonicadvance1/libgl_thunks
Thunks: Adds a few missing libGL thunk functions
2021-10-08 02:02:17 -07:00
Ryan Houdek 351412a3e3 Thunks: Install a global thunksDB for our current thunks
This covers the x86_64 definitions, in the future we can have the 32-bit
versions in here as well.
2021-10-07 20:20:34 -07:00
Ryan Houdek 842ab169ce Thunks: Adds a few missing libGL thunk functions
These should really be autogenerated by we aren't there yet.
For now this misses some function aliases which fixes Mangohud when
using GL thunking.
2021-10-07 19:15:57 -07:00
Ryan Houdek b3efb1d2d7 Merge pull request #1296 from Sonicadvance1/vulkan_thunks
Vulkan thunks
2021-10-07 12:49:08 -07:00
Ryan Houdek 0d0ce38050 Thunks: Disable malloc libraries
Keeping this in the commit history to go back to later.
While these are required for static-pie builds to work, with the glibc
bug we can't use those yet.
Disable for now since it is is unnecessary.
2021-10-06 11:53:48 -07:00
Ryan Houdek cb21f52f93 Thunks: Have the ThunkHandler load the FEX malloc symbol libraries
If the thunk configuration is enabled then preemptively load the
libraries. Since they will need to be loaded for any thunking library.

This is because we need to ALWAYS share the allocators to the thunks.
2021-10-05 23:51:26 -07:00
Ryan Houdek 14b0cc5af4 Thunks: Wires up all the new thunks to the generators 2021-10-05 23:51:26 -07:00
Ryan Houdek c0fc4c4623 Thunks: Adds libvulkan
This is specifically the device loader rather than the libvulkan loader
library.
This is meant to override what is provided in the ICD files, not the
loader.

There's no real need to replace the loader, it's quite complex
2021-10-05 23:51:26 -07:00
Ryan Houdek c3230f6a91 Thunks: Adds libdrm 2021-10-05 23:51:26 -07:00
Ryan Houdek 4a3bd7cd13 Thunks: Adds libxshmfence 2021-10-05 23:51:26 -07:00
Ryan Houdek f98627da32 Thunks: Adds libxcb_xfixes 2021-10-05 23:51:26 -07:00
Ryan Houdek f3c20f2743 Thunks: Adds libxcb_sync 2021-10-05 23:51:26 -07:00
Ryan Houdek ec6140fde0 Thunks: Adds libxcb_shm 2021-10-05 23:51:25 -07:00
Ryan Houdek c5490821fc Thunks: Adds libxcb_randr 2021-10-05 23:51:25 -07:00
Ryan Houdek c0aad64578 Thunks: Adds libxcb_present 2021-10-05 23:51:25 -07:00
Ryan Houdek 7381240fb6 Thunks: Adds libxcb_glx 2021-10-05 23:51:25 -07:00
Ryan Houdek 3ea8e4864d Thunks: Adds libxcb_dri3 2021-10-05 23:51:25 -07:00
Ryan Houdek 7bf8d09391 Thunks: Adds libxcb_dri2 2021-10-05 23:51:25 -07:00
Ryan Houdek 798c6772b7 Thunks: Adds libxcb 2021-10-05 23:51:25 -07:00
Ryan Houdek 29debcda3d Thunks: Adds FEX malloc libraries
These are required to expose FEX's allocators to the guest.
Which is required when we are compiled with jemalloc, otherwise
the libraries will crash on allocations.

This needs to be done in stages otherwise glibc will load the library
and as it is setting up symbols, replace malloc, which isn't set
currently and cause a crash
2021-10-02 18:12:07 -07:00
Ryan Houdek 39c1751215 Thunks: Adds a few more features to Generator python file
This will be necessary for Vulkan
2021-10-02 18:05:03 -07:00
Ryan Houdek 9451bc5273 Thunks: Pass Vulkan XML to the thunks generators 2021-10-02 18:03:24 -07:00
Ryan Houdek e2b24f7f59 Thunks: Support thunk init function on host
Allows library to have an additional initialization function
2021-10-02 17:54:30 -07:00
Ryan Houdek 4589876ebc Adds Vulkan-Docs repo to externals 2021-10-02 17:09:23 -07:00
Ryan Houdek e0b878f1fd Merge pull request #1289 from Sonicadvance1/thunks_debugging_changes
Thunks: Some minor X related thunk changes
2021-10-02 11:13:22 -07:00
Ryan Houdek 073224ffea Merge pull request #1287 from Sonicadvance1/expand_thunk_Xext
Thunks: Expands Xext thunked functions listo
2021-10-02 11:13:14 -07:00
Ryan Houdek 00511c16a4 Merge pull request #1286 from Sonicadvance1/expand_thunk_asound
Thunks: Expands what asound thunking supports
2021-10-02 11:13:04 -07:00
Ryan Houdek 66c7fdb6de Merge pull request #1293 from Sonicadvance1/thunks_database
Thunks: Adds a new Thunks database config file
2021-10-02 11:11:55 -07:00
Ryan Houdek a48ed376d8 Merge pull request #1291 from Sonicadvance1/thunk_file_search
Thunks: Makes file searches a bit easier
2021-10-02 11:10:08 -07:00
Ryan Houdek 98bee5a6dd Merge pull request #1290 from Sonicadvance1/thunk_init_constructor
Thunks: Adds init helpers with function call
2021-10-02 11:09:23 -07:00
Ryan Houdek 8d13261ab6 Merge pull request #1292 from Sonicadvance1/thunks_script
Thunks: ThunkHelpers script improvements
2021-10-02 11:08:23 -07:00
Ryan Houdek 2b757a9b9a Merge pull request #1284 from Sonicadvance1/fix_struct_match
StructVerifier: Fix struct match and minor fixes
2021-10-02 11:07:51 -07:00
Ryan Houdek 08f56540d2 Merge pull request #1282 from Sonicadvance1/InterpreterLoadStores
Interpreter: Changes basic loadstores to sized accesses
2021-10-02 11:07:38 -07:00
Ryan Houdek fb01e8bf28 Merge pull request #1283 from Sonicadvance1/minor_core_cleanup
Core: Minor documentation and code splitting
2021-10-02 11:06:56 -07:00
Ryan Houdek d5f9f43ab0 Merge pull request #1281 from Sonicadvance1/SupportHostEnv
Adds support for setting host environment variables from config
2021-10-02 11:06:22 -07:00
Ryan Houdek 9d460e807c Merge pull request #1288 from Sonicadvance1/libclang_definition_extract
Scripts: Adds a new script for extracting function definitions
2021-10-02 11:05:36 -07:00
Ryan Houdek 185f265f5c Thunks: Adds a new Thunks database config file
This adds a new `ThunksDB.json` file to the config folder.
This file lets users describe thunks in a meaningful way without
duplicating it amongst multiple configuration files.

eg:
```
{
  "DB": {
    "GL": {
      "Library" : "libGL-guest.so",
      "Depends": [
        "X11"
      ],
      "Overlay": [
        "/usr/lib/x86_64-linux-gnu/libGL.so",
        "/usr/lib/x86_64-linux-gnu/libGL.so.1",
        "/usr/lib/x86_64-linux-gnu/libGL.so.1.2.0",
        "/usr/lib/x86_64-linux-gnu/libGL.so.1.7.0",
        "/lib/x86_64-linux-gnu/libGL.so",
        "/lib/x86_64-linux-gnu/libGL.so.1",
        "/lib/x86_64-linux-gnu/libGL.so.1.2.0",
        "/lib/x86_64-linux-gnu/libGL.so.1.7.0"
      ]
    },
    "X11": {
      "Library": "libX11-guest.so",
      "Overlay": [
        "/usr/lib/x86_64-linux-gnu/libX11.so.6",
        "/lib/x86_64-linux-gnu/libX11.so.6"
      ]
    }
  }
}
```

This file lets the user describe the library with an nicer name, in this instance
`GL` instead of `libGL-guest.so`.
It also tracks depedencies, like how GL currently has a hard dependency on X11.
This allows the loader to automatically enable the dependencies if described.
The `Overlays` array is like the regular Thunks config file but now in this DB file.

With the DB file now describing the libraries, this allows us to then stick a lighter
description inside of the Thunk Config file.

```
{
 "ThunksDB": {
   "GL": 1
 }
}
```

With this example Thunk config file (Which can be configured per application), There is
a new property of name `ThunksDB`.
All this takes is key:value pairs which describe the user friendly library name and an Integer
to state if the thunk should be enabled or not.
This allows very quick toggling of thunks directly inside of the configuration files rather than
breaking the configuration to disable it.
2021-10-02 11:03:08 -07:00
Ryan Houdek 591fc001cc Merge pull request #1294 from neobrain/fix_jitsymbols_build
Build fix for ENABLE_JITSYMBOLS
2021-10-01 10:25:50 -07:00
Tony Wasserka d71add78ae Build fix for ENABLE_JITSYMBOLS 2021-10-01 17:20:26 +02:00
Ryan Houdek d93edb2da8 Merge pull request #1285 from Sonicadvance1/thunk_config_fixes
Thunks: Minor fixes to the config and loading
2021-09-30 22:25:33 -07:00
Ryan Houdek 2360f9cec1 Thunks: ThunkHelpers script improvements
Allows having a library filename be different from the name.
This will fix a quirk in future thunks where the library name and
filename don't match

Also allow custom callback unpacks. A future thunk will need this
2021-09-30 19:13:28 -07:00
Ryan Houdek 10d596314e Thunks: Makes file searches a bit easier
Instead of searching inside the thunk folder for Guest and Host files
with a library name attached to it, also search for ones That are just
named `Host.cpp` and `Guest.cpp`.

Makes quickly pounding out a bunch of thunks significantly less tedious.
2021-09-30 18:48:32 -07:00
Ryan Houdek cab0cf6a6b Thunks: Adds init helpers with function call
This will be used in the future.
2021-09-30 18:43:17 -07:00
Ryan Houdek 6162a8c7f4 Thunks: Some minor X related thunk changes
Makes it print to stderr instead of stdout.
Also sets the mutex symbols to something that can be debugged.
Found something linking to them but not actually using them.
2021-09-30 18:39:22 -07:00
Ryan Houdek 1b0d2bbf9f Scripts: Adds a new script for extracting function definitions
This is very useful for extracting function definitions for thunks.
Keep it in upstream to not get lost.

Sometimes it can munge a definition but it is usually fine.
2021-09-30 18:35:43 -07:00
Ryan Houdek 121023fb72 Thunks: Expands Xext thunked functions listo
Adds in the function definitions from extutil and Xlibint.

Xlibint has some particularly nasty functions that aren't supported.
2021-09-30 18:23:12 -07:00
Ryan Houdek c30cb87b01 Thunks: Expands what asound thunking supports
These are pulled from headers using an automated script.
Almost all functions are easy drop in, only a handful are commented out.

Works in every game that I've tested.
2021-09-30 18:20:20 -07:00
Ryan Houdek d8edbba71c Thunks: Minor fixes to the config and loading
There was a hard upper limit to 128 json elements, this has now been
removed.
The debugging text for the thunk overlay is now disabled. It can get
very spammy but it is useful for debugging purposes.

Fixes an issue where thunks would be entirely disabled if a RootFS isn't
set.
This meant debugging in a chroot or x86_64 host was breaking rootfs
2021-09-30 18:15:41 -07:00
Ryan Houdek 7c553f3508 Linux: Minor fixes to 32-bit epoll
epoll_pwait has a sigsetsize argument, which was unused in this case but
it caused strace to look ugly.
On epoll_ctl, don't write back the resulting event. The event isn't
written and depending on how the guest allocated the object, we could
have a 4byte overwrite.
2021-09-30 18:10:51 -07:00
Ryan Houdek 5b2d944886 Linux: Minor struct verifier and types fixes
fex-match annotation was actually not doing anything due to python typo.
Fixes the minor warnings that cropped up. Nothing actually broken
2021-09-30 18:08:05 -07:00
Ryan Houdek 9ab7de56ef Core: Minor documentation and code splitting
This will change slightly in the future. Clean this up and document.
2021-09-30 18:01:57 -07:00
Ryan Houdek fc46cb9390 Interpreter: Changes basic loadstores to sized accesses
This makes debugging a bit more clear as to what is happening on crash
2021-09-30 17:56:36 -07:00
Ryan Houdek 8cf4b285bf Adds support for setting host environment variables from config
This can be useful for edge case environment variable setting
2021-09-30 17:51:49 -07:00
Ryan Houdek 3744ec2a44 Merge pull request #1262 from phire/RAValidation
RA validation
2021-09-13 01:39:24 -07:00
Scott Mansell 486c62f77c RAValidation: Improve comments 2021-09-13 20:25:33 +12:00
Ryan Houdek 327c4d550a Merge pull request #1277 from phire/aotir_use_after_free
AOTIR: fix use after free
2021-09-12 23:36:55 -07:00
Scott Mansell f14b73689f AOTIR: fix use after free
These variables are owned by other parts of the code, and AOTIR
should not be freeing/deleting them.
2021-09-13 17:54:26 +12:00
Scott Mansell 93de38d13c FillRegister: Keep refrence to original SSA
This allows the RA Validation pass to verify that a spill slot contains
the correct SSA.

I was originally planning to do a IR equlivent test between the IR
before and after register allocation, mostly to catch this type of error.
But this approach was much faster to implement and gives 90%
of the benefits.
2021-09-13 15:43:26 +12:00
Scott Mansell 3107898f06 Fix ReplaceUsesWithAfter:
There were two issues:
 1. The OrderedNode *After overload created the wrong iterator
 2. Despite being named After, they were actually inclusive

Kept ReplaceAllUsesWithRange with the current inclusive behaviour and
adjusted the argument name to match
2021-09-13 15:43:26 +12:00
Ryan Houdek 21ff433999 Merge pull request #1276 from Sonicadvance1/implement_message_queue
Linux: Implements support for POSIX message queues on 32-bit
2021-09-12 17:46:03 -07:00
Ryan Houdek 217e4764c4 Linux: Implements support for POSIX message queues on 32-bit
These would have worked for 64-bit but wasn't working on 32-bit.
This now passes my unit test.

Relies on #1275 to be merged first.
Fixes #1260
2021-09-12 17:36:26 -07:00
Ryan Houdek f232dcebfc Merge pull request #1275 from Sonicadvance1/implement_timer_create
Linux: Implements support for timer_create
2021-09-12 17:34:06 -07:00
Ryan Houdek 76dd09ea60 Linux: Implements support for timer_create
Needed to fix rt_sigtimedwait to use the raw syscall.
Needed to have sigtimedwait and sigtimedwait_time64 parse siginfo_t
correctly.
glibc uses this to ensure it is sending the correct signal across from
their helper thread.

Needed to correctly parse sigval and sigevent for 32-bit.

Passes my unit test for timer_create
2021-09-12 17:25:12 -07:00
Ryan Houdek 368095f96f Merge pull request #1274 from Sonicadvance1/fix_timex
Linux: Fixes timex definition for 32-bit syscalls
2021-09-12 17:14:36 -07:00
Scott Mansell 2cd844bbcb RAValidation: Validate Spill slots
This can prove that the ssa in the Spill slots are consistant based on
control flow.

It's one major blind spot is that it can't prove the spill slot actually
contains the correct ssa, if it has the same wrong ssa on every cfg
path.
2021-09-13 11:56:39 +12:00
Scott Mansell 55a9ee702b Register Allocator Validation
This is a validation pass that attempts to prove the output from the RA pass is valid.

The current design should be able to prove that the RA result is internally
consistent. That no-matter what control flow path you take thought the control
flow graph, the physical registers and spill slots will always contain a single
possible SSA value.

It also checks that the SSA values in the IR actually line up with the SSA value
in the physical register.
2021-09-13 11:56:29 +12:00
Scott Mansell cb57797550 PassManager: Lookup pass by name 2021-09-13 11:46:11 +12:00
Ryan Houdek 1b6cbd39a4 Linux: Fixes timex definition for 32-bit syscalls
Fixes #1251
2021-09-11 18:39:46 -07:00
Ryan Houdek d004fee7e4 Merge pull request #1273 from Sonicadvance1/softfloat_x86_debug
Softfloat: Allow forcing use of some x87 on x86 host
2021-09-11 18:08:41 -07:00
Ryan Houdek 1fa3afd0c1 Merge pull request #1272 from Sonicadvance1/ensure_long
OpcodeDispatcher: Ensure some x87 templates get passed long constants
2021-09-11 18:08:29 -07:00
Ryan Houdek ba174e5c32 Merge pull request #1271 from Sonicadvance1/fix_wine_working_dir
FileManager: Allow reading real root
2021-09-11 18:08:20 -07:00
Ryan Houdek e91061f7ff Merge pull request #1268 from Sonicadvance1/fix_rlimit_x32
Linux: Fixes rlimit syscalls for 32-bit
2021-09-11 18:08:11 -07:00
Ryan Houdek a56463a7f1 Merge pull request #1269 from Sonicadvance1/change_app_config_shortcut
FEXConfig: Change shortcut for opening application profile
2021-09-11 18:08:01 -07:00
Ryan Houdek 56bddd3a22 Merge pull request #1266 from Sonicadvance1/pressure_vessel_checks
Config: Check for container-manager and redirect
2021-09-11 18:07:50 -07:00
Ryan Houdek 829db6c30d gvisor: Updates getdents test behaviour
This now passes on x86-64 host but still fails on AArch64 because we
don't emulate the getdents syscall
2021-09-11 05:51:10 -07:00
Ryan Houdek 7855b58c73 Softfloat: Allow forcing use of some x87 on x86 host
This is useful for testing the ops that are emulated using different
precision.
Haven't found anything that changes behaviour but useful to keep around
2021-09-11 05:42:18 -07:00
Ryan Houdek 2b91108255 OpcodeDispatcher: Ensure some x87 templates get passed long constants
Just to ensure these don't get truncated and intent
2021-09-11 05:41:14 -07:00
Ryan Houdek 593be950de FileManager: Allow reading real root
wine walks the file structure to ensure the cwdir is safe for use. It
does this with a combination of `getcwd` and `statx`.

Once it reachs `/` then in rootfs environments it would fail to find
directories we've deleted.
Thus making it impossible to find folders like `/mnt` and `/home`

Should be safe since it just gives the application a larger world view
2021-09-11 05:37:28 -07:00
Ryan Houdek 360c4a2060 Merge pull request #1267 from Sonicadvance1/flush_logs
FEXLoader: Flush log output
2021-09-11 05:34:33 -07:00
Ryan Houdek a5046e92cd Config: Check for container-manager and redirect
In the case of running inside of a container then we need to redirect
where we look for configuration.
Currently we only care about pressure vessel so we just redirect some
options to check inside of `/run/host/`

This will resolve an issue where installed thunks wouldn't be found.
2021-09-11 02:16:08 -07:00
Ryan Houdek 793b25f93b Thunks: Set default config option to our install location
These don't really need to be changed unless doing development
2021-09-11 02:16:08 -07:00
Ryan Houdek acfcfa127a FEXConfig: Change shortcut for opening application profile
CTRL+A is used for selecting all in a text box.
Don't override that, it's annoying
2021-09-11 02:13:00 -07:00
Ryan Houdek 35d09de8c6 Linux: Fixes rlimit syscalls for 32-bit
32-bit versions of these syscalls saturate on the upper limit.
Depending on which syscall it'll saturate signed or unsigned.

With set if the 32-bit value is UINT32_MAX then it'll saturate to the
maximum 64-bit value
2021-09-11 01:17:38 -07:00
Ryan Houdek 1f11e307ec FEXLoader: Flush log output
This was removed when we switched from FILE to raw fd.
This fixes an annoying issue where we would assert and not get any
output.
2021-09-11 00:53:49 -07:00
Ryan Houdek c9a62658e3 Merge pull request #1259 from Sonicadvance1/fix_fexmountdaemon_races
FEXMountDaemon: Fixes shutdown race conditions
2021-09-07 22:13:08 -07:00
Ryan Houdek 4ccd68af8a FEXMountDaemon: Fixes shutdown race conditions
This solves a problem where sometimes FEX would spin up a new process
while FEXMountDaemon was in the process of shutting down. Breaking
things on both sides.

Now the race conditions are squashed that I could see.

Also fixes one race where FEX is starting up and FEXMountDaemon is
spinning up. This case is where FEX managed to pull the lock file just
before it got deleted. Then sent the FEXMountDaemon a request to be
observed. With DGRAM sockets we would fire and forget. Use a STREAM with
a ack result so we know we can return.
2021-09-07 19:30:38 -07:00
Ryan Houdek cd4586b67d Docs: Update for release FEX-2109 2021-09-05 01:57:02 -07:00
Ryan Houdek 2c02dcac9f Merge pull request #1257 from CallumDev/caspair_fix_armv8
Fix unaligned CASPair on ARMv8.0
2021-09-05 01:35:43 -07:00
CallumDev dec512187e JIT: Remove nops in ARMv8.0 CASPair 2021-09-05 17:49:57 +09:30
Ryan Houdek dd34316562 Merge pull request #1256 from Sonicadvance1/stabilize_fexmountdaemon
FEXMountDaemon: Fixes dangling mounts problem
2021-09-05 01:13:01 -07:00
CallumDev 5f7532c569 Interpreter: Lower 4 byte CASPair to inline assembly 2021-09-05 17:30:18 +09:30
Ryan Houdek 0fa7af15b2 FEXMountDaemon: Fixes dangling mounts problem
The FEXMountDaemon no longer uses the inotify interface for refcounting
instances of FEX.
The inotify interface fails to send close events when an application
crashes. Which is either an API oversight or intentional choice.

Now to use FEXMountDaemon the FEX process must send the daemon a pipe
fd.
The FEXMountDaemon then uses the write end of the pipe to determine if
the read end of the pipe is still open. It does this using the epoll API
and ref counting how many pipes are still active.
This is possible since epoll will tell us if pipe status has changed to
error. Signalling to the write end that the read end has closed for
whatever reason.

Now we only use the "lock" file to remove races and tell the new
instances of FEX where the rootfs is mounted
2021-09-05 00:36:44 -07:00
CallumDev 8654d19f02 JIT: Fix unaligned CASPair on ARMv8.0 2021-09-05 16:54:53 +09:30
Ryan Houdek befd0aa9cf Merge pull request #1255 from Sonicadvance1/syscall_fixes
Syscall fixes
2021-09-04 18:39:33 -07:00
Ryan Houdek 96cef80a25 Linux: Adds missing Namespace handlers
Missed committing this at some point
2021-09-03 17:04:15 -07:00
Ryan Houdek fd3a88389b Linux: Fixes some 32-bit syscalls 2021-09-03 16:53:39 -07:00
Ryan Houdek d8350353d6 Linux: Creates a Types.h header that matches between architectures 2021-09-03 16:48:53 -07:00
Ryan Houdek 083d3a464a StructPackVerifier: Add a couple missing defines 2021-09-03 16:46:16 -07:00
Ryan Houdek 70931bf388 Merge pull request #1250 from Sonicadvance1/fix_signal_nodefer
Linux: Setup signal mask correctly to block signal-in-signal situations
2021-09-03 16:45:39 -07:00
Ryan Houdek 4f93259332 Linux: Setup signal mask correctly to block signal-in-signal situations
Due to how we emulate the guest signal handlers, we do the state setup
in the real host signal handler, then we jump out after state setup.
This was causing a situation where we were setting up the guest signal
handler state with the correct sa_mask.
Then after setting up the guest state we would leave the FEX signal
handler, restoring the signal mask to our original mask.

Instead now as we are setting up the guest state, we save our host
signal mask. Then on signal handler return we modify our host signal
mask to match what the guest wants.

Once we hit our sigreturn emulation we then restore the original signal
mask.

This looks to improve some stability problems regarding how wine uses
signals but it still doesn't fix the gvisor test sadly.
2021-09-02 20:21:10 -07:00
Ryan Houdek ad34cddbf2 Merge pull request #1249 from Sonicadvance1/fix_fexmountdaemon_messages
FEXMountDaemon: Fixes some minor issues
2021-09-02 16:32:41 -07:00
Ryan Houdek c74620f083 Merge pull request #1248 from Sonicadvance1/sigaltstack_ignore_onstack
Linux: sigaltstack ignore SS_ONSTACK
2021-09-02 16:32:24 -07:00
Ryan Houdek eefcde369a FEXMountDaemon: Fixes some minor issues
Now that we deparent FEXMountDaemon we can no longer use
PR_SET_PDEATHSIG.
Instead we rely on the ref counting and checking the pipe status to see
if the original parent has left us.
Further improvements that could be done in the future is that every user
of the mount point talks to the daemon to give it a pipe to check is
still live, since the ref counting sometimes is incorrect.
2021-09-02 15:42:55 -07:00
Ryan Houdek 005818177c unittests: Update posix tests known failures
This test is relying on legacy behaviour which is no longer true.
Keep running it but expect it to fail
2021-09-02 15:23:10 -07:00
Ryan Houdek 2ae47eae48 Linux: sigaltstack ignore SS_ONSTACK
This flag is ignored with sigaltstack.
Fixes an early assert in:
- Splice
- No Time to Explain Remastered
- Ittledew
- Hyperdrive Massacre
- English Country Tune
2021-09-02 15:15:20 -07:00
Ryan Houdek 1c4503e26a Merge pull request #1247 from Sonicadvance1/handle_xmm_state_32bit
Linux: Handle fpstate in the signal delegator correctly
2021-09-02 12:36:02 -07:00
Ryan Houdek 425ee98f81 Merge pull request #1246 from Sonicadvance1/itimer_32_fixes
Linux: Fixes 32-bit interval timers
2021-09-02 12:35:56 -07:00
Ryan Houdek 9945542375 SignalDelegator: Minor fix with SA_NODEFER
if a test application set NODEFER then changed the signal handler to
have one without it then we weren't correctly removing it
2021-09-02 03:31:58 -07:00
Ryan Houdek 0468bb4496 Linux: Handle fpstate in the signal delegator correctly
We were using the glibc context structure layout which doesn't match
what the kernel is doing.

Switches over to allocating fpstate independentally of the ucontext_t,
then pointing to it from uc_mcontext how we're supposed to.

glibc uses the __fpregs_mem region for other purposes.

This also fixes xmm state being stored on the 32-bit side, and also
fixes SIGALRM and SIGVTALRM siginfo overwriting structure data.
On 32-bit we can't just memcpy the siginfo_t over because the sizes
don't match. Throw a message instead.
2021-09-02 03:29:24 -07:00
Ryan Houdek e55e3a58b1 Linux: Fixes 32-bit interval timers
getitimer and setitimer use an itimerval struct with 32-bit members in
it.

Handles this case and now the timers work
2021-09-02 03:26:50 -07:00
Ryan Houdek 25e9585564 Merge pull request #1244 from CallumDev/fix_cas_armv8
Properly implement single CAS on ARMv8.0
2021-09-01 17:37:47 -07:00
CallumDev 9ab294b2f8 Interpreter: Templated AtomicCompareAndSwap 2021-09-01 21:02:07 +09:30
CallumDev d7f4fe7564 Interpreter: Fix x86 build 2021-09-01 19:24:58 +09:30
CallumDev e09219e5e5 Properly implement single CAS on ARMv8.0 2021-09-01 18:13:18 +09:30
Ryan Houdek c645d8683a Merge pull request #1243 from phire/24bit_assert
RA: Add max NoteCount assert
2021-08-31 06:43:13 -07:00
Scott Mansell eb9d3b11f2 RA: Add max NoteCount assert
The chance of hitting this is near zero, but still safer to have
an assert.
2021-09-01 01:32:41 +12:00
Ryan Houdek 7795078f7e Merge pull request #1242 from Sonicadvance1/deparent_fexmountdaemon
FEXMountDaemon: Early fork to deparent child
2021-08-30 23:07:19 -07:00
Ryan Houdek 0b564652d2 FEXMountDaemon: Early fork to deparent child
This allows FEXMountDaemon to remove FEXInterpreter as its parent.
Instead becoming the parent of whatever the current reaper process is.

Do it as early as possible this way FEXInterpreter won't get an
erroneous SIGCHLD.
2021-08-30 22:57:45 -07:00
Ryan Houdek 4f66d3e9dc Merge pull request #1240 from phire/extract_bucketlist
Move BucketList into it's own file
2021-08-30 19:21:12 -07:00
Scott Mansell bd7822bbe9 Move BucketList into it's own file 2021-08-31 14:06:07 +12:00
Ryan Houdek 0c484ac49c Merge pull request #1239 from Sonicadvance1/fix_ioctl_definition
Linux: Fix V3d and VC4 ioctl definitions
2021-08-30 19:02:46 -07:00
Ryan Houdek 5d21a1e6d6 Linux: Fix V3d and VC4 ioctl definitions
Oops, used the wrong definition

Also fix two struct definitions
2021-08-30 18:54:26 -07:00
Ryan Houdek d600b34b8e Merge pull request #1237 from Sonicadvance1/gvisor_fixes
Gvisor fixes
2021-08-30 18:08:47 -07:00
Ryan Houdek 4003ede7ee unittests: gvisor: Remove stale tmp files
Sometimes a test leaves a tmp file in the root of the rootfs.
This should be fixed but to ensure it stops happening, make sure to
delete it
2021-08-30 18:00:10 -07:00
Ryan Houdek 127d7c1a9e Merge pull request #1238 from Sonicadvance1/vc4_v3d_ioctl
Linux: x86: Initial V3D and VC4 ioctl emulation
2021-08-30 17:45:37 -07:00
Ryan Houdek f45de45568 unittests: Updates gvisor lists on changed behaviour 2021-08-30 17:43:12 -07:00
Ryan Houdek 5fdb66249b Linux: x86: Initial V3D and VC4 ioctl emulation
Untested but with how the structs are laid out, it likely just works
2021-08-30 17:35:58 -07:00
Ryan Houdek bb525e2291 unittests: Updates posix tests with expected failures
These are broken due to how sa_mask currently works. Will need to
resolve this.
2021-08-30 03:00:12 -07:00
Ryan Houdek de29c65585 Linux: Be more verbose about VFORK in code
With a comment claiming we don't support it yet since it causes problems
2021-08-30 02:27:54 -07:00
Ryan Houdek ef0f2246ac Linux: Fixes tkill syscall
There is no glibc wrapper for tkill and tgkill requires a tgid.
Kernel will reject us if we tried using -1 or 0 even though that is what
it does internally.
2021-08-30 02:26:46 -07:00
Ryan Houdek 76877e08bc Linux: Add some comments claiming shmctl and msqctl is incorrect on x86
These aren't correctly handled, so they will get garbage data for now
2021-08-30 02:25:48 -07:00
Ryan Houdek d2636f63f2 Linux: Return -EPERM on non-canonical TLS
If it lives outside of the canonical range then immediately reject with
-EPERM. Just like the official kernel.
2021-08-30 02:24:51 -07:00
Ryan Houdek c241c2f5e7 Linux: Report that we don't support RSEQ
This way the gvisor tests skip rather than just breaking
2021-08-30 02:24:19 -07:00
Ryan Houdek 99e24c284c Linux: Let an application correctly reset its signal handlers
If it is going back to SIG_IGN or SIG_DFL then let them unregister.
This is useful for when they are wanting to ignore a signal after a
while.
Or only capture a fault during a time, then afterwards want the
application to crash on error.

Additionally clears up some other minor logic which wasn't being used
anymore
2021-08-30 02:22:52 -07:00
Ryan Houdek bb3bd3faa1 Linux: Register guest signal delegators for all signals including SIGRT32
Had accidentally missed the last one.
2021-08-30 02:21:41 -07:00
Ryan Houdek 6b3a5470c4 Linux: Fixes tracking of /proc/self with dirfd
Previously we would miss this. Causing things like lscpu to leak state.
2021-08-30 02:20:39 -07:00
Ryan Houdek d241c925c0 Linux: Fixes /proc/self/cmdline arguments
With some changes in the frontend this had gotten out of sync.
2021-08-30 02:19:50 -07:00
Ryan Houdek 378dfcf164 Linux: Fixes semctl syscall on AArch64
Turns out the semid_ds struct is a different layout on x86-64 versus
AArch64.
This was causing some minor failures
2021-08-30 02:19:07 -07:00
Ryan Houdek 9fd558e173 Linux: Fixes signalfd
Was using the wrong signal mask for this.
We need to use the one passed in to the syscall, not the current active
mask
2021-08-30 02:16:26 -07:00
Ryan Houdek 3fbc3c347a Linux: Fixes the pread/pwrite family of syscalls
These syscall arguments are not laid out in a sane way.
Looks like they wanted to keep the interface the same for both x86-64
and 32-bit x86 so they split the offset argument in to two values.

Weirdly enough, even though these are 32-bit offsets on 32-bit x86; On
x86-64 these are still 64-bit. Which means the kernel weirdly allows you
to overlap the two 64-bit halves.

eg: `uint64_t Offset = (pos_high << 32) | pos_low;`
So you can have a 64-bit value that is the full range, but if you have
data in the lower 32-bits of pos_high then you corrupt the offset.
Additionally this allows you to interleave low and high if you want to
be obtuse.
2021-08-30 02:12:35 -07:00
Ryan Houdek fb0b03808f Linux: Fixes waitid syscall
The raw syscall has an rusage argument that can give you child process
rusage similar to wait4
2021-08-30 02:11:07 -07:00
Ryan Houdek a1aeb0d7ee Linux: Fixes execve with missing arguments
Most of the arguments to execve can be nullptr.
Fixes this from crashing
2021-08-30 02:09:51 -07:00
Ryan Houdek 4b6b7495df Signals: Fix incorrect SIGINFO check
Initially this was a small hack to work around sigqueueinfo sending over
siginfo_t. Now this is unnecessary and would crash if an application
used sigqueueinfo to a signal without the SIGINFO flag.
2021-08-30 02:06:55 -07:00
Ryan Houdek 29e30773ee Signals: On jumping to signal frame FPU state is reset
All registers are set to zero and setup to be able to do float
operations.
2021-08-30 02:05:56 -07:00
Ryan Houdek 3e05544e78 Merge pull request #1236 from Sonicadvance1/fexconfig_advanced
FEXConfig: Load application config and advanced tab
2021-08-29 13:05:19 -07:00
Ryan Houdek 61260c2a62 FEXConfig: Load application config and advanced tab
Loading a preexisting application config was impossible through the gui.
Adds a way to do so.

Also adds an advanced tab that just displays all the items.
Allows pruning of application config options to be fairly efficient
2021-08-29 12:30:39 -07:00
Ryan Houdek 109c42a629 Merge pull request #1235 from Sonicadvance1/fix_emulated_openat
EmulatedFiles: Fixes openat for emulated files not using FDCWD
2021-08-28 18:09:42 -07:00
Ryan Houdek 9140ba28b7 Merge pull request #1234 from Sonicadvance1/FEXBash_init
FEXBash: Allow creating a bash instance easily
2021-08-28 17:55:33 -07:00
Ryan Houdek 1c3be542f7 Merge pull request #1233 from Sonicadvance1/fix_execve_escape
Linux: Fixes accidental execve escape
2021-08-28 17:55:22 -07:00
Ryan Houdek 8a331202ee EmulatedFiles: Fixes openat for emulated files not using FDCWD
Fixes lscpu and other applications that open the directory first and
then files inside of that directory.
2021-08-28 17:54:05 -07:00
Ryan Houdek f72ecacd86 FEXBash: Allow creating a bash instance easily
If not passing in any arguments then just immediately create a bash
instance. Incredibly nice little helper
2021-08-28 16:54:25 -07:00
Ryan Houdek 057c1de69d Linux: Fixes accidental execve escape
If the binfmt_misc interpreter was installed then we were running execve
directly.
This allowed shebang programs to escape and see the host architecture
when we weren't planning on it.

Fixes `FEXBash steam` from complaining about missing packages.
2021-08-28 16:37:29 -07:00
Ryan Houdek 10793e89a9 Merge pull request #1232 from Sonicadvance1/iwyu_fixes
Massive amount of IWYU cleanup
2021-08-28 09:54:45 -07:00
Ryan Houdek f161e3bfb0 Massive amount of IWYU cleanup
This isn't quite a 100% clean sweep of IWYU.
There are some false positives where clang fails.
Additionally there are still a few missed in the frontend side of things
that I didn't get to
2021-08-28 00:32:15 -07:00
Ryan Houdek c206942b59 Merge pull request #1231 from Sonicadvance1/Arm64Emitter_iwyu
Arm64Emitter: Resolves some IWYU warnings
2021-08-27 17:18:10 -07:00
Ryan Houdek 10922293c9 Merge pull request #1230 from Sonicadvance1/fix_stdio_log
FEXLoader: Fixes potential bug in log output to stdout/stderr
2021-08-27 17:09:04 -07:00
Ryan Houdek 38ce5876f8 Arm64Emitter: Resolves some IWYU warnings
Should help some build errors
2021-08-27 17:08:29 -07:00
Ryan Houdek de64db5852 FEXLoader: Fixes potential bug in log output to stdout/stderr
Comment in the file for why this can be a bug.

Noticed a game opening a file and our logs were ending up in their
files.
2021-08-27 15:23:17 -07:00
Ryan Houdek 097b3ad881 Merge pull request #1229 from Sonicadvance1/more_signal_splitting
SignalDelegator: More splitting and cleanup
2021-08-27 13:57:07 -07:00
Ryan Houdek c25ae9b2c3 Dispatcher: Changes SRA assert in to an error message instead
In some instances this is safe but we can't currently distinguish safe
or not.
2021-08-27 12:58:40 -07:00
Ryan Houdek b625437071 SignalDelegator: More splitting and cleanup
This is working towards getting the stack frame for guest signals being
pushed over to the Frontend.
Still some more work to do but this is the first step that can be split
up.
2021-08-27 12:46:41 -07:00
Ryan Houdek b0c8710b2c X86Enums: Adds some more defines around signals 2021-08-27 12:42:26 -07:00
Ryan Houdek dfce1dc476 SignalDispatcher: Minor fix for SRA
In the case of SRA plus a signal not using siginfo then we were not
spilling SRA registers.
In this case we would then shift the signal frame and fill the guest
context with garbage register data.
This would potentially cause some issues but haven't noticed anything
outside outside of my test applications.
2021-08-27 12:38:19 -07:00
Ryan Houdek 77db25fc24 Merge pull request #1227 from Sonicadvance1/initial_hangover
Hangover: Initial support for the syscall handling.
2021-08-26 01:49:59 -07:00
Ryan Houdek fa224a3557 Merge pull request #1226 from Sonicadvance1/fix_callback
Fixes Callback interface to take a thread argument
2021-08-26 01:41:16 -07:00
Ryan Houdek b373d0fcfe Merge pull request #1225 from Sonicadvance1/fix_libs
Fixes jemalloc library ordering
2021-08-26 01:40:08 -07:00
Ryan Houdek a3490aad5b Hangover: Initial support for the syscall handling.
Hangover currently abuses the syscall op for thunking purposes.
Very likely this will be changed in the future but for now make sure to
support that use case.
2021-08-26 01:34:13 -07:00
Ryan Houdek 304db72d6b Fixes Callback interface to take a thread argument
This was using the implicit thread TLS object. All users of this
have access to the thread object directly.

Use that instead. Fixes a subtle bug were the frontend could be trying
to do a callback and TLS sections weren't correctly set.
2021-08-26 01:23:17 -07:00
Ryan Houdek b15e0c5f6c Fixes jemalloc library ordering
FEXCore relies on jemalloc symbols if compiled with it.
Have FEXCore link to jemalloc instead of the frontend.

Fixes a missing symbol if someone loads libFEXCore

Additionally, stop trying to compile JEMalloc if not enabled
2021-08-26 01:21:56 -07:00
Ryan Houdek 97f413cfec Merge pull request #1224 from Sonicadvance1/expose_parent_thread
FEXCore: Return the ParentThread with InitCore
2021-08-25 22:47:37 -07:00
Ryan Houdek 6da3330646 Merge pull request #1223 from Sonicadvance1/move_config_to_fexcore
Config: Moves non-OS specific configuration loading to FEXCore
2021-08-25 22:47:30 -07:00
Ryan Houdek 5debdf8d57 Merge pull request #1222 from Sonicadvance1/minor_visibility_fixes
FEXCore: Minor symbol visibility fixes
2021-08-25 22:47:23 -07:00
Ryan Houdek 6483740553 FEXCore: Return the ParentThread with InitCore
This always returns success and for easier state management, just return
our parent thread.

Frontend needs full visibility of this state anyway for thread
management.
2021-08-24 23:20:44 -07:00
Ryan Houdek f131f07612 Config: Moves non-OS specific configuration loading to FEXCore
Puts the visibility of the main layer, application layers, and
environment in to FEXCore instead of FEX.
These layers aren't specific to FEX/FEXLoader and should live in
FEXCore.

Only the EmptyMapper remains in FEX, which should eventually move over
to FEXConfig since that is the only user.
2021-08-24 23:17:32 -07:00
Ryan Houdek f268a28caf FEXCore: Minor symbol visibility fixes
Noticed these missing
2021-08-24 22:35:38 -07:00
Ryan Houdek 6afc3ca13a Merge pull request #1219 from Sonicadvance1/fix_large_offset_syscalls
Linux: Fixes 32-bit syscalls that use 64-bit values
2021-08-23 22:42:08 -07:00
Ryan Houdek 35c664295d Merge pull request #1215 from Sonicadvance1/fix_arm64_signal_handling
Arm64: Fixes SRA spilling on signal
2021-08-23 00:47:10 -07:00
Ryan Houdek b7af5c641c Linux: Fixes 32-bit syscalls that use 64-bit values
A bunch of these were defined incorrectly. I tested a few of these
locally to ensure they were correct after the fact.

Shows that most 32-bit applications that we've encountered aren't
dealing with files larger than 4GB.
2021-08-23 00:44:04 -07:00
Ryan Houdek 6b87839437 Merge pull request #1216 from Sonicadvance1/fix_32bit_sigsegv
x86: Fixes siginfo_t si_addr for SIGBUS/SIGSEGV
2021-08-22 22:26:27 -07:00
Ryan Houdek 9563b5aa3a Merge pull request #1218 from Sonicadvance1/hotfix_nasm_fix
unittests: Hotfix for older nasm
2021-08-22 22:26:07 -07:00
Ryan Houdek 28b3bc3508 unittests: Hotfix for older nasm
Newer nasm takes the size specifier for LEA, older ones do not
2021-08-22 21:59:57 -07:00
Ryan Houdek 8099dfc830 Merge pull request #1210 from Sonicadvance1/proton_6.3_fixes
Proton 6.3 32-bit fixes
2021-08-22 21:32:26 -07:00
Ryan Houdek 926ddabbd1 Merge pull request #1211 from Sonicadvance1/implement_repne_strings
OpcodeDispatcher: Implements undocumented repne on string ops
2021-08-22 21:32:17 -07:00
Ryan Houdek af3af9d048 x86: Fixes siginfo_t si_addr for SIGBUS/SIGSEGV
si_addr is set to the address that is trying to be accessed, not the RIP
that is trying to access it.
We just need to copy our host value over for this.
SIGFPE and SIGILL we still don't have a good answer for.
2021-08-22 21:10:51 -07:00
Ryan Houdek 3b8f24d74f Arm64: Fixes SRA spilling on signal
We were checking the PC after we set the new location in the data
structure.
Didn't matter for x86-64 since it doesn't use SRA but it does matter for
ARM.

Now on signal while in JIT code it will spill correctly
2021-08-22 19:14:04 -07:00
Ryan Houdek 8931ddc382 unittests: Duplicates rep string unit tests with repne
These are exactly the same except using the other prefix.
Hardware tests confirm that this behave the same
2021-08-21 20:19:38 -07:00
Ryan Houdek dd8225be8a OpcodeDispatcher: Implements undocumented repne on string ops
MOVS, LODS, and STOS ops are only documented to support the REP prefix.
These instructions actually repeat correctly with the REPNE prefix as
well.
Behaviour is the same.

Fixes POD Gold
2021-08-21 20:16:25 -07:00
Ryan Houdek 02c10b9671 unittests: Adds tests for FXSave/FXRStor 2021-08-21 17:45:52 -07:00
Ryan Houdek 4d7455989c OpcodeDispatcher: Ensure FXSave/FXRStor doesn't store too many XMM registers in 32-bit
32-bit doesn't store XMM registers 8-15 since they don't exist there.
This technically falls under the "reserved" slot so applications can't
rely on them to not be written.
Saves us a bit of CPU overhead at the very least
2021-08-21 17:44:06 -07:00
Ryan Houdek 23fb4baf46 unittests: Adds unit tests for storing segment register sizes 2021-08-21 17:43:39 -07:00
Ryan Houdek 6e5fc5cdde Fixes segment register storing to memory
These were storing to memory as 32-bits but when storing to memory these
are only ever stored as 16-bit
2021-08-21 17:41:57 -07:00
Ryan Houdek 6a08587d2e Merge pull request #1209 from Sonicadvance1/wine_fixes2
32-bit wine fixes
2021-08-21 17:41:16 -07:00
Ryan Houdek 7bfa1c4838 Linux: Fixes 32-bit sigaltstack
Due to an incorrect pointer check, we were never setting the sigaltstack
on 32-bit applications.
Additionally the stack_t type didn't have the members in the correct
order.

This fixes wine 32-bit applications where wine sends an application
SIGUSR1 and expects the altstack to be used. This is because the
altstack has the thread's TEB region at the start of the stack.

Without this when the application was getting sent a SIGUSR1, it would
remain in the application stack, thus getting an invalid TEB and loading
an FS register with zero.
2021-08-20 23:55:56 -07:00
Ryan Houdek 84f42f6155 Dispatcher: Fixes alt stack check and redzone offset
There are two checks to the alt stack that the signal handler needs to
check.
First it needs to check if the signal handler was registered with teh
flag SA_ONSTACK.

Then it needs to check if the alt stack is actually enabled by not
having flag SS_DISABLE.

Additionally, 32-bit x86 doesn't have a redzone so stop offsetting by
128
2021-08-20 23:54:09 -07:00
Ryan Houdek e4a230c3ec OpcodeDispatcher: Fixes FTW saving/storing in FXSave/FXRStor
Missed this when implementing FTW
2021-08-20 23:53:13 -07:00
Ryan Houdek 09296fe73e unittests: Adds IRET unit tests 2021-08-20 23:52:42 -07:00
Ryan Houdek d2130e1df3 OpcodeDispatcher: Fixes 32-bit IRET
We were failing to store the updated ESP on 32-bit.
On 64-bit or CPL change (which we don't support) the stack pointer is
pulled from the frame.

This resolves an issue in wine's 32-bit loader where it was getting zero
for one of the context pointers since the stack pointer wasn't adjusted
2021-08-20 23:50:04 -07:00
Ryan Houdek 01c49dbb9c FEXLoader: Updates RanAsInterpreter check for FD exec
Only the interpreter can run when executed as FD.
This happens when executed with binfmt_misc and can resolve an issue if
someone sets up the hardlink incorrectly.
2021-08-20 23:48:42 -07:00
Ryan Houdek 6d60689ad4 FileManagement: Stop calling getpid for every file access
We can cache the pid result, we know every case in which the pid is
changed.
This makes watching strace significantly less annoying.
2021-08-20 23:47:48 -07:00
Ryan Houdek 63af80fce3 Merge pull request #1205 from Sonicadvance1/fix_jemalloc_missing_alias
Updates jemalloc to fix missing alias posix_memalign
2021-08-15 03:19:31 -07:00
Ryan Houdek 43052a5707 Merge pull request #1204 from Sonicadvance1/pressure_vessel_option_program
Adds new FEXGetConfig program
2021-08-15 03:19:24 -07:00
Ryan Houdek db5a26991f Updates jemalloc to fix missing alias posix_memalign
This was causing us to fail compiling in debug.
xbyak uses this.
2021-08-14 02:59:40 -07:00
Ryan Houdek c07b5e480b Adds new FEXGetConfig program
This is a simple program to get a few configuration options that are necessary
to expose for pressure-vessel
2021-08-13 20:53:30 -07:00
Ryan Houdek 7aae9b7e44 Merge pull request #1201 from Sonicadvance1/pressure-vessel_fixes
Linux: Implements support for clone with namespaces
2021-08-12 12:57:48 -07:00
Ryan Houdek e90892a20b unittests: Disable gvisor test that doesn't pass when namespaces are disabled
It's expecting EPERM (Which is what we used to return) but it is also
valid for the kernel to return EINVAL when not compiled with namespaces enabled
2021-08-10 19:13:32 -07:00
Ryan Houdek a46773a9ff unittests: Disables gvisor test that uses unsupported clone flags
It uses CLONE_VM which breaks our cloning
2021-08-10 18:39:40 -07:00
Ryan Houdek 740270c05f Linux: Implements support for clone with namespaces
This is very tricky to handle and it has a bunch of rough edges.
One of the major problems that we can't workaround is that if we receive a
clone flag that pthreads can't support with THREAD, then we are required to fall down
the pthreads code path.
This is because threads going down the clone path will break TLS and we don't have
a way to work around it currently.

So this adds a clone path, a clone3 path, and keeps the legacy path as well.
Which makes this fairly convoluted but it gets pressure-vessel working on x86-64 host.
It's a bit tricky to setup but it does work.

Still some work necessary to get pressure-vessel working on AArch64 host, but I'm working on that.
2021-08-10 18:31:04 -07:00
Ryan Houdek 83d20d8f34 Allocator: Leak the allocator object until we fix static initializer allocations
When we are running a 32-bit process we end up mixing VMA Region allocators which
can cause crashes on shutdown.
This is because if a statically initialized object allocates memory, and then frees that
memory in the atexit handler. There is a chance that if it had to reallocate memory during the
VMA allocator switch, that the atexit handler will try freeing memory from the FEX VMA region allocator
AFTER it has already been deallocated.

This can't be safely worked around with atexit handlers.
So until we resolve this issue, we HAVE to leak the allocator so it can safely clean up and then let the kernel
clean up after us
2021-08-10 18:23:39 -07:00
Ryan Houdek e388729403 Config: Changes string config default values to be string_view
These were causing static initializer construction all over the codebase.
Scope of these variables has also been changed over to the module instead of the header as well
2021-08-10 18:20:15 -07:00
Ryan Houdek e3fda9f232 Telemetry: Changes telemetry names map to use string_view
These don't need to be std::string and was causing allocations to occur in the static
initializer
2021-08-10 18:19:03 -07:00
Ryan Houdek f5940df822 RAPass: Removes static initialization of INVALID_REGCLASS
We were just using this as a reference and it was causing a static initializer
2021-08-10 18:18:07 -07:00
Ryan Houdek e6c4f9aad1 IRParser: Removes static initialization of map
This didn't need to be global
2021-08-10 18:16:49 -07:00
Ryan Houdek 8eb0df96f9 Merge pull request #1199 from MerryMage/GetCursorAddress
FEXCore: Use GetCursorAddress when able
2021-08-08 13:26:09 -07:00
Merry 55d981fcb0 FEXCore: Use GetCursorAddress when able 2021-08-08 21:06:06 +01:00
Ryan Houdek b120a8ea84 Merge pull request #1196 from Sonicadvance1/offline_telemetry
Implements support for offline *only* telemetry
2021-08-06 23:23:38 -07:00
Ryan Houdek 5d73ac3234 Merge pull request #1198 from Sonicadvance1/fix_binfmt_misc_arch
Arm64: Reimplements support for binfmt_misc without update-binfmts
2021-08-06 23:23:25 -07:00
Ryan Houdek b05adaeba3 Arm64: Reimplements support for binfmt_misc without update-binfmts
Arch doesn't have update-binfmts. Fall back to the classic approach
without it.
2021-08-06 23:14:25 -07:00
Ryan Houdek 0be16baebf Merge pull request #1197 from Sonicadvance1/rebase_skmp/no-sra
Rebase skmp/no sra
2021-08-06 23:09:13 -07:00
Ryan Houdek 8dd41e7fcd Merge pull request #1193 from Sonicadvance1/fix_readlinkat_self
Linux: Fixes readlinkat for self
2021-08-06 22:59:47 -07:00
Ryan Houdek 83cdf0f377 Fix no-sra crash 2021-08-06 22:57:33 -07:00
Stefanos Kornilios Mitsis Poiitidis b010ab42c7 JIT: Add an option to disable SRA 2021-08-06 22:52:58 -07:00
Ryan Houdek 1c1f40e5af Linux: Fixes readlinkat for self
Similar code to readlinkat. Needs to be correct for self otherwise we
return EINVAL which is unexpected since self should always be a symlink

Fixes a bug in running bwrap
2021-08-06 22:51:31 -07:00
Ryan Houdek c6c94570b4 Implements support for offline *only* telemetry
This information is only ever going to be offline. Will be useful for multiple reasons.

1) Searching for split lock usage in applications, which can be a programming bug.
  a) This isn't visible on AMD systems and on Intel is a fairly new linux feature
2) Having more information about when an application breaks.
3) Useful for some minor profiling for devs looking for statistical data
2021-08-06 22:40:19 -07:00
Ryan Houdek cce3f365cc Merge pull request #1194 from Sonicadvance1/implement_pivot_root
Linux: Implements pivot_root syscall
2021-08-06 22:36:30 -07:00
Ryan Houdek 4641e44276 Linux: Implements pivot_root syscall
Somehow missed this one. Easy enough and matches between architectures.
Used by bubblewrap
2021-08-03 23:30:38 -07:00
Ryan Houdek 1f65bf9e47 Docs: Update for release FEX-2108 2021-08-02 23:21:04 -07:00
Ryan Houdek 4c431518b0 Merge pull request #1192 from Sonicadvance1/workaround_static_pie
AArch64: Workaround static-pie crashing
2021-08-02 23:19:45 -07:00
Ryan Houdek b586592643 Make sure our std::filesystem users use the std::error_code versions
Otherwise these crash out.
2021-08-02 23:10:23 -07:00
Ryan Houdek 83bdbc8392 AArch64: Check if we launched with FD argument
This will give us another way to check if the interpreter is installed rather than file checks.
The binfmt_misc files won't exist inside of a rootfs
2021-08-02 22:47:49 -07:00
Ryan Houdek c49fa5a64f Safely check if we can create paths rather than crashing on failure. 2021-08-02 22:47:42 -07:00
Ryan Houdek d052e87d47 AArch64: Workaround static-pie crashing
Static-pie can only work with GNU ld instead of lld on AArch64 hosts.
This is a known problem on the llvm side: https://bugs.llvm.org/show_bug.cgi?id=49672
2021-08-02 22:02:12 -07:00
Ryan Houdek ff98a43f70 Merge pull request #1191 from Sonicadvance1/improve_fexmountdaemon_stability
FEXMountDaemon: Make squashfs mounting more robust
2021-07-30 16:49:20 -07:00
Ryan Houdek a6c4ad889b FEXMountDaemon: Make squashfs mounting more robust
Instead of watching to ensure our parent process is still alive. Mount the
squashfs once and use a combination of file leases and inotify to
ref count how many processes are using the rootfs.

This makes it so in the common case, the FEXMountDaemon only ever executes once
and runs until all FEX processes stop running.
In the rare edge case there is a race condition where multiple FEXMountDaemon
applications will start, but only one will end up mounting the squashfs.
In this case, one application wins and the one that failed to grab the lease
will wait until the other one completes.

With this change, squashfs should be reasonable to use now.
2021-07-29 21:45:30 -07:00
Ryan Houdek 7e8577a50a Merge pull request #1180 from Sonicadvance1/support_setxid
Linux: Implements support for signals 32 and 33
2021-07-28 00:22:30 -07:00
Ryan Houdek a21cead9dc Fixes memcpy of siginfo_t data
_sifields are offset by a uint32_t between a 32-bit and 64-bit arch.

Fixes thread cancelling on 32-bit processes
2021-07-28 00:10:15 -07:00
Ryan Houdek 555cdb558b Linux: Implements support for signals 32 and 33
When a guest tries to use the setxid syscalls, the guest glibc has
a mechanism in place to ensure that the process wide setxid is handled.
The mechanism is fairly complex but it uses signal 33 and sends the signal to all
active threads to ensure every thread sets the correct state here.

We need to intercept this and pass the context information correctly to the guest instead.
Otherwise FEX just crashes when the glibc HOST handler tries handling the guest applications signal.

Fixes the game SOMA https://store.steampowered.com/app/282140/SOMA/
2021-07-28 00:10:08 -07:00
Ryan Houdek ceaaac3b7a Linux: Pass setxid syscalls directly to the kernel
We don't want these to be captured by the host glibc handlers around these
2021-07-27 18:55:30 -07:00
Ryan Houdek de0337db47 Merge pull request #1188 from Sonicadvance1/wine_fixes
Wine fixes
2021-07-27 18:07:53 -07:00
Ryan Houdek a48e41e4a9 Merge pull request #1187 from Sonicadvance1/cleanup_compile_service_on_fork
Cleanup compile service on fork
2021-07-27 18:07:42 -07:00
Ryan Houdek 896aa52851 Merge pull request #1186 from Sonicadvance1/fix_partial_rclse
RCLSE: Fixes an assumption in RCLSE
2021-07-27 18:07:32 -07:00
Ryan Houdek 0315964848 Merge pull request #1185 from Sonicadvance1/allow_stall_config
Adds an option to stall processes on launch
2021-07-27 18:07:19 -07:00
Ryan Houdek dec822192a Merge pull request #1182 from Sonicadvance1/disable_jemalloc_option
Adds an option to disable jemalloc from cmake
2021-07-27 18:07:09 -07:00
Ryan Houdek a5b142cf02 Merge pull request #1178 from Sonicadvance1/implement_vsyscall
Linux: Implements support for vsyscall
2021-07-27 18:06:54 -07:00
Ryan Houdek 007b1cbe9d Merge pull request #1174 from Sonicadvance1/binfmt_support_preserve_and_fd
binfmt_misc: Support loading ELFs from FD and support preserve
2021-07-27 18:06:39 -07:00
Ryan Houdek b0a28e595f Merge pull request #1181 from Sonicadvance1/fix_rapass
Fixes bug in RAPass
2021-07-27 18:06:27 -07:00
Ryan Houdek b8c5cf2736 Adds FXAM unit tests for ABI conformance
Wine was doing a fun x87 stack clean by checking tag bits for empty.
We hadn't previously supported anything but valid. Which cause wine to overflow the stack
trying to save all of the x87 values.

We now support valid and empty well enough that this doesn't break and we can unit test it
2021-07-27 08:25:40 -07:00
Ryan Houdek 2803113e35 Improves 32-bit signal handling
A decent amount of this was broken.
To the point that the ABI was incorrect even, causing 32-bit signals to completely crash in most instances.
This is enough to get my 32-bit signal handler test application to work and basic SIGUSR behaviour
2021-07-27 08:23:25 -07:00
Ryan Houdek 7a84ab8cfc Implements partial support for x87 FTW
This only implements the Empty and Valid tags.
Anything that is not empty is considered valid currently.
2021-07-27 08:22:11 -07:00
Ryan Houdek 77a7ecbae0 Add x87 FTW to CoreState
This will be needed for x87 FTW emulation
2021-07-27 07:57:41 -07:00
Ryan Houdek 2310ea83bb Cleanup compile service on fork
On fork this thread will be gone. Make sure to delete the shared_ptr.
2021-07-27 07:53:02 -07:00
Ryan Houdek 109df27bba Amd64: Very minor optimization in Push/Pop Regs
Just generates a bit nicer code to look at. Doesn't really change much.
2021-07-24 08:45:38 -07:00
Ryan Houdek 2afa14d1f3 ConstProp: Removes VMOV assumption
ConstProp has an assumption that a VMOV of the same size arguments is a no-op.
This isn't quite 100% true as this could be behaving as a zext and the pass can't see through the op correctly.

Something to be said that the vector ops should be looked over to see which op is leaving dirty bits in the upper bits
of the register. But that's a battle for a different time
2021-07-24 08:43:35 -07:00
Ryan Houdek 744b778013 RCLSE: Fixes an assumption in RCLSE
There is an assumption in the RCLSE pass that if a StoreContext + LoadContext pair of the same size
with a vector register that it is safe to remove the LoadContext.
This isn't quite safe since it can't entirely see through the IR to determine if the store op
had already zext the upper bits of the register.

This was causing 8byte loadcontext operations to "load" with garbage data in the upper
bits when it was expecting a zext.

Which granted it is likely a bug that not all vector ops zero their upper bits but that's a battle for a different day.

Additionally:
- Removes a couple dead optimizations in the GPR path that was never hit.
- Cleans up how access types are checked
- Uses ReplaceAllUsesWithRange to reinforce that this is purely a per block optimization still
2021-07-24 08:37:03 -07:00
Ryan Houdek b3965ab9b4 Adds an option to stall processes on launch
For misbehaving applications before we can fully diagnose, allow
a config that hangs a process on load.
2021-07-23 15:18:52 -07:00
Ryan Houdek ec6cc5cb4f Merge pull request #1184 from lioncash/vec-alloc
64BitAllocator: Convert std::vector table into std::array
2021-07-23 14:29:03 -07:00
Ryan Houdek 7931dc2c09 Merge pull request #1183 from lioncash/size
FlexBitSet: Use non-template type parameter for index parameters
2021-07-23 14:17:13 -07:00
Lioncash 22590806a0 64BitAllocator: Convert std::vector table into std::array
This is small enough that it can be converted over to a std::array so
the heap use isn't necessary
2021-07-23 17:12:37 -04:00
Lioncash f9313f4000 FlexBitSet: Use non-template type parameter for index parameters
These parameters are used as indexes into the tracked memory, so sizing
the index variable relative to the type being stored is kind of sketchy.

e.g. If the tracked types were uint8_t for example, we'd still want the
indexing parameters to be regularly sized so that we aren't implicitly
truncating values all the time when passing values to a uint8_t
parameter (and while all usages are currently using uint64_t as type
T, we may as well address this).

While we're at it, we can make both Get() and operator[] const member
functions, since they don't directly modify any underlying data, they
only read it.
2021-07-23 17:07:41 -04:00
Ryan Houdek ddcc18277b Check for valgrind for memory region check
If running under valgrind then ignore memory region check.
Otherwise you'd need to modify source to get past this check
2021-07-22 18:17:16 -07:00
Ryan Houdek 19de33f9ca Adds an option to disable jemalloc from cmake
While not recommended. It is necessary to allow disabling jemalloc if you want to run asan or tsan
2021-07-22 18:16:24 -07:00
Ryan Houdek a9ee4123ef Fixes bug in RAPass
RegisterNode isn't a POD so we can't memset it.
This was overwriting some unique_ptrs which were memory leaking and causing crashes in other PRs
2021-07-22 18:13:44 -07:00
Ryan Houdek 138e88bc80 Linux: Implements support for vsyscall
vsyscall is a legacy interface to expose three syscalls in a faster fashion.
Most applications should have moved to vDSO but this isn't entirely the case.

golang specifically still used vsyscall in the case that vdso isn't available.
Which that isn't completely safe since vsyscall can be completely disabled.
This is fixed in a newer version of their runtime but we must support legacy behaviour.

This makes it so FEX checks if the incoming RIP is a "special" region and adjusts where it
loads instructions from accordingly. Lying to the guest application about the instruction stream
while maintaining the illusion that its RIP is in the correct location.
This was fairly straightforward since we already decoupled the instruction stream from the RIP location.

This fixes golang execution as long as you also disable their preempt signaling.
environment variable to do so is `GODEBUG=asyncpreemptoff=1`
2021-07-22 16:58:17 -07:00
Ryan Houdek 62775161fe Merge pull request #1176 from Sonicadvance1/fix_static_pie_crashing
Works around static-pie crashing
2021-07-20 19:37:56 -07:00
Ryan Houdek 310cfcbcc4 Works around static-pie crashing
static-pie is crashing early due to an issue with pthread symbols being exposed as weak.
For some reason the weak symbols never get resolved and jump to zero or ELF base.

When linking static-pie, force search for the pthread symbols using an undefined glob.

With this, static-pie works as long as you link with a new enough version of lld (version 13.0)
2021-07-20 05:24:57 -07:00
Ryan Houdek 2fd41de5e0 Merge pull request #1153 from Sonicadvance1/split_opdispatcher
OpcodeDispatcher: Split the opcode handling to multiple files
2021-07-20 04:43:53 -07:00
Ryan Houdek c2a97d4fc1 Minor header shuffling to improve compilation time
Shaves a few seconds off compile time
2021-07-20 04:27:28 -07:00
Ryan Houdek ebb21e86b0 OpcodeDispatcher: Split the opcode handling to multiple files
These are some fairly large separate files still.
Don't want these to be too terribly small but helps significantly with compile time when working
in the OpcodeDispatcher
2021-07-20 04:27:28 -07:00
Ryan Houdek d8da4ce2a5 binfmt_misc: Support loading ELFs from FD and support preserve
This lets us support the final two flags in binfmt_misc that we needed.
1) Support open_binary
2) Support preserve

1)
We already supported the credentials flag, which implied open_binary, but we weren't handling half of it.
With the open_binary flag, the Linux kernel passes us the executable in an FD instead of as a pathname.
This can be found inside of auxv on startup, inside of AT_EXECFD.
If AT_EXECFD is available then we prioritize using that instead of the pathname passed in.
This fixes a potential permissions issue where an executable is executed without read permissions.

2)
The preserve flag has the Linux kernel preserve the original argv[0] that was passed to the application.
Prior to supporting this flag, the kernel would provide us with a resolved program path.
This can happen in the instance where something like `blah` resolves to `/usr/loca/bin/blah` which
isn't what the user originally typed.
This works around this problem by handing the interpreter both the resolved path and the original typed path.

Alongside open_binary, we can just use the FD passed in instead of the resolved path, this means
we can just drop the argv[0] for the guest (which is the kernel resolved path) and pass through arguments
unmangled.
We do have to make a minor assumption here that if we are using EXECFD that we assume preserve.
It isn't until kernel v5.12 that we can actually check AT_FLAGS to see if that was true.
2021-07-20 01:35:23 -07:00
Ryan Houdek aa767b3922 Merge pull request #1166 from Sonicadvance1/static_pie_support
Adds cmake option ENABLE_STATIC_PIE
2021-07-19 23:33:46 -07:00
Ryan Houdek 14480f3ee0 Merge pull request #1167 from Sonicadvance1/tsan_problems
Fixes some issues I found when running under tsan and asan
2021-07-19 23:30:25 -07:00
Ryan Houdek 6ac23003e1 Merge pull request #1168 from Sonicadvance1/fix_spill_slot
Arm64: Fixes fill and spill slot offset calculation
2021-07-19 23:29:01 -07:00
Ryan Houdek 04aacea58b Merge pull request #1173 from Sonicadvance1/fix_binfmt_misc_not_usr
Fixes binfmt_misc install when not installed to /usr
2021-07-19 23:26:33 -07:00
Ryan Houdek 3d46c40aaf Fixes binfmt_misc install when not installed to /usr
We had hardcoded paths in our binfmt_misc files. Change it to a generated
file instead.

Additionally for the binfmt_misc install targets, since we aren't installing to the
global binfmt_misc folder, we need to pass in the import directory

Fixes #1169
2021-07-18 13:05:12 -07:00
Ryan Houdek 7052d022c1 Merge pull request #1172 from lioncash/jits
Jits: Migrate logs over to fmt where applicable
2021-07-16 11:10:27 -07:00
Lioncash 9c7ebca235 JIT/x86_64: Migrate logs over to fmt 2021-07-16 13:57:15 -04:00
Lioncash 22656aedb2 JIT/Arm64: Migrate logs over to fmt 2021-07-16 13:57:12 -04:00
Ryan Houdek 80caf9038f Merge pull request #1171 from lioncash/string
General: Remove redundant string constructions in log calls related to IR op name retrieval
2021-07-16 10:09:43 -07:00
Lioncash 7fddac7b22 General: Remove redundant string constructions in log calls
Prior to the introduction of fmt, there wasn't a nice and easy way to
format std::string_view, but now that we have it in place, we can modify
relevant log calls to make use of it and get rid of the need to
construct a string around the view.

While we're in the same area, we can make the rest of the modified file
use fmt where appropriate.
2021-07-16 09:53:13 -04:00
Ryan Houdek 7f5b352453 Merge pull request #1170 from lioncash/dead
Validation Passes: Remove unused variables
2021-07-16 06:13:35 -07:00
Lioncash f8968b8296 IRValidation: Move stringstream int HadError conditonal body
Same behavior, but now we only construct it when we need it.
2021-07-16 08:54:09 -04:00
Lioncash 914861ba42 PhiValidation: Move stringstream int HadError conditonal body
Same behavior, but now we don't construct it unless we need to.
2021-07-16 08:52:51 -04:00
Lioncash 5d4efa8caa ValueDominanceValidation: Remove unused variables
No warnings are ever appended to the warning string stream, and
HadWarning is never set to anything other than false, so we can remove
it to bring it in line with the PhiValidation pass.
2021-07-16 08:50:33 -04:00
Ryan Houdek 0075e4ed19 Arm64: Fixes fill and spill slot offset calculation
+16 doesn't matter anymore. This can cause stack corruption
2021-07-15 06:47:28 -07:00
Scott Mansell 3a0da68b94 Merge pull request #1165 from Sonicadvance1/fix_lock_flags_ops
Fixes flag setting for 8bit and 16bit LOCK ALU ops
2021-07-15 14:00:33 +12:00
Ryan Houdek d0edd04e58 Amd64: Fixes X86Dispatcher allocator
The entire object didn't need to be an Allocator.
Additionally there was an overwrite happening on the Size variable.
This was causing the shutdown to free a size of zero.
2021-07-14 13:25:29 -07:00
Ryan Houdek 4332c22797 Linux: Fixes BRK deallocation
The SyscallHandler takes over ownership of BRK from the ELFLoader.
We were failing to deallocate the full range.
Additionally if the guest changed BRK size to something smaller than initial load then the size would underflow.
2021-07-14 13:23:58 -07:00
Ryan Houdek 5e12c38298 SignalDelegator: Switch Required flag to atomic bool
This is shared between multiple threads but it isn't never changed beyond first set.
Change it to atomic bool with relaxed loads to quiet tsan
2021-07-14 13:22:38 -07:00
Ryan Houdek bf8e28f935 Fixes TLS for softfloat
softfloat uses TLS for setting the rounding mode per thread.
We had never set the `THREAD_LOCAL` environment variable, so this
rounding mode was being set globally rather than per thread.
We are expecting per thread state here.

Noticed this while running through tsan
2021-07-14 13:21:02 -07:00
Ryan Houdek 6b90663be3 Adds cmake option ENABLE_STATIC_PIE
This option does cmake checks to determine if your system can handle static-pie.
With upstream projects static-pie only works if you use the binutils linker.
Using lld doesn't currently work because it defines __rela_iplt_{start,end} symbols.

Our cmake file will now compile a test application and check for these symbols.
Either the symbols will not exist at all or they will exist but be a null address

Once your system passes the checks then it will allow you to enable static-pie
2021-07-14 09:56:47 -07:00
Ryan Houdek c362bd883d Update jemalloc for more symbols 2021-07-14 08:21:29 -07:00
Ryan Houdek 0ade57614a Fixes flag setting for 8bit and 16bit LOCK ALU ops
MUL is handled in a different function, so this isn't needed there anymore.
This was causing a problem where if you force enabled LOCK on all capable ops it was causing a crash.

Issue #1164 is set to track getting unit tests for these.
2021-07-13 01:37:17 -07:00
Ryan Houdek f4e044a53c Merge pull request #1163 from Sonicadvance1/add_named_thunk_configs
Adds a convenience of named thunk configs
2021-07-13 00:26:36 -07:00
Ryan Houdek 108081e2cd Merge pull request #1162 from Sonicadvance1/update_man_page_with_json_key
Update man page to show JSON key for config options
2021-07-13 00:26:31 -07:00
Ryan Houdek 6ff9b9664d Merge pull request #1161 from Sonicadvance1/fix_thunks_finding_gl
Fixes Host thunks lib not being able to GL
2021-07-13 00:26:26 -07:00
Ryan Houdek 0a0d8f8885 Adds a convenience of named thunk configs
If the thunk config is not a path then search for the filename in $XDG_DATA_DIR/.fex-emu/ThunkConfigs/
for the file.

Just makes it easier to use rather than having full paths
2021-07-12 19:30:03 -07:00
Ryan Houdek bc09d73b9d Update man page to show JSON key for config options
Allows users to reference the man page for the json key instead of code diving
2021-07-12 19:22:57 -07:00
Ryan Houdek ed891f09f1 Fixes Host thunks lib not being able to GL 2021-07-12 19:21:58 -07:00
Ryan Houdek a24d63b5f9 Merge pull request #1160 from Sonicadvance1/allocator_fixes
Allocator fixes
2021-07-12 16:38:22 -07:00
Ryan Houdek 9434a59902 Remove some logs that aren't worth seeing anymore 2021-07-12 00:25:09 -07:00
Ryan Houdek 786a341652 BRKHandler: Don't rely on 64Bit Allocator 32-bit hack.
Use the actual 32-bit allocator if we are running a 32-bit process.
This hack in the 64bit allocator is likely to go away in the future
2021-07-12 00:22:59 -07:00
Ryan Houdek 74bff6c3a5 64BitAllocator: Add a backward scan path
First thing, do a backwards scan to try and fill any holes in the allocator
This reduces the number of holes we have in our VA space
2021-07-12 00:22:44 -07:00
Ryan Houdek 8cfda9c86d RAPass: Mostly revert the PR switching this over to mmap.
1) We needed to switch it to the FEXCore allocator so it didn't live in 32-bit space
2) This generates holes in the VA space due to how the allocations line up.
This makes us almost immediate hit the 65k VMA region.
Switching over to jemalloc allocations means we don't hit that.

Not worth creating an mmap based allocator that can work around that.
2021-07-12 00:16:20 -07:00
Ryan Houdek f79c832f88 X86HelperGen: Use System mmap for 32-bit fixed location
This needs to end up 32-bit space which means we shouldn't use the FEXCore Allocator.
FEXCore allocator has a hack in it for fixed offset in 32-bit space, but we shouldn't rely on it
2021-07-12 00:13:32 -07:00
Ryan Houdek 1fa51f7f74 Frontend: Use mmap helper for DecodeBuffer
This ensures that on 32-bit applications we don't consume 416KB per thread inside of 32-bit space
2021-07-12 00:12:30 -07:00
Ryan Houdek 53417e5322 Merge pull request #1158 from Sonicadvance1/atomic_improvements
Atomic improvements
2021-07-11 18:17:16 -07:00
Ryan Houdek 94e38bb556 Arm64: Fixes duplicated symbols problem in the interpreter
Uses local symbol numbers so the compiler doesn't get upset if this is inlined.
2021-07-10 23:51:02 -07:00
Ryan Houdek 59a4481e5b unittests: Adds explicit 16bit, 32bit, 64bit unaligned unit tests 2021-07-10 23:40:18 -07:00
Ryan Houdek 31a672b1d5 unittests: Adds explicit 128bit vector unaligned load test 2021-07-10 23:39:04 -07:00
Ryan Houdek e03ece0ee6 unittests: Adds LOCK NEG unit tests 2021-07-10 23:38:19 -07:00
Ryan Houdek b1151428e8 Arm64: Stop backpatching loadstores in paranoid TSO mode
When paroid tso is enabled, stop backpatching almost entirely.
Only code still backpatching is 128bit loadstores
2021-07-10 23:37:10 -07:00
Ryan Houdek 818808c3e1 Arm64: Implements unaligned atomic loads store handler
In the case that paranoid TSO is enabled, we can force all unaligned loadstores
down this path.

Instead of backpatching, just do the unaligned loadstore in the handler
2021-07-10 23:32:18 -07:00
Ryan Houdek ee40300eb5 OpcodeDispatcher: Implements support for LOCK NEG
Previously this was unimplemented since it isn't really used on x86.
Fixes #844
2021-07-10 23:30:42 -07:00
Ryan Houdek 484d756a26 Arm64: Implements AtomicFetchNeg IR op 2021-07-10 23:29:49 -07:00
Ryan Houdek d0bcdeda6b amd64: Implements AtomicFetchNeg IR op 2021-07-10 23:29:04 -07:00
Ryan Houdek ea218f8758 Interpreter: Implements AtomicFetchNeg
Does a two's complement negation of what is in memory atomically

Requires a loadstore exclusive loop to implement correctly
2021-07-10 23:27:45 -07:00
Ryan Houdek 821634e300 Arm64: Make sure ARMv8.0 Atomic Swap follows our pattern
The most minor of optimizations as well
2021-07-10 23:26:26 -07:00
Ryan Houdek 5df8354339 Arm64: Allow capturing loadstore exclusive pairing for atomic memory ops
This allows us to capture LDAXR* instructions which appear in ARMv8.0 memory ops.
These scan the memory space to see what the loop is doing and then make an atomic decision to handle
the loadstore exclusive loop with our signal handler based CAS atomics.

This will be necessary to support `LOCK NEG` on ARMv8.1+
2021-07-10 23:23:16 -07:00
Ryan Houdek fe10db9396 Adds an assert if we tried restoring a signal without known frames
Noticed this happen while tinkering. Best to get a debug only message in that case.
2021-07-10 23:19:46 -07:00
Ryan Houdek 3b4119dd31 Arm64: Add some helpers to unaligned atomic load
If we want to do atomic memory loads that are unaligned then we need some helpers to do these.

LoadAcquire128 needs to specifically avoid the std::atomic helper because it can try storing to read only memory
It does this to clear the exclusive lock from its ldaxp but we can just use clrex.
2021-07-10 23:18:46 -07:00
Ryan Houdek 60da70b07e ARMv8.0: Adds some NOPs around CASPair loadstore exclusive pair
The unaligned signal handler will backpatch these with DMBs.
Previously generating corrupted code. This will fix that corruption but not the atomicity problem
2021-07-10 23:15:49 -07:00
Ryan Houdek 35a4f52d7e Merge pull request #1156 from Sonicadvance1/actually_fix_jemalloc
Actually fix jemalloc for FEXConfig
2021-07-09 04:58:51 -07:00
Ryan Houdek 78a9dcf505 Actually fix jemalloc for FEXConfig 2021-07-09 03:48:29 -07:00
Ryan Houdek 1eee2dd004 Merge pull request #1155 from Sonicadvance1/improve_startup_time
Switch some memory allocations over to mmap
2021-07-09 03:47:05 -07:00
Ryan Houdek 25866a00ed Disable rtsignal gvisor test
This test is getting frustrating and is becoming MORE flakey as threads become faster
2021-07-09 01:57:48 -07:00
Ryan Houdek a27c05f577 Switch some memory allocations over to mmap
This cuts start up time from 8.962ms to 3.949ms and lets us use less physical memory.
We pay a small amortized cost from the faulting to populate pages which will be fairly low
2021-07-09 01:57:47 -07:00
Ryan Houdek 0d9ab9ca42 Merge pull request #1154 from Sonicadvance1/fix_jemalloc_override
Update jemalloc to fix glibc override.
2021-07-08 14:48:43 -07:00
Ryan Houdek 9c4c9ea566 Update jemalloc to fix glibc override.
For some reason jemalloc was missing an override
2021-07-08 14:33:35 -07:00
Stefanos Kornilios Mitsis Poiitidis a9bb33b8c2 Merge pull request #1152 from Sonicadvance1/fix_jemalloc_malloc_replace
Fix jemalloc malloc replace
2021-07-08 18:13:20 +03:00
Ryan Houdek faf09fec5f Removes glibc symbol replacement inside FEX
FEX_jemalloc now handles this entirely by itself.
It now replaces glibc during startup rather than when we call for it.
2021-07-06 21:52:57 -07:00
Ryan Houdek 4cd2b1a959 Update Externals jemalloc 2021-07-06 21:20:17 -07:00
Stefanos Kornilios Mitsis Poiitidis 8f09bd9215 Merge pull request #1150 from Sonicadvance1/fix_old_kernel_defines
Fixes old kernel defines for sockios
2021-07-06 15:39:54 +03:00
Stefanos Kornilios Mitsis Poiitidis 84dee03a16 Merge pull request #1151 from Sonicadvance1/error_on_immediate_failure
FEXLoader: stderr on immediate failure
2021-07-06 14:36:11 +03:00
Stefanos Kornilios Mitsis Poiitidis 0a8ee43081 Merge pull request #1149 from Sonicadvance1/enable_local_xxhash
Use externals xxhash if not found installed
2021-07-06 14:35:18 +03:00
Ryan Houdek 9fbeba3b79 FEXLoader: stderr on immediate failure
In the case that there is an immediate configuration failure. Use stderr specifically for outputting.
These errors won't be output typically because silent logging is enabled by default.
In the case of executable missing or rootfs configuration failure, print directly to stderr.
Previously it looked like FEX just exited for no reason.
We had multiple users encounter this and be confused
2021-07-05 18:52:06 -07:00
Ryan Houdek b3bab814de Fixes old kernel defines for sockios
These defines were changed at in kernel v5.2 but the debian defines didn't seem to change?
Redefine them if missing
2021-07-05 18:26:44 -07:00
Ryan Houdek 728e9fc187 Use externals xxhash if not found installed
Also in the case that you don't have v0.8.0 minimum installed
2021-07-05 17:42:35 -07:00
Ryan Houdek e01e6d9293 Adds xxhash external submodule 2021-07-05 17:39:19 -07:00
Stefanos Kornilios Mitsis Poiitidis 1a64c908e5 Merge pull request #1146 from Sonicadvance1/implement_sigqueueinfo
Linux: Implements rt_{tg,}sigqueueinfo
2021-07-05 16:55:34 +03:00
Stefanos Kornilios Mitsis Poiitidis 7dd700394e Merge pull request #1147 from Sonicadvance1/fix_imul_flags
OpDispatcher: Fixes imul flags calculations
2021-07-05 16:55:16 +03:00
Ryan Houdek b7b9d4089f unittests: Adds more imul unit tests for flags 2021-07-03 21:48:18 -07:00
Ryan Houdek 5a7dd844b0 OpDispatcher: Fixes imul flags calculations
We were calculating the high bits incorrectly in a couple variants
2021-07-03 21:45:15 -07:00
Ryan Houdek 80da08b15b Fixes Signal handler check for queue signals
Any signals sent with sigqueue or kill/sigsend are safe on the siginfo path.
2021-07-03 17:12:42 -07:00
Ryan Houdek 20a519a250 unittests: Update posix tests that have been fixed 2021-07-02 23:00:48 -07:00
Ryan Houdek b9efbd6159 Linux: Implements rt_{tg,}sigqueueinfo
We already support receiving user signals in our signal handlers for this.
Just need to push through syscalls.

This fixes Dead Island Definitive Edition so it runs.
2021-07-02 22:53:27 -07:00
Ryan Houdek 749f1eba51 Merge pull request #1145 from lioncash/stub
GdbStub: Fix memory leak in GdbServerLoop()
2021-07-02 03:21:00 -07:00
Lioncash bc52a16fde GdbStub: Fix memory leak in GdbServerLoop()
Previously the opened socket stream would be leaked.
2021-07-02 05:53:59 -04:00
Stefanos Kornilios Mitsis Poiitidis 2db44cf2ed Merge pull request #1142 from Sonicadvance1/implement_base_signalfd
Linux: Implements a base implementation of signalfd{4,}
2021-07-02 10:04:26 +03:00
Ryan Houdek 336eb9a8b4 Merge pull request #1143 from Sonicadvance1/remove_fork_log
Linux: Remove logs about parent or child stack usage
2021-07-01 12:46:18 -07:00
Ryan Houdek b923a82c39 unittests: fallocate gvisor test no longer fails 2021-07-01 07:11:00 -07:00
Ryan Houdek 1304279eb5 Linux: Remove logs about parent or child stack usage
We don't need to care about what the guest stacks are doing now
2021-07-01 07:06:14 -07:00
Ryan Houdek 1f6926a245 Linux: Implements a base implementation of signalfd{4,}
This is a base implementation of signalfd.
Signalfd allows the application to receive siginfo_t information through an FD.
The FD is either provided by the application or created by the kernel depending.
This specifically doesn't pick up *true* synchronous signals. tgkill of the number
should theoretically go through this interface.

This very specifically skips our internal required signals for now.
This means it won't pick up SIGILL, SIGBUS, or SIG63.

This is enough to capture an application that just wants to poll for SIGCHLD.
Anything more complex has the same problems of the guest handling a siginfo_t.
2021-07-01 06:51:39 -07:00
Stefanos Kornilios Mitsis Poiitidis 2d11df03cb Merge pull request #1141 from Sonicadvance1/modify_cpack_name
CPack: Update package name to remove conflict
2021-07-01 16:44:44 +03:00
Stefanos Kornilios Mitsis Poiitidis d142d7c363 Merge pull request #1140 from Sonicadvance1/cpuid_tm
CPUID: Update TM and TM2 CPUID bits
2021-07-01 16:44:09 +03:00
Ryan Houdek b546869d9d CPack: Update package name to remove conflict
'fex' is already taken by another projects. Use 'fex-emu' instead; Similar to other emulator package names.
2021-07-01 05:08:57 -07:00
Ryan Houdek 2a49ad941f CPUID: Update TM and TM2 CPUID bits
These are in place to let applications know that we do our own thermal management

TM was introduced with Pentium M, it would dispatch idle uops to counter thermals.
TM2 was later and is what allows the CPU to dynamically change its multiplier and frequency.
All in an effort to not cook themselves.

Our ARM CPUs also do all this. Claim it as well.
2021-07-01 04:26:39 -07:00
393 changed files with 39180 additions and 13149 deletions

No files matched your search

+7
View File
@@ -39,3 +39,10 @@
[submodule "External/drm-headers"]
path = External/drm-headers
url = https://github.com/FEX-Emu/drm-headers.git
[submodule "External/xxhash"]
path = External/xxhash
url = https://github.com/FEX-Emu/xxHash.git
[submodule "External/Vulkan-Docs"]
shallow = true
path = External/Vulkan-Docs
url = https://github.com/KhronosGroup/Vulkan-Docs.git
+207 -43
View File
@@ -14,6 +14,9 @@ option(ENABLE_ASSERTIONS "Enables assertions in build" FALSE)
option(ENABLE_VISUAL_DEBUGGER "Enables the visual debugger for compiling" FALSE)
option(ENABLE_STRICT_WERROR "Enables stricter -Werror for CI" FALSE)
option(ENABLE_WERROR "Enables -Werror" FALSE)
option(ENABLE_STATIC_PIE "Enables static-pie build" FALSE)
option(ENABLE_JEMALLOC "Enables jemalloc allocator" TRUE)
option(ENABLE_OFFLINE_TELEMETRY "Enables FEX offline telemetry" TRUE)
set (X86_C_COMPILER "x86_64-linux-gnu-gcc" CACHE STRING "c compiler for compiling x86 guest libs")
set (X86_CXX_COMPILER "x86_64-linux-gnu-g++" CACHE STRING "c++ compiler for compiling x86 guest libs")
@@ -44,38 +47,6 @@ else()
set(CMAKE_INTERPROCEDURAL_OPTIMIZATION FALSE)
endif()
find_program(CCACHE_PROGRAM ccache)
if(CCACHE_PROGRAM)
message(STATUS "CCache enabled")
set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE "${CCACHE_PROGRAM}")
endif()
if (ENABLE_XRAY)
add_compile_options(-fxray-instrument)
link_libraries(-fxray-instrument)
endif()
if (ENABLE_LLD)
link_libraries(-fuse-ld=lld)
endif()
if (ENABLE_ASAN)
add_definitions(-DENABLE_ASAN=1)
add_compile_options(-fno-omit-frame-pointer -fsanitize=address -fsanitize-address-use-after-scope)
link_libraries(-fno-omit-frame-pointer -fsanitize=address -fsanitize-address-use-after-scope)
endif()
if (ENABLE_TSAN)
add_compile_options(-fno-omit-frame-pointer -fsanitize=thread)
link_libraries(-fno-omit-frame-pointer -fsanitize=thread)
endif()
set (CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} -fno-omit-frame-pointer")
set (CMAKE_LINKER_FLAGS_RELWITHDEBINFO "${CMAKE_LINKER_FLAGS_RELWITHDEBINFO} -fno-omit-frame-pointer")
set (CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -fomit-frame-pointer")
set (CMAKE_LINKER_FLAGS_RELEASE "${CMAKE_LINKER_FLAGS_RELEASE} -fomit-frame-pointer")
if (CMAKE_SYSTEM_PROCESSOR MATCHES "x86_64")
option(ENABLE_X86_HOST_DEBUG "Enables compiling on x86_64 host" FALSE)
if (NOT ENABLE_X86_HOST_DEBUG)
@@ -95,6 +66,181 @@ if (CMAKE_SYSTEM_PROCESSOR MATCHES "aarch64")
add_definitions(-D_M_ARM_64=1)
endif()
find_program(CCACHE_PROGRAM ccache)
if(CCACHE_PROGRAM)
message(STATUS "CCache enabled")
set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE "${CCACHE_PROGRAM}")
endif()
if (ENABLE_XRAY)
add_compile_options(-fxray-instrument)
link_libraries(-fxray-instrument)
endif()
set (PTHREAD_LIB pthread)
if (ENABLE_LLD)
set (LD_OVERRIDE "-fuse-ld=lld")
link_libraries(${LD_OVERRIDE})
endif()
if (NOT ENABLE_OFFLINE_TELEMETRY)
# Disable FEX offline telemetry entirely if asked
add_definitions(-DFEX_DISABLE_TELEMETRY=1)
endif()
if (ENABLE_STATIC_PIE)
if (_M_ARM_64 AND ENABLE_LLD)
message (FATAL_ERROR "Static linking does not currently work with AArch64+LLD. Use GNU ld for now.")
endif()
file(WRITE ${PROJECT_BINARY_DIR}/CMakeFiles/CMakeTmp/Determine_iplt.c
"int main(int argc, char* argv[])
{
return 0;
}")
# Compile the test application with our LD_OVERRIDE and static-pie options
try_compile(
COMPILE_RESULT
${PROJECT_BINARY_DIR}/CMakeFiles/CMakeTmp
${PROJECT_BINARY_DIR}/CMakeFiles/CMakeTmp/Determine_iplt.c
COMPILE_DEFINITIONS "-fPIE ${LD_OVERRIDE}"
LINK_LIBRARIES "-static-pie ${LD_OVERRIDE}"
COPY_FILE ${PROJECT_BINARY_DIR}/CMakeFiles/CMakeTmp/Determine_iplt
)
if (${COMPILE_RESULT})
# Read the symbols from the elf
execute_process(COMMAND
readelf -s ${PROJECT_BINARY_DIR}/CMakeFiles/CMakeTmp/Determine_iplt
OUTPUT_FILE ${PROJECT_BINARY_DIR}/CMakeFiles/CMakeTmp/plt_out.txt
OUTPUT_VARIABLE PLT_SYMBOLS)
# Pull out the __rela_iplt_{start,end} symbols if they exist
execute_process(COMMAND
"grep" "__rela_iplt" ${PROJECT_BINARY_DIR}/CMakeFiles/CMakeTmp/plt_out.txt
OUTPUT_VARIABLE PLT_SYMBOLS)
set (SYMBOLS_FINE TRUE)
set (HAS_IPLT -1)
# Check if we have any symbols in our grep output
# The symbols must either not exist at all OR the symbols are zero
if (PLT_SYMBOLS)
string(FIND ${PLT_SYMBOLS} "__rela_iplt_start" HAS_IPLT)
endif()
if (NOT HAS_IPLT EQUAL -1)
# We have some symbols from readelf. Let's parse the results to check if they are zero
# Format: '35: 0000000000000000 0 NOTYPE LOCAL HIDDEN UND __rela_iplt_start'
string(REPLACE "\n" ";" SYMBOL_LIST ${PLT_SYMBOLS})
foreach (SYMBOL ${SYMBOL_LIST})
# strip any leading and trailing whitespace
string (STRIP ${SYMBOL} SYMBOL)
# Convert string to a list
string(REPLACE " " ";" SYMBOL_VALUES ${SYMBOL}})
# Pull out the address argument
list(GET SYMBOL_VALUES 1 OFFSET)
# Check against integer zero
if (NOT ${OFFSET} EQUAL 0)
# Symbol wasn't zero, this now fails
set (SYMBOLS_FINE FALSE)
endif()
endforeach()
endif()
if (SYMBOLS_FINE)
# We can now exnable static-pie
set (STATIC_PIE_OPTIONS "-static-pie")
# Pthreads has an issue with exposing symbols
# We need to make some concessions to the pthread gods
if (ENABLE_LLD)
set (PTHREAD_LIB
-Wl,--undefined-glob=pthread_*
-Wl,--undefined=__cxa_finalize
-Wl,--undefined=_pthread_cleanup_push_defer
-Wl,--undefined=_pthread_cleanup_pop_restore
-Wl,--undefined=__pthread_cleanup_upto
pthread)
else()
set (PTHREAD_LIB
-Wl,--undefined=pthread_join
-Wl,--undefined=pthread_attr_getdetachstate
-Wl,--undefined=pthread_sigmask
-Wl,--undefined=pthread_mutex_lock
-Wl,--undefined=pthread_cond_init
-Wl,--undefined=pthread_attr_init
-Wl,--undefined=pthread_mutex_unlock
-Wl,--undefined=pthread_mutexattr_destroy
-Wl,--undefined=pthread_detach
-Wl,--undefined=pthread_mutex_init
-Wl,--undefined=pthread_getattr_np
-Wl,--undefined=pthread_cond_timedwait
-Wl,--undefined=pthread_attr_destroy
-Wl,--undefined=pthread_mutexattr_settype
-Wl,--undefined=pthread_rwlock_unlock
-Wl,--undefined=pthread_rwlock_wrlock
-Wl,--undefined=pthread_setspecific
-Wl,--undefined=pthread_create
-Wl,--undefined=pthread_cond_clockwait
-Wl,--undefined=pthread_key_create
-Wl,--undefined=pthread_rwlock_rdlock
-Wl,--undefined=pthread_setname_np
-Wl,--undefined=pthread_cond_signal
-Wl,--undefined=pthread_mutexattr_init
-Wl,--undefined=pthread_attr_setstack
-Wl,--undefined=pthread_self
-Wl,--undefined=pthread_getaffinity_np
-Wl,--undefined=pthread_cond_wait
-Wl,--undefined=pthread_mutex_trylock
-Wl,--undefined=pthread_cond_broadcast
-Wl,--undefined=pthread_cond_destroy
-Wl,--undefined=pthread_getspecific
-Wl,--undefined=pthread_key_delete
-Wl,--undefined=pthread_once
-Wl,--undefined=__cxa_finalize
-Wl,--undefined=_pthread_cleanup_push_defer
-Wl,--undefined=_pthread_cleanup_pop_restore
-Wl,--undefined=__pthread_cleanup_upto
pthread)
endif()
else()
message (FATAL_ERROR "Application has __rela_iplt_{start,end} symbols. Which means static-pie can't be enabled")
endif()
else()
message (FATAL_ERROR "Couldn't compile static-pie test. Static-pie can't be enabled! Is your glibc compiled without static-pie?")
endif()
endif()
if (ENABLE_ASAN)
add_definitions(-DENABLE_ASAN=1)
add_compile_options(-fno-omit-frame-pointer -fsanitize=address -fsanitize-address-use-after-scope)
link_libraries(-fno-omit-frame-pointer -fsanitize=address -fsanitize-address-use-after-scope)
endif()
if (ENABLE_TSAN)
add_compile_options(-fno-omit-frame-pointer -fsanitize=thread)
link_libraries(-fno-omit-frame-pointer -fsanitize=thread)
endif()
if (ENABLE_JEMALLOC)
add_definitions(-DENABLE_JEMALLOC=1)
add_subdirectory(External/jemalloc/)
include_directories(External/jemalloc/pregen/include/)
else()
message (STATUS
" jemalloc disabled!\n"
" This is not a recommended configuration!\n"
" This will very explicitly break 32-bit application execution!\n"
" Use at your own risk!")
endif()
set (CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} -fno-omit-frame-pointer")
set (CMAKE_LINKER_FLAGS_RELWITHDEBINFO "${CMAKE_LINKER_FLAGS_RELWITHDEBINFO} -fno-omit-frame-pointer")
set (CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -fomit-frame-pointer")
set (CMAKE_LINKER_FLAGS_RELEASE "${CMAKE_LINKER_FLAGS_RELEASE} -fomit-frame-pointer")
add_subdirectory(External/vixl/)
include_directories(External/vixl/src/)
@@ -105,14 +251,17 @@ endif()
find_package(PkgConfig REQUIRED)
find_package(Python 3.0 REQUIRED COMPONENTS Interpreter)
pkg_check_modules(XXHASH libxxhash REQUIRED)
pkg_check_modules(XXHASH libxxhash>=0.8.0 QUIET)
if (NOT XXHASH_FOUND)
message(STATUS "xxHash not found. Using Externals")
add_subdirectory(External/xxhash/)
include_directories(External/xxhash/)
endif()
add_definitions(-Wno-trigraphs)
add_definitions(-DGLOBAL_DATA_DIRECTORY="${DATA_DIRECTORY}/")
add_subdirectory(External/jemalloc/)
include_directories(External/jemalloc/pregen/include/)
add_subdirectory(External/cpp-optparse/)
include_directories(External/cpp-optparse/)
@@ -151,11 +300,6 @@ if(ENUM_ENUM_WARNING)
add_compile_options(-Wno-deprecated-enum-enum-conversion)
endif()
check_cxx_compiler_flag("-march=native" COMPILER_SUPPORTS_MARCH_NATIVE)
if(COMPILER_SUPPORTS_MARCH_NATIVE)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -march=native")
endif()
if(ENABLE_WERROR OR ENABLE_STRICT_WERROR)
add_compile_options(-Werror)
if (NOT ENABLE_STRICT_WERROR)
@@ -187,6 +331,11 @@ if(_M_ARM_64)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -mcpu=${AARCH64_CPU}")
endif()
endif()
else()
check_cxx_compiler_flag("-march=native" COMPILER_SUPPORTS_MARCH_NATIVE)
if(COMPILER_SUPPORTS_MARCH_NATIVE)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -march=native")
endif()
endif()
if (ENABLE_IWYU)
@@ -267,6 +416,11 @@ add_subdirectory(Data/binfmts/)
add_subdirectory(Source/)
add_subdirectory(Data/AppConfig/)
# Install the ThunksDB file
install(
FILES ${CMAKE_CURRENT_SOURCE_DIR}/Data/ThunksDB.json
DESTINATION ${DATA_DIRECTORY}/)
if (BUILD_TESTS)
add_subdirectory(unittests/)
endif()
@@ -278,7 +432,10 @@ if (BUILD_THUNKS)
PREFIX host-libs
SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/ThunkLibs/HostLibs"
BINARY_DIR "Host"
CMAKE_ARGS "-DCMAKE_INSTALL_PREFIX=${CMAKE_INSTALL_PREFIX}"
CMAKE_ARGS
"-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}"
"-DCMAKE_INSTALL_PREFIX=${CMAKE_INSTALL_PREFIX}"
"-DVULKAN_XML=${CMAKE_SOURCE_DIR}/External/Vulkan-Docs/xml/vk.xml"
INSTALL_COMMAND ""
BUILD_ALWAYS ON
)
@@ -296,7 +453,13 @@ if (BUILD_THUNKS)
PREFIX guest-libs
SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/ThunkLibs/GuestLibs"
BINARY_DIR "Guest"
CMAKE_ARGS "-DX86_C_COMPILER:STRING=${X86_C_COMPILER}" "-DX86_CXX_COMPILER:STRING=${X86_CXX_COMPILER}" "-DCMAKE_INSTALL_PREFIX=${CMAKE_INSTALL_PREFIX}"
CMAKE_ARGS
"-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}"
"-DX86_C_COMPILER:STRING=${X86_C_COMPILER}"
"-DX86_CXX_COMPILER:STRING=${X86_CXX_COMPILER}"
"-DCMAKE_INSTALL_PREFIX=${CMAKE_INSTALL_PREFIX}"
"-DSTRUCT_VERIFIER=${CMAKE_SOURCE_DIR}/Scripts/StructPackVerifier.py"
"-DVULKAN_XML=${CMAKE_SOURCE_DIR}/External/Vulkan-Docs/xml/vk.xml"
INSTALL_COMMAND ""
BUILD_ALWAYS ON
)
@@ -356,6 +519,7 @@ endif()
# Package creation
set (CPACK_GENERATOR "DEB")
set (CPACK_PACKAGE_NAME fex-emu)
set (CPACK_PACKAGE_CONTACT "team@fex-emu.org")
set (CPACK_PACKAGE_VERSION_MAJOR "${FEX_VERSION_MAJOR}")
set (CPACK_PACKAGE_VERSION_MINOR "${FEX_VERSION_MINOR}")
+5
View File
@@ -0,0 +1,5 @@
{
"Config": {
"StallProcess": "1"
}
}
+287
View File
@@ -0,0 +1,287 @@
{
"DB": {
"GL": {
"Library" : "libGL-guest.so",
"Depends": [
"X11"
],
"Overlay": [
"/usr/lib/x86_64-linux-gnu/libGL.so",
"/usr/lib/x86_64-linux-gnu/libGL.so.1",
"/usr/lib/x86_64-linux-gnu/libGL.so.1.2.0",
"/usr/lib/x86_64-linux-gnu/libGL.so.1.7.0",
"/lib/x86_64-linux-gnu/libGL.so",
"/lib/x86_64-linux-gnu/libGL.so.1",
"/lib/x86_64-linux-gnu/libGL.so.1.2.0",
"/lib/x86_64-linux-gnu/libGL.so.1.7.0"
]
},
"GLESv2": {
"Library": "libGLESv2-guest.so",
"Depends": [
"X11"
],
"Overlay": [
"/usr/lib/x86_64-linux-gnu/libGLESv2.so",
"/usr/lib/x86_64-linux-gnu/libGLESv2.so.2",
"/usr/lib/x86_64-linux-gnu/libGLESv2.so.2.0.0",
"/lib/x86_64-linux-gnu/libGLESv2.so",
"/lib/x86_64-linux-gnu/libGLESv2.so.2",
"/lib/x86_64-linux-gnu/libGLESv2.so.2.0.0"
]
},
"X11": {
"Library": "libX11-guest.so",
"Overlay": [
"/usr/lib/x86_64-linux-gnu/libX11.so",
"/usr/lib/x86_64-linux-gnu/libX11.so.6",
"/usr/lib/x86_64-linux-gnu/libX11.so.6.4.0",
"/lib/x86_64-linux-gnu/libX11.so",
"/lib/x86_64-linux-gnu/libX11.so.6",
"/lib/x86_64-linux-gnu/libX11.so.6.4.0"
]
},
"Vulkan-radeon": {
"Library": "libvulkan_radeon-guest.so",
"Depends": [
"xcb"
],
"Overlay": [
"/usr/lib/x86_64-linux-gnu/libvulkan_radeon.so",
"/lib/x86_64-linux-gnu/libvulkan_radeon.so"
],
"Comment": [
"Vulkan library relies on xcb, otherwise it crashes with jemalloc"
]
},
"Vulkan-lavapipe": {
"Library": "libvulkan_lvp-guest.so",
"Depends": [
"xcb"
],
"Overlay": [
"/usr/lib/x86_64-linux-gnu/libvulkan_lvp.so",
"/lib/x86_64-linux-gnu/libvulkan_lvp.so"
]
},
"Vulkan-freedreno": {
"Library": "libvulkan_freedreno-guest.so",
"Depends": [
"xcb"
],
"Overlay": [
"/usr/lib/x86_64-linux-gnu/libvulkan_freedreno.so",
"/lib/x86_64-linux-gnu/libvulkan_freedreno.so"
]
},
"Vulkan-intel": {
"Library": "libvulkan_intel-guest.so",
"Depends": [
"xcb"
],
"Overlay": [
"/usr/lib/x86_64-linux-gnu/libvulkan_intel.so",
"/lib/x86_64-linux-gnu/libvulkan_intel.so"
]
},
"Vulkan-panfrost": {
"Library": "libvulkan_panfrost-guest.so",
"Depends": [
"xcb"
],
"Overlay": [
"/usr/lib/x86_64-linux-gnu/libvulkan_panfrost.so",
"/lib/x86_64-linux-gnu/libvulkan_panfrost.so"
]
},
"Vulkan-nvidia": {
"Library": "libvulkan_nvidia-guest.so",
"Depends": [
"xcb"
],
"Overlay": [
"/usr/lib/x86_64-linux-gnu/libGLX_nvidia.so.0",
"/lib/x86_64-linux-gnu/libGLX_nvidia.so.0"
],
"Comment": [
"Not currently wired up"
]
},
"Vulkan-virtio": {
"Library": "libvulkan_virtio-guest.so",
"Depends": [
"xcb"
],
"Overlay": [
"/usr/lib/x86_64-linux-gnu/libvulkan_virtio.so",
"/lib/x86_64-linux-gnu/libvulkan_virtio.so"
]
},
"xcb": {
"Library": "libxcb-guest.so",
"Overlay": [
"/usr/lib/x86_64-linux-gnu/libxcb.so",
"/usr/lib/x86_64-linux-gnu/libxcb.so.1",
"/usr/lib/x86_64-linux-gnu/libxcb.so.1.1.0",
"/lib/x86_64-linux-gnu/libxcb.so",
"/lib/x86_64-linux-gnu/libxcb.so.1",
"/lib/x86_64-linux-gnu/libxcb.so.1.1.0"
]
},
"xcb-dri2": {
"Library": "libxcb_dri2-guest.so",
"Overlay": [
"/usr/lib/x86_64-linux-gnu/libxcb-dri2.so",
"/usr/lib/x86_64-linux-gnu/libxcb-dri2.so.0",
"/usr/lib/x86_64-linux-gnu/libxcb-dri2.so.0.0.0",
"/lib/x86_64-linux-gnu/libxcb-dri2.so",
"/lib/x86_64-linux-gnu/libxcb-dri2.so.0",
"/lib/x86_64-linux-gnu/libxcb-dri2.so.0.0.0"
]
},
"xcb-dri3": {
"Library": "libxcb_dri3-guest.so",
"Overlay": [
"/usr/lib/x86_64-linux-gnu/libxcb-dri3.so",
"/usr/lib/x86_64-linux-gnu/libxcb-dri3.so.0",
"/usr/lib/x86_64-linux-gnu/libxcb-dri3.so.0.0.0",
"/lib/x86_64-linux-gnu/libxcb-dri3.so",
"/lib/x86_64-linux-gnu/libxcb-dri3.so.0",
"/lib/x86_64-linux-gnu/libxcb-dri3.so.0.0.0"
]
},
"xcb-xfixes": {
"Library": "libxcb_xfixes-guest.so",
"Overlay": [
"/usr/lib/x86_64-linux-gnu/libxcb-xfixes.so",
"/usr/lib/x86_64-linux-gnu/libxcb-xfixes.so.0",
"/usr/lib/x86_64-linux-gnu/libxcb-xfixes.so.0.0.0",
"/lib/x86_64-linux-gnu/libxcb-xfixes.so",
"/lib/x86_64-linux-gnu/libxcb-xfixes.so.0",
"/lib/x86_64-linux-gnu/libxcb-xfixes.so.0.0.0"
]
},
"xcb-shm": {
"Library": "libxcb_shm-guest.so",
"Overlay": [
"/usr/lib/x86_64-linux-gnu/libxcb-shm.so",
"/usr/lib/x86_64-linux-gnu/libxcb-shm.so.0",
"/usr/lib/x86_64-linux-gnu/libxcb-shm.so.0.0.0",
"/lib/x86_64-linux-gnu/libxcb-shm.so",
"/lib/x86_64-linux-gnu/libxcb-shm.so.0",
"/lib/x86_64-linux-gnu/libxcb-shm.so.0.0.0"
]
},
"xcb-sync": {
"Library": "libxcb_sync-guest.so",
"Overlay": [
"/usr/lib/x86_64-linux-gnu/libxcb-sync.so",
"/usr/lib/x86_64-linux-gnu/libxcb-sync.so.1",
"/usr/lib/x86_64-linux-gnu/libxcb-sync.so.1.0.0",
"/lib/x86_64-linux-gnu/libxcb-sync.so",
"/lib/x86_64-linux-gnu/libxcb-sync.so.1",
"/lib/x86_64-linux-gnu/libxcb-sync.so.1.0.0"
]
},
"xcb-randr": {
"Library": "libxcb_randr-guest.so",
"Overlay": [
"/usr/lib/x86_64-linux-gnu/libxcb-randr.so",
"/usr/lib/x86_64-linux-gnu/libxcb-randr.so.0",
"/usr/lib/x86_64-linux-gnu/libxcb-randr.so.0.1.0",
"/lib/x86_64-linux-gnu/libxcb-randr.so",
"/lib/x86_64-linux-gnu/libxcb-randr.so.0",
"/lib/x86_64-linux-gnu/libxcb-randr.so.0.1.0"
]
},
"xcb-present": {
"Library": "libxcb_present-guest.so",
"Overlay": [
"/usr/lib/x86_64-linux-gnu/libxcb-present.so",
"/usr/lib/x86_64-linux-gnu/libxcb-present.so.0",
"/usr/lib/x86_64-linux-gnu/libxcb-present.so.0.0.0",
"/lib/x86_64-linux-gnu/libxcb-present.so",
"/lib/x86_64-linux-gnu/libxcb-present.so.0",
"/lib/x86_64-linux-gnu/libxcb-present.so.0.0.0"
]
},
"xcb-glx": {
"Library": "libxcb_glx-guest.so",
"Overlay": [
"/usr/lib/x86_64-linux-gnu/libxcb-glx.so",
"/usr/lib/x86_64-linux-gnu/libxcb-glx.so.0",
"/usr/lib/x86_64-linux-gnu/libxcb-glx.so.0.0.0",
"/lib/x86_64-linux-gnu/libxcb-glx.so",
"/lib/x86_64-linux-gnu/libxcb-glx.so.0",
"/lib/x86_64-linux-gnu/libxcb-glx.so.0.0.0"
]
},
"xshmfence": {
"Library": "libshmfence-guest.so",
"Overlay": [
"/usr/lib/x86_64-linux-gnu/libxshmfence.so",
"/usr/lib/x86_64-linux-gnu/libxshmfence.so.1",
"/usr/lib/x86_64-linux-gnu/libxshmfence.so.1.0.0",
"/lib/x86_64-linux-gnu/libxshmfence.so",
"/lib/x86_64-linux-gnu/libxshmfence.so.1",
"/lib/x86_64-linux-gnu/libxshmfence.so.1.0.0"
]
},
"drm": {
"Library": "libdrm-guest.so",
"Overlay": [
"/usr/lib/x86_64-linux-gnu/libdrm.so",
"/usr/lib/x86_64-linux-gnu/libdrm.so.2",
"/usr/lib/x86_64-linux-gnu/libdrm.so.2.4.0",
"/lib/x86_64-linux-gnu/libdrm.so",
"/lib/x86_64-linux-gnu/libdrm.so.2",
"/lib/x86_64-linux-gnu/libdrm.so.2.4.0"
]
},
"asound": {
"Library": "libasound-guest.so",
"Overlay": [
"/usr/lib/x86_64-linux-gnu/libasound.so",
"/usr/lib/x86_64-linux-gnu/libasound.so.2",
"/usr/lib/x86_64-linux-gnu/libasound.so.2.0.0",
"/lib/x86_64-linux-gnu/libasound.so",
"/lib/x86_64-linux-gnu/libasound.so.2",
"/lib/x86_64-linux-gnu/libasound.so.2.0.0"
]
},
"Xrender": {
"Library": "libXrender-guest.so",
"Overlay": [
"/usr/lib/x86_64-linux-gnu/libXrender.so",
"/usr/lib/x86_64-linux-gnu/libXrender.so.1",
"/usr/lib/x86_64-linux-gnu/libXrender.so.1.3.0",
"/lib/x86_64-linux-gnu/libXrender.so",
"/lib/x86_64-linux-gnu/libXrender.so.1",
"/lib/x86_64-linux-gnu/libXrender.so.1.3.0"
]
},
"Xext": {
"Library": "libXext-guest.so",
"Overlay": [
"/usr/lib/x86_64-linux-gnu/libXext.so",
"/usr/lib/x86_64-linux-gnu/libXext.so.6",
"/usr/lib/x86_64-linux-gnu/libXext.so.6.4.0",
"/lib/x86_64-linux-gnu/libXext.so",
"/lib/x86_64-linux-gnu/libXext.so.6",
"/lib/x86_64-linux-gnu/libXext.so.6.4.0"
]
},
"Xfixes": {
"Library": "libXfixes-guest.so",
"Overlay": [
"/usr/lib/x86_64-linux-gnu/libXfixes.so",
"/usr/lib/x86_64-linux-gnu/libXfixes.so.3",
"/usr/lib/x86_64-linux-gnu/libXfixes.so.3.1.0",
"/lib/x86_64-linux-gnu/libXfixes.so",
"/lib/x86_64-linux-gnu/libXfixes.so.3",
"/lib/x86_64-linux-gnu/libXfixes.so.3.1.0"
]
},
"":{}
}
}
+17 -4
View File
@@ -1,4 +1,17 @@
install(FILES FEX-x86
DESTINATION ${CMAKE_INSTALL_PREFIX}/share/binfmts/)
install(FILES FEX-x86_64
DESTINATION ${CMAKE_INSTALL_PREFIX}/share/binfmts/)
function(GenBinFmt Name)
# Get the filename only component
get_filename_component(FMT_NAME ${Name} NAME_WE)
# Configure it
configure_file(
${Name}
${CMAKE_BINARY_DIR}/Data/binfmts/${FMT_NAME})
# Then install the configured binfmt
install(
FILES ${CMAKE_BINARY_DIR}/Data/binfmts/${FMT_NAME}
DESTINATION ${CMAKE_INSTALL_PREFIX}/share/binfmts/)
endfunction()
GenBinFmt(FEX-x86.in)
GenBinFmt(FEX-x86_64.in)
@@ -1,9 +1,8 @@
package fex
interpreter /usr/bin/FEXInterpreter
interpreter @CMAKE_INSTALL_PREFIX@/bin/FEXInterpreter
magic \x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x03\x00
offset 0
mask \xff\xff\xff\xff\xff\xfe\xfe\x00\x00\x00\x00\xff\xff\xff\xff\xff\xfe\xff\xff\xff
credentials yes
fix_binary yes
preserve no
preserve yes
@@ -1,8 +1,8 @@
package fex
interpreter /usr/bin/FEXInterpreter
interpreter @CMAKE_INSTALL_PREFIX@/bin/FEXInterpreter
magic \x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x3e\x00
offset 0
mask \xff\xff\xff\xff\xff\xfe\xfe\x00\x00\x00\x00\xff\xff\xff\xff\xff\xfe\xff\xff\xff
credentials yes
fix_binary yes
preserve no
preserve yes
-1
View File
@@ -16,7 +16,6 @@ endif()
set(ENABLE_JIT_X86_64 ${_M_X86_64} CACHE BOOL "Enable the x86_64 JIT")
set(ENABLE_JIT_ARM64 ${_M_ARM_64} CACHE BOOL "Enable the ARM64 JIT")
option(ENABLE_CLANG_FORMAT "Run clang format over the source" FALSE)
option(ENABLE_JITSYMBOLS "Enable visibility of JITSymbols in profiling tools" FALSE)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
cmake_policy(SET CMP0083 NEW) # Follow new PIE policy
+33 -8
View File
@@ -98,14 +98,19 @@ def print_man_option(short, long, desc, default):
output_man.write("\\fBdefault:\\fR {0}\n".format(default))
output_man.write(".Pp\n\n")
def print_man_env_option(name, desc, default):
output_man.write("\\fBFEX_{0}\\fR\n".format(name))
def print_man_env_option(name, desc, default, no_json_key):
output_man.write("\\fBFEX_{0}\\fR\n".format(name.upper()))
# Print description
for line in desc:
output_man.write(".Pp\n")
output_man.write("{0}\n".format(line))
if (not no_json_key):
output_man.write(".Pp\n")
output_man.write("\\fBJSON key:\\fR '{0}'\n".format(name))
output_man.write(".Pp\n\n")
output_man.write(".Pp\n")
output_man.write("\\fBdefault:\\fR {0}\n".format(default))
output_man.write(".Pp\n\n")
@@ -154,9 +159,10 @@ def print_man_environment(options):
# Wrap the string argument in quotes
default = "'" + default + "'"
print_man_env_option(
op_key.upper(),
op_key,
op_vals["Desc"],
default
default,
False
)
print_man_environment_tail()
@@ -172,7 +178,7 @@ def print_man_environment_tail():
"By default FEX will look in {$HOME, $XDG_CONFIG_HOME}/.fex-emu/",
"This will override the full path",
],
"''")
"''", True)
print_man_env_option(
"FEX_APP_CONFIG",
@@ -183,7 +189,7 @@ def print_man_environment_tail():
"One must be careful with this option as it will override any applications that load with execve as well"
"If you need to support applications that execve then use FEX_APP_CONFIG_LOCATION instead"
],
"''")
"''", True)
print_man_env_option(
"FEX_APP_DATA_LOCATION",
@@ -193,7 +199,7 @@ def print_man_environment_tail():
"This will override the full path",
"This is the folder where FEX stores generated files like IR cache"
],
"''")
"''", True)
def print_man_header():
header ='''.Dd {0}
@@ -368,7 +374,7 @@ def print_parse_argloader_options(options):
conversion_func = "std::to_string"
if ("ArgumentHandler" in op_vals):
NeedsString = True
conversion_func = "FEX::Handler::{0}".format(op_vals["ArgumentHandler"])
conversion_func = "FEXCore::Config::Handler::{0}".format(op_vals["ArgumentHandler"])
if (value_type == "str"):
NeedsString = True
conversion_func = ""
@@ -390,6 +396,21 @@ def print_parse_argloader_options(options):
output_argloader.write("#endif\n")
def print_parse_envloader_options(options):
output_argloader.write("#ifdef ENVLOADER\n")
output_argloader.write("#undef ENVLOADER\n")
output_argloader.write("if (false) {}\n")
for op_group, group_vals in options.items():
for op_key, op_vals in group_vals.items():
if ("ArgumentHandler" in op_vals):
conversion_func = "FEXCore::Config::Handler::{0}".format(op_vals["ArgumentHandler"])
output_argloader.write("else if (Key == \"FEX_{0}\") {{\n".format(op_key.upper()))
output_argloader.write("Value = {0}(Value);\n".format(conversion_func))
output_argloader.write("}\n")
output_argloader.write("#endif\n")
def check_for_duplicate_options(options):
short_map = []
long_map = []
@@ -464,4 +485,8 @@ output_man.close()
output_argloader = open(output_argumentloader_filename, "w")
print_argloader_options(options);
print_parse_argloader_options(options);
# Generate environment loader code
print_parse_envloader_options(options);
output_argloader.close()
+31 -6
View File
@@ -81,7 +81,12 @@ set (SRCS
Interface/Core/Frontend.cpp
Interface/Core/GdbServer.cpp
Interface/Core/HostFeatures.cpp
Interface/Core/OpcodeDispatcher/Crypto.cpp
Interface/Core/OpcodeDispatcher/Flags.cpp
Interface/Core/OpcodeDispatcher/Vector.cpp
Interface/Core/OpcodeDispatcher/X87.cpp
Interface/Core/OpcodeDispatcher.cpp
Interface/Core/SignalDelegator.cpp
Interface/Core/X86Tables.cpp
Interface/Core/X86DebugInfo.cpp
Interface/Core/X86HelperGen.cpp
@@ -92,6 +97,17 @@ set (SRCS
Interface/Core/Dispatcher/Arm64Dispatcher.cpp
Interface/Core/Interpreter/InterpreterCore.cpp
Interface/Core/Interpreter/InterpreterOps.cpp
Interface/Core/Interpreter/ALUOps.cpp
Interface/Core/Interpreter/AtomicOps.cpp
Interface/Core/Interpreter/BranchOps.cpp
Interface/Core/Interpreter/ConversionOps.cpp
Interface/Core/Interpreter/EncryptionOps.cpp
Interface/Core/Interpreter/F80Ops.cpp
Interface/Core/Interpreter/FlagOps.cpp
Interface/Core/Interpreter/MemoryOps.cpp
Interface/Core/Interpreter/MiscOps.cpp
Interface/Core/Interpreter/MoveOps.cpp
Interface/Core/Interpreter/VectorOps.cpp
Interface/Core/X86Tables/BaseTables.cpp
Interface/Core/X86Tables/DDDTables.cpp
Interface/Core/X86Tables/EVEXTables.cpp
@@ -114,6 +130,7 @@ set (SRCS
Interface/IR/Passes/DeadContextStoreElimination.cpp
Interface/IR/Passes/IRCompaction.cpp
Interface/IR/Passes/IRValidation.cpp
Interface/IR/Passes/RAValidation.cpp
Interface/IR/Passes/LongDivideRemovalPass.cpp
Interface/IR/Passes/ValueDominanceValidation.cpp
Interface/IR/Passes/PhiValidation.cpp
@@ -124,7 +141,9 @@ set (SRCS
Interface/IR/Passes/SyscallOptimization.cpp
Utils/Allocator.cpp
Utils/Allocator/64BitAllocator.cpp
Utils/FileLoading.cpp
Utils/LogManager.cpp
Utils/Telemetry.cpp
Utils/Threads.cpp
)
@@ -133,7 +152,7 @@ if(_M_ARM_64)
Interface/Core/ArchHelpers/Arm64.cpp)
endif()
set(DEFINES )
set(DEFINES -DTHREAD_LOCAL=_Thread_local)
if (_M_X86_64)
list(APPEND DEFINES -D_M_X86_64=1)
@@ -175,10 +194,16 @@ if (ENABLE_JIT_ARM64)
Interface/Core/JIT/Arm64/VectorOps.cpp)
endif()
if (ENABLE_JITSYMBOLS)
list(APPEND DEFINES -DENABLE_JITSYMBOLS=1)
set (LIBS vixl dl fmt::fmt xxhash tiny-json)
if (ENABLE_JEMALLOC)
list (APPEND LIBS FEX_jemalloc)
endif()
# Generate config
configure_file(
${CMAKE_CURRENT_SOURCE_DIR}/Interface/Config/Config.json.in
${CMAKE_BINARY_DIR}/generated/Config/Config.json)
# Generate IR include file
set(OUTPUT_IR_FOLDER "${CMAKE_BINARY_DIR}/include/FEXCore/IR")
set(OUTPUT_NAME "${OUTPUT_IR_FOLDER}/IRDefines.inc")
@@ -221,7 +246,7 @@ add_custom_target(IR_INC
set(OUTPUT_CONFIG_FOLDER "${CMAKE_BINARY_DIR}/include/FEXCore/Config")
set(OUTPUT_CONFIG_NAME "${OUTPUT_CONFIG_FOLDER}/ConfigValues.inl")
set(OUTPUT_CONFIG_OPTION_NAME "${OUTPUT_CONFIG_FOLDER}/ConfigOptions.inl")
set(INPUT_CONFIG_NAME "${CMAKE_CURRENT_SOURCE_DIR}/Interface/Config/Config.json")
set(INPUT_CONFIG_NAME "${CMAKE_BINARY_DIR}/generated/Config/Config.json")
set(OUTPUT_MAN_NAME "${CMAKE_BINARY_DIR}/generated/FEX.1")
add_custom_target(CREATE_CONFIG_FOLDER ALL
@@ -265,7 +290,7 @@ function(AddObject Name Type)
add_dependencies(${Name} IR_INC)
add_dependencies(${Name} CONFIG_INC)
target_link_libraries(${Name} pthread vixl dl fmt::fmt xxhash FEX_jemalloc)
target_link_libraries(${Name} ${LIBS})
set_target_properties(${Name} PROPERTIES OUTPUT_NAME FEXCore)
set_target_properties(${Name} PROPERTIES C_VISIBILITY_PRESET hidden)
set_target_properties(${Name} PROPERTIES CXX_VISIBILITY_PRESET hidden)
@@ -306,7 +331,7 @@ endfunction()
function(AddLibrary Name Type)
add_library(${Name} ${Type} $<TARGET_OBJECTS:${PROJECT_NAME}_object>)
target_link_libraries(${Name} pthread vixl dl fmt::fmt xxhash FEX_jemalloc)
target_link_libraries(${Name} ${LIBS})
set_target_properties(${Name} PROPERTIES OUTPUT_NAME FEXCore)
set_target_properties(${Name} PROPERTIES C_VISIBILITY_PRESET hidden)
set_target_properties(${Name} PROPERTIES CXX_VISIBILITY_PRESET hidden)
+21
View File
@@ -41,5 +41,26 @@ namespace FEXCore {
String << std::hex << HostAddr << " " << CodeSize << " " << Name << "_" << HostAddr << std::endl;
fwrite(String.str().c_str(), 1, String.str().size(), fp);
}
void JITSymbols::RegisterNamedRegion(void *HostAddr, uint32_t CodeSize, std::string const &Name) {
if (!fp) return;
// Linux perf format is very straightforward
// `<HostPtr> <Size> <Name>\n`
std::stringstream String;
String << std::hex << HostAddr << " " << CodeSize << " " << Name << std::endl;
fwrite(String.str().c_str(), 1, String.str().size(), fp);
}
void JITSymbols::RegisterJITSpace(void *HostAddr, uint32_t CodeSize) {
if (!fp) return;
// Linux perf format is very straightforward
// `<HostPtr> <Size> <Name>\n`
std::stringstream String;
String << std::hex << HostAddr << " " << CodeSize << " FEXJIT" << std::endl;
fwrite(String.str().c_str(), 1, String.str().size(), fp);
}
}
+2
View File
@@ -10,6 +10,8 @@ public:
~JITSymbols();
void Register(void *HostAddr, uint64_t GuestAddr, uint32_t CodeSize);
void Register(void *HostAddr, uint32_t CodeSize, std::string const &Name);
void RegisterNamedRegion(void *HostAddr, uint32_t CodeSize, std::string const &Name);
void RegisterJITSpace(void *HostAddr, uint32_t CodeSize);
private:
FILE* fp{};
+1 -5
View File
@@ -1,11 +1,7 @@
#include "NetStream.h"
#include <cstring>
#include <sys/types.h>
#include <sys/socket.h>
#include <stdio.h>
#include <unistd.h>
int NetStream::NetBuf::flushBuffer(const char *buffer, size_t size) {
@@ -29,7 +25,7 @@ std::streamsize NetStream::NetBuf::xsputn(const char* buffer, std::streamsize si
// Check if the string fits neatly in our buffer
if (size <= buf_remaining) {
std::memcpy(pptr(), buffer, size);
::memcpy(pptr(), buffer, size);
pbump(size);
return size;
}
+1
View File
@@ -2,6 +2,7 @@
#include <array>
#include <iostream>
#include <iterator>
#include <string.h>
class NetStream : public std::iostream {
+36 -3
View File
@@ -3,12 +3,44 @@
#include <cstdlib>
#include <filesystem>
#include <sys/stat.h>
#include <memory>
#include <pwd.h>
#include <system_error>
#include <unistd.h>
namespace FEXCore::Paths {
std::unique_ptr<std::string> CachePath;
std::unique_ptr<std::string> EntryCache;
char const* FindUserHomeThroughUID() {
auto passwd = getpwuid(geteuid());
if (passwd) {
return passwd->pw_dir;
}
return nullptr;
}
const char *GetHomeDirectory() {
char const *HomeDir = getenv("HOME");
// Try to get home directory from uid
if (!HomeDir) {
HomeDir = FindUserHomeThroughUID();
}
// try the PWD
if (!HomeDir) {
HomeDir = getenv("PWD");
}
// Still doesn't exit? You get local
if (!HomeDir) {
HomeDir = ".";
}
return HomeDir;
}
void InitializePaths() {
CachePath = std::make_unique<std::string>();
EntryCache = std::make_unique<std::string>();
@@ -36,9 +68,10 @@ namespace FEXCore::Paths {
*CachePath += "/.fex-emu/";
*EntryCache = *CachePath + "/EntryCache/";
std::error_code ec{};
// Ensure the folder structure is created for our Data
if (!std::filesystem::exists(*EntryCache) &&
!std::filesystem::create_directories(*EntryCache)) {
if (!std::filesystem::exists(*EntryCache, ec) &&
!std::filesystem::create_directories(*EntryCache, ec)) {
LogMan::Msg::D("Couldn't create EntryCache directory: '%s'", EntryCache->c_str());
}
}
+3
View File
@@ -4,6 +4,9 @@
namespace FEXCore::Paths {
void InitializePaths();
void ShutdownPaths();
const char *GetHomeDirectory();
std::string GetCachePath();
std::string GetEntryCachePath();
}
+26
View File
@@ -16,9 +16,19 @@ extern "C" {
struct X80SoftFloat {
#ifdef _M_X86_64
// Define this to push some operations to x87
// Only useful to see if precision loss is killing something
// #define DEBUG_X86_FLOAT
#ifdef DEBUG_X86_FLOAT
#define BIGFLOAT long double
#define BIGFLOATSIZE 10
#else
#define BIGFLOAT __float128
#define BIGFLOATSIZE 16
#endif
#elif defined(_M_ARM_64)
#define BIGFLOAT long double
#define BIGFLOATSIZE 16
#else
#error No 128bit float for this target!
#endif
@@ -170,8 +180,14 @@ struct X80SoftFloat {
}
operator BIGFLOAT() const {
#if BIGFLOATSIZE == 16
const float128_t Result = extF80_to_f128(*this);
return FEXCore::BitCast<BIGFLOAT>(Result);
#else
BIGFLOAT result{};
memcpy(&result, this, sizeof(result));
return result;
#endif
}
operator int16_t() const {
@@ -217,6 +233,12 @@ struct X80SoftFloat {
*this = ui64_to_extF80(rhs);
}
#if BIGFLOATSIZE == 10
void operator=(const long double rhs) {
memcpy(this, &rhs, sizeof(rhs));
}
#endif
operator void*() {
return reinterpret_cast<void*>(this);
}
@@ -236,7 +258,11 @@ struct X80SoftFloat {
}
X80SoftFloat(BIGFLOAT rhs) {
#if BIGFLOATSIZE == 16
*this = f128_to_extF80(FEXCore::BitCast<float128_t>(rhs));
#else
*this = FEXCore::BitCast<long double>(rhs);
#endif
}
X80SoftFloat(const int16_t rhs) {
+358 -59
View File
@@ -1,43 +1,126 @@
#include "Common/StringConv.h"
#include <FEXCore/Utils/LogManager.h>
#include "Interface/Context/Context.h"
#include "Common/Paths.h"
#include "Utils/FileLoading.h"
#include <FEXCore/Config/Config.h>
#include <FEXCore/Utils/LogManager.h>
#include <array>
#include <assert.h>
#include <cstdlib>
#include <filesystem>
#include <pwd.h>
#include <fstream>
#include <functional>
#include <map>
#include <memory>
#include <list>
#include <optional>
#include <stddef.h>
#include <stdint.h>
#include <string>
#include <string_view>
#include <sys/sysinfo.h>
#include <unistd.h>
#include <system_error>
#include <type_traits>
#include <unordered_map>
#include <utility>
#include <vector>
#include <tiny-json.h>
namespace FEXCore::Context {
struct Context;
}
namespace FEXCore::Config {
char const* FindUserHomeThroughUID() {
auto passwd = getpwuid(geteuid());
if (passwd) {
return passwd->pw_dir;
}
return nullptr;
namespace DefaultValues {
#define P(x) x
#define OPT_BASE(type, group, enum, json, default) const P(type) P(enum) = P(default);
#define OPT_STR(group, enum, json, default) const std::string_view P(enum) = P(default);
#define OPT_STRARRAY(group, enum, json, default) OPT_STR(group, enum, json, default)
#include <FEXCore/Config/ConfigValues.inl>
}
namespace JSON {
struct JsonAllocator {
jsonPool_t PoolObject;
std::unique_ptr<std::list<json_t>> json_objects;
};
static_assert(offsetof(JsonAllocator, PoolObject) == 0, "This needs to be at offset zero");
json_t* PoolInit(jsonPool_t* Pool) {
JsonAllocator* alloc = reinterpret_cast<JsonAllocator*>(Pool);
alloc->json_objects = std::make_unique<std::list<json_t>>();
return &*alloc->json_objects->emplace(alloc->json_objects->end());
}
const char *GetHomeDirectory() {
char const *HomeDir = getenv("HOME");
json_t* PoolAlloc(jsonPool_t* Pool) {
JsonAllocator* alloc = reinterpret_cast<JsonAllocator*>(Pool);
return &*alloc->json_objects->emplace(alloc->json_objects->end());
}
// Try to get home directory from uid
if (!HomeDir) {
HomeDir = FindUserHomeThroughUID();
static void LoadJSonConfig(const std::string &Config, std::function<void(const char *Name, const char *ConfigSring)> Func) {
std::vector<char> Data;
if (!FEXCore::FileLoading::LoadFile(Data, Config)) {
return;
}
// try the PWD
if (!HomeDir) {
HomeDir = getenv("PWD");
JsonAllocator Pool {
.PoolObject = {
.init = PoolInit,
.alloc = PoolAlloc,
},
};
json_t const *json = json_createWithPool(&Data.at(0), &Pool.PoolObject);
if (!json) {
LogMan::Msg::E("Couldn't create json");
return;
}
// Still doesn't exit? You get local
if (!HomeDir) {
HomeDir = ".";
json_t const* ConfigList = json_getProperty(json, "Config");
if (!ConfigList) {
LogMan::Msg::E("Couldn't get config list");
return;
}
return HomeDir;
for (json_t const* ConfigItem = json_getChild(ConfigList);
ConfigItem != nullptr;
ConfigItem = json_getSibling(ConfigItem)) {
const char* ConfigName = json_getName(ConfigItem);
const char* ConfigString = json_getValue(ConfigItem);
if (!ConfigName) {
LogMan::Msg::E("Couldn't get config name");
return;
}
if (!ConfigString) {
LogMan::Msg::E("Couldn't get ConfigString for '%s'", ConfigName);
return;
}
Func(ConfigName, ConfigString);
}
}
}
std::string GetDataDirectory() {
std::string DataDir{};
char const *HomeDir = Paths::GetHomeDirectory();
char const *DataXDG = getenv("XDG_DATA_HOME");
char const *DataOverride = getenv("FEX_APP_DATA_LOCATION");
if (DataOverride) {
// Data override will override the complete directory
DataDir = DataOverride;
}
else {
DataDir = DataXDG ?: HomeDir;
DataDir += "/.fex-emu/";
}
return DataDir;
}
std::string GetConfigDirectory(bool Global) {
@@ -46,7 +129,7 @@ namespace FEXCore::Config {
ConfigDir = GLOBAL_DATA_DIRECTORY;
}
else {
char const *HomeDir = GetHomeDirectory();
char const *HomeDir = Paths::GetHomeDirectory();
char const *ConfigXDG = getenv("XDG_CONFIG_HOME");
char const *ConfigOverride = getenv("FEX_APP_CONFIG_LOCATION");
if (ConfigOverride) {
@@ -59,9 +142,9 @@ namespace FEXCore::Config {
}
// Ensure the folder structure is created for our configuration
if (!std::filesystem::exists(ConfigDir) &&
!std::filesystem::create_directories(ConfigDir)) {
LogMan::Msg::D("Couldn't create config directory: '%s'", ConfigDir.c_str());
std::error_code ec{};
if (!std::filesystem::exists(ConfigDir, ec) &&
!std::filesystem::create_directories(ConfigDir, ec)) {
// Let's go local in this case
return "./";
}
@@ -85,9 +168,11 @@ namespace FEXCore::Config {
std::string GetApplicationConfig(const std::string &Filename, bool Global) {
std::string ConfigFile = GetConfigDirectory(Global);
std::error_code ec{};
if (!Global &&
!std::filesystem::exists(ConfigFile) &&
!std::filesystem::create_directories(ConfigFile)) {
!std::filesystem::exists(ConfigFile, ec) &&
!std::filesystem::create_directories(ConfigFile, ec)) {
LogMan::Msg::D("Couldn't create config directory: '%s'", ConfigFile.c_str());
// Let's go local in this case
return "./" + Filename + ".json";
@@ -97,9 +182,8 @@ namespace FEXCore::Config {
// Attempt to create the local folder if it doesn't exist
if (!Global &&
!std::filesystem::exists(ConfigFile) &&
!std::filesystem::create_directories(ConfigFile)) {
LogMan::Msg::D("Couldn't create AppConfig directory: '%s'", ConfigFile.c_str());
!std::filesystem::exists(ConfigFile, ec) &&
!std::filesystem::create_directories(ConfigFile, ec)) {
// Let's go local in this case
return "./" + Filename + ".json";
}
@@ -108,23 +192,6 @@ namespace FEXCore::Config {
return ConfigFile;
}
std::string GetDataDirectory() {
std::string DataDir{};
char const *HomeDir = GetHomeDirectory();
char const *DataXDG = getenv("XDG_DATA_HOME");
char const *DataOverride = getenv("FEX_APP_DATA_LOCATION");
if (DataOverride) {
// Data override will override the complete directory
DataDir = DataOverride;
}
else {
DataDir = DataXDG ?: HomeDir;
DataDir += "/.fex-emu/";
}
return DataDir;
}
void SetConfig(FEXCore::Context::Context *CTX, ConfigOption Option, uint64_t Config) {
}
@@ -224,7 +291,8 @@ namespace FEXCore::Config {
void MetaLayer::MergeConfigMap(const LayerOptions &Options) {
// Insert this layer's options, overlaying previous options that exist here
for (auto &it : Options) {
if (it.first == FEXCore::Config::ConfigOption::CONFIG_ENV) {
if (it.first == FEXCore::Config::ConfigOption::CONFIG_ENV ||
it.first == FEXCore::Config::ConfigOption::CONFIG_HOSTENV) {
MergeEnvironmentVariables(it.first, it.second);
}
else {
@@ -252,7 +320,7 @@ namespace FEXCore::Config {
}
}
std::string ExpandPath(std::string PathName) {
std::string ExpandPath(std::string const &ContainerPrefix, std::string PathName) {
if (PathName.empty()) {
return {};
}
@@ -273,10 +341,74 @@ namespace FEXCore::Config {
Path = std::filesystem::absolute(Path);
// Only return if it exists
if (std::filesystem::exists(Path)) {
std::error_code ec{};
if (std::filesystem::exists(Path, ec)) {
return Path;
}
}
else {
// If the containerprefix and pathname isn't empty
// Then we check if the pathname exists in our current namespace
// If the path DOESN'T exist but DOES exist with the prefix applied
// then redirect to the prefix
//
// This might not be expected behaviour for some edge cases but since
// all paths aren't mounted inside the container, then it'll be fine
//
// Main catch case for this is the default thunk install folders
// HostThunks: $CMAKE_INSTALL_PREFIX/lib/fex-emu/HostThunks/
// GuestThunks: $CMAKE_INSTALL_PREFIX/share/fex-emu/GuestThunks/
if (!ContainerPrefix.empty() && !PathName.empty()) {
if (!std::filesystem::exists(PathName)) {
auto ContainerPath = ContainerPrefix + PathName;
if (std::filesystem::exists(ContainerPath)) {
return ContainerPath;
}
}
}
}
return {};
}
std::string ltrim(std::string String) {
size_t pos = std::string::npos;
if ((pos = String.find_first_not_of(" \t\n\r")) != std::string::npos) {
String.erase(0, pos);
}
return String;
}
std::string rtrim(std::string String) {
size_t pos = std::string::npos;
if ((pos = String.find_last_not_of(" \t\n\r")) != std::string::npos) {
String.erase(String.begin() + pos + 1, String.end());
}
return String;
}
std::string trim(std::string String) {
return rtrim(ltrim(String));
}
std::string FindContainerPrefix() {
// We only support pressure-vessel at the moment
const static std::string ContainerManager = "/run/host/container-manager";
if (std::filesystem::exists(ContainerManager)) {
std::vector<char> Manager{};
if (FEXCore::FileLoading::LoadFile(Manager, ContainerManager)) {
// Trim the whitespace, may contain a newline
std::string ManagerStr = Manager.data();
ManagerStr = trim(ManagerStr);
if (strncmp(ManagerStr.data(), "pressure-vessel", Manager.size()) == 0) {
// We are running inside of pressure vessel
// Our $CMAKE_INSTALL_PREFIX paths are now inside of /run/host/$CMAKE_INSTALL_PREFIX
return "/run/host/";
}
}
}
return {};
}
@@ -292,8 +424,9 @@ namespace FEXCore::Config {
}
}
auto ExpandPathIfExists = [](FEXCore::Config::ConfigOption Config, std::string PathName) {
auto NewPath = ExpandPath(PathName);
std::string ContainerPrefix { FindContainerPrefix() };
auto ExpandPathIfExists = [&ContainerPrefix](FEXCore::Config::ConfigOption Config, std::string PathName) {
auto NewPath = ExpandPath(ContainerPrefix, PathName);
if (!NewPath.empty()) {
FEXCore::Config::EraseSet(Config, NewPath);
}
@@ -301,7 +434,7 @@ namespace FEXCore::Config {
if (FEXCore::Config::Exists(FEXCore::Config::CONFIG_ROOTFS)) {
FEX_CONFIG_OPT(PathName, ROOTFS);
auto ExpandedString = ExpandPath(PathName());
auto ExpandedString = ExpandPath(ContainerPrefix, PathName());
if (!ExpandedString.empty()) {
// Adjust the path if it ended up being relative
FEXCore::Config::EraseSet(FEXCore::Config::CONFIG_ROOTFS, ExpandedString);
@@ -309,7 +442,8 @@ namespace FEXCore::Config {
else if (!PathName().empty()) {
// If the filesystem doesn't exist then let's see if it exists in the fex-emu folder
std::string NamedRootFS = GetDataDirectory() + "RootFS/" + PathName();
if (std::filesystem::exists(NamedRootFS)) {
std::error_code ec{};
if (std::filesystem::exists(NamedRootFS, ec)) {
FEXCore::Config::EraseSet(FEXCore::Config::CONFIG_ROOTFS, NamedRootFS);
}
}
@@ -324,7 +458,19 @@ namespace FEXCore::Config {
}
if (FEXCore::Config::Exists(FEXCore::Config::CONFIG_THUNKCONFIG)) {
FEX_CONFIG_OPT(PathName, THUNKCONFIG);
ExpandPathIfExists(FEXCore::Config::CONFIG_THUNKCONFIG, PathName());
auto ExpandedString = ExpandPath(ContainerPrefix, PathName());
if (!ExpandedString.empty()) {
// Adjust the path if it ended up being relative
FEXCore::Config::EraseSet(FEXCore::Config::CONFIG_THUNKCONFIG, ExpandedString);
}
else if (!PathName().empty()) {
// If the filesystem doesn't exist then let's see if it exists in the fex-emu folder
std::string NamedConfig = GetDataDirectory() + "ThunkConfigs/" + PathName();
std::error_code ec{};
if (std::filesystem::exists(NamedConfig, ec)) {
FEXCore::Config::EraseSet(FEXCore::Config::CONFIG_THUNKCONFIG, NamedConfig);
}
}
}
if (FEXCore::Config::Exists(FEXCore::Config::CONFIG_OUTPUTLOG)) {
FEX_CONFIG_OPT(PathName, OUTPUTLOG);
@@ -355,7 +501,7 @@ namespace FEXCore::Config {
return Meta->Get(Option);
}
void Set(ConfigOption Option, std::string Data) {
void Set(ConfigOption Option, std::string_view Data) {
Meta->Set(Option, Data);
}
@@ -363,7 +509,7 @@ namespace FEXCore::Config {
Meta->Erase(Option);
}
void EraseSet(ConfigOption Option, std::string Data) {
void EraseSet(ConfigOption Option, std::string_view Data) {
Meta->EraseSet(Option, Data);
}
@@ -402,6 +548,17 @@ namespace FEXCore::Config {
}
}
template<>
std::string Value<std::string>::GetIfExists(FEXCore::Config::ConfigOption Option, std::string_view Default) {
auto Value = FEXCore::Config::Get(Option);
if (Value) {
return **Value;
}
else {
return std::string(Default);
}
}
template bool Value<bool>::GetIfExists(FEXCore::Config::ConfigOption Option, bool Default);
template int8_t Value<int8_t>::GetIfExists(FEXCore::Config::ConfigOption Option, int8_t Default);
template uint8_t Value<uint8_t>::GetIfExists(FEXCore::Config::ConfigOption Option, uint8_t Default);
@@ -427,5 +584,147 @@ namespace FEXCore::Config {
}
}
template void Value<std::string>::GetListIfExists(FEXCore::Config::ConfigOption Option, std::list<std::string> *List);
// Application loaders
class MainLoader final : public FEXCore::Config::OptionMapper {
public:
explicit MainLoader();
explicit MainLoader(std::string ConfigFile);
void Load() override;
private:
std::string Config;
};
class AppLoader final : public FEXCore::Config::OptionMapper {
public:
explicit AppLoader(const std::string& Filename, bool Global);
void Load();
private:
std::string Config;
};
class EnvLoader final : public FEXCore::Config::Layer {
public:
explicit EnvLoader(char *const _envp[]);
void Load() override;
private:
char *const *envp;
};
static const std::map<std::string, FEXCore::Config::ConfigOption, std::less<>> ConfigLookup = {{
#define OPT_BASE(type, group, enum, json, default) {#json, FEXCore::Config::ConfigOption::CONFIG_##enum},
#include <FEXCore/Config/ConfigValues.inl>
}};
static const std::vector<std::pair<const char*, FEXCore::Config::ConfigOption>> EnvConfigLookup = {{
#define OPT_BASE(type, group, enum, json, default) {"FEX_" #enum, FEXCore::Config::ConfigOption::CONFIG_##enum},
#include <FEXCore/Config/ConfigValues.inl>
}};
OptionMapper::OptionMapper(FEXCore::Config::LayerType Layer)
: FEXCore::Config::Layer(Layer) {
}
void OptionMapper::MapNameToOption(const char *ConfigName, const char *ConfigString) {
auto it = ConfigLookup.find(ConfigName);
if (it != ConfigLookup.end()) {
Set(it->second, ConfigString);
}
}
MainLoader::MainLoader()
: FEXCore::Config::OptionMapper(FEXCore::Config::LayerType::LAYER_MAIN)
, Config{FEXCore::Config::GetConfigFileLocation()} {
}
MainLoader::MainLoader(std::string ConfigFile)
: FEXCore::Config::OptionMapper(FEXCore::Config::LayerType::LAYER_MAIN)
, Config{std::move(ConfigFile)} {
}
void MainLoader::Load() {
JSON::LoadJSonConfig(Config, [this](const char *Name, const char *ConfigString) {
MapNameToOption(Name, ConfigString);
});
}
AppLoader::AppLoader(const std::string& Filename, bool Global)
: FEXCore::Config::OptionMapper(Global ? FEXCore::Config::LayerType::LAYER_GLOBAL_APP : FEXCore::Config::LayerType::LAYER_LOCAL_APP) {
Config = FEXCore::Config::GetApplicationConfig(Filename, Global);
// Immediately load so we can reload the meta layer
Load();
}
void AppLoader::Load() {
JSON::LoadJSonConfig(Config, [this](const char *Name, const char *ConfigString) {
MapNameToOption(Name, ConfigString);
});
}
EnvLoader::EnvLoader(char *const _envp[])
: FEXCore::Config::Layer(FEXCore::Config::LayerType::LAYER_ENVIRONMENT)
, envp {_envp} {
}
void EnvLoader::Load() {
std::unordered_map<std::string_view, std::string_view> EnvMap;
for(const char *const *pvar=envp; pvar && *pvar; pvar++) {
std::string_view Var(*pvar);
size_t pos = Var.rfind('=');
if (std::string::npos == pos)
continue;
std::string_view Key = Var.substr(0,pos);
std::string_view Value {Var.substr(pos+1)};
#define ENVLOADER
#include <FEXCore/Config/ConfigOptions.inl>
EnvMap[Key]=Value;
}
std::function GetVar = [=](const std::string_view id) -> std::optional<std::string_view> {
if (EnvMap.find(id) != EnvMap.end())
return EnvMap.at(id);
// If envp[] was empty, search using std::getenv()
const char* vs = std::getenv(id.data());
if (vs) {
return vs;
}
else {
return std::nullopt;
}
};
std::optional<std::string_view> Value;
for (auto &it : EnvConfigLookup) {
if ((Value = GetVar(it.first)).has_value()) {
Set(it.second, std::string(*Value));
}
}
}
std::unique_ptr<FEXCore::Config::Layer> CreateMainLayer(std::string const *File) {
if (File) {
return std::make_unique<FEXCore::Config::MainLoader>(*File);
}
else {
return std::make_unique<FEXCore::Config::MainLoader>();
}
}
std::unique_ptr<FEXCore::Config::Layer> CreateAppLayer(const std::string& Filename, bool Global) {
return std::make_unique<FEXCore::Config::AppLoader>(Filename, Global);
}
std::unique_ptr<FEXCore::Config::Layer> CreateEnvironmentLayer(char *const _envp[]) {
return std::make_unique<FEXCore::Config::EnvLoader>(_envp);
}
}
@@ -58,7 +58,7 @@
},
"ThunkHostLibs": {
"Type": "str",
"Default": "",
"Default": "@CMAKE_INSTALL_PREFIX@/lib/fex-emu/HostThunks/",
"ShortArg": "t",
"Desc": [
"Folder to find the host-side thunking libraries."
@@ -66,7 +66,7 @@
},
"ThunkGuestLibs": {
"Type": "str",
"Default": "",
"Default": "@CMAKE_INSTALL_PREFIX@/share/fex-emu/GuestThunks/",
"ShortArg": "j",
"Desc": [
"Folder to find the guest-side thunking libraries."
@@ -77,7 +77,14 @@
"Default": "",
"ShortArg": "k",
"Desc": [
"A json file specifying where to overlay the thunks."
"A json file specifying where to overlay the thunks.",
"This can be a filesystem path",
"\teg: ~/MyThunkConfig.json",
"Or this can be a named of a Thunk config file",
"If the named config file exists in the FEX data folder folder the it will use that one",
"\teg: $HOME/.fex-emu/ThunkConfigs/<ThunkConfig name>",
"Or if you have XDG_DATA_HOME the config will search in that directory",
"\teg: $XDG_DATA_HOME/.fex-emu/ThunkConfigs/<ThunkConfig name>"
]
},
"Env": {
@@ -87,6 +94,16 @@
"Desc": [
"Adds an environment variable to the emulated environment."
]
},
"HostEnv": {
"Type": "strarray",
"Default": "",
"ShortArg": "H",
"Desc": [
"Adds an environment variable to the host environment.",
"This can be useful for setting environment variables that thunks can pick up.",
"Typically isn't necessary since the guest libc isn't thunked. But is possible."
]
}
},
"Debug": {
@@ -130,6 +147,13 @@
"Disables optimizations passes for debugging."
]
},
"SRA": {
"Type": "bool",
"Default": "true",
"Desc": [
"Set to false to disable Static Register Allocation"
]
},
"Force32BitAllocator": {
"Type": "bool",
"Default": "false",
@@ -139,8 +163,34 @@
"Potentially useful for debugging memory problems",
"32-bit allocator is always used if your host kernel is older than 4.17"
]
},
"GlobalJITNaming": {
"Type": "bool",
"Default": "false",
"Desc": [
"Uses JITSymbols to name all JIT state as one symbol",
"Useful for querying how much time is spent inside of the JIT",
"Profiling tools will show JIT time as FEXJIT"
]
},
"LibraryJITNaming": {
"Type": "bool",
"Default": "false",
"Desc": [
"Uses JITSymbols to name JIT symbols grouped by library",
"Useful for querying how much time is spent in each guest library",
"Can be used to help guide thunk generation"
]
},
"BlockJITNaming": {
"Type": "bool",
"Default": "false",
"Desc": [
"Uses JITSymbols to name JIT symbols",
"Useful for determining hot blocks of code",
"Has some file writing overhead per JIT block"
]
}
},
"Logging": {
"SilentLog": {
@@ -207,6 +257,14 @@
"Makes TSO operations even more strict.",
"Forces vector loadstores to also become atomic."
]
},
"StallProcess": {
"Type": "bool",
"Default": "false",
"Desc": [
"Forces a process to stall out on initialization",
"Useful for a process that keeps restarting and doesn't work"
]
}
},
"Misc": {
+19 -5
View File
@@ -2,10 +2,20 @@
#include "Interface/Context/Context.h"
#include "Interface/Core/Core.h"
#include "Interface/Core/OpcodeDispatcher.h"
#include "Interface/Core/X86Tables/X86Tables.h"
#include <FEXCore/Config/Config.h>
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/Debug/X86Tables.h>
#include <FEXCore/Core/Context.h>
#include <FEXCore/Core/CPUID.h>
#include <FEXCore/Core/SignalDelegator.h>
#include "FEXCore/Debug/InternalThreadState.h"
#include <string.h>
#include <utility>
namespace FEXCore::HLE {
class SyscallVisitor;
}
namespace FEXCore::Context {
void InitializeStaticTables(OperatingMode Mode) {
@@ -33,7 +43,7 @@ namespace FEXCore::Context {
delete CTX;
}
bool InitCore(FEXCore::Context::Context *CTX, FEXCore::CodeLoader *Loader) {
FEXCore::Core::InternalThreadState* InitCore(FEXCore::Context::Context *CTX, FEXCore::CodeLoader *Loader) {
return CTX->InitCore(Loader);
}
@@ -100,8 +110,8 @@ namespace FEXCore::Context {
void RegisterExternalSyscallVisitor(FEXCore::Context::Context *CTX, [[maybe_unused]] uint64_t Syscall, [[maybe_unused]] FEXCore::HLE::SyscallVisitor *Visitor) {
}
void HandleCallback(FEXCore::Context::Context *CTX, uint64_t RIP) {
CTX->HandleCallback(RIP);
void HandleCallback(FEXCore::Context::Context *CTX, FEXCore::Core::InternalThreadState *Thread, uint64_t RIP) {
CTX->HandleCallback(Thread, RIP);
}
void RegisterHostSignalHandler(FEXCore::Context::Context *CTX, int Signal, HostSignalDelegatorFunction Func, bool Required) {
@@ -116,6 +126,10 @@ namespace FEXCore::Context {
return CTX->CreateThread(NewThreadState, ParentTID);
}
void ExecutionThread(FEXCore::Context::Context *CTX, FEXCore::Core::InternalThreadState *Thread) {
return CTX->ExecutionThread(Thread);
}
void InitializeThread(FEXCore::Context::Context *CTX, FEXCore::Core::InternalThreadState *Thread) {
return CTX->InitializeThread(Thread);
}
+94 -21
View File
@@ -2,47 +2,48 @@
#include "Common/JitSymbols.h"
#include "Interface/Core/CPUID.h"
#include "Interface/Core/Frontend.h"
#include "Interface/Core/HostFeatures.h"
#include "Interface/Core/InternalThreadState.h"
#include "Interface/Core/X86HelperGen.h"
#include "Interface/IR/PassManager.h"
#include "Interface/IR/Passes/RegisterAllocationPass.h"
#include <FEXCore/Config/Config.h>
#include <FEXCore/Core/CPUBackend.h>
#include <FEXCore/Core/Context.h>
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/Core/SignalDelegator.h>
#include <FEXCore/Debug/InternalThreadState.h>
#include <FEXCore/Utils/CompilerDefs.h>
#include <FEXCore/Utils/Event.h>
#include <stdint.h>
#include <atomic>
#include <condition_variable>
#include <functional>
#include <istream>
#include <map>
#include <memory>
#include <mutex>
#include <optional>
#include <ostream>
#include <set>
#include <shared_mutex>
#include <stddef.h>
#include <string>
#include <unordered_map>
#include <queue>
#include <vector>
namespace FEXCore {
class CodeLoader;
class ThunkHandler;
class BlockSamplingData;
class GdbServer;
class SiganlDelegator;
namespace CPU {
class Arm64JITCore;
class X86JITCore;
}
namespace HLE {
struct SyscallArguments;
class SyscallHandler;
}
}
namespace FEXCore::IR {
class RegisterAllocationPass;
class RegisterAllocationData;
class IRListView;
namespace Validation {
@@ -121,7 +122,13 @@ namespace FEXCore::Context {
FEX_CONFIG_OPT(MaxInstPerBlock, MAXINST);
FEX_CONFIG_OPT(RootFSPath, ROOTFS);
FEX_CONFIG_OPT(ThunkHostLibsPath, THUNKHOSTLIBS);
FEX_CONFIG_OPT(ThunkConfigFile, THUNKCONFIG);
FEX_CONFIG_OPT(DumpIR, DUMPIR);
FEX_CONFIG_OPT(StaticRegisterAllocation, SRA);
FEX_CONFIG_OPT(GlobalJITNaming, GLOBALJITNAMING);
FEX_CONFIG_OPT(LibraryJITNaming, LIBRARYJITNAMING);
FEX_CONFIG_OPT(BlockJITNaming, BLOCKJITNAMING);
FEX_CONFIG_OPT(ParanoidTSO, PARANOIDTSO);
} Config;
using IntCallbackReturn = FEX_NAKED void(*)(FEXCore::Core::InternalThreadState *Thread, volatile void *Host_RSP);
@@ -170,9 +177,11 @@ namespace FEXCore::Context {
bool ContainsCode;
};
std::map<uint64_t, AddrToFileEntry> AddrToFile;
using AddrToFileMapType = std::map<uint64_t, AddrToFileEntry>;
AddrToFileMapType AddrToFile;
std::map<std::string, std::string> FilesWithCode;
AddrToFileMapType::iterator FindAddrForFile(uint64_t Entry, uint64_t Length);
#ifdef BLOCKSTATS
std::unique_ptr<FEXCore::BlockSamplingData> BlockData;
#endif
@@ -183,7 +192,7 @@ namespace FEXCore::Context {
Context();
~Context();
bool InitCore(FEXCore::CodeLoader *Loader);
FEXCore::Core::InternalThreadState* InitCore(FEXCore::CodeLoader *Loader);
FEXCore::Context::ExitReason RunUntilExit();
int GetProgramStatus() const;
bool IsPaused() const { return !Running; }
@@ -199,7 +208,7 @@ namespace FEXCore::Context {
bool GetGdbServerStatus() const { return DebugServer != nullptr; }
void StartGdbServer();
void StopGdbServer();
void HandleCallback(uint64_t RIP);
void HandleCallback(FEXCore::Core::InternalThreadState *Thread, uint64_t RIP);
void RegisterHostSignalHandler(int Signal, HostSignalDelegatorFunction Func, bool Required);
void RegisterFrontendHostSignalHandler(int Signal, HostSignalDelegatorFunction Func, bool Required);
@@ -240,23 +249,80 @@ namespace FEXCore::Context {
};
[[nodiscard]] CompileCodeResult CompileCode(FEXCore::Core::InternalThreadState *Thread, uint64_t GuestRIP);
uintptr_t CompileBlock(FEXCore::Core::CpuStateFrame *Frame, uint64_t GuestRIP);
// same as CompileBlock, but aborts on failure
void CompileBlockJit(FEXCore::Core::CpuStateFrame *Frame, uint64_t GuestRIP);
bool LoadAOTIRCache(int streamfd);
void FinalizeAOTIRCache();
void WriteFilesWithCode(std::function<void(const std::string& fileid, const std::string& filename)> Writer);
// Used for thread creation from syscalls
/**
* @brief Initializes the JIT compilers for the thread
*
* @param State The internal FEX thread state object
* @param CompileThread Is this for the compile service or not?
*
* InitializeCompiler is called inside of CreateThread, so you likely don't need this
* This is exposed because the CompileService needs to initialize compilers while copying data from
* the paired InternalThreadState that it is compiling code for
*/
void InitializeCompiler(FEXCore::Core::InternalThreadState* State, bool CompileThread);
// Used for thread creation from syscalls
/**
* @brief Used to create FEX thread objects in preparation for creating a true OS thread
*
* @param NewThreadState The initial thread state to setup for our state
* @param ParentTID The PID that was the parent thread that created this
*
* @return The InternalThreadState object that tracks all of the emulated thread's state
*
* Usecases:
* OS thread Creation:
* - Thread = CreateThread(NewState, PPID);
* - InitializeThread(Thread);
* OS fork (New thread created with a clone of thread state):
* - clone{2, 3}
* - Thread = CreateThread(CopyOfThreadState, PPID);
* - ExecutionThread(Thread); // Starts executing without creating another host thread
* Thunk callback executing guest code from native host thread
* - Thread = CreateThread(NewState, PPID);
* - InitializeThreadTLSData(Thread);
* - HandleCallback(Thread, RIP);
*/
FEXCore::Core::InternalThreadState* CreateThread(FEXCore::Core::CPUState *NewThreadState, uint64_t ParentTID);
void InitializeThreadData(FEXCore::Core::InternalThreadState *Thread);
/**
* @brief Initializes the TLS data for a thread
*
* @param Thread The internal FEX thread state object
*/
void InitializeThreadTLSData(FEXCore::Core::InternalThreadState *Thread);
/**
* @brief Initializes the OS thread object and prepares to start executing on that new OS thread
*
* @param Thread The internal FEX thread state object
*
* The OS thread will wait until RunThread is executed
*/
void InitializeThread(FEXCore::Core::InternalThreadState *Thread);
void CopyMemoryMapping(FEXCore::Core::InternalThreadState *ParentThread, FEXCore::Core::InternalThreadState *ChildThread);
/**
* @brief Starts the OS thread object to start executing guest code
*
* @param Thread The internal FEX thread state object
*/
void RunThread(FEXCore::Core::InternalThreadState *Thread);
/**
* @brief Destroys this FEX thread object and stops tracking it internally
*
* @param Thread The internal FEX thread state object
*/
void DestroyThread(FEXCore::Core::InternalThreadState *Thread);
void CopyMemoryMapping(FEXCore::Core::InternalThreadState *ParentThread, FEXCore::Core::InternalThreadState *ChildThread);
void CleanupAfterFork(FEXCore::Core::InternalThreadState *ExceptForThread);
std::vector<FEXCore::Core::InternalThreadState*>* GetThreads() { return &Threads; }
@@ -266,9 +332,7 @@ namespace FEXCore::Context {
void AddNamedRegion(uintptr_t Base, uintptr_t Size, uintptr_t Offset, const std::string &filename);
void RemoveNamedRegion(uintptr_t Base, uintptr_t Size);
#if ENABLE_JITSYMBOLS
FEXCore::JITSymbols Symbols;
#endif
// Public for threading
void ExecutionThread(FEXCore::Core::InternalThreadState *Thread);
@@ -277,6 +341,15 @@ namespace FEXCore::Context {
void ClearCodeCache(FEXCore::Core::InternalThreadState *Thread, bool AlsoClearIRCache);
private:
/**
* @brief Does some final thread initialization
*
* @param Thread The internal FEX thread state object
*
* InitCore and CreateThread both call this to finish up thread object initialization
*/
void InitializeThreadData(FEXCore::Core::InternalThreadState *Thread);
void WaitForIdleWithTimeout();
void NotifyPause();
File diff suppressed because it is too large. Load diff
@@ -18,6 +18,39 @@ namespace FEXCore::ArchHelpers::Arm64 {
constexpr uint32_t STLXP_MASK = 0xBF'E0'80'00;
constexpr uint32_t STLXP_INST = 0x88'20'80'00;
constexpr uint32_t LDAXR_MASK = 0x3F'FF'FC'00;
constexpr uint32_t LDAXR_INST = 0x08'5F'FC'00;
constexpr uint32_t STLXR_MASK = 0x3F'E0'FC'00;
constexpr uint32_t STLXR_INST = 0x08'00'FC'00;
constexpr uint32_t CBNZ_MASK = 0x7F'00'00'00;
constexpr uint32_t CBNZ_INST = 0x35'00'00'00;
constexpr uint32_t ALU_OP_MASK = 0x7F'00'00'00;
constexpr uint32_t ADD_INST = 0x0B'00'00'00;
constexpr uint32_t SUB_INST = 0x4B'00'00'00;
constexpr uint32_t CMP_INST = 0x6B'00'00'00;
constexpr uint32_t AND_INST = 0x0A'00'00'00;
constexpr uint32_t OR_INST = 0x2A'00'00'00;
constexpr uint32_t EOR_INST = 0x4A'00'00'00;
constexpr uint32_t CCMP_MASK = 0x7F'E0'0C'10;
constexpr uint32_t CCMP_INST = 0x7A'40'00'00;
constexpr uint32_t CLREX_MASK = 0xFF'FF'F0'FF;
constexpr uint32_t CLREX_INST = 0xD5'03'30'5F;
enum ExclusiveAtomicPairType {
TYPE_SWAP,
TYPE_ADD,
TYPE_SUB,
TYPE_AND,
TYPE_OR,
TYPE_EOR,
TYPE_NEG, // This is just a sub with zero. Need to know the differences
};
// Load ops are 4 bits
// Acquire and release bits are independent on the instruction
constexpr uint32_t ATOMIC_ADD_OP = 0b0000;
@@ -30,7 +63,34 @@ namespace FEXCore::ArchHelpers::Arm64 {
constexpr uint32_t ATOMIC_UMIN_OP = 0b0111;
constexpr uint32_t ATOMIC_SWAP_OP = 0b1000;
constexpr uint32_t REGISTER_MASK = 0b11111;
constexpr uint32_t RD_OFFSET = 0;
constexpr uint32_t RN_OFFSET = 5;
constexpr uint32_t RM_OFFSET = 16;
constexpr uint32_t DMB = 0b1101'0101'0000'0011'0011'0000'1011'1111 |
0b1011'0000'0000; // Inner shareable all
inline uint32_t GetRdReg(uint32_t Instr) {
return (Instr >> RD_OFFSET) & REGISTER_MASK;
}
inline uint32_t GetRnReg(uint32_t Instr) {
return (Instr >> RN_OFFSET) & REGISTER_MASK;
}
inline uint32_t GetRmReg(uint32_t Instr) {
return (Instr >> RM_OFFSET) & REGISTER_MASK;
}
bool HandleAtomicLoad(void *_ucontext, void *_info, uint32_t Instr);
bool HandleAtomicStore(void *_ucontext, void *_info, uint32_t Instr);
bool HandleAtomicLoad128(void *_ucontext, void *_info, uint32_t Instr);
uint64_t HandleAtomicLoadstoreExclusive(void *_ucontext, void *_info);
bool HandleCASPAL(void *_ucontext, void *_info, uint32_t Instr);
uint64_t HandleCASPAL_ARMv8(void *_ucontext, void *_info, uint32_t Instr);
bool HandleAtomicVectorStore(void *_ucontext, void *_info, uint32_t Instr);
bool HandleCASAL(void *_ucontext, void *_info, uint32_t Instr);
bool HandleAtomicMemOp(void *_ucontext, void *_info, uint32_t Instr);
[[nodiscard]] bool HandleSIGBUS(bool ParanoidTSO, int Signal, void *info, void *ucontext);
}
@@ -4,6 +4,11 @@
#include <FEXCore/Core/CoreState.h>
#include "aarch64/cpu-aarch64.h"
#include "cpu-features.h"
#include "aarch64/instructions-aarch64.h"
#include "utils-vixl.h"
#include <tuple>
namespace FEXCore::CPU {
#define STATE x28
@@ -143,22 +148,26 @@ void Arm64Emitter::PopCalleeSavedRegisters() {
void Arm64Emitter::SpillStaticRegs() {
for (size_t i = 0; i < SRA64.size(); i+=2) {
stp(SRA64[i], SRA64[i+1], MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[i])));
}
if (StaticRegisterAllocation()) {
for (size_t i = 0; i < SRA64.size(); i+=2) {
stp(SRA64[i], SRA64[i+1], MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[i])));
}
for (size_t i = 0; i < SRAFPR.size(); i+=2) {
stp(SRAFPR[i].Q(), SRAFPR[i+1].Q(), MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.xmm[i][0])));
for (size_t i = 0; i < SRAFPR.size(); i+=2) {
stp(SRAFPR[i].Q(), SRAFPR[i+1].Q(), MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.xmm[i][0])));
}
}
}
void Arm64Emitter::FillStaticRegs() {
for (size_t i = 0; i < SRA64.size(); i+=2) {
ldp(SRA64[i], SRA64[i+1], MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[i])));
}
if (StaticRegisterAllocation()) {
for (size_t i = 0; i < SRA64.size(); i+=2) {
ldp(SRA64[i], SRA64[i+1], MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[i])));
}
for (size_t i = 0; i < SRAFPR.size(); i+=2) {
ldp(SRAFPR[i].Q(), SRAFPR[i+1].Q(), MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.xmm[i][0])));
for (size_t i = 0; i < SRAFPR.size(); i+=2) {
ldp(SRAFPR[i].Q(), SRAFPR[i+1].Q(), MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.xmm[i][0])));
}
}
}
@@ -222,7 +231,7 @@ void Arm64Emitter::ResetStack() {
}
void Arm64Emitter::Align16B() {
uint64_t CurrentOffset = GetBuffer()->GetOffsetAddress<uint64_t>(GetCursorOffset());
uint64_t CurrentOffset = GetCursorAddress<uint64_t>();
for (uint64_t i = (16 - (CurrentOffset & 0xF)); i != 0; i -= 4) {
nop();
}
@@ -1,7 +1,16 @@
#pragma once
#include "aarch64/assembler-aarch64.h"
#include "aarch64/constants-aarch64.h"
#include "aarch64/cpu-aarch64.h"
#include "aarch64/operands-aarch64.h"
#include "platform-vixl.h"
#include "FEXCore/Config/Config.h"
#include <array>
#include <stddef.h>
#include <stdint.h>
#include <utility>
namespace FEXCore::CPU {
using namespace vixl;
@@ -72,6 +81,8 @@ protected:
uint32_t DCacheLineSize{};
uint32_t ICacheLineSize{};
FEX_CONFIG_OPT(StaticRegisterAllocation, SRA);
};
}
@@ -1,6 +1,7 @@
#include "Interface/Core/ArchHelpers/Arm64.h"
#include <FEXCore/Utils/LogManager.h>
#include <stdint.h>
namespace FEXCore::ArchHelpers::Arm64 {
@@ -23,4 +24,4 @@ bool HandleAtomicMemOp(void *_ucontext, void *_info, uint32_t Instr) {
}
#endif
}
}
+48 -13
View File
@@ -18,6 +18,7 @@ struct X86ContextBackup {
// RIP and RSP is stored in GPRs here
uint64_t GPRs[23];
FEXCore::x86_64::_libc_fpstate FPRState;
uint64_t sa_mask;
// Guest state
int Signal;
@@ -35,6 +36,7 @@ struct ArmContextBackup {
uint32_t FPSR;
uint32_t FPCR;
__uint128_t FPRs[32];
uint64_t sa_mask;
// Guest state
int Signal;
@@ -44,6 +46,11 @@ struct ArmContextBackup {
static constexpr int RedZoneSize = 0;
};
static inline ucontext_t* GetUContext(void* ucontext) {
ucontext_t* _context = (ucontext_t*)ucontext;
return _context;
}
static inline mcontext_t* GetMContext(void* ucontext) {
ucontext_t* _context = (ucontext_t*)ucontext;
return &_context->uc_mcontext;
@@ -52,6 +59,20 @@ static inline mcontext_t* GetMContext(void* ucontext) {
#ifdef _M_ARM_64
constexpr uint32_t FPR_MAGIC = 0x46508001U;
struct HostCTXHeader {
uint32_t Magic;
uint32_t Size;
};
struct HostFPRState {
HostCTXHeader Head;
uint32_t FPSR;
uint32_t FPCR;
__uint128_t FPRs[32];
};
static inline uint64_t GetSp(void* ucontext) {
return GetMContext(ucontext)->sp;
}
@@ -84,24 +105,19 @@ static inline void SetArmReg(void* ucontext, uint32_t id, uint64_t val) {
GetMContext(ucontext)->regs[id] = val;
}
constexpr uint32_t FPR_MAGIC = 0x46508001U;
static inline __uint128_t GetArmFPR(void* ucontext, uint32_t id) {
auto MContext = GetMContext(ucontext);
HostFPRState *HostState = reinterpret_cast<HostFPRState*>(&MContext->__reserved[0]);
LOGMAN_THROW_A(HostState->Head.Magic == FPR_MAGIC, "Wrong FPR Magic: 0x%08x", HostState->Head.Magic);
struct HostCTXHeader {
uint32_t Magic;
uint32_t Size;
};
struct HostFPRState {
HostCTXHeader Head;
uint32_t FPSR;
uint32_t FPCR;
__uint128_t FPRs[32];
};
return HostState->FPRs[id];
}
using ContextBackup = ArmContextBackup;
template <typename T>
static inline void BackupContext(void* ucontext, T *Backup) {
if constexpr (std::is_same<T, ArmContextBackup>::value) {
auto _ucontext = GetUContext(ucontext);
auto _mcontext = GetMContext(ucontext);
memcpy(&Backup->GPRs[0], &_mcontext->regs[0], 31 * sizeof(uint64_t));
@@ -115,6 +131,9 @@ static inline void BackupContext(void* ucontext, T *Backup) {
Backup->FPSR = HostState->FPSR;
Backup->FPCR = HostState->FPCR;
memcpy(&Backup->FPRs[0], &HostState->FPRs[0], 32 * sizeof(__uint128_t));
// Save the signal mask so we can restore it
memcpy(&Backup->sa_mask, &_ucontext->uc_sigmask, sizeof(uint64_t));
} else {
ERROR_AND_DIE("Wrong context type"); // This must be a runtime error
}
@@ -123,6 +142,7 @@ static inline void BackupContext(void* ucontext, T *Backup) {
template <typename T>
static inline void RestoreContext(void* ucontext, T *Backup) {
if constexpr (std::is_same<T, ArmContextBackup>::value) {
auto _ucontext = GetUContext(ucontext);
auto _mcontext = GetMContext(ucontext);
HostFPRState *HostState = reinterpret_cast<HostFPRState*>(&_mcontext->__reserved[0]);
@@ -136,6 +156,9 @@ static inline void RestoreContext(void* ucontext, T *Backup) {
ArchHelpers::Context::SetPc(ucontext, Backup->PrevPC);
ArchHelpers::Context::SetSp(ucontext, Backup->PrevSP);
memcpy(&_mcontext->regs[0], &Backup->GPRs[0], 31 * sizeof(uint64_t));
// Restore the signal mask now
memcpy(&_ucontext->uc_sigmask, &Backup->sa_mask, sizeof(uint64_t));
} else {
ERROR_AND_DIE("Wrong context type"); // This must be a runtime error
}
@@ -177,10 +200,15 @@ static inline void SetArmReg(void* ucontext, uint32_t id, uint64_t val) {
ERROR_AND_DIE("Not impelented for x86 host");
}
static inline __uint128_t GetArmFPR(void* ucontext, uint32_t id) {
ERROR_AND_DIE("Not implemented for x86 host");
}
using ContextBackup = X86ContextBackup;
template <typename T>
static inline void BackupContext(void* ucontext, T *Backup) {
if constexpr (std::is_same<T, X86ContextBackup>::value) {
auto _ucontext = GetUContext(ucontext);
auto _mcontext = GetMContext(ucontext);
// Copy the GPRs
@@ -188,6 +216,9 @@ static inline void BackupContext(void* ucontext, T *Backup) {
// Copy the FPRState
memcpy(&Backup->FPRState, _mcontext->fpregs, sizeof(X86ContextBackup::FPRState));
// XXX: Save 256bit and 512bit AVX register state
// Save the signal mask so we can restore it
memcpy(&Backup->sa_mask, &_ucontext->uc_sigmask, sizeof(uint64_t));
} else {
ERROR_AND_DIE("Wrong context type"); // This must be a runtime error
}
@@ -196,12 +227,16 @@ static inline void BackupContext(void* ucontext, T *Backup) {
template <typename T>
static inline void RestoreContext(void* ucontext, T *Backup) {
if constexpr (std::is_same<T, X86ContextBackup>::value) {
auto _ucontext = GetUContext(ucontext);
auto _mcontext = GetMContext(ucontext);
// Copy the GPRs
memcpy(&_mcontext->gregs[0], &Backup->GPRs[0], sizeof(X86ContextBackup::GPRs));
// Copy the FPRState
memcpy(_mcontext->fpregs, &Backup->FPRState, sizeof(X86ContextBackup::FPRState));
// Restore the signal mask now
memcpy(&_ucontext->uc_sigmask, &Backup->sa_mask, sizeof(uint64_t));
} else {
ERROR_AND_DIE("Wrong context type"); // This must be a runtime error
}
@@ -209,4 +244,4 @@ static inline void RestoreContext(void* ucontext, T *Backup) {
#endif
} // namespace FEXCore::ArchHelpers::Context
} // namespace FEXCore::ArchHelpers::Context
@@ -2,6 +2,7 @@
#include <FEXCore/Utils/LogManager.h>
#include <cstring>
#include <fstream>
#include <utility>
namespace FEXCore {
void BlockSamplingData::DumpBlockData() {
+2 -1
View File
@@ -1,6 +1,7 @@
#pragma once
#include <cstdint>
#include <unordered_map>
#include <stdint.h>
namespace FEXCore {
class BlockSamplingData {
+95 -16
View File
@@ -5,8 +5,14 @@ desc: Handles presented capability bits for guest cpu
$end_info$
*/
#include <FEXCore/Config/Config.h>
#include <FEXCore/Core/CPUID.h>
#include "Common/StringConv.h"
#include "Interface/Context/Context.h"
#include "Interface/Core/CPUID.h"
#include "Interface/Core/HostFeatures.h"
#include "Utils/FileLoading.h"
#include "git_version.h"
#include <cstring>
@@ -42,6 +48,63 @@ static uint32_t GetCycleCounterFrequency() {
: [Res] "=r" (Result));
return Result;
}
static bool GetHostHybridFlag() {
int MaxCPUs = 64;
size_t AllocSize = CPU_ALLOC_SIZE(MaxCPUs);
cpu_set_t *Set = CPU_ALLOC(MaxCPUs);
CPU_ZERO_S(AllocSize, Set);
int Result{};
for (;;) {
Result = sched_getaffinity(0, AllocSize, Set);
if (Result == 0 ||
(Result == -1 && errno != EINVAL)) {
break;
}
MaxCPUs <<= 1;
CPU_FREE(Set);
Set = CPU_ALLOC(MaxCPUs);
AllocSize = CPU_ALLOC_SIZE(MaxCPUs);
CPU_ZERO_S(AllocSize, Set);
}
if (Result != 0) {
return false;
}
int CPUs = CPU_COUNT_S(AllocSize, Set);
bool Hybrid = false;
uint64_t MIDR{};
for (int i = 0; i < CPUs; ++i) {
if (CPU_ISSET_S(i, AllocSize, Set)) {
std::error_code ec{};
std::string MIDRPath = "/sys/devices/system/cpu/cpu" + std::to_string(i) + "/regs/identification/midr_el1";
if (std::filesystem::exists(MIDRPath, ec)) {
std::vector<char> Data{};
// Needs to be a fixed size since depending on kernel it will try to read a full page of data and fail
// Only read 18 bytes for a 64bit value prefixed with 0x
if (FEXCore::FileLoading::LoadFile(Data, MIDRPath, 18)) {
uint64_t NewMIDR{};
if (FEXCore::StrConv::Conv(&Data.at(0), &NewMIDR)) {
if (MIDR != 0 && MIDR != NewMIDR) {
// CPU mismatch, claim hybrid
Hybrid = true;
break;
}
MIDR = NewMIDR;
}
}
}
}
}
CPU_FREE(Set);
return Hybrid;
}
#else
static uint32_t GetCycleCounterFrequency() {
uint32_t eax, ebx, ecx, edx;
@@ -55,6 +118,19 @@ static uint32_t GetCycleCounterFrequency() {
}
return 0;
}
static bool GetHostHybridFlag() {
uint32_t eax, ebx, ecx, edx;
__cpuid(0, eax, ebx, ecx, edx);
if (eax >= 0x7) {
__cpuid(0x7, eax, ebx, ecx, edx);
// Bit 15 of edx claims hybrid CPU
return (edx & (1U << 15)) != 0;
}
return false;
}
#endif
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_0h(uint32_t Leaf) {
@@ -96,7 +172,7 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_01h(uint32_t Leaf) {
(0 << 5) | // VMX
(0 << 6) | // SMX
(0 << 7) | // Intel SpeedStep
(0 << 8) | // Thermal Monitor 2
(1 << 8) | // Thermal Monitor 2
(1 << 9) | // SSSE3
(0 << 10) | // L1 context ID
(0 << 11) | // Silicon debug
@@ -151,7 +227,7 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_01h(uint32_t Leaf) {
(1 << 26) | // SSE2
(0 << 27) | // Self Snoop
(1 << 28) | // Max APIC IDs reserved field is valid
(0 << 29) | // Thermal monitor
(1 << 29) | // Thermal monitor
(0 << 30) | // Reserved
(0 << 31); // Pending break enable
return Res;
@@ -305,7 +381,7 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_07h(uint32_t Leaf) {
(1 << 0) | // FS/GS support
(0 << 1) | // TSC adjust MSR
(0 << 2) | // SGX
(0 << 3) | // BMI1
(1 << 3) | // BMI1
(0 << 4) | // Intel Hardware Lock Elison
(0 << 5) | // AVX2 support
(1 << 6) | // FPU data pointer updated only on exception
@@ -379,29 +455,29 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_07h(uint32_t Leaf) {
(0 << 6) | // Reserved
(0 << 7) | // Reserved
(0 << 8) | // AVX512_VP2INTERSECT
(0 << 9) | // Reserved
(0 << 9) | // SRBDS_CTRL (Special Register Buffer Data Sampling Mitigations)
(0 << 10) | // VERW clears CPU buffers
(0 << 11) | // Reserved
(0 << 12) | // Reserved
(0 << 13) | // Reserved
(0 << 13) | // TSX Force Abort (TSX will force abort if attempted)
(0 << 14) | // SERIALIZE instruction
(0 << 15) | // Reserved
(0 << 16) | // Reserved
((Hybrid ? 1U : 0U) << 15) | // Hybrid
(0 << 16) | // TSXLDTRK (TSX Suspend load address tracking) - Allows untracked memory loads inside TSX region
(0 << 17) | // Reserved
(0 << 18) | // Intel PCONFIG
(0 << 19) | // Intel Architectural LBR
(0 << 20) | // Intel CET
(0 << 21) | // Reserved
(0 << 22) | // Reserved
(0 << 23) | // Reserved
(0 << 24) | // Reserved
(0 << 25) | // Reserved
(0 << 26) | // Reserved
(0 << 27) | // Reserved
(0 << 22) | // AMX-BF16 - Tile computation on bfloat16
(0 << 23) | // AVX512_FP16 - FP16 AVX512 instructions
(0 << 24) | // AMX-tile - If AMX is implemented
(0 << 25) | // AMX-int8 - AMX on 8-bit integers
(0 << 26) | // IBRS_IBPB - Speculation control
(0 << 27) | // STIBP - Single Thread Indirect Branch Predictor, Part of IBC
(0 << 28) | // L1D Flush
(0 << 29) | // Arch capabilities
(0 << 30) | // Reserved
(0 << 31); // Reserved
(0 << 29) | // Arch capabilities - Speculative side channel mitigations
(0 << 30) | // Arch capabilities - MSR module specific
(0 << 31); // SSBD - Speculative Store Bypass Disable
}
return Res;
@@ -882,6 +958,9 @@ void CPUIDEmu::Init(FEXCore::Context::Context *ctx) {
#endif
// 0x8000'001E: Extended APIC ID
// 0x8000'001F: AMD Secure Encryption
// Setup some state tracking
Hybrid = GetHostHybridFlag();
}
}
+4 -4
View File
@@ -4,7 +4,9 @@
#include <FEXCore/Core/CPUID.h>
#include <FEXCore/Config/Config.h>
#include <FEXCore/Utils/LogManager.h>
#include <cstdint>
#include <utility>
namespace FEXCore {
namespace Context {
@@ -28,9 +30,6 @@ public:
auto Handler = FunctionHandlers.find(Function);
if (Handler == FunctionHandlers.end()) {
#ifndef NDEBUG
LogMan::Msg::E("Unhandled CPU ID function, 0x%x-0x%x", Function, Leaf);
#endif
return Function_Reserved(Leaf);
}
@@ -38,6 +37,7 @@ public:
}
private:
FEXCore::Context::Context *CTX;
bool Hybrid{};
FEX_CONFIG_OPT(Cores, THREADS);
using FunctionHandler = std::function<FEXCore::CPUID::FunctionResults(uint32_t Leaf)>;
+14 -1
View File
@@ -1,8 +1,21 @@
#include "Interface/Context/Context.h"
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/CompileService.h"
#include "Interface/Core/InternalThreadState.h"
#include "Interface/Core/OpcodeDispatcher.h"
#include "FEXCore/Debug/InternalThreadState.h"
#include "FEXCore/HLE/Linux/ThreadManagement.h"
#include "Interface/IR/PassManager.h"
#include <FEXCore/Core/CPUBackend.h>
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/Core/SignalDelegator.h>
#include <FEXCore/Utils/Event.h>
#include <FEXCore/Utils/LogManager.h>
#include <FEXCore/Utils/Threads.h>
#include <memory>
#include <pthread.h>
#include <stdio.h>
namespace FEXCore {
static void* ThreadHandler(void *Arg) {
+6 -7
View File
@@ -1,23 +1,22 @@
#pragma once
#include <FEXCore/Core/CPUBackend.h>
#include <FEXCore/Debug/InternalThreadState.h>
#include <FEXCore/Utils/Event.h>
#include <FEXCore/Utils/Threads.h>
#include <atomic>
#include <memory>
#include <thread>
#include <unordered_map>
#include <mutex>
#include <queue>
#include <stdint.h>
#include <vector>
namespace FEXCore {
namespace Context {
struct Context;
}
namespace Core {
struct InternalThreadState;
}
namespace IR {
class IRListView;
class RegisterAllocationData;
};
class CompileService final {
+150 -81
View File
@@ -7,42 +7,70 @@ desc: Glues Frontend, OpDispatcher and IR Opts & Compilation, LookupCache, Dispa
$end_info$
*/
#include "Common/MathUtils.h"
#include "Common/Paths.h"
#include "Interface/Context/Context.h"
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/BlockSamplingData.h"
#include "Interface/Core/CompileService.h"
#include "Interface/Core/Core.h"
#include "Interface/Core/DebugData.h"
#include "Interface/Core/CPUID.h"
#include "Interface/Core/Frontend.h"
#include "Interface/Core/GdbServer.h"
#include "Interface/Core/OpcodeDispatcher.h"
#include "Interface/Core/Interpreter/InterpreterCore.h"
#include "Interface/Core/JIT/JITCore.h"
#include "Interface/HLE/Thunks/Thunks.h"
#include "Interface/IR/Passes/RegisterAllocationPass.h"
#include "Interface/IR/Passes.h"
#include "Interface/IR/PassManager.h"
#include <FEXCore/Config/Config.h>
#include <FEXCore/Core/CodeLoader.h>
#include <FEXCore/Core/Context.h>
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/Core/CPUBackend.h>
#include <FEXCore/Core/SignalDelegator.h>
#include <FEXCore/Core/X86Enums.h>
#include <FEXCore/Debug/InternalThreadState.h>
#include <FEXCore/Debug/X86Tables.h>
#include <FEXCore/HLE/SyscallHandler.h>
#include <FEXCore/HLE/Linux/ThreadManagement.h>
#include <FEXCore/IR/IR.h>
#include <FEXCore/IR/IREmitter.h>
#include <FEXCore/IR/IntrusiveIRList.h>
#include <FEXCore/IR/RegisterAllocationData.h>
#include <FEXCore/Utils/Allocator.h>
#include <FEXCore/Utils/Event.h>
#include <FEXCore/Utils/LogManager.h>
#include <FEXCore/Utils/Threads.h>
#include "Interface/HLE/Thunks/Thunks.h"
#include "FEXCore/Utils/Allocator.h"
#include <xxh3.h>
#include <fstream>
#include <unistd.h>
#include <filesystem>
#include <algorithm>
#include <array>
#include <atomic>
#include <chrono>
#include <condition_variable>
#include <cstdint>
#include <filesystem>
#include <functional>
#include <map>
#include <memory>
#include <mutex>
#include <queue>
#include <set>
#include <shared_mutex>
#include <signal.h>
#include <stdio.h>
#include <string.h>
#include <string>
#include <string_view>
#include <sstream>
#include <sys/mman.h>
#include <unistd.h>
#include <sys/stat.h>
#include "Interface/Core/GdbServer.h"
#include <sys/syscall.h>
#include <type_traits>
#include <unistd.h>
#include <unordered_map>
#include <utility>
#include <vector>
#include <xxhash.h>
namespace FEXCore::CPU {
bool CreateCPUCore(FEXCore::Context::Context *CTX) {
@@ -195,11 +223,7 @@ namespace FEXCore::Context {
}
}
bool Context::InitCore(FEXCore::CodeLoader *Loader) {
ThunkHandler.reset(FEXCore::ThunkHandler::Create());
LocalLoader = Loader;
using namespace FEXCore::Core;
static FEXCore::Core::CPUState CreateDefaultCPUState() {
FEXCore::Core::CPUState NewThreadState{};
// Initialize default CPU state
@@ -216,7 +240,18 @@ namespace FEXCore::Context {
NewThreadState.flags[1] = 1;
NewThreadState.flags[9] = 1;
NewThreadState.FCW = 0x37F;
NewThreadState.FTW = 0xFFFF;
return NewThreadState;
}
FEXCore::Core::InternalThreadState* Context::InitCore(FEXCore::CodeLoader *Loader) {
ThunkHandler.reset(FEXCore::ThunkHandler::Create());
LocalLoader = Loader;
using namespace FEXCore::Core;
FEXCore::CPU::InitializeInterpreterOpHandlers();
FEXCore::Core::CPUState NewThreadState = CreateDefaultCPUState();
FEXCore::Core::InternalThreadState *Thread = CreateThread(&NewThreadState, 0);
// We are the parent thread
@@ -227,8 +262,7 @@ namespace FEXCore::Context {
Thread->CurrentFrame->State.rip = StartingRIP = Loader->DefaultRIP();
InitializeThreadData(Thread);
return true;
return Thread;
}
void Context::StartGdbServer() {
@@ -242,8 +276,7 @@ namespace FEXCore::Context {
DebugServer.reset();
}
void Context::HandleCallback(uint64_t RIP) {
auto Thread = Core::ThreadData.Thread;
void Context::HandleCallback(FEXCore::Core::InternalThreadState *Thread, uint64_t RIP) {
Thread->CPUBackend->CallbackPtr(Thread->CurrentFrame, RIP);
}
@@ -424,7 +457,13 @@ namespace FEXCore::Context {
auto IRHandler = [Thread](uint64_t Addr, IR::IREmitter *IR) -> void {
// Run the passmanager over the IR from the dispatcher
Thread->PassManager->Run(IR);
Core::LocalIREntry Entry = {Addr, 0ULL, decltype(Entry.IR)(IR->CreateIRCopy()), decltype(Entry.RAData)(Thread->PassManager->GetRAPass() ? Thread->PassManager->GetRAPass()->PullAllocationData() : nullptr), decltype(Entry.DebugData)(new Core::DebugData())};
Core::LocalIREntry Entry = {Addr, 0ULL,
decltype(Entry.IR)(IR->CreateIRCopy()),
decltype(Entry.RAData)(Thread->PassManager->HasPass("RA")
? Thread->PassManager->GetPass<IR::RegisterAllocationPass>("RA")->PullAllocationData()
: nullptr),
decltype(Entry.DebugData)(new Core::DebugData())
};
Thread->LocalIRCache.insert({Addr, std::move(Entry)});
};
@@ -455,6 +494,14 @@ namespace FEXCore::Context {
Thread->ThreadWaiting.Wait();
}
void Context::InitializeThreadTLSData(FEXCore::Core::InternalThreadState *Thread) {
// Let's do some initial bookkeeping here
Thread->ThreadManager.TID = ::gettid();
Thread->ThreadManager.PID = ::getpid();
SignalDelegation->RegisterTLSState(Thread);
ThunkHandler->RegisterTLSState(Thread);
}
void Context::RunThread(FEXCore::Core::InternalThreadState *Thread) {
// Tell the thread to start executing
Thread->StartRunning.NotifyAll();
@@ -470,8 +517,10 @@ namespace FEXCore::Context {
Stop(false /* Ignore current thread */);
});
State->CTX = this;
#if _M_ARM_64
bool DoSRA = true;
bool DoSRA = State->CTX->Config.StaticRegisterAllocation;
#else
bool DoSRA = false;
#endif
@@ -481,8 +530,6 @@ namespace FEXCore::Context {
State->PassManager->RegisterSyscallHandler(SyscallHandler);
State->CTX = this;
// Create CPU backend
switch (Config.Core) {
case FEXCore::Config::CONFIG_INTERPRETER:
@@ -586,6 +633,12 @@ namespace FEXCore::Context {
// Clean up dead stacks
FEXCore::Threads::Thread::CleanupAfterFork();
if (LiveThread->CompileService) {
// If this live thread had a compile service then it no longer exists
// Erase the shared_ptr
LiveThread->CompileService.reset();
}
}
void Context::AddBlockMapping(FEXCore::Core::InternalThreadState *Thread, uint64_t Address, void *Ptr, uint64_t Start, uint64_t Length) {
@@ -777,17 +830,17 @@ namespace FEXCore::Context {
Thread->PassManager->Run(Thread->OpDispatcher.get());
if (Thread->CTX->Config.DumpIR() != "no") {
IRDumper(Thread->PassManager->GetRAPass() ? Thread->PassManager->GetRAPass()->GetAllocationData() : nullptr);
IRDumper(Thread->PassManager->HasPass("RA") ? Thread->PassManager->GetPass<IR::RegisterAllocationPass>("RA")->GetAllocationData() : nullptr);
}
if (Thread->OpDispatcher->ShouldDump) {
std::stringstream out;
auto NewIR = Thread->OpDispatcher->ViewIR();
FEXCore::IR::Dump(&out, &NewIR, Thread->PassManager->GetRAPass() ? Thread->PassManager->GetRAPass()->GetAllocationData() : nullptr);
FEXCore::IR::Dump(&out, &NewIR, Thread->PassManager->HasPass("RA") ? Thread->PassManager->GetPass<IR::RegisterAllocationPass>("RA")->GetAllocationData() : nullptr);
LogMan::Msg::I("IR 0x%lx:\n%s\n@@@@@\n", GuestRIP, out.str().c_str());
}
auto RAData = Thread->PassManager->GetRAPass() ? Thread->PassManager->GetRAPass()->PullAllocationData() : nullptr;
auto RAData = Thread->PassManager->HasPass("RA") ? Thread->PassManager->GetPass<IR::RegisterAllocationPass>("RA")->PullAllocationData() : nullptr;
auto IRList = Thread->OpDispatcher->CreateIRCopy();
Thread->OpDispatcher->ResetWorkingList();
@@ -1084,10 +1137,25 @@ namespace FEXCore::Context {
if (NewBlock == 0) {
LogMan::Msg::E("CompileBlockJit: Failed to compile code %lX - aborting process", GuestRIP);
abort();
// Return similar behaviour of SIGILL abort
Frame->Thread->StatusCode = 128 + SIGILL;
Stop(false /* Ignore current thread */);
}
}
Context::AddrToFileMapType::iterator Context::FindAddrForFile(uint64_t Entry, uint64_t Length) {
// Thread safety here! We are returning an iterator to the map object
// This needs the AOTIRCacheLock locked prior to coming in to the function
auto file = AddrToFile.lower_bound(Entry);
if (file != AddrToFile.begin()) {
--file;
if (file->second.Start <= Entry && (file->second.Start + file->second.Len) >= (Entry + Length)) {
return file;
}
}
return AddrToFile.end();
}
uintptr_t Context::CompileBlock(FEXCore::Core::CpuStateFrame *Frame, uint64_t GuestRIP) {
auto Thread = Frame->Thread;
@@ -1147,63 +1215,68 @@ namespace FEXCore::Context {
}
// The core managed to compile the code.
#if ENABLE_JITSYMBOLS
if (DebugData) {
if (DebugData->Subblocks.size()) {
for (auto& Subblock: DebugData->Subblocks) {
Symbols.Register((void*)Subblock.HostCodeStart, GuestRIP, Subblock.HostCodeSize);
if (Config.BlockJITNaming()) {
if (DebugData) {
if (DebugData->Subblocks.size()) {
for (auto& Subblock: DebugData->Subblocks) {
Symbols.Register((void*)Subblock.HostCodeStart, GuestRIP, Subblock.HostCodeSize);
}
} else {
Symbols.Register(CodePtr, GuestRIP, DebugData->HostCodeSize);
}
} else {
Symbols.Register(CodePtr, GuestRIP, DebugData->HostCodeSize);
}
}
#endif
// Insert to caches if we generated IR
if (GeneratedIR) {
// Add to AOT cache if aot generation is enabled
if ((Config.AOTIRCapture() || Config.AOTIRGenerate()) && RAData) {
auto hash = XXH3_64bits((void*)StartAddr, Length);
// Both generated ir and LibraryJITName need a named region lookup
if (GeneratedIR || Config.LibraryJITNaming()) {
std::shared_lock lk(AOTIRCacheLock);
std::shared_lock lk(AOTIRCacheLock);
auto file = FindAddrForFile(StartAddr, Length);
auto file = AddrToFile.lower_bound(StartAddr);
if (file != AddrToFile.begin()) {
--file;
if (file->second.Start <= StartAddr && (file->second.Start + file->second.Len) >= (StartAddr + Length)) {
auto LocalRIP = GuestRIP - file->second.Start + file->second.Offset;
auto LocalStartAddr = StartAddr - file->second.Start + file->second.Offset;
auto fileid = file->second.fileid;
AOTIRCaptureCacheWriteoutQueue_Append([this, LocalRIP, LocalStartAddr, Length, hash, IRList, RAData, fileid]() {
auto *AotFile = &AOTIRCaptureCache[fileid];
if (!AotFile->Stream) {
AotFile->Stream = AOTIRWriter(fileid);
uint64_t tag = 0xDEADBEEFC0D30004;
AotFile->Stream->write((char*)&tag, sizeof(tag));
}
AotFile->AppendAOTIRCaptureCache(LocalRIP, LocalStartAddr, Length, hash, IRList, RAData);
delete IRList;
FEXCore::Allocator::free(RAData);
});
}
// Only go down this path if we actually found a library region
if (file != AddrToFile.end()) {
if (DebugData && Config.LibraryJITNaming()) {
Symbols.RegisterNamedRegion(CodePtr, DebugData->HostCodeSize, file->second.filename);
}
if (Config.AOTIRGenerate()) {
// cleanup memory and early exit here -- we're not running the application
// Add to AOT cache if aot generation is enabled
if (GeneratedIR && RAData &&
(Config.AOTIRCapture() || Config.AOTIRGenerate())) {
auto hash = XXH3_64bits((void*)StartAddr, Length);
if (DecrementRefCount)
--Thread->CompileBlockReentrantRefCount;
auto LocalRIP = GuestRIP - file->second.Start + file->second.Offset;
auto LocalStartAddr = StartAddr - file->second.Start + file->second.Offset;
auto fileid = file->second.fileid;
AOTIRCaptureCacheWriteoutQueue_Append([this, LocalRIP, LocalStartAddr, Length, hash, IRList, RAData, fileid]() {
auto *AotFile = &AOTIRCaptureCache[fileid];
Thread->CPUBackend->ClearCache();
if (!AotFile->Stream) {
AotFile->Stream = AOTIRWriter(fileid);
uint64_t tag = 0xDEADBEEFC0D30004;
AotFile->Stream->write((char*)&tag, sizeof(tag));
}
AotFile->AppendAOTIRCaptureCache(LocalRIP, LocalStartAddr, Length, hash, IRList, RAData);
});
return (uintptr_t)CodePtr;
if (Config.AOTIRGenerate()) {
// cleanup memory and early exit here -- we're not running the application
if (DecrementRefCount)
--Thread->CompileBlockReentrantRefCount;
Thread->CPUBackend->ClearCache();
return (uintptr_t)CodePtr;
}
}
}
// Add to thread local ir cache
Core::LocalIREntry Entry = {StartAddr, Length, decltype(Entry.IR)(IRList), decltype(Entry.RAData)(RAData), decltype(Entry.DebugData)(DebugData)};
Thread->LocalIRCache.insert({GuestRIP, std::move(Entry)});
// Insert to caches if we generated IR
if (GeneratedIR) {
// Add to thread local ir cache
Core::LocalIREntry Entry = {StartAddr, Length, decltype(Entry.IR)(IRList), decltype(Entry.RAData)(RAData), decltype(Entry.DebugData)(DebugData)};
Thread->LocalIRCache.insert({GuestRIP, std::move(Entry)});
}
}
if (DecrementRefCount)
@@ -1219,11 +1292,7 @@ namespace FEXCore::Context {
Core::ThreadData.Thread = Thread;
Thread->ExitReason = FEXCore::Context::ExitReason::EXIT_WAITING;
// Let's do some initial bookkeeping here
Thread->ThreadManager.TID = ::gettid();
Thread->ThreadManager.PID = ::getpid();
SignalDelegation->RegisterTLSState(Thread);
ThunkHandler->RegisterTLSState(Thread);
InitializeThreadTLSData(Thread);
++IdleWaitRefCount;
@@ -1337,7 +1406,7 @@ namespace FEXCore::Context {
// TODO: Support overlapping maps and region splitting
auto base_filename = std::filesystem::path(filename).filename().string();
if (base_filename.size()) {
if (!base_filename.empty()) {
auto filename_hash = XXH3_64bits(filename.c_str(), filename.size());
auto fileid = base_filename + "-" + std::to_string(filename_hash) + "-";
@@ -1,16 +1,31 @@
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/ArchHelpers/MContext.h"
#include "Interface/Core/Dispatcher/Arm64Dispatcher.h"
#include "Interface/Core/Interpreter/InterpreterClass.h"
#include "Interface/Context/Context.h"
#include "Interface/Context/Context.h"
#include "Interface/Core/X86HelperGen.h"
#include <FEXCore/Core/CPUBackend.h>
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/Core/X86Enums.h>
#include <FEXCore/Debug/InternalThreadState.h>
#include <array>
#include <bit>
#include <cmath>
#include <cstdint>
#include <memory>
#include <stddef.h>
#include "aarch64/assembler-aarch64.h"
#include "aarch64/constants-aarch64.h"
#include "aarch64/operands-aarch64.h"
#include "aarch64/cpu-aarch64.h"
#include "aarch64/disasm-aarch64.h"
#include "code-buffer-vixl.h"
#include "platform-vixl.h"
#include <unistd.h>
namespace FEXCore::CPU {
@@ -25,8 +40,7 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::
SRAEnabled = config.StaticRegisterAssignment;
SetAllowAssembler(true);
auto Buffer = GetBuffer();
DispatchPtr = Buffer->GetOffsetAddress<CPUBackend::AsmDispatch>(GetCursorOffset());
DispatchPtr = GetCursorAddress<CPUBackend::AsmDispatch>();
// while (true) {
// Ptr = FindBlock(RIP)
@@ -59,7 +73,7 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::
add(x0, sp, 0);
str(x0, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, ReturningStackLocation)));
AbsoluteLoopTopAddressFillSRA = Buffer->GetOffsetAddress<uint64_t>(GetCursorOffset());
AbsoluteLoopTopAddressFillSRA = GetCursorAddress<uint64_t>();
if (SRAEnabled) {
FillStaticRegs();
@@ -178,11 +192,11 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::
{
bind(&ExitSpillSRA);
ThreadStopHandlerAddressSpillSRA = Buffer->GetOffsetAddress<uint64_t>(GetCursorOffset());
ThreadStopHandlerAddressSpillSRA = GetCursorAddress<uint64_t>();
if (SRAEnabled)
SpillStaticRegs();
ThreadStopHandlerAddress = Buffer->GetOffsetAddress<uint64_t>(GetCursorOffset());
ThreadStopHandlerAddress = GetCursorAddress<uint64_t>();
PopCalleeSavedRegisters();
@@ -192,7 +206,7 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::
}
{
ExitFunctionLinkerAddress = Buffer->GetOffsetAddress<uint64_t>(GetCursorOffset());
ExitFunctionLinkerAddress = GetCursorAddress<uint64_t>();
if (SRAEnabled)
SpillStaticRegs();
@@ -229,7 +243,7 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::
}
{
SignalHandlerReturnAddress = Buffer->GetOffsetAddress<uint64_t>(GetCursorOffset());
SignalHandlerReturnAddress = GetCursorAddress<uint64_t>();
// Now to get back to our old location we need to do a fault dance
// We can't use SIGTRAP here since gdb catches it and never gives it to the application!
@@ -237,12 +251,12 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::
}
{
ThreadPauseHandlerAddressSpillSRA = Buffer->GetOffsetAddress<uint64_t>(GetCursorOffset());
ThreadPauseHandlerAddressSpillSRA = GetCursorAddress<uint64_t>();
if (SRAEnabled)
SpillStaticRegs();
bind(&ThreadPauseHandler);
ThreadPauseHandlerAddress = Buffer->GetOffsetAddress<uint64_t>(GetCursorOffset());
ThreadPauseHandlerAddress = GetCursorAddress<uint64_t>();
// We are pausing, this means the frontend should be waiting for this thread to idle
// We will have faulted and jumped to this location at this point
@@ -252,7 +266,7 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::
ldr(x2, &l_Sleep);
blr(x2);
PauseReturnInstruction = Buffer->GetOffsetAddress<uint64_t>(GetCursorOffset());
PauseReturnInstruction = GetCursorAddress<uint64_t>();
// Fault to start running again
hlt(0);
}
@@ -273,7 +287,7 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::
// On return to the thunk, the thunk can get whatever its return value is from the thread context depending on ABI handling on its end
// When the thunk itself returns, it'll do its regular return logic there
// void ReentrantCallback(FEXCore::Core::InternalThreadState *Thread, uint64_t RIP);
CallbackPtr = Buffer->GetOffsetAddress<CPUBackend::JITCallback>(GetCursorOffset());
CallbackPtr = GetCursorAddress<CPUBackend::JITCallback>();
// We expect the thunk to have previously pushed the registers it was using
PushCalleeSavedRegisters();
@@ -322,28 +336,28 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::
FinalizeCode();
Start = reinterpret_cast<uint64_t>(DispatchPtr);
End = Buffer->GetOffsetAddress<uint64_t>(GetCursorOffset());
End = GetCursorAddress<uint64_t>();
vixl::aarch64::CPU::EnsureIAndDCacheCoherency(reinterpret_cast<void*>(DispatchPtr), End - reinterpret_cast<uint64_t>(DispatchPtr));
GetBuffer()->SetExecutable();
#if ENABLE_JITSYMBOLS
std::string Name = "Dispatch_" + std::to_string(::gettid());
CTX->Symbols.Register(reinterpret_cast<void*>(DispatchPtr), End - reinterpret_cast<uint64_t>(DispatchPtr), Name);
#endif
if (CTX->Config.BlockJITNaming()) {
std::string Name = "Dispatch_" + std::to_string(::gettid());
CTX->Symbols.Register(reinterpret_cast<void*>(DispatchPtr), End - reinterpret_cast<uint64_t>(DispatchPtr), Name);
}
if (CTX->Config.GlobalJITNaming()) {
CTX->Symbols.RegisterJITSpace(reinterpret_cast<void*>(DispatchPtr), End - reinterpret_cast<uint64_t>(DispatchPtr));
}
}
void Arm64Dispatcher::SpillSRA(void *ucontext) {
for(int i = 0; i < SRA64.size(); i++) {
ThreadState->CurrentFrame->State.gregs[i] = ArchHelpers::Context::GetArmReg(ucontext, SRA64[i].GetCode());
}
// TODO: Also recover FPRs, not sure where the neon context is
// This is usually not needed
/*
for(int i = 0; i < SRAFPR.size(); i++) {
State->State.State.xmm[i][0] = _mcontext.neon[SRAFPR[i].GetCode()];
State->State.State.xmm[i][0] = _mcontext.neon[SRAFPR[i].GetCode()];
auto FPR = ArchHelpers::Context::GetArmFPR(ucontext, SRAFPR[i].GetCode());
memcpy(&ThreadState->CurrentFrame->State.xmm[i][0], &FPR, sizeof(__uint128_t));
}
*/
}
#ifdef _M_ARM_64
@@ -3,7 +3,13 @@
#include "Interface/Core/ArchHelpers/Arm64Emitter.h"
#include "Interface/Core/Dispatcher/Dispatcher.h"
#include "aarch64/assembler-aarch64.h"
namespace FEXCore::Context {
struct Context;
}
namespace FEXCore::Core {
struct InternalThreadState;
}
namespace FEXCore::CPU {
@@ -15,4 +21,4 @@ class Arm64Dispatcher final : public Dispatcher, public Arm64Emitter {
void SpillSRA(void *ucontext) override;
};
}
}
@@ -1,8 +1,23 @@
#include "Interface/Core/Dispatcher/Dispatcher.h"
#include "Common/MathUtils.h"
#include "Interface/Core/ArchHelpers/MContext.h"
#include "Interface/Core/Dispatcher/Dispatcher.h"
#include "Interface/Core/X86HelperGen.h"
#include <FEXCore/Config/Config.h>
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/Core/SignalDelegator.h>
#include <FEXCore/Core/UContext.h>
#include <FEXCore/Core/X86Enums.h>
#include <FEXCore/Debug/InternalThreadState.h>
#include <FEXCore/Utils/Event.h>
#include <FEXCore/Utils/LogManager.h>
#include <atomic>
#include <condition_variable>
#include <bits/types/siginfo_t.h>
#include <signal.h>
#include <string.h>
namespace FEXCore::CPU {
@@ -54,6 +69,7 @@ void Dispatcher::StoreThreadState(int Signal, void *ucontext) {
}
void Dispatcher::RestoreThreadState(void *ucontext) {
LOGMAN_THROW_A(!SignalFrames.empty(), "Trying to restore a signal frame when we don't have any");
uint64_t OldSP = SignalFrames.top();
SignalFrames.pop();
uintptr_t NewSP = OldSP;
@@ -64,10 +80,38 @@ void Dispatcher::RestoreThreadState(void *ucontext) {
// Now restore host state
ArchHelpers::Context::RestoreContext(ucontext, Context);
}
// Restore the previous signal state
// This allows recursive signals to properly handle signal masking as we are walking back up the list of signals
CTX->SignalDelegation->SetCurrentSignal(Context->Signal);
static uint32_t ConvertSignalToTrapNo(int Signal, siginfo_t *HostSigInfo) {
switch (Signal) {
case SIGSEGV:
if (HostSigInfo->si_code == SEGV_MAPERR ||
HostSigInfo->si_code == SEGV_ACCERR) {
// Protection fault
return X86State::X86_TRAPNO_PF;
}
break;
}
// Unknown mapping, fall back to old behaviour and just pass signal
return Signal;
}
static uint32_t ConvertSignalToError(int Signal, siginfo_t *HostSigInfo) {
switch (Signal) {
case SIGSEGV:
if (HostSigInfo->si_code == SEGV_MAPERR ||
HostSigInfo->si_code == SEGV_ACCERR) {
// Protection fault
// Always a user fault for us
// XXX: PF_PROT and PF_WRITE
return X86State::X86_PF_USER;
}
break;
}
// Not a page fault issue
return 0;
}
bool Dispatcher::HandleGuestSignal(int Signal, void *info, void *ucontext, GuestSigAction *GuestAction, stack_t *GuestStack) {
@@ -78,60 +122,97 @@ bool Dispatcher::HandleGuestSignal(int Signal, void *info, void *ucontext, Guest
// We use this to track if it is safe to clear cache
++SignalHandlerRefCounter;
uint64_t OldPC = ArchHelpers::Context::GetPc(ucontext);
// Set the new PC
ArchHelpers::Context::SetPc(ucontext, AbsoluteLoopTopAddressFillSRA);
// Set our state register to point to our guest thread data
ArchHelpers::Context::SetState(ucontext, reinterpret_cast<uint64_t>(Frame));
uint64_t OldGuestSP = Frame->State.gregs[X86State::REG_RSP];
uint64_t NewGuestSP = OldGuestSP;
if (!(GuestStack->ss_flags & SS_DISABLE)) {
// If our guest is already inside of the alternative stack
// Then that means we are hitting recursive signals and we need to walk back the stack correctly
uint64_t AltStackBase = reinterpret_cast<uint64_t>(GuestStack->ss_sp);
uint64_t AltStackEnd = AltStackBase + GuestStack->ss_size;
if (OldGuestSP >= AltStackBase &&
OldGuestSP <= AltStackEnd) {
// We are already in the alt stack, the rest of the code will handle adjusting this
}
else {
NewGuestSP = AltStackEnd;
// Pulling from context here
bool Is64BitMode = CTX->Config.Is64BitMode;
uint64_t SignalReturn = CTX->X86CodeGen.SignalReturn;
// Spill the SRA regardless of signal handler type
// We are going to be returning to the top of the dispatcher which will fill again
// Otherwise we might load garbage
if (SRAEnabled) {
if (IsAddressInJITCode(OldPC, false)) {
// We are in jit, SRA must be spilled
SpillSRA(ucontext);
} else {
if (!IsAddressInJITCode(OldPC, true)) {
// This is likely to cause issues but in some cases it isn't fatal
// This can also happen if we have put a signal on hold, then we just reenabled the signal
// So we are in the syscall handler
// Only throw a log message in this case
LogMan::Msg::E("Signals in dispatcher have unsynchronized context");
}
}
}
// Back up past the redzone, which is 128bytes
// Don't need this offset if we aren't going to be putting siginfo in to it
NewGuestSP -= 128;
if (GuestAction->sa_flags & SA_SIGINFO) {
if (SRAEnabled) {
if (!IsAddressInJITCode(ArchHelpers::Context::GetPc(ucontext), false)) {
LOGMAN_THROW_A(!IsAddressInJITCode(ArchHelpers::Context::GetPc(ucontext), true), "Signals in dispatcher have unsynchronized context");
} else {
// We are in jit, SRA must be spilled
SpillSRA(ucontext);
// altstack is only used if the signal handler was setup with SA_ONSTACK
if (GuestAction->sa_flags & SA_ONSTACK) {
// Additionally the altstack is only used if the enabled (SS_DISABLE flag is not set)
if (!(GuestStack->ss_flags & SS_DISABLE)) {
// If our guest is already inside of the alternative stack
// Then that means we are hitting recursive signals and we need to walk back the stack correctly
uint64_t AltStackBase = reinterpret_cast<uint64_t>(GuestStack->ss_sp);
uint64_t AltStackEnd = AltStackBase + GuestStack->ss_size;
if (OldGuestSP >= AltStackBase &&
OldGuestSP <= AltStackEnd) {
// We are already in the alt stack, the rest of the code will handle adjusting this
}
else {
NewGuestSP = AltStackEnd;
}
}
}
if (Is64BitMode) {
// Back up past the redzone, which is 128bytes
// 32-bit doesn't have a redzone
NewGuestSP -= 128;
}
// siginfo_t
siginfo_t *HostSigInfo = reinterpret_cast<siginfo_t*>(info);
if (GuestAction->sa_flags & SA_SIGINFO) {
// Setup ucontext a bit
if (CTX->Config.Is64BitMode) {
if (Is64BitMode) {
NewGuestSP -= sizeof(FEXCore::x86_64::_libc_fpstate);
NewGuestSP = AlignDown(NewGuestSP, alignof(FEXCore::x86_64::_libc_fpstate));
uint64_t FPStateLocation = NewGuestSP;
NewGuestSP -= sizeof(FEXCore::x86_64::ucontext_t);
NewGuestSP = AlignDown(NewGuestSP, alignof(FEXCore::x86_64::ucontext_t));
uint64_t UContextLocation = NewGuestSP;
NewGuestSP -= sizeof(siginfo_t);
NewGuestSP = AlignDown(NewGuestSP, alignof(siginfo_t));
uint64_t SigInfoLocation = NewGuestSP;
FEXCore::x86_64::ucontext_t *guest_uctx = reinterpret_cast<FEXCore::x86_64::ucontext_t*>(UContextLocation);
siginfo_t *guest_siginfo = reinterpret_cast<siginfo_t*>(SigInfoLocation);
// We have extended float information
guest_uctx->uc_flags |= FEXCore::x86_64::UC_FP_XSTATE;
guest_uctx->uc_flags = FEXCore::x86_64::UC_FP_XSTATE;
// Pointer to where the fpreg memory is
guest_uctx->uc_mcontext.fpregs = &guest_uctx->__fpregs_mem;
guest_uctx->uc_mcontext.fpregs = reinterpret_cast<FEXCore::x86_64::_libc_fpstate*>(FPStateLocation);
FEXCore::x86_64::_libc_fpstate *fpstate = reinterpret_cast<FEXCore::x86_64::_libc_fpstate*>(FPStateLocation);
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_RIP] = Frame->State.rip;
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_EFL] = 0;
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_CSGSFS] = 0;
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_ERR] = ConvertSignalToError(Signal, HostSigInfo);
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_TRAPNO] = ConvertSignalToTrapNo(Signal, HostSigInfo);
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_OLDMASK] = 0;
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_CR2] = 0;
#define COPY_REG(x) \
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_##x] = Frame->State.gregs[X86State::REG_##x];
@@ -154,14 +235,15 @@ bool Dispatcher::HandleGuestSignal(int Signal, void *info, void *ucontext, Guest
#undef COPY_REG
// Copy float registers
memcpy(guest_uctx->__fpregs_mem._st, Frame->State.mm, sizeof(Frame->State.mm));
memcpy(guest_uctx->__fpregs_mem._xmm, Frame->State.xmm, sizeof(Frame->State.xmm));
memcpy(fpstate->_st, Frame->State.mm, sizeof(Frame->State.mm));
memcpy(fpstate->_xmm, Frame->State.xmm, sizeof(Frame->State.xmm));
// FCW store default
guest_uctx->__fpregs_mem.fcw = Frame->State.FCW;
fpstate->fcw = Frame->State.FCW;
fpstate->ftw = Frame->State.FTW;
// Reconstruct FSW
guest_uctx->__fpregs_mem.fsw =
fpstate->fsw =
(Frame->State.flags[FEXCore::X86State::X87FLAG_TOP_LOC] << 11) |
(Frame->State.flags[FEXCore::X86State::X87FLAG_C0_LOC] << 8) |
(Frame->State.flags[FEXCore::X86State::X87FLAG_C1_LOC] << 9) |
@@ -173,8 +255,6 @@ bool Dispatcher::HandleGuestSignal(int Signal, void *info, void *ucontext, Guest
guest_uctx->uc_stack.ss_sp = GuestStack->ss_sp;
guest_uctx->uc_stack.ss_size = GuestStack->ss_size;
// siginfo_t
siginfo_t *HostSigInfo = reinterpret_cast<siginfo_t*>(info);
// aarch64 and x86_64 siginfo_t matches. We can just copy this over
// SI_USER could also potentially have random data in it, needs to be bit perfect
// For guest faults we don't have a real way to reconstruct state to a real guest RIP
@@ -184,14 +264,77 @@ bool Dispatcher::HandleGuestSignal(int Signal, void *info, void *ucontext, Guest
Frame->State.gregs[X86State::REG_RDX] = UContextLocation;
}
else {
// XXX: 32bit Support
NewGuestSP -= sizeof(FEXCore::x86::_libc_fpstate);
NewGuestSP = AlignDown(NewGuestSP, alignof(FEXCore::x86::_libc_fpstate));
uint64_t FPStateLocation = NewGuestSP;
NewGuestSP -= sizeof(FEXCore::x86::ucontext_t);
uint64_t UContextLocation = 0; // NewGuestSP;
NewGuestSP = AlignDown(NewGuestSP, alignof(FEXCore::x86::ucontext_t));
uint64_t UContextLocation = NewGuestSP;
NewGuestSP -= sizeof(FEXCore::x86::siginfo_t);
NewGuestSP = AlignDown(NewGuestSP, alignof(FEXCore::x86::siginfo_t));
uint64_t SigInfoLocation = NewGuestSP;
FEXCore::x86::ucontext_t *guest_uctx = reinterpret_cast<FEXCore::x86::ucontext_t*>(UContextLocation);
FEXCore::x86::siginfo_t *guest_siginfo = reinterpret_cast<FEXCore::x86::siginfo_t*>(SigInfoLocation);
siginfo_t *HostSigInfo = reinterpret_cast<siginfo_t*>(info);
// We have extended float information
guest_uctx->uc_flags = FEXCore::x86::UC_FP_XSTATE;
// Pointer to where the fpreg memory is
guest_uctx->uc_mcontext.fpregs = static_cast<uint32_t>(FPStateLocation);
FEXCore::x86::_libc_fpstate *fpstate = reinterpret_cast<FEXCore::x86::_libc_fpstate*>(FPStateLocation);
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_GS] = Frame->State.gs;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_FS] = Frame->State.fs;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_ES] = Frame->State.es;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_DS] = Frame->State.ds;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_TRAPNO] = ConvertSignalToTrapNo(Signal, HostSigInfo);
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_ERR] = ConvertSignalToError(Signal, HostSigInfo);
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_EIP] = Frame->State.rip;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_CS] = Frame->State.cs;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_EFL] = 0;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_UESP] = 0;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_SS] = Frame->State.ss;
#define COPY_REG(x) \
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_##x] = Frame->State.gregs[X86State::REG_##x];
COPY_REG(RDI);
COPY_REG(RSI);
COPY_REG(RBP);
COPY_REG(RBX);
COPY_REG(RDX);
COPY_REG(RAX);
COPY_REG(RCX);
COPY_REG(RSP);
#undef COPY_REG
// Copy float registers
for (size_t i = 0; i < 8; ++i) {
// 32-bit st register size is only 10 bytes. Not padded to 16byte like x86-64
memcpy(&fpstate->_st[i], &Frame->State.mm[i], 10);
}
// Extended XMM state
fpstate->status = FEXCore::x86::fpstate_magic::MAGIC_XFPSTATE;
memcpy(fpstate->_xmm, Frame->State.xmm, sizeof(Frame->State.xmm));
// FCW store default
fpstate->fcw = Frame->State.FCW;
fpstate->ftw = Frame->State.FTW;
// Reconstruct FSW
fpstate->fsw =
(Frame->State.flags[FEXCore::X86State::X87FLAG_TOP_LOC] << 11) |
(Frame->State.flags[FEXCore::X86State::X87FLAG_C0_LOC] << 8) |
(Frame->State.flags[FEXCore::X86State::X87FLAG_C1_LOC] << 9) |
(Frame->State.flags[FEXCore::X86State::X87FLAG_C2_LOC] << 10) |
(Frame->State.flags[FEXCore::X86State::X87FLAG_C3_LOC] << 14);
// Copy over signal stack information
guest_uctx->uc_stack.ss_flags = GuestStack->ss_flags;
guest_uctx->uc_stack.ss_sp = static_cast<uint32_t>(reinterpret_cast<uint64_t>(GuestStack->ss_sp));
guest_uctx->uc_stack.ss_size = GuestStack->ss_size;
// These three elements are in every siginfo
guest_siginfo->si_signo = HostSigInfo->si_signo;
@@ -201,6 +344,12 @@ bool Dispatcher::HandleGuestSignal(int Signal, void *info, void *ucontext, Guest
switch (Signal) {
case SIGSEGV:
case SIGBUS:
// Macro expansion to get the si_addr
// This is the address trying to be accessed, not the RIP
guest_siginfo->_sifields._sigfault.addr = static_cast<uint32_t>(reinterpret_cast<uintptr_t>(HostSigInfo->si_addr));
break;
case SIGFPE:
case SIGILL:
// Macro expansion to get the si_addr
// Can't really give a real result here. Pull from the context for now
guest_siginfo->_sifields._sigfault.addr = Frame->State.rip;
@@ -212,40 +361,59 @@ bool Dispatcher::HandleGuestSignal(int Signal, void *info, void *ucontext, Guest
guest_siginfo->_sifields._sigchld.utime = HostSigInfo->si_utime;
guest_siginfo->_sifields._sigchld.stime = HostSigInfo->si_stime;
break;
default:
LogMan::Msg::D("Unhandled siginfo_t signal: %d", Signal);
// Hope for the best, most things just copy over
memcpy(guest_siginfo, info, sizeof(siginfo_t));
break;
case SIGALRM:
case SIGVTALRM:
guest_siginfo->_sifields._timer.tid = HostSigInfo->si_timerid;
guest_siginfo->_sifields._timer.overrun = HostSigInfo->si_overrun;
guest_siginfo->_sifields._timer.sigval.sival_int = HostSigInfo->si_int;
break;
default:
LogMan::Msg::E("Unhandled siginfo_t for signal: %d\n", Signal);
break;
}
NewGuestSP -= 4;
*(uint32_t*)NewGuestSP = UContextLocation;
NewGuestSP -= 4;
*(uint32_t*)NewGuestSP = SigInfoLocation;
NewGuestSP -= 4;
*(uint32_t*)NewGuestSP = Signal;
}
Frame->State.rip = reinterpret_cast<uint64_t>(GuestAction->sigaction_handler.sigaction);
}
else {
if (!Is64BitMode) {
NewGuestSP -= 4;
*(uint32_t*)NewGuestSP = Signal;
}
Frame->State.rip = reinterpret_cast<uint64_t>(GuestAction->sigaction_handler.handler);
}
if (CTX->Config.Is64BitMode) {
Frame->State.gregs[X86State::REG_RDI] = Signal;
if (Is64BitMode) {
Frame->State.gregs[FEXCore::X86State::REG_RDI] = Signal;
// Set up the new SP for stack handling
NewGuestSP -= 8;
*(uint64_t*)NewGuestSP = CTX->X86CodeGen.SignalReturn;
Frame->State.gregs[X86State::REG_RSP] = NewGuestSP;
*(uint64_t*)NewGuestSP = SignalReturn;
Frame->State.gregs[FEXCore::X86State::REG_RSP] = NewGuestSP;
}
else {
NewGuestSP -= 4;
*(uint32_t*)NewGuestSP = CTX->X86CodeGen.SignalReturn;
LOGMAN_THROW_A(CTX->X86CodeGen.SignalReturn < 0x1'0000'0000ULL, "This needs to be below 4GB");
Frame->State.gregs[X86State::REG_RSP] = NewGuestSP;
*(uint32_t*)NewGuestSP = SignalReturn;
LOGMAN_THROW_A(SignalReturn < 0x1'0000'0000ULL, "This needs to be below 4GB");
Frame->State.gregs[FEXCore::X86State::REG_RSP] = NewGuestSP;
}
// The guest starts its signal frame with a zero initialized FPU
// Set that up now. Little bit costly but it's a requirement
// This state will be restored on rt_sigreturn
memset(Frame->State.xmm, 0, sizeof(Frame->State.xmm));
memset(Frame->State.mm, 0, sizeof(Frame->State.mm));
Frame->State.FCW = 0x37F;
Frame->State.FTW = 0xFFFF;
return true;
}
@@ -291,9 +459,6 @@ bool Dispatcher::HandleSignalPause(int Signal, void *info, void *ucontext) {
ArchHelpers::Context::SetPc(ucontext, ThreadPauseHandlerAddress);
}
// Set the new PC
ArchHelpers::Context::SetPc(ucontext, ThreadPauseHandlerAddress);
// Set our state register to point to our guest thread data
ArchHelpers::Context::SetState(ucontext, reinterpret_cast<uint64_t>(Frame));
@@ -1,11 +1,24 @@
#pragma once
#include <FEXCore/Core/CPUBackend.h>
#include <FEXCore/Core/SignalDelegator.h>
#include "Interface/Context/Context.h"
#include <bits/types/stack_t.h>
#include <cstdint>
#include <stddef.h>
#include <stack>
#include <tuple>
#include <vector>
namespace FEXCore {
struct GuestSigAction;
}
namespace FEXCore::Core {
struct CpuStateFrame;
struct InternalThreadState;
}
namespace FEXCore::CPU {
@@ -1,10 +1,23 @@
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/Dispatcher/X86Dispatcher.h"
#include "Interface/Core/Interpreter/InterpreterClass.h"
#include "Interface/Core/X86HelperGen.h"
#include "Interface/Context/Context.h"
#include <FEXCore/Core/X86Enums.h>
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/Core/CPUBackend.h>
#include <FEXCore/Debug/InternalThreadState.h>
#include <FEXCore/Utils/Allocator.h>
#include <cmath>
#include <memory>
#include <stddef.h>
#include <stdint.h>
#include <sys/mman.h>
#include "xbyak/xbyak.h"
namespace FEXCore::CPU {
static constexpr size_t MAX_DISPATCHER_CODE_SIZE = 4096;
@@ -12,7 +25,9 @@ static constexpr size_t MAX_DISPATCHER_CODE_SIZE = 4096;
X86Dispatcher::X86Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::InternalThreadState *Thread, DispatcherConfig &config)
: Dispatcher(ctx, Thread)
, Xbyak::CodeGenerator(MAX_DISPATCHER_CODE_SIZE, nullptr, this) {
, Xbyak::CodeGenerator(MAX_DISPATCHER_CODE_SIZE,
FEXCore::Allocator::mmap(nullptr, MAX_DISPATCHER_CODE_SIZE, PROT_READ | PROT_WRITE | PROT_EXEC, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0),
nullptr) {
using namespace Xbyak;
using namespace Xbyak::util;
@@ -291,14 +306,17 @@ X86Dispatcher::X86Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::Inte
Start = reinterpret_cast<uint64_t>(getCode());
End = Start + getSize();
#if ENABLE_JITSYMBOLS
if (CTX->Config.BlockJITNaming()) {
std::string Name = "Dispatch_" + std::to_string(::gettid());
CTX->Symbols.Register(reinterpret_cast<void*>(Start), End-Start, Name);
#endif
}
if (CTX->Config.GlobalJITNaming()) {
CTX->Symbols.RegisterJITSpace(reinterpret_cast<void*>(Start), End-Start);
}
}
X86Dispatcher::~X86Dispatcher() {
FEXCore::Allocator::munmap(top_, MAX_DISPATCHER_CODE_SIZE);
}
#ifdef _M_X86_64
@@ -2,26 +2,24 @@
#include "Interface/Core/Dispatcher/Dispatcher.h"
#include <FEXCore/Utils/Allocator.h>
#define XBYAK64
#include <xbyak/xbyak.h>
namespace FEXCore::Context {
struct Context;
}
namespace FEXCore::Core {
struct InternalThreadState;
}
namespace FEXCore::CPU {
class X86Dispatcher final : public Dispatcher, public Xbyak::CodeGenerator, public Xbyak::Allocator {
class X86Dispatcher final : public Dispatcher, public Xbyak::CodeGenerator {
public:
X86Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::InternalThreadState *Thread, DispatcherConfig &config);
virtual ~X86Dispatcher() override;
// Xbyak::Allocator
Xbyak::uint8 *alloc(size_t size) override { Size = size; return reinterpret_cast<uint8_t*>(FEXCore::Allocator::mmap(nullptr, size, PROT_READ | PROT_WRITE | PROT_EXEC, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0)); }
void free(Xbyak::uint8 *p) override { FEXCore::Allocator::munmap(p, Size); }
bool useProtect() const override { return false; }
private:
size_t Size{};
};
}
+142 -30
View File
@@ -7,17 +7,24 @@ $end_info$
#include "Interface/Context/Context.h"
#include "Interface/Core/Frontend.h"
#include "Interface/Core/InternalThreadState.h"
#include <array>
#include <assert.h>
#include <algorithm>
#include <cstring>
#include <FEXCore/Config/Config.h>
#include <FEXCore/Core/X86Enums.h>
#include <FEXCore/Debug/X86Tables.h>
#include <FEXCore/HLE/SyscallHandler.h>
#include <FEXCore/Utils/Allocator.h>
#include <FEXCore/Utils/LogManager.h>
#include <FEXCore/Utils/Telemetry.h>
#include <set>
#include <sys/mman.h>
namespace FEXCore::Frontend {
#include "Interface/Core/VSyscall/VSyscall.inc"
using namespace FEXCore::X86Tables;
static uint32_t MapModRMToReg(uint8_t REX, uint8_t bits, bool HighBits, bool HasREX, bool HasXMM, bool HasMM, uint8_t InvalidOffset = 16) {
@@ -120,9 +127,67 @@ static uint32_t MapModRMToReg(uint8_t REX, uint8_t bits, bool HighBits, bool Has
return (*GPRs)[(REX << 3) | bits];
}
static uint32_t MapVEXToReg(uint8_t vvvv, bool HasXMM) {
using GPRArray = std::array<uint32_t, 16>;
static constexpr GPRArray GPRIndexes = {
FEXCore::X86State::REG_RAX,
FEXCore::X86State::REG_RCX,
FEXCore::X86State::REG_RDX,
FEXCore::X86State::REG_RBX,
FEXCore::X86State::REG_RSP,
FEXCore::X86State::REG_RBP,
FEXCore::X86State::REG_RSI,
FEXCore::X86State::REG_RDI,
FEXCore::X86State::REG_R8,
FEXCore::X86State::REG_R9,
FEXCore::X86State::REG_R10,
FEXCore::X86State::REG_R11,
FEXCore::X86State::REG_R12,
FEXCore::X86State::REG_R13,
FEXCore::X86State::REG_R14,
FEXCore::X86State::REG_R15,
};
static constexpr GPRArray XMMIndexes = {
FEXCore::X86State::REG_XMM_0,
FEXCore::X86State::REG_XMM_1,
FEXCore::X86State::REG_XMM_2,
FEXCore::X86State::REG_XMM_3,
FEXCore::X86State::REG_XMM_4,
FEXCore::X86State::REG_XMM_5,
FEXCore::X86State::REG_XMM_6,
FEXCore::X86State::REG_XMM_7,
FEXCore::X86State::REG_XMM_8,
FEXCore::X86State::REG_XMM_9,
FEXCore::X86State::REG_XMM_10,
FEXCore::X86State::REG_XMM_11,
FEXCore::X86State::REG_XMM_12,
FEXCore::X86State::REG_XMM_13,
FEXCore::X86State::REG_XMM_14,
FEXCore::X86State::REG_XMM_15,
};
if (HasXMM) {
return XMMIndexes[vvvv];
} else {
return GPRIndexes[vvvv];
}
}
Decoder::Decoder(FEXCore::Context::Context *ctx)
: CTX {ctx} {
DecodedBuffer.resize(DefaultDecodedBufferSize);
: CTX {ctx}
, OSABI { ctx->SyscallHandler ? ctx->SyscallHandler->GetOSABI() : FEXCore::HLE::SyscallOSABI::OS_UNKNOWN } {
// Using mmap is a start-up time optimization
// Take advantage of page faulting to reduce startup time for minimal runtime cost
DecodedBuffer =
reinterpret_cast<FEXCore::X86Tables::DecodedInst *>(
FEXCore::Allocator::mmap(0, sizeof(FEXCore::X86Tables::DecodedInst) * DefaultDecodedBufferSize,
PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0));
}
Decoder::~Decoder() {
FEXCore::Allocator::munmap(DecodedBuffer, sizeof(FEXCore::X86Tables::DecodedInst) * DefaultDecodedBufferSize);
}
uint8_t Decoder::ReadByte() {
@@ -214,28 +279,28 @@ void Decoder::DecodeModRM_16(X86Tables::DecodedOperand *Operand, X86Tables::ModR
{FEXCore::X86State::REG_RBX, FEXCore::X86State::REG_RDI},
{FEXCore::X86State::REG_RBP, FEXCore::X86State::REG_RSI},
{FEXCore::X86State::REG_RBP, FEXCore::X86State::REG_RDI},
{FEXCore::X86State::REG_RSI, 255},
{FEXCore::X86State::REG_RDI, 255},
{255, 255},
{FEXCore::X86State::REG_RBX, 255},
{FEXCore::X86State::REG_RSI, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_RDI, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_INVALID, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_RBX, FEXCore::X86State::REG_INVALID},
// Mod = 0b01
{FEXCore::X86State::REG_RBX, FEXCore::X86State::REG_RSI},
{FEXCore::X86State::REG_RBX, FEXCore::X86State::REG_RDI},
{FEXCore::X86State::REG_RBP, FEXCore::X86State::REG_RSI},
{FEXCore::X86State::REG_RBP, FEXCore::X86State::REG_RDI},
{FEXCore::X86State::REG_RSI, 255},
{FEXCore::X86State::REG_RDI, 255},
{FEXCore::X86State::REG_RBP, 255},
{FEXCore::X86State::REG_RBX, 255},
{FEXCore::X86State::REG_RSI, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_RDI, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_RBP, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_RBX, FEXCore::X86State::REG_INVALID},
// Mod = 0b10
{FEXCore::X86State::REG_RBX, FEXCore::X86State::REG_RSI},
{FEXCore::X86State::REG_RBX, FEXCore::X86State::REG_RDI},
{FEXCore::X86State::REG_RBP, FEXCore::X86State::REG_RSI},
{FEXCore::X86State::REG_RBP, FEXCore::X86State::REG_RDI},
{FEXCore::X86State::REG_RSI, 255},
{FEXCore::X86State::REG_RDI, 255},
{FEXCore::X86State::REG_RBP, 255},
{FEXCore::X86State::REG_RBX, 255},
{FEXCore::X86State::REG_RSI, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_RDI, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_RBP, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_RBX, FEXCore::X86State::REG_INVALID},
}};
uint8_t LookupIndex = ModRM.mod << 3 | ModRM.rm;
@@ -326,7 +391,7 @@ void Decoder::DecodeModRM_64(X86Tables::DecodedOperand *Operand, X86Tables::ModR
}
}
bool Decoder::NormalOp(FEXCore::X86Tables::X86InstInfo const *Info, uint16_t Op) {
bool Decoder::NormalOp(FEXCore::X86Tables::X86InstInfo const *Info, uint16_t Op, DecodedHeader Options) {
DecodeInst->OP = Op;
DecodeInst->TableInfo = Info;
@@ -350,8 +415,9 @@ bool Decoder::NormalOp(FEXCore::X86Tables::X86InstInfo const *Info, uint16_t Op)
"Group Ops should have been decoded before this!");
uint8_t DestSize{};
bool HasWideningDisplacement = FEXCore::X86Tables::DecodeFlags::GetOpAddr(DecodeInst->Flags, 0) & FEXCore::X86Tables::DecodeFlags::FLAG_WIDENING_SIZE_LAST;
bool HasNarrowingDisplacement = FEXCore::X86Tables::DecodeFlags::GetOpAddr(DecodeInst->Flags, 0) & FEXCore::X86Tables::DecodeFlags::FLAG_OPERAND_SIZE_LAST;
const bool HasWideningDisplacement = (FEXCore::X86Tables::DecodeFlags::GetOpAddr(DecodeInst->Flags, 0) & FEXCore::X86Tables::DecodeFlags::FLAG_WIDENING_SIZE_LAST) != 0 ||
Options.w;
const bool HasNarrowingDisplacement = (FEXCore::X86Tables::DecodeFlags::GetOpAddr(DecodeInst->Flags, 0) & FEXCore::X86Tables::DecodeFlags::FLAG_OPERAND_SIZE_LAST) != 0;
bool HasXMMSrc = !!(Info->Flags & FEXCore::X86Tables::InstFlags::FLAGS_XMM_FLAGS) &&
!HAS_XMM_SUBFLAG(Info->Flags, FEXCore::X86Tables::InstFlags::FLAGS_SF_SRC_GPR) &&
@@ -384,8 +450,8 @@ bool Decoder::NormalOp(FEXCore::X86Tables::X86InstInfo const *Info, uint16_t Op)
// New instruction size decoding
{
// Decode destinations first
uint32_t DstSizeFlag = FEXCore::X86Tables::InstFlags::GetSizeDstFlags(Info->Flags);
uint32_t SrcSizeFlag = FEXCore::X86Tables::InstFlags::GetSizeSrcFlags(Info->Flags);
const auto DstSizeFlag = FEXCore::X86Tables::InstFlags::GetSizeDstFlags(Info->Flags);
const auto SrcSizeFlag = FEXCore::X86Tables::InstFlags::GetSizeSrcFlags(Info->Flags);
if (DstSizeFlag == FEXCore::X86Tables::InstFlags::SIZE_8BIT) {
DecodeInst->Flags |= DecodeFlags::GenSizeDstSize(DecodeFlags::SIZE_8BIT);
@@ -529,6 +595,13 @@ bool Decoder::NormalOp(FEXCore::X86Tables::X86InstInfo const *Info, uint16_t Op)
size_t CurrentSrc = 0;
if ((Info->Flags & FEXCore::X86Tables::InstFlags::FLAGS_VEX_1ST_SRC) != 0) {
DecodeInst->Src[CurrentSrc].Type = DecodedOperand::OpType::GPR;
DecodeInst->Src[CurrentSrc].Data.GPR.HighBits = false;
DecodeInst->Src[CurrentSrc].Data.GPR.GPR = MapVEXToReg(Options.vvvv, HasXMMSrc);
++CurrentSrc;
}
if (Info->Flags & FEXCore::X86Tables::InstFlags::FLAGS_MODRM) {
if (Info->Flags & FEXCore::X86Tables::InstFlags::FLAGS_SF_MOD_DST) {
if (!ModRMOperand(DecodeInst->Src[CurrentSrc], DecodeInst->Dest, HasXMMSrc, HasXMMDst, HasMMSrc, HasMMDst, Is8BitSrc, Is8BitDest))
@@ -541,6 +614,13 @@ bool Decoder::NormalOp(FEXCore::X86Tables::X86InstInfo const *Info, uint16_t Op)
++CurrentSrc;
}
if ((Info->Flags & FEXCore::X86Tables::InstFlags::FLAGS_VEX_2ND_SRC) != 0) {
DecodeInst->Src[CurrentSrc].Type = DecodedOperand::OpType::GPR;
DecodeInst->Src[CurrentSrc].Data.GPR.HighBits = false;
DecodeInst->Src[CurrentSrc].Data.GPR.GPR = MapVEXToReg(Options.vvvv, HasXMMSrc);
++CurrentSrc;
}
if (HAS_NON_XMM_SUBFLAG(Info->Flags, FEXCore::X86Tables::InstFlags::FLAGS_SF_SRC_RAX)) {
DecodeInst->Src[CurrentSrc].Type = DecodedOperand::OpType::GPR;
DecodeInst->Src[CurrentSrc].Data.GPR.HighBits = false;
@@ -554,6 +634,12 @@ bool Decoder::NormalOp(FEXCore::X86Tables::X86InstInfo const *Info, uint16_t Op)
++CurrentSrc;
}
if ((Info->Flags & FEXCore::X86Tables::InstFlags::FLAGS_VEX_DST) != 0) {
CurrentDest->Type = DecodedOperand::OpType::GPR;
CurrentDest->Data.GPR.HighBits = false;
CurrentDest->Data.GPR.GPR = MapVEXToReg(Options.vvvv, HasXMMDst);
}
if (Bytes != 0) {
LOGMAN_THROW_A(Bytes <= 8, "Number of bytes should be <= 8 for literal src");
@@ -683,18 +769,22 @@ bool Decoder::NormalOpHeader(FEXCore::X86Tables::X86InstInfo const *Info, uint16
return NormalOp(&X87Ops[X87Op], X87Op);
}
else if (Info->Type == FEXCore::X86Tables::TYPE_VEX_TABLE_PREFIX) {
FEXCORE_TELEMETRY_SET(VEXOpTelem, 1);
uint16_t map_select = 1;
uint16_t pp = 0;
uint8_t Byte1 = ReadByte();
const uint8_t Byte1 = ReadByte();
DecodedHeader options{};
if (Op == 0xC5) { // Two byte VEX
pp = Byte1 & 0b11;
options.vvvv = 15 - ((Byte1 & 0b01111000) >> 3);
}
else { // 0xC4 = Three byte VEX
uint8_t Byte2 = ReadByte();
const uint8_t Byte2 = ReadByte();
pp = Byte2 & 0b11;
map_select = Byte1 & 0b11111;
options.vvvv = 15 - ((Byte2 & 0b01111000) >> 3);
options.w = (Byte2 & 0b10000000) != 0;
if (!(map_select >= 1 && map_select <= 3)) {
LogMan::Msg::E("We don't understand a map_select of: %d", map_select);
return false;
@@ -710,6 +800,7 @@ bool Decoder::NormalOpHeader(FEXCore::X86Tables::X86InstInfo const *Info, uint16
if (LocalInfo->Type >= FEXCore::X86Tables::TYPE_VEX_GROUP_12 &&
LocalInfo->Type <= FEXCore::X86Tables::TYPE_VEX_GROUP_17) {
FEXCORE_TELEMETRY_SET(VEXOpTelem, 1);
// We have ModRM
uint8_t ModRMByte = ReadByte();
DecodeInst->ModRM = ModRMByte;
@@ -721,12 +812,14 @@ bool Decoder::NormalOpHeader(FEXCore::X86Tables::X86InstInfo const *Info, uint16
#define OPD(group, pp, opcode) (((group - TYPE_VEX_GROUP_12) << 4) | (pp << 3) | (opcode))
Op = OPD(LocalInfo->Type, pp, ModRM.reg);
#undef OPD
return NormalOp(&VEXTableGroupOps[Op], Op);
return NormalOp(&VEXTableGroupOps[Op], Op, options);
} else {
return NormalOp(LocalInfo, Op, options);
}
else
return NormalOp(LocalInfo, Op);
}
else if (Info->Type == FEXCore::X86Tables::TYPE_GROUP_EVEX) {
FEXCORE_TELEMETRY_SET(EVEXOpTelem, 1);
/* uint8_t P1 = */ ReadByte();
/* uint8_t P2 = */ ReadByte();
/* uint8_t P3 = */ ReadByte();
@@ -929,7 +1022,7 @@ bool Decoder::DecodeInstruction(uint64_t PC) {
}
if (DecodeInst->Dest.IsGPR()) {
assert(DecodeInst->Dest.Data.GPR.GPR != 255);
assert(DecodeInst->Dest.Data.GPR.GPR != FEXCore::X86State::REG_INVALID);
}
return true;
@@ -1004,6 +1097,25 @@ void Decoder::BranchTargetInMultiblockRange() {
}
}
const uint8_t *Decoder::AdjustAddrForSpecialRegion(uint8_t const* _InstStream, uint64_t EntryPoint, uint64_t RIP) {
constexpr uint64_t VSyscall_Base = 0xFFFF'FFFF'FF60'0000ULL;
constexpr uint64_t VSyscall_End = VSyscall_Base + 0x1000;
if (OSABI == FEXCore::HLE::SyscallOSABI::OS_LINUX64 &&
RIP >= VSyscall_Base &&
RIP < VSyscall_End) {
// VSyscall
// This doesn't exist on AArch64 and on x86_64 hosts this is emulated with faults to a region mapped with --xp permissions
// Offset 0: vgettimeofday
// Offset 0x400: vtime
// Offset 0x800: vgetcpu
uint64_t Offset = RIP - VSyscall_Base;
return VSyscallData + Offset;
}
return _InstStream - EntryPoint + RIP;
}
bool Decoder::DecodeInstructionsAtEntry(uint8_t const* _InstStream, uint64_t PC) {
Blocks.clear();
BlocksToDecode.clear();
@@ -1047,7 +1159,7 @@ bool Decoder::DecodeInstructionsAtEntry(uint8_t const* _InstStream, uint64_t PC)
uint64_t BlockStartOffset = DecodedSize;
// Do a bit of pointer math to figure out where we are in code
InstStream = _InstStream - EntryPoint + RIPToDecode;
InstStream = AdjustAddrForSpecialRegion(_InstStream, EntryPoint, RIPToDecode);
while (1) {
ErrorDuringDecoding = !DecodeInstruction(RIPToDecode + PCOffset);
@@ -1091,7 +1203,7 @@ bool Decoder::DecodeInstructionsAtEntry(uint8_t const* _InstStream, uint64_t PC)
}
if (DecodedSize >= CTX->Config.MaxInstPerBlock ||
DecodedSize >= DecodedBuffer.size()) {
DecodedSize >= DefaultDecodedBufferSize) {
break;
}
@@ -1108,7 +1220,7 @@ bool Decoder::DecodeInstructionsAtEntry(uint8_t const* _InstStream, uint64_t PC)
// Copy over only the number of instructions we decoded
CurrentBlockDecoding.NumInstructions = BlockNumberOfInstructions;
CurrentBlockDecoding.DecodedInstructions = &DecodedBuffer.at(BlockStartOffset);
CurrentBlockDecoding.DecodedInstructions = &DecodedBuffer[BlockStartOffset];
}
+21 -4
View File
@@ -1,11 +1,13 @@
#pragma once
#include <FEXCore/Debug/X86Tables.h>
#include <FEXCore/HLE/SyscallHandler.h>
#include <FEXCore/Utils/Telemetry.h>
#include <array>
#include <cstdint>
#include <utility>
#include <set>
#include <stack>
#include <stddef.h>
#include <vector>
namespace FEXCore::Context {
@@ -24,6 +26,7 @@ public:
};
Decoder(FEXCore::Context::Context *ctx);
~Decoder();
bool DecodeInstructionsAtEntry(uint8_t const* InstStream, uint64_t PC);
std::vector<DecodedBlocks> const *GetDecodedBlocks() const {
@@ -36,7 +39,15 @@ public:
void SetSectionMaxAddress(uint64_t v) { SectionMaxAddress = v; }
void SetExternalBranches(std::set<uint64_t> *v) { ExternalBranches = v; }
private:
// To pass any information from instruction prefixes
// down into the actual instruction handling machinery.
struct DecodedHeader {
uint8_t vvvv; // Encoded operand in a VEX prefix.
bool w; // VEX.W bit.
};
FEXCore::Context::Context *CTX;
const FEXCore::HLE::SyscallOSABI OSABI{};
bool DecodeInstruction(uint64_t PC);
@@ -46,11 +57,12 @@ private:
uint8_t PeekByte(uint8_t Offset) const;
uint64_t ReadData(uint8_t Size);
void SkipBytes(uint8_t Size) { InstructionSize += Size; }
bool NormalOp(FEXCore::X86Tables::X86InstInfo const *Info, uint16_t Op);
bool NormalOp(FEXCore::X86Tables::X86InstInfo const *Info, uint16_t Op, DecodedHeader Options = {});
bool NormalOpHeader(FEXCore::X86Tables::X86InstInfo const *Info, uint16_t Op);
static constexpr size_t DefaultDecodedBufferSize = 0x10000;
std::vector<FEXCore::X86Tables::DecodedInst> DecodedBuffer;
FEXCore::X86Tables::DecodedInst *DecodedBuffer{};
size_t DecodedSize {};
uint8_t const *InstStream;
@@ -83,5 +95,10 @@ private:
&FEXCore::Frontend::Decoder::DecodeModRM_64,
&FEXCore::Frontend::Decoder::DecodeModRM_16,
};
const uint8_t *AdjustAddrForSpecialRegion(uint8_t const* _InstStream, uint64_t EntryPoint, uint64_t RIP);
FEXCORE_TELEMETRY_INIT(VEXOpTelem, TYPE_USES_VEX_OPS);
FEXCORE_TELEMETRY_INIT(EVEXOpTelem, TYPE_USES_EVEX_OPS);
};
}
+18 -7
View File
@@ -8,30 +8,41 @@ $end_info$
#include <cstdlib>
#include <cstdio>
#include <iomanip>
#include <iostream>
#include <sstream>
#include <string>
#include <memory>
#include <optional>
#include "Common/NetStream.h"
#include "Common/SoftFloat.h"
#include "Interface/Context/Context.h"
#include <FEXCore/Config/Config.h>
#include <FEXCore/Core/Context.h>
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/Core/SignalDelegator.h>
#include <FEXCore/Core/X86Enums.h>
#include <FEXCore/Debug/InternalThreadState.h>
#include <FEXCore/HLE/Linux/ThreadManagement.h>
#include <FEXCore/Utils/CompilerDefs.h>
#include <FEXCore/Utils/LogManager.h>
#include <FEXCore/Utils/Threads.h>
#include <atomic>
#include <cstring>
#include <errno.h>
#include <fcntl.h>
#include <fmt/format.h>
#include <fstream>
#include <fmt/format.h>
#include <netdb.h>
#include <signal.h>
#include <stddef.h>
#include <string_view>
#include <sys/socket.h>
#include <sys/types.h>
#include <unistd.h>
#include <utility>
#include <vector>
#include "GdbServer.h"
#include <FEXCore/Core/CodeLoader.h>
#include <FEXCore/Core/X86Enums.h>
namespace FEXCore
{
@@ -939,7 +950,7 @@ void GdbServer::GdbServerLoop() {
{
std::lock_guard lk(sendMutex);
CommsStream.release();
CommsStream.reset();
}
}
}
+9 -6
View File
@@ -5,18 +5,21 @@ $end_info$
*/
#pragma once
#include <mutex>
#include <thread>
#include "Interface/Context/Context.h"
#include "Common/NetStream.h"
#include <FEXCore/Config/Config.h>
#include <FEXCore/Utils/Threads.h>
#include <istream>
#include <memory>
#include <mutex>
#include <stdint.h>
#include <string>
namespace FEXCore {
namespace Context {
struct Context;
}
class GdbServer {
public:
GdbServer(FEXCore::Context::Context *ctx);
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,796 @@
/*
$info$
tags: backend|interpreter
$end_info$
*/
#include "Interface/Core/Interpreter/InterpreterClass.h"
#include "Interface/Core/Interpreter/InterpreterOps.h"
#include "Interface/Core/Interpreter/InterpreterDefines.h"
#include <FEXCore/Utils/BitUtils.h>
#include <cstdint>
namespace FEXCore::CPU {
#ifdef _M_X86_64
uint8_t AtomicFetchNeg(uint8_t *Addr) {
using Type = uint8_t;
std::atomic<Type> *MemData = reinterpret_cast<std::atomic<Type>*>(Addr);
Type Expected = MemData->load();
Type Desired = -Expected;
do {
Desired = -Expected;
} while (!MemData->compare_exchange_strong(Expected, Desired, std::memory_order_seq_cst));
return Expected;
}
uint16_t AtomicFetchNeg(uint16_t *Addr) {
using Type = uint16_t;
std::atomic<Type> *MemData = reinterpret_cast<std::atomic<Type>*>(Addr);
Type Expected = MemData->load();
Type Desired = -Expected;
do {
Desired = -Expected;
} while (!MemData->compare_exchange_strong(Expected, Desired, std::memory_order_seq_cst));
return Expected;
}
uint32_t AtomicFetchNeg(uint32_t *Addr) {
using Type = uint32_t;
std::atomic<Type> *MemData = reinterpret_cast<std::atomic<Type>*>(Addr);
Type Expected = MemData->load();
Type Desired = -Expected;
do {
Desired = -Expected;
} while (!MemData->compare_exchange_strong(Expected, Desired, std::memory_order_seq_cst));
return Expected;
}
uint64_t AtomicFetchNeg(uint64_t *Addr) {
using Type = uint64_t;
std::atomic<Type> *MemData = reinterpret_cast<std::atomic<Type>*>(Addr);
Type Expected = MemData->load();
Type Desired = -Expected;
do {
Desired = -Expected;
} while (!MemData->compare_exchange_strong(Expected, Desired, std::memory_order_seq_cst));
return Expected;
}
template<typename T>
T AtomicCompareAndSwap(T expected, T desired, T *addr)
{
std::atomic<T> *MemData = reinterpret_cast<std::atomic<T>*>(addr);
T Src1 = expected;
T Src2 = desired;
T Expected = Src1;
bool Result = MemData->compare_exchange_strong(Expected, Src2);
return Result ? Src1 : Expected;
}
template uint8_t AtomicCompareAndSwap<uint8_t>(uint8_t expected, uint8_t desired, uint8_t *addr);
template uint16_t AtomicCompareAndSwap<uint16_t>(uint16_t expected, uint16_t desired, uint16_t *addr);
template uint32_t AtomicCompareAndSwap<uint32_t>(uint32_t expected, uint32_t desired, uint32_t *addr);
template uint64_t AtomicCompareAndSwap<uint64_t>(uint64_t expected, uint64_t desired, uint64_t *addr);
#else
// Needs to match what the AArch64 JIT and unaligned signal handler expects
uint8_t AtomicFetchNeg(uint8_t *Addr) {
using Type = uint8_t;
Type Result{};
Type Tmp{};
Type TmpStatus{};
__asm__ volatile(
R"(
1:
ldaxrb %w[Result], [%[Memory]];
neg %w[Tmp], %w[Result];
stlxrb %w[TmpStatus], %w[Tmp], [%[Memory]];
cbnz %w[TmpStatus], 1b;
)"
: [Result] "=r" (Result)
, [Tmp] "=r" (Tmp)
, [TmpStatus] "=r" (TmpStatus)
, [Memory] "+r" (Addr)
:: "memory"
);
return Result;
}
uint16_t AtomicFetchNeg(uint16_t *Addr) {
using Type = uint16_t;
Type Result{};
Type Tmp{};
Type TmpStatus{};
__asm__ volatile(
R"(
1:
ldaxrh %w[Result], [%[Memory]];
neg %w[Tmp], %w[Result];
stlxrh %w[TmpStatus], %w[Tmp], [%[Memory]];
cbnz %w[TmpStatus], 1b;
)"
: [Result] "=r" (Result)
, [Tmp] "=r" (Tmp)
, [TmpStatus] "=r" (TmpStatus)
, [Memory] "+r" (Addr)
:: "memory"
);
return Result;
}
uint32_t AtomicFetchNeg(uint32_t *Addr) {
using Type = uint32_t;
Type Result{};
Type Tmp{};
Type TmpStatus{};
__asm__ volatile(
R"(
1:
ldaxr %w[Result], [%[Memory]];
neg %w[Tmp], %w[Result];
stlxr %w[TmpStatus], %w[Tmp], [%[Memory]];
cbnz %w[TmpStatus], 1b;
)"
: [Result] "=r" (Result)
, [Tmp] "=r" (Tmp)
, [TmpStatus] "=r" (TmpStatus)
, [Memory] "+r" (Addr)
:: "memory"
);
return Result;
}
uint64_t AtomicFetchNeg(uint64_t *Addr) {
using Type = uint64_t;
Type Result{};
Type Tmp{};
Type TmpStatus{};
__asm__ volatile(
R"(
1:
ldaxr %[Result], [%[Memory]];
neg %[Tmp], %[Result];
stlxr %w[TmpStatus], %[Tmp], [%[Memory]];
cbnz %w[TmpStatus], 1b;
)"
: [Result] "=r" (Result)
, [Tmp] "=r" (Tmp)
, [TmpStatus] "=r" (TmpStatus)
, [Memory] "+r" (Addr)
:: "memory"
);
return Result;
}
template<>
uint8_t AtomicCompareAndSwap(uint8_t expected, uint8_t desired, uint8_t *addr) {
using Type = uint8_t;
//force Result to r9 (scratch register) or clang spills to stack
register Type Result asm("r9"){};
Type Tmp{};
Type Tmp2{};
__asm__ volatile(
R"(
1:
ldaxrb %w[Tmp], [%[Memory]];
cmp %w[Tmp], %w[Expected], uxtb;
b.ne 2f;
stlxrb %w[Tmp2], %w[Desired], [%[Memory]];
cbnz %w[Tmp2], 1b;
mov %w[Result], %w[Expected];
b 3f;
2:
mov %w[Result], %w[Tmp];
clrex;
3:
)"
: [Tmp] "=r" (Tmp)
, [Tmp2] "=r" (Tmp2)
, [Desired] "+r" (desired)
, [Expected] "+r" (expected)
, [Result] "=r" (Result)
, [Memory] "+r" (addr)
:: "memory"
);
return Result;
}
template<>
uint16_t AtomicCompareAndSwap(uint16_t expected, uint16_t desired, uint16_t *addr) {
using Type = uint16_t;
//force Result to r9 (scratch register) or clang spills to stack
register Type Result asm("r9"){};
Type Tmp{};
Type Tmp2{};
__asm__ volatile(
R"(
1:
ldaxrh %w[Tmp], [%[Memory]];
cmp %w[Tmp], %w[Expected], uxth;
b.ne 2f;
stlxrh %w[Tmp2], %w[Desired], [%[Memory]];
cbnz %w[Tmp2], 1b;
mov %w[Result], %w[Expected];
b 3f;
2:
mov %w[Result], %w[Tmp];
clrex;
3:
)"
: [Tmp] "=r" (Tmp)
, [Tmp2] "=r" (Tmp2)
, [Desired] "+r" (desired)
, [Expected] "+r" (expected)
, [Result] "=r" (Result)
, [Memory] "+r" (addr)
:: "memory"
);
return Result;
}
template<>
uint32_t AtomicCompareAndSwap(uint32_t expected, uint32_t desired, uint32_t *addr) {
using Type = uint32_t;
//force Result to r9 (scratch register) or clang spills to stack
register Type Result asm("r9"){};
Type Tmp{};
Type Tmp2{};
__asm__ volatile(
R"(
1:
ldaxr %w[Tmp], [%[Memory]];
cmp %w[Tmp], %w[Expected];
b.ne 2f;
stlxr %w[Tmp2], %w[Desired], [%[Memory]];
cbnz %w[Tmp2], 1b;
mov %w[Result], %w[Expected];
b 3f;
2:
mov %w[Result], %w[Tmp];
clrex;
3:
)"
: [Tmp] "=r" (Tmp)
, [Tmp2] "=r" (Tmp2)
, [Desired] "+r" (desired)
, [Expected] "+r" (expected)
, [Result] "=r" (Result)
, [Memory] "+r" (addr)
:: "memory"
);
return Result;
}
template<>
uint64_t AtomicCompareAndSwap(uint64_t expected, uint64_t desired, uint64_t *addr) {
using Type = uint64_t;
//force Result to r9 (scratch register) or clang spills to stack
register Type Result asm("r9"){};
Type Tmp{};
Type Tmp2{};
__asm__ volatile(
R"(
1:
ldaxr %[Tmp], [%[Memory]];
cmp %[Tmp], %[Expected];
b.ne 2f;
stlxr %w[Tmp2], %[Desired], [%[Memory]];
cbnz %w[Tmp2], 1b;
mov %[Result], %[Expected];
b 3f;
2:
mov %[Result], %[Tmp];
clrex;
3:
)"
: [Tmp] "=r" (Tmp)
, [Tmp2] "=r" (Tmp2)
, [Desired] "+r" (desired)
, [Expected] "+r" (expected)
, [Result] "=r" (Result)
, [Memory] "+r" (addr)
:: "memory"
);
return Result;
}
#endif
#define DEF_OP(x) void InterpreterOps::Op_##x(FEXCore::IR::IROp_Header *IROp, IROpData *Data, uint32_t Node)
DEF_OP(CASPair) {
auto Op = IROp->C<IR::IROp_CASPair>();
uint8_t OpSize = IROp->Size;
// Size is the size of each pair element
switch (OpSize) {
case 4: {
GD = AtomicCompareAndSwap(
*GetSrc<uint64_t*>(Data->SSAData, Op->Header.Args[0]),
*GetSrc<uint64_t*>(Data->SSAData, Op->Header.Args[1]),
*GetSrc<uint64_t**>(Data->SSAData, Op->Header.Args[2])
);
break;
}
case 8: {
std::atomic<__uint128_t> *MemData = *GetSrc<std::atomic<__uint128_t> **>(Data->SSAData, Op->Header.Args[2]);
__uint128_t Src1 = *GetSrc<__uint128_t*>(Data->SSAData, Op->Header.Args[0]);
__uint128_t Src2 = *GetSrc<__uint128_t*>(Data->SSAData, Op->Header.Args[1]);
__uint128_t Expected = Src1;
bool Result = MemData->compare_exchange_strong(Expected, Src2);
memcpy(GDP, Result ? &Src1 : &Expected, 16);
break;
}
default: LOGMAN_MSG_A_FMT("Unknown CAS size: {}", OpSize); break;
}
}
DEF_OP(CAS) {
auto Op = IROp->C<IR::IROp_CAS>();
uint8_t OpSize = IROp->Size;
switch (OpSize) {
case 1: {
GD = AtomicCompareAndSwap(
*GetSrc<uint8_t*>(Data->SSAData, Op->Header.Args[0]),
*GetSrc<uint8_t*>(Data->SSAData, Op->Header.Args[1]),
*GetSrc<uint8_t**>(Data->SSAData, Op->Header.Args[2])
);
break;
}
case 2: {
GD = AtomicCompareAndSwap(
*GetSrc<uint16_t*>(Data->SSAData, Op->Header.Args[0]),
*GetSrc<uint16_t*>(Data->SSAData, Op->Header.Args[1]),
*GetSrc<uint16_t**>(Data->SSAData, Op->Header.Args[2])
);
break;
}
case 4: {
GD = AtomicCompareAndSwap(
*GetSrc<uint32_t*>(Data->SSAData, Op->Header.Args[0]),
*GetSrc<uint32_t*>(Data->SSAData, Op->Header.Args[1]),
*GetSrc<uint32_t**>(Data->SSAData, Op->Header.Args[2])
);
break;
}
case 8: {
GD = AtomicCompareAndSwap(
*GetSrc<uint64_t*>(Data->SSAData, Op->Header.Args[0]),
*GetSrc<uint64_t*>(Data->SSAData, Op->Header.Args[1]),
*GetSrc<uint64_t**>(Data->SSAData, Op->Header.Args[2])
);
break;
}
default: LOGMAN_MSG_A_FMT("Unknown CAS size: {}", OpSize); break;
}
}
DEF_OP(AtomicAdd) {
auto Op = IROp->C<IR::IROp_AtomicAdd>();
switch (IROp->Size) {
case 1: {
std::atomic<uint8_t> *MemData = *GetSrc<std::atomic<uint8_t> **>(Data->SSAData, Op->Header.Args[0]);
uint8_t Src = *GetSrc<uint8_t*>(Data->SSAData, Op->Header.Args[1]);
*MemData += Src;
break;
}
case 2: {
std::atomic<uint16_t> *MemData = *GetSrc<std::atomic<uint16_t> **>(Data->SSAData, Op->Header.Args[0]);
uint16_t Src = *GetSrc<uint16_t*>(Data->SSAData, Op->Header.Args[1]);
*MemData += Src;
break;
}
case 4: {
std::atomic<uint32_t> *MemData = *GetSrc<std::atomic<uint32_t> **>(Data->SSAData, Op->Header.Args[0]);
uint32_t Src = *GetSrc<uint32_t*>(Data->SSAData, Op->Header.Args[1]);
*MemData += Src;
break;
}
case 8: {
std::atomic<uint64_t> *MemData = *GetSrc<std::atomic<uint64_t> **>(Data->SSAData, Op->Header.Args[0]);
uint64_t Src = *GetSrc<uint64_t*>(Data->SSAData, Op->Header.Args[1]);
*MemData += Src;
break;
}
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
DEF_OP(AtomicSub) {
auto Op = IROp->C<IR::IROp_AtomicSub>();
switch (IROp->Size) {
case 1: {
std::atomic<uint8_t> *MemData = *GetSrc<std::atomic<uint8_t> **>(Data->SSAData, Op->Header.Args[0]);
uint8_t Src = *GetSrc<uint8_t*>(Data->SSAData, Op->Header.Args[1]);
*MemData -= Src;
break;
}
case 2: {
std::atomic<uint16_t> *MemData = *GetSrc<std::atomic<uint16_t> **>(Data->SSAData, Op->Header.Args[0]);
uint16_t Src = *GetSrc<uint16_t*>(Data->SSAData, Op->Header.Args[1]);
*MemData -= Src;
break;
}
case 4: {
std::atomic<uint32_t> *MemData = *GetSrc<std::atomic<uint32_t> **>(Data->SSAData, Op->Header.Args[0]);
uint32_t Src = *GetSrc<uint32_t*>(Data->SSAData, Op->Header.Args[1]);
*MemData -= Src;
break;
}
case 8: {
std::atomic<uint64_t> *MemData = *GetSrc<std::atomic<uint64_t> **>(Data->SSAData, Op->Header.Args[0]);
uint64_t Src = *GetSrc<uint64_t*>(Data->SSAData, Op->Header.Args[1]);
*MemData -= Src;
break;
}
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
DEF_OP(AtomicAnd) {
auto Op = IROp->C<IR::IROp_AtomicAnd>();
switch (IROp->Size) {
case 1: {
std::atomic<uint8_t> *MemData = *GetSrc<std::atomic<uint8_t> **>(Data->SSAData, Op->Header.Args[0]);
uint8_t Src = *GetSrc<uint8_t*>(Data->SSAData, Op->Header.Args[1]);
*MemData &= Src;
break;
}
case 2: {
std::atomic<uint16_t> *MemData = *GetSrc<std::atomic<uint16_t> **>(Data->SSAData, Op->Header.Args[0]);
uint16_t Src = *GetSrc<uint16_t*>(Data->SSAData, Op->Header.Args[1]);
*MemData &= Src;
break;
}
case 4: {
std::atomic<uint32_t> *MemData = *GetSrc<std::atomic<uint32_t> **>(Data->SSAData, Op->Header.Args[0]);
uint32_t Src = *GetSrc<uint32_t*>(Data->SSAData, Op->Header.Args[1]);
*MemData &= Src;
break;
}
case 8: {
std::atomic<uint64_t> *MemData = *GetSrc<std::atomic<uint64_t> **>(Data->SSAData, Op->Header.Args[0]);
uint64_t Src = *GetSrc<uint64_t*>(Data->SSAData, Op->Header.Args[1]);
*MemData &= Src;
break;
}
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
DEF_OP(AtomicOr) {
auto Op = IROp->C<IR::IROp_AtomicOr>();
switch (IROp->Size) {
case 1: {
std::atomic<uint8_t> *MemData = *GetSrc<std::atomic<uint8_t> **>(Data->SSAData, Op->Header.Args[0]);
uint8_t Src = *GetSrc<uint8_t*>(Data->SSAData, Op->Header.Args[1]);
*MemData |= Src;
break;
}
case 2: {
std::atomic<uint16_t> *MemData = *GetSrc<std::atomic<uint16_t> **>(Data->SSAData, Op->Header.Args[0]);
uint16_t Src = *GetSrc<uint16_t*>(Data->SSAData, Op->Header.Args[1]);
*MemData |= Src;
break;
}
case 4: {
std::atomic<uint32_t> *MemData = *GetSrc<std::atomic<uint32_t> **>(Data->SSAData, Op->Header.Args[0]);
uint32_t Src = *GetSrc<uint32_t*>(Data->SSAData, Op->Header.Args[1]);
*MemData |= Src;
break;
}
case 8: {
std::atomic<uint64_t> *MemData = *GetSrc<std::atomic<uint64_t> **>(Data->SSAData, Op->Header.Args[0]);
uint64_t Src = *GetSrc<uint64_t*>(Data->SSAData, Op->Header.Args[1]);
*MemData |= Src;
break;
}
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
DEF_OP(AtomicXor) {
auto Op = IROp->C<IR::IROp_AtomicXor>();
switch (IROp->Size) {
case 1: {
std::atomic<uint8_t> *MemData = *GetSrc<std::atomic<uint8_t> **>(Data->SSAData, Op->Header.Args[0]);
uint8_t Src = *GetSrc<uint8_t*>(Data->SSAData, Op->Header.Args[1]);
*MemData ^= Src;
break;
}
case 2: {
std::atomic<uint16_t> *MemData = *GetSrc<std::atomic<uint16_t> **>(Data->SSAData, Op->Header.Args[0]);
uint16_t Src = *GetSrc<uint16_t*>(Data->SSAData, Op->Header.Args[1]);
*MemData ^= Src;
break;
}
case 4: {
std::atomic<uint32_t> *MemData = *GetSrc<std::atomic<uint32_t> **>(Data->SSAData, Op->Header.Args[0]);
uint32_t Src = *GetSrc<uint32_t*>(Data->SSAData, Op->Header.Args[1]);
*MemData ^= Src;
break;
}
case 8: {
std::atomic<uint64_t> *MemData = *GetSrc<std::atomic<uint64_t> **>(Data->SSAData, Op->Header.Args[0]);
uint64_t Src = *GetSrc<uint64_t*>(Data->SSAData, Op->Header.Args[1]);
*MemData ^= Src;
break;
}
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
DEF_OP(AtomicSwap) {
auto Op = IROp->C<IR::IROp_AtomicSwap>();
switch (IROp->Size) {
case 1: {
std::atomic<uint8_t> *MemData = *GetSrc<std::atomic<uint8_t> **>(Data->SSAData, Op->Header.Args[0]);
uint8_t Src = *GetSrc<uint8_t*>(Data->SSAData, Op->Header.Args[1]);
uint8_t Previous = MemData->exchange(Src);
GD = Previous;
break;
}
case 2: {
std::atomic<uint16_t> *MemData = *GetSrc<std::atomic<uint16_t> **>(Data->SSAData, Op->Header.Args[0]);
uint16_t Src = *GetSrc<uint16_t*>(Data->SSAData, Op->Header.Args[1]);
uint16_t Previous = MemData->exchange(Src);
GD = Previous;
break;
}
case 4: {
std::atomic<uint32_t> *MemData = *GetSrc<std::atomic<uint32_t> **>(Data->SSAData, Op->Header.Args[0]);
uint32_t Src = *GetSrc<uint32_t*>(Data->SSAData, Op->Header.Args[1]);
uint32_t Previous = MemData->exchange(Src);
GD = Previous;
break;
}
case 8: {
std::atomic<uint64_t> *MemData = *GetSrc<std::atomic<uint64_t> **>(Data->SSAData, Op->Header.Args[0]);
uint64_t Src = *GetSrc<uint64_t*>(Data->SSAData, Op->Header.Args[1]);
uint64_t Previous = MemData->exchange(Src);
GD = Previous;
break;
}
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
DEF_OP(AtomicFetchAdd) {
auto Op = IROp->C<IR::IROp_AtomicFetchAdd>();
switch (IROp->Size) {
case 1: {
std::atomic<uint8_t> *MemData = *GetSrc<std::atomic<uint8_t> **>(Data->SSAData, Op->Header.Args[0]);
uint8_t Src = *GetSrc<uint8_t*>(Data->SSAData, Op->Header.Args[1]);
uint8_t Previous = MemData->fetch_add(Src);
GD = Previous;
break;
}
case 2: {
std::atomic<uint16_t> *MemData = *GetSrc<std::atomic<uint16_t> **>(Data->SSAData, Op->Header.Args[0]);
uint16_t Src = *GetSrc<uint16_t*>(Data->SSAData, Op->Header.Args[1]);
uint16_t Previous = MemData->fetch_add(Src);
GD = Previous;
break;
}
case 4: {
std::atomic<uint32_t> *MemData = *GetSrc<std::atomic<uint32_t> **>(Data->SSAData, Op->Header.Args[0]);
uint32_t Src = *GetSrc<uint32_t*>(Data->SSAData, Op->Header.Args[1]);
uint32_t Previous = MemData->fetch_add(Src);
GD = Previous;
break;
}
case 8: {
std::atomic<uint64_t> *MemData = *GetSrc<std::atomic<uint64_t> **>(Data->SSAData, Op->Header.Args[0]);
uint64_t Src = *GetSrc<uint64_t*>(Data->SSAData, Op->Header.Args[1]);
uint64_t Previous = MemData->fetch_add(Src);
GD = Previous;
break;
}
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
DEF_OP(AtomicFetchSub) {
auto Op = IROp->C<IR::IROp_AtomicFetchSub>();
switch (IROp->Size) {
case 1: {
std::atomic<uint8_t> *MemData = *GetSrc<std::atomic<uint8_t> **>(Data->SSAData, Op->Header.Args[0]);
uint8_t Src = *GetSrc<uint8_t*>(Data->SSAData, Op->Header.Args[1]);
uint8_t Previous = MemData->fetch_sub(Src);
GD = Previous;
break;
}
case 2: {
std::atomic<uint16_t> *MemData = *GetSrc<std::atomic<uint16_t> **>(Data->SSAData, Op->Header.Args[0]);
uint16_t Src = *GetSrc<uint16_t*>(Data->SSAData, Op->Header.Args[1]);
uint16_t Previous = MemData->fetch_sub(Src);
GD = Previous;
break;
}
case 4: {
std::atomic<uint32_t> *MemData = *GetSrc<std::atomic<uint32_t> **>(Data->SSAData, Op->Header.Args[0]);
uint32_t Src = *GetSrc<uint32_t*>(Data->SSAData, Op->Header.Args[1]);
uint32_t Previous = MemData->fetch_sub(Src);
GD = Previous;
break;
}
case 8: {
std::atomic<uint64_t> *MemData = *GetSrc<std::atomic<uint64_t> **>(Data->SSAData, Op->Header.Args[0]);
uint64_t Src = *GetSrc<uint64_t*>(Data->SSAData, Op->Header.Args[1]);
uint64_t Previous = MemData->fetch_sub(Src);
GD = Previous;
break;
}
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
DEF_OP(AtomicFetchAnd) {
auto Op = IROp->C<IR::IROp_AtomicFetchAnd>();
switch (IROp->Size) {
case 1: {
std::atomic<uint8_t> *MemData = *GetSrc<std::atomic<uint8_t> **>(Data->SSAData, Op->Header.Args[0]);
uint8_t Src = *GetSrc<uint8_t*>(Data->SSAData, Op->Header.Args[1]);
uint8_t Previous = MemData->fetch_and(Src);
GD = Previous;
break;
}
case 2: {
std::atomic<uint16_t> *MemData = *GetSrc<std::atomic<uint16_t> **>(Data->SSAData, Op->Header.Args[0]);
uint16_t Src = *GetSrc<uint16_t*>(Data->SSAData, Op->Header.Args[1]);
uint16_t Previous = MemData->fetch_and(Src);
GD = Previous;
break;
}
case 4: {
std::atomic<uint32_t> *MemData = *GetSrc<std::atomic<uint32_t> **>(Data->SSAData, Op->Header.Args[0]);
uint32_t Src = *GetSrc<uint32_t*>(Data->SSAData, Op->Header.Args[1]);
uint32_t Previous = MemData->fetch_and(Src);
GD = Previous;
break;
}
case 8: {
std::atomic<uint64_t> *MemData = *GetSrc<std::atomic<uint64_t> **>(Data->SSAData, Op->Header.Args[0]);
uint64_t Src = *GetSrc<uint64_t*>(Data->SSAData, Op->Header.Args[1]);
uint64_t Previous = MemData->fetch_and(Src);
GD = Previous;
break;
}
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
DEF_OP(AtomicFetchOr) {
auto Op = IROp->C<IR::IROp_AtomicFetchOr>();
switch (IROp->Size) {
case 1: {
std::atomic<uint8_t> *MemData = *GetSrc<std::atomic<uint8_t> **>(Data->SSAData, Op->Header.Args[0]);
uint8_t Src = *GetSrc<uint8_t*>(Data->SSAData, Op->Header.Args[1]);
uint8_t Previous = MemData->fetch_or(Src);
GD = Previous;
break;
}
case 2: {
std::atomic<uint16_t> *MemData = *GetSrc<std::atomic<uint16_t> **>(Data->SSAData, Op->Header.Args[0]);
uint16_t Src = *GetSrc<uint16_t*>(Data->SSAData, Op->Header.Args[1]);
uint16_t Previous = MemData->fetch_or(Src);
GD = Previous;
break;
}
case 4: {
std::atomic<uint32_t> *MemData = *GetSrc<std::atomic<uint32_t> **>(Data->SSAData, Op->Header.Args[0]);
uint32_t Src = *GetSrc<uint32_t*>(Data->SSAData, Op->Header.Args[1]);
uint32_t Previous = MemData->fetch_or(Src);
GD = Previous;
break;
}
case 8: {
std::atomic<uint64_t> *MemData = *GetSrc<std::atomic<uint64_t> **>(Data->SSAData, Op->Header.Args[0]);
uint64_t Src = *GetSrc<uint64_t*>(Data->SSAData, Op->Header.Args[1]);
uint64_t Previous = MemData->fetch_or(Src);
GD = Previous;
break;
}
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
DEF_OP(AtomicFetchXor) {
auto Op = IROp->C<IR::IROp_AtomicFetchXor>();
switch (IROp->Size) {
case 1: {
std::atomic<uint8_t> *MemData = *GetSrc<std::atomic<uint8_t> **>(Data->SSAData, Op->Header.Args[0]);
uint8_t Src = *GetSrc<uint8_t*>(Data->SSAData, Op->Header.Args[1]);
uint8_t Previous = MemData->fetch_xor(Src);
GD = Previous;
break;
}
case 2: {
std::atomic<uint16_t> *MemData = *GetSrc<std::atomic<uint16_t> **>(Data->SSAData, Op->Header.Args[0]);
uint16_t Src = *GetSrc<uint16_t*>(Data->SSAData, Op->Header.Args[1]);
uint16_t Previous = MemData->fetch_xor(Src);
GD = Previous;
break;
}
case 4: {
std::atomic<uint32_t> *MemData = *GetSrc<std::atomic<uint32_t> **>(Data->SSAData, Op->Header.Args[0]);
uint32_t Src = *GetSrc<uint32_t*>(Data->SSAData, Op->Header.Args[1]);
uint32_t Previous = MemData->fetch_xor(Src);
GD = Previous;
break;
}
case 8: {
std::atomic<uint64_t> *MemData = *GetSrc<std::atomic<uint64_t> **>(Data->SSAData, Op->Header.Args[0]);
uint64_t Src = *GetSrc<uint64_t*>(Data->SSAData, Op->Header.Args[1]);
uint64_t Previous = MemData->fetch_xor(Src);
GD = Previous;
break;
}
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
DEF_OP(AtomicFetchNeg) {
auto Op = IROp->C<IR::IROp_AtomicFetchNeg>();
switch (IROp->Size) {
case 1: {
using Type = uint8_t;
GD = AtomicFetchNeg(*GetSrc<Type**>(Data->SSAData, Op->Header.Args[0]));
break;
}
case 2: {
using Type = uint16_t;
GD = AtomicFetchNeg(*GetSrc<Type**>(Data->SSAData, Op->Header.Args[0]));
break;
}
case 4: {
using Type = uint32_t;
GD = AtomicFetchNeg(*GetSrc<Type**>(Data->SSAData, Op->Header.Args[0]));
break;
}
case 8: {
using Type = uint64_t;
GD = AtomicFetchNeg(*GetSrc<Type**>(Data->SSAData, Op->Header.Args[0]));
break;
}
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
#undef DEF_OP
void InterpreterOps::RegisterAtomicHandlers() {
#define REGISTER_OP(op, x) OpHandlers[FEXCore::IR::IROps::OP_##op] = &InterpreterOps::Op_##x
REGISTER_OP(CASPAIR, CASPair);
REGISTER_OP(CAS, CAS);
REGISTER_OP(ATOMICADD, AtomicAdd);
REGISTER_OP(ATOMICSUB, AtomicSub);
REGISTER_OP(ATOMICAND, AtomicAnd);
REGISTER_OP(ATOMICOR, AtomicOr);
REGISTER_OP(ATOMICXOR, AtomicXor);
REGISTER_OP(ATOMICSWAP, AtomicSwap);
REGISTER_OP(ATOMICFETCHADD, AtomicFetchAdd);
REGISTER_OP(ATOMICFETCHSUB, AtomicFetchSub);
REGISTER_OP(ATOMICFETCHAND, AtomicFetchAnd);
REGISTER_OP(ATOMICFETCHOR, AtomicFetchOr);
REGISTER_OP(ATOMICFETCHXOR, AtomicFetchXor);
REGISTER_OP(ATOMICFETCHNEG, AtomicFetchNeg);
#undef REGISTER_OP
}
}
@@ -0,0 +1,157 @@
/*
$info$
tags: backend|interpreter
$end_info$
*/
#include "Interface/Core/Interpreter/InterpreterClass.h"
#include "Interface/Core/Interpreter/InterpreterOps.h"
#include "Interface/Core/Interpreter/InterpreterDefines.h"
#include "Interface/HLE/Thunks/Thunks.h"
#include <FEXCore/Utils/BitUtils.h>
#include <FEXCore/HLE/SyscallHandler.h>
#include <cstdint>
namespace FEXCore::CPU {
[[noreturn]]
static void SignalReturn(FEXCore::Core::InternalThreadState *Thread) {
Thread->CTX->SignalThread(Thread, FEXCore::Core::SignalEvent::Return);
LOGMAN_MSG_A_FMT("unreachable");
FEX_UNREACHABLE;
}
#define DEF_OP(x) void InterpreterOps::Op_##x(FEXCore::IR::IROp_Header *IROp, IROpData *Data, uint32_t Node)
DEF_OP(GuestCallDirect) {
LogMan::Msg::DFmt("Unimplemented");
}
DEF_OP(GuestCallIndirect) {
LogMan::Msg::DFmt("Unimplemented");
}
DEF_OP(GuestReturn) {
LogMan::Msg::DFmt("Unimplemented");
}
DEF_OP(SignalReturn) {
SignalReturn(Data->State);
}
DEF_OP(CallbackReturn) {
Data->State->CTX->InterpreterCallbackReturn(Data->State, Data->StackEntry);
}
DEF_OP(ExitFunction) {
auto Op = IROp->C<IR::IROp_ExitFunction>();
uint8_t OpSize = IROp->Size;
uintptr_t* ContextPtr = reinterpret_cast<uintptr_t*>(Data->State->CurrentFrame);
void *ContextData = reinterpret_cast<void*>(ContextPtr);
void *Src = GetSrc<void*>(Data->SSAData, Op->Header.Args[0]);
memcpy(ContextData, Src, OpSize);
Data->BlockResults.Quit = true;
}
DEF_OP(Jump) {
auto Op = IROp->C<IR::IROp_Jump>();
uintptr_t ListBegin = Data->CurrentIR->GetListData();
uintptr_t DataBegin = Data->CurrentIR->GetData();
Data->BlockIterator = IR::NodeIterator(ListBegin, DataBegin, Op->Header.Args[0]);
Data->BlockResults.Redo = true;
}
DEF_OP(CondJump) {
auto Op = IROp->C<IR::IROp_CondJump>();
uintptr_t ListBegin = Data->CurrentIR->GetListData();
uintptr_t DataBegin = Data->CurrentIR->GetData();
bool CompResult;
uint64_t Src1 = *GetSrc<uint64_t*>(Data->SSAData, Op->Cmp1);
uint64_t Src2 = *GetSrc<uint64_t*>(Data->SSAData, Op->Cmp2);
if (Op->CompareSize == 4)
CompResult = IsConditionTrue<uint32_t, int32_t, float>(Op->Cond.Val, Src1, Src2);
else
CompResult = IsConditionTrue<uint64_t, int64_t, double>(Op->Cond.Val, Src1, Src2);
if (CompResult) {
Data->BlockIterator = IR::NodeIterator(ListBegin, DataBegin, Op->TrueBlock);
}
else {
Data->BlockIterator = IR::NodeIterator(ListBegin, DataBegin, Op->FalseBlock);
}
Data->BlockResults.Redo = true;
}
DEF_OP(Syscall) {
auto Op = IROp->C<IR::IROp_Syscall>();
FEXCore::HLE::SyscallArguments Args;
for (size_t j = 0; j < FEXCore::HLE::SyscallArguments::MAX_ARGS; ++j) {
if (Op->Header.Args[j].IsInvalid()) break;
Args.Argument[j] = *GetSrc<uint64_t*>(Data->SSAData, Op->Header.Args[j]);
}
uint64_t Res = FEXCore::Context::HandleSyscall(Data->State->CTX->SyscallHandler, Data->State->CurrentFrame, &Args);
GD = Res;
}
DEF_OP(Thunk) {
auto Op = IROp->C<IR::IROp_Thunk>();
auto thunkFn = Data->State->CTX->ThunkHandler->LookupThunk(Op->ThunkNameHash);
thunkFn(*GetSrc<void**>(Data->SSAData, Op->Header.Args[0]));
}
DEF_OP(ValidateCode) {
auto Op = IROp->C<IR::IROp_ValidateCode>();
auto CodePtr = Data->CurrentEntry + Op->Offset;
if (memcmp((void*)CodePtr, &Op->CodeOriginalLow, Op->CodeLength) != 0) {
GD = 1;
} else {
GD = 0;
}
}
DEF_OP(RemoveCodeEntry) {
Data->State->CTX->RemoveCodeEntry(Data->State, Data->CurrentEntry);
}
DEF_OP(CPUID) {
auto Op = IROp->C<IR::IROp_CPUID>();
uint64_t *DstPtr = GetDest<uint64_t*>(Data->SSAData, Node);
uint64_t Arg = *GetSrc<uint64_t*>(Data->SSAData, Op->Header.Args[0]);
uint64_t Leaf = *GetSrc<uint64_t*>(Data->SSAData, Op->Header.Args[1]);
auto Results = Data->State->CTX->CPUID.RunFunction(Arg, Leaf);
memcpy(DstPtr, &Results, sizeof(uint32_t) * 4);
}
#undef DEF_OP
void InterpreterOps::RegisterBranchHandlers() {
#define REGISTER_OP(op, x) OpHandlers[FEXCore::IR::IROps::OP_##op] = &InterpreterOps::Op_##x
REGISTER_OP(GUESTCALLDIRECT, GuestCallDirect);
REGISTER_OP(GUESTCALLINDIRECT, GuestCallIndirect);
REGISTER_OP(GUESTRETURN, GuestReturn);
REGISTER_OP(SIGNALRETURN, SignalReturn);
REGISTER_OP(CALLBACKRETURN, CallbackReturn);
REGISTER_OP(EXITFUNCTION, ExitFunction);
REGISTER_OP(JUMP, Jump);
REGISTER_OP(CONDJUMP, CondJump);
REGISTER_OP(SYSCALL, Syscall);
REGISTER_OP(THUNK, Thunk);
REGISTER_OP(VALIDATECODE, ValidateCode);
REGISTER_OP(REMOVECODEENTRY, RemoveCodeEntry);
REGISTER_OP(CPUID, CPUID);
#undef REGISTER_OP
}
}
@@ -0,0 +1,237 @@
/*
$info$
tags: backend|interpreter
$end_info$
*/
#include "Interface/Core/Interpreter/InterpreterClass.h"
#include "Interface/Core/Interpreter/InterpreterOps.h"
#include "Interface/Core/Interpreter/InterpreterDefines.h"
#include <cstdint>
namespace FEXCore::CPU {
#define DEF_OP(x) void InterpreterOps::Op_##x(FEXCore::IR::IROp_Header *IROp, IROpData *Data, uint32_t Node)
DEF_OP(VInsGPR) {
auto Op = IROp->C<IR::IROp_VInsGPR>();
uint8_t OpSize = IROp->Size;
__uint128_t Src1 = *GetSrc<__uint128_t*>(Data->SSAData, Op->Header.Args[0]);
__uint128_t Src2 = *GetSrc<__uint128_t*>(Data->SSAData, Op->Header.Args[1]);
uint64_t Offset = Op->Index * Op->Header.ElementSize * 8;
__uint128_t Mask = (1ULL << (Op->Header.ElementSize * 8)) - 1;
if (Op->Header.ElementSize == 8) {
Mask = ~0ULL;
}
Src2 = Src2 & Mask;
Mask <<= Offset;
Mask = ~Mask;
__uint128_t Dst = Src1 & Mask;
Dst |= Src2 << Offset;
memcpy(GDP, &Dst, OpSize);
}
DEF_OP(VCastFromGPR) {
auto Op = IROp->C<IR::IROp_VCastFromGPR>();
memcpy(GDP, GetSrc<void*>(Data->SSAData, Op->Header.Args[0]), Op->Header.ElementSize);
}
DEF_OP(Float_FromGPR_S) {
auto Op = IROp->C<IR::IROp_Float_FromGPR_S>();
uint16_t Conv = (Op->Header.ElementSize << 8) | Op->SrcElementSize;
switch (Conv) {
case 0x0404: { // Float <- int32_t
float Dst = (float)*GetSrc<int32_t*>(Data->SSAData, Op->Header.Args[0]);
memcpy(GDP, &Dst, Op->Header.ElementSize);
break;
}
case 0x0408: { // Float <- int64_t
float Dst = (float)*GetSrc<int64_t*>(Data->SSAData, Op->Header.Args[0]);
memcpy(GDP, &Dst, Op->Header.ElementSize);
break;
}
case 0x0804: { // Double <- int32_t
double Dst = (double)*GetSrc<int32_t*>(Data->SSAData, Op->Header.Args[0]);
memcpy(GDP, &Dst, Op->Header.ElementSize);
break;
}
case 0x0808: { // Double <- int64_t
double Dst = (double)*GetSrc<int64_t*>(Data->SSAData, Op->Header.Args[0]);
memcpy(GDP, &Dst, Op->Header.ElementSize);
break;
}
}
}
DEF_OP(Float_FToF) {
auto Op = IROp->C<IR::IROp_Float_FToF>();
uint16_t Conv = (Op->Header.ElementSize << 8) | Op->SrcElementSize;
switch (Conv) {
case 0x0804: { // Double <- Float
double Dst = (double)*GetSrc<float*>(Data->SSAData, Op->Header.Args[0]);
memcpy(GDP, &Dst, 8);
break;
}
case 0x0408: { // Float <- Double
float Dst = (float)*GetSrc<double*>(Data->SSAData, Op->Header.Args[0]);
memcpy(GDP, &Dst, 4);
break;
}
default: LOGMAN_MSG_A_FMT("Unknown FCVT sizes: 0x{:x}", Conv);
}
}
DEF_OP(Vector_SToF) {
auto Op = IROp->C<IR::IROp_Vector_SToF>();
uint8_t OpSize = IROp->Size;
void *Src = GetSrc<void*>(Data->SSAData, Op->Header.Args[0]);
uint8_t Tmp[16]{};
uint8_t Elements = OpSize / Op->Header.ElementSize;
auto Func = [](auto a, auto min, auto max) { return a; };
switch (Op->Header.ElementSize) {
DO_VECTOR_1SRC_2TYPE_OP(4, float, int32_t, Func, 0, 0)
DO_VECTOR_1SRC_2TYPE_OP(8, double, int64_t, Func, 0, 0)
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
memcpy(GDP, Tmp, OpSize);
}
DEF_OP(Vector_FToZS) {
auto Op = IROp->C<IR::IROp_Vector_FToZS>();
uint8_t OpSize = IROp->Size;
void *Src = GetSrc<void*>(Data->SSAData, Op->Header.Args[0]);
uint8_t Tmp[16]{};
uint8_t Elements = OpSize / Op->Header.ElementSize;
auto Func = [](auto a, auto min, auto max) { return std::trunc(a); };
switch (Op->Header.ElementSize) {
DO_VECTOR_1SRC_2TYPE_OP(4, int32_t, float, Func, 0, 0)
DO_VECTOR_1SRC_2TYPE_OP(8, int64_t, double, Func, 0, 0)
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
memcpy(GDP, Tmp, OpSize);
}
DEF_OP(Vector_FToS) {
auto Op = IROp->C<IR::IROp_Vector_FToS>();
uint8_t OpSize = IROp->Size;
void *Src = GetSrc<void*>(Data->SSAData, Op->Header.Args[0]);
uint8_t Tmp[16]{};
uint8_t Elements = OpSize / Op->Header.ElementSize;
auto Func = [](auto a, auto min, auto max) { return std::nearbyint(a); };
switch (Op->Header.ElementSize) {
DO_VECTOR_1SRC_2TYPE_OP(4, int32_t, float, Func, 0, 0)
DO_VECTOR_1SRC_2TYPE_OP(8, int64_t, double, Func, 0, 0)
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
memcpy(GDP, Tmp, OpSize);
}
DEF_OP(Vector_FToF) {
auto Op = IROp->C<IR::IROp_Vector_FToF>();
uint8_t OpSize = IROp->Size;
void *Src = GetSrc<void*>(Data->SSAData, Op->Header.Args[0]);
uint8_t Tmp[16]{};
uint16_t Conv = (Op->Header.ElementSize << 8) | Op->SrcElementSize;
auto Func = [](auto a, auto min, auto max) { return a; };
switch (Conv) {
case 0x0804: { // Double <- float
// Only the lower elements from the source
// This uses half the source elements
uint8_t Elements = OpSize / 8;
DO_VECTOR_1SRC_2TYPE_OP_NOSIZE(double, float, Func, 0, 0)
break;
}
case 0x0408: { // Float <- Double
// Little bit tricky here
// Sometimes is used to convert from a 128bit vector register
// in to a 64bit vector register with different sized elements
// eg: %ssa5 i32v2 = Vector_FToF %ssa4 i128, #0x8
uint8_t Elements = (OpSize << 1) / Op->SrcElementSize;
DO_VECTOR_1SRC_2TYPE_OP_NOSIZE(float, double, Func, 0, 0)
break;
}
default: LOGMAN_MSG_A_FMT("Unknown Conversion Type : 0x{:04x}", Conv); break;
}
memcpy(GDP, Tmp, OpSize);
}
DEF_OP(Vector_FToI) {
auto Op = IROp->C<IR::IROp_Vector_FToI>();
uint8_t OpSize = IROp->Size;
void *Src = GetSrc<void*>(Data->SSAData, Op->Header.Args[0]);
uint8_t Tmp[16]{};
uint8_t Elements = OpSize / Op->Header.ElementSize;
auto Func_Nearest = [](auto a) { return std::rint(a); };
auto Func_Neg = [](auto a) { return std::floor(a); };
auto Func_Pos = [](auto a) { return std::ceil(a); };
auto Func_Trunc = [](auto a) { return std::trunc(a); };
auto Func_Host = [](auto a) { return std::rint(a); };
switch (Op->Round) {
case FEXCore::IR::Round_Nearest.Val:
switch (Op->Header.ElementSize) {
DO_VECTOR_1SRC_OP(4, float, Func_Nearest)
DO_VECTOR_1SRC_OP(8, double, Func_Nearest)
}
break;
case FEXCore::IR::Round_Negative_Infinity.Val:
switch (Op->Header.ElementSize) {
DO_VECTOR_1SRC_OP(4, float, Func_Neg)
DO_VECTOR_1SRC_OP(8, double, Func_Neg)
}
break;
case FEXCore::IR::Round_Positive_Infinity.Val:
switch (Op->Header.ElementSize) {
DO_VECTOR_1SRC_OP(4, float, Func_Pos)
DO_VECTOR_1SRC_OP(8, double, Func_Pos)
}
break;
case FEXCore::IR::Round_Towards_Zero.Val:
switch (Op->Header.ElementSize) {
DO_VECTOR_1SRC_OP(4, float, Func_Trunc)
DO_VECTOR_1SRC_OP(8, double, Func_Trunc)
}
break;
case FEXCore::IR::Round_Host.Val:
switch (Op->Header.ElementSize) {
DO_VECTOR_1SRC_OP(4, float, Func_Host)
DO_VECTOR_1SRC_OP(8, double, Func_Host)
}
break;
}
memcpy(GDP, Tmp, OpSize);
}
#undef DEF_OP
void InterpreterOps::RegisterConversionHandlers() {
#define REGISTER_OP(op, x) OpHandlers[FEXCore::IR::IROps::OP_##op] = &InterpreterOps::Op_##x
REGISTER_OP(VINSGPR, VInsGPR);
REGISTER_OP(VCASTFROMGPR, VCastFromGPR);
REGISTER_OP(FLOAT_FROMGPR_S, Float_FromGPR_S);
REGISTER_OP(FLOAT_FTOF, Float_FToF);
REGISTER_OP(VECTOR_STOF, Vector_SToF);
REGISTER_OP(VECTOR_FTOZS, Vector_FToZS);
REGISTER_OP(VECTOR_FTOS, Vector_FToS);
REGISTER_OP(VECTOR_FTOF, Vector_FToF);
REGISTER_OP(VECTOR_FTOI, Vector_FToI);
#undef REGISTER_OP
}
}
@@ -0,0 +1,443 @@
/*
$info$
tags: backend|interpreter
$end_info$
*/
#include "Interface/Core/Interpreter/InterpreterClass.h"
#include "Interface/Core/Interpreter/InterpreterOps.h"
#include "Interface/Core/Interpreter/InterpreterDefines.h"
#include <cstdint>
namespace AES {
static __uint128_t InvShiftRows(uint8_t *State) {
uint8_t Shifted[16] = {
State[0], State[13], State[10], State[7],
State[4], State[1], State[14], State[11],
State[8], State[5], State[2], State[15],
State[12], State[9], State[6], State[3],
};
__uint128_t Res{};
memcpy(&Res, Shifted, 16);
return Res;
}
static __uint128_t InvSubBytes(uint8_t *State) {
// 16x16 matrix table
static const uint8_t InvSubstitutionTable[256] = {
0x52, 0x09, 0x6a, 0xd5, 0x30, 0x36, 0xa5, 0x38, 0xbf, 0x40, 0xa3, 0x9e, 0x81, 0xf3, 0xd7, 0xfb,
0x7c, 0xe3, 0x39, 0x82, 0x9b, 0x2f, 0xff, 0x87, 0x34, 0x8e, 0x43, 0x44, 0xc4, 0xde, 0xe9, 0xcb,
0x54, 0x7b, 0x94, 0x32, 0xa6, 0xc2, 0x23, 0x3d, 0xee, 0x4c, 0x95, 0x0b, 0x42, 0xfa, 0xc3, 0x4e,
0x08, 0x2e, 0xa1, 0x66, 0x28, 0xd9, 0x24, 0xb2, 0x76, 0x5b, 0xa2, 0x49, 0x6d, 0x8b, 0xd1, 0x25,
0x72, 0xf8, 0xf6, 0x64, 0x86, 0x68, 0x98, 0x16, 0xd4, 0xa4, 0x5c, 0xcc, 0x5d, 0x65, 0xb6, 0x92,
0x6c, 0x70, 0x48, 0x50, 0xfd, 0xed, 0xb9, 0xda, 0x5e, 0x15, 0x46, 0x57, 0xa7, 0x8d, 0x9d, 0x84,
0x90, 0xd8, 0xab, 0x00, 0x8c, 0xbc, 0xd3, 0x0a, 0xf7, 0xe4, 0x58, 0x05, 0xb8, 0xb3, 0x45, 0x06,
0xd0, 0x2c, 0x1e, 0x8f, 0xca, 0x3f, 0x0f, 0x02, 0xc1, 0xaf, 0xbd, 0x03, 0x01, 0x13, 0x8a, 0x6b,
0x3a, 0x91, 0x11, 0x41, 0x4f, 0x67, 0xdc, 0xea, 0x97, 0xf2, 0xcf, 0xce, 0xf0, 0xb4, 0xe6, 0x73,
0x96, 0xac, 0x74, 0x22, 0xe7, 0xad, 0x35, 0x85, 0xe2, 0xf9, 0x37, 0xe8, 0x1c, 0x75, 0xdf, 0x6e,
0x47, 0xf1, 0x1a, 0x71, 0x1d, 0x29, 0xc5, 0x89, 0x6f, 0xb7, 0x62, 0x0e, 0xaa, 0x18, 0xbe, 0x1b,
0xfc, 0x56, 0x3e, 0x4b, 0xc6, 0xd2, 0x79, 0x20, 0x9a, 0xdb, 0xc0, 0xfe, 0x78, 0xcd, 0x5a, 0xf4,
0x1f, 0xdd, 0xa8, 0x33, 0x88, 0x07, 0xc7, 0x31, 0xb1, 0x12, 0x10, 0x59, 0x27, 0x80, 0xec, 0x5f,
0x60, 0x51, 0x7f, 0xa9, 0x19, 0xb5, 0x4a, 0x0d, 0x2d, 0xe5, 0x7a, 0x9f, 0x93, 0xc9, 0x9c, 0xef,
0xa0, 0xe0, 0x3b, 0x4d, 0xae, 0x2a, 0xf5, 0xb0, 0xc8, 0xeb, 0xbb, 0x3c, 0x83, 0x53, 0x99, 0x61,
0x17, 0x2b, 0x04, 0x7e, 0xba, 0x77, 0xd6, 0x26, 0xe1, 0x69, 0x14, 0x63, 0x55, 0x21, 0x0c, 0x7d,
};
// Uses a byte substitution table with a constant set of values
// Needs to do a table look up
uint8_t Substituted[16];
for (size_t i = 0; i < 16; ++i) {
Substituted[i] = InvSubstitutionTable[State[i]];
}
__uint128_t Res{};
memcpy(&Res, Substituted, 16);
return Res;
}
static __uint128_t ShiftRows(uint8_t *State) {
uint8_t Shifted[16] = {
State[0], State[5], State[10], State[15],
State[4], State[9], State[14], State[3],
State[8], State[13], State[2], State[7],
State[12], State[1], State[6], State[11],
};
__uint128_t Res{};
memcpy(&Res, Shifted, 16);
return Res;
}
static __uint128_t SubBytes(uint8_t *State, size_t Bytes) {
// 16x16 matrix table
static const uint8_t SubstitutionTable[256] = {
0x63, 0x7c, 0x77, 0x7b, 0xf2, 0x6b, 0x6f, 0xc5, 0x30, 0x01, 0x67, 0x2b, 0xfe, 0xd7, 0xab, 0x76,
0xca, 0x82, 0xc9, 0x7d, 0xfa, 0x59, 0x47, 0xf0, 0xad, 0xd4, 0xa2, 0xaf, 0x9c, 0xa4, 0x72, 0xc0,
0xb7, 0xfd, 0x93, 0x26, 0x36, 0x3f, 0xf7, 0xcc, 0x34, 0xa5, 0xe5, 0xf1, 0x71, 0xd8, 0x31, 0x15,
0x04, 0xc7, 0x23, 0xc3, 0x18, 0x96, 0x05, 0x9a, 0x07, 0x12, 0x80, 0xe2, 0xeb, 0x27, 0xb2, 0x75,
0x09, 0x83, 0x2c, 0x1a, 0x1b, 0x6e, 0x5a, 0xa0, 0x52, 0x3b, 0xd6, 0xb3, 0x29, 0xe3, 0x2f, 0x84,
0x53, 0xd1, 0x00, 0xed, 0x20, 0xfc, 0xb1, 0x5b, 0x6a, 0xcb, 0xbe, 0x39, 0x4a, 0x4c, 0x58, 0xcf,
0xd0, 0xef, 0xaa, 0xfb, 0x43, 0x4d, 0x33, 0x85, 0x45, 0xf9, 0x02, 0x7f, 0x50, 0x3c, 0x9f, 0xa8,
0x51, 0xa3, 0x40, 0x8f, 0x92, 0x9d, 0x38, 0xf5, 0xbc, 0xb6, 0xda, 0x21, 0x10, 0xff, 0xf3, 0xd2,
0xcd, 0x0c, 0x13, 0xec, 0x5f, 0x97, 0x44, 0x17, 0xc4, 0xa7, 0x7e, 0x3d, 0x64, 0x5d, 0x19, 0x73,
0x60, 0x81, 0x4f, 0xdc, 0x22, 0x2a, 0x90, 0x88, 0x46, 0xee, 0xb8, 0x14, 0xde, 0x5e, 0x0b, 0xdb,
0xe0, 0x32, 0x3a, 0x0a, 0x49, 0x06, 0x24, 0x5c, 0xc2, 0xd3, 0xac, 0x62, 0x91, 0x95, 0xe4, 0x79,
0xe7, 0xc8, 0x37, 0x6d, 0x8d, 0xd5, 0x4e, 0xa9, 0x6c, 0x56, 0xf4, 0xea, 0x65, 0x7a, 0xae, 0x08,
0xba, 0x78, 0x25, 0x2e, 0x1c, 0xa6, 0xb4, 0xc6, 0xe8, 0xdd, 0x74, 0x1f, 0x4b, 0xbd, 0x8b, 0x8a,
0x70, 0x3e, 0xb5, 0x66, 0x48, 0x03, 0xf6, 0x0e, 0x61, 0x35, 0x57, 0xb9, 0x86, 0xc1, 0x1d, 0x9e,
0xe1, 0xf8, 0x98, 0x11, 0x69, 0xd9, 0x8e, 0x94, 0x9b, 0x1e, 0x87, 0xe9, 0xce, 0x55, 0x28, 0xdf,
0x8c, 0xa1, 0x89, 0x0d, 0xbf, 0xe6, 0x42, 0x68, 0x41, 0x99, 0x2d, 0x0f, 0xb0, 0x54, 0xbb, 0x16,
};
// Uses a byte substitution table with a constant set of values
// Needs to do a table look up
uint8_t Substituted[16];
Bytes = std::min(Bytes, (size_t)16);
for (size_t i = 0; i < Bytes; ++i) {
Substituted[i] = SubstitutionTable[State[i]];
}
__uint128_t Res{};
memcpy(&Res, Substituted, Bytes);
return Res;
}
static uint8_t FFMul02(uint8_t in) {
static const uint8_t FFMul02[256] = {
0x00, 0x02, 0x04, 0x06, 0x08, 0x0a, 0x0c, 0x0e, 0x10, 0x12, 0x14, 0x16, 0x18, 0x1a, 0x1c, 0x1e,
0x20, 0x22, 0x24, 0x26, 0x28, 0x2a, 0x2c, 0x2e, 0x30, 0x32, 0x34, 0x36, 0x38, 0x3a, 0x3c, 0x3e,
0x40, 0x42, 0x44, 0x46, 0x48, 0x4a, 0x4c, 0x4e, 0x50, 0x52, 0x54, 0x56, 0x58, 0x5a, 0x5c, 0x5e,
0x60, 0x62, 0x64, 0x66, 0x68, 0x6a, 0x6c, 0x6e, 0x70, 0x72, 0x74, 0x76, 0x78, 0x7a, 0x7c, 0x7e,
0x80, 0x82, 0x84, 0x86, 0x88, 0x8a, 0x8c, 0x8e, 0x90, 0x92, 0x94, 0x96, 0x98, 0x9a, 0x9c, 0x9e,
0xa0, 0xa2, 0xa4, 0xa6, 0xa8, 0xaa, 0xac, 0xae, 0xb0, 0xb2, 0xb4, 0xb6, 0xb8, 0xba, 0xbc, 0xbe,
0xc0, 0xc2, 0xc4, 0xc6, 0xc8, 0xca, 0xcc, 0xce, 0xd0, 0xd2, 0xd4, 0xd6, 0xd8, 0xda, 0xdc, 0xde,
0xe0, 0xe2, 0xe4, 0xe6, 0xe8, 0xea, 0xec, 0xee, 0xf0, 0xf2, 0xf4, 0xf6, 0xf8, 0xfa, 0xfc, 0xfe,
0x1b, 0x19, 0x1f, 0x1d, 0x13, 0x11, 0x17, 0x15, 0x0b, 0x09, 0x0f, 0x0d, 0x03, 0x01, 0x07, 0x05,
0x3b, 0x39, 0x3f, 0x3d, 0x33, 0x31, 0x37, 0x35, 0x2b, 0x29, 0x2f, 0x2d, 0x23, 0x21, 0x27, 0x25,
0x5b, 0x59, 0x5f, 0x5d, 0x53, 0x51, 0x57, 0x55, 0x4b, 0x49, 0x4f, 0x4d, 0x43, 0x41, 0x47, 0x45,
0x7b, 0x79, 0x7f, 0x7d, 0x73, 0x71, 0x77, 0x75, 0x6b, 0x69, 0x6f, 0x6d, 0x63, 0x61, 0x67, 0x65,
0x9b, 0x99, 0x9f, 0x9d, 0x93, 0x91, 0x97, 0x95, 0x8b, 0x89, 0x8f, 0x8d, 0x83, 0x81, 0x87, 0x85,
0xbb, 0xb9, 0xbf, 0xbd, 0xb3, 0xb1, 0xb7, 0xb5, 0xab, 0xa9, 0xaf, 0xad, 0xa3, 0xa1, 0xa7, 0xa5,
0xdb, 0xd9, 0xdf, 0xdd, 0xd3, 0xd1, 0xd7, 0xd5, 0xcb, 0xc9, 0xcf, 0xcd, 0xc3, 0xc1, 0xc7, 0xc5,
0xfb, 0xf9, 0xff, 0xfd, 0xf3, 0xf1, 0xf7, 0xf5, 0xeb, 0xe9, 0xef, 0xed, 0xe3, 0xe1, 0xe7, 0xe5,
};
return FFMul02[in];
}
static uint8_t FFMul03(uint8_t in) {
static const uint8_t FFMul03[256] = {
0x00, 0x03, 0x06, 0x05, 0x0c, 0x0f, 0x0a, 0x09, 0x18, 0x1b, 0x1e, 0x1d, 0x14, 0x17, 0x12, 0x11,
0x30, 0x33, 0x36, 0x35, 0x3c, 0x3f, 0x3a, 0x39, 0x28, 0x2b, 0x2e, 0x2d, 0x24, 0x27, 0x22, 0x21,
0x60, 0x63, 0x66, 0x65, 0x6c, 0x6f, 0x6a, 0x69, 0x78, 0x7b, 0x7e, 0x7d, 0x74, 0x77, 0x72, 0x71,
0x50, 0x53, 0x56, 0x55, 0x5c, 0x5f, 0x5a, 0x59, 0x48, 0x4b, 0x4e, 0x4d, 0x44, 0x47, 0x42, 0x41,
0xc0, 0xc3, 0xc6, 0xc5, 0xcc, 0xcf, 0xca, 0xc9, 0xd8, 0xdb, 0xde, 0xdd, 0xd4, 0xd7, 0xd2, 0xd1,
0xf0, 0xf3, 0xf6, 0xf5, 0xfc, 0xff, 0xfa, 0xf9, 0xe8, 0xeb, 0xee, 0xed, 0xe4, 0xe7, 0xe2, 0xe1,
0xa0, 0xa3, 0xa6, 0xa5, 0xac, 0xaf, 0xaa, 0xa9, 0xb8, 0xbb, 0xbe, 0xbd, 0xb4, 0xb7, 0xb2, 0xb1,
0x90, 0x93, 0x96, 0x95, 0x9c, 0x9f, 0x9a, 0x99, 0x88, 0x8b, 0x8e, 0x8d, 0x84, 0x87, 0x82, 0x81,
0x9b, 0x98, 0x9d, 0x9e, 0x97, 0x94, 0x91, 0x92, 0x83, 0x80, 0x85, 0x86, 0x8f, 0x8c, 0x89, 0x8a,
0xab, 0xa8, 0xad, 0xae, 0xa7, 0xa4, 0xa1, 0xa2, 0xb3, 0xb0, 0xb5, 0xb6, 0xbf, 0xbc, 0xb9, 0xba,
0xfb, 0xf8, 0xfd, 0xfe, 0xf7, 0xf4, 0xf1, 0xf2, 0xe3, 0xe0, 0xe5, 0xe6, 0xef, 0xec, 0xe9, 0xea,
0xcb, 0xc8, 0xcd, 0xce, 0xc7, 0xc4, 0xc1, 0xc2, 0xd3, 0xd0, 0xd5, 0xd6, 0xdf, 0xdc, 0xd9, 0xda,
0x5b, 0x58, 0x5d, 0x5e, 0x57, 0x54, 0x51, 0x52, 0x43, 0x40, 0x45, 0x46, 0x4f, 0x4c, 0x49, 0x4a,
0x6b, 0x68, 0x6d, 0x6e, 0x67, 0x64, 0x61, 0x62, 0x73, 0x70, 0x75, 0x76, 0x7f, 0x7c, 0x79, 0x7a,
0x3b, 0x38, 0x3d, 0x3e, 0x37, 0x34, 0x31, 0x32, 0x23, 0x20, 0x25, 0x26, 0x2f, 0x2c, 0x29, 0x2a,
0x0b, 0x08, 0x0d, 0x0e, 0x07, 0x04, 0x01, 0x02, 0x13, 0x10, 0x15, 0x16, 0x1f, 0x1c, 0x19, 0x1a,
};
return FFMul03[in];
}
static __uint128_t MixColumns(uint8_t *State) {
uint8_t In0[16] = {
State[0], State[4], State[8], State[12],
State[1], State[5], State[9], State[13],
State[2], State[6], State[10], State[14],
State[3], State[7], State[11], State[15],
};
uint8_t Out0[4]{};
uint8_t Out1[4]{};
uint8_t Out2[4]{};
uint8_t Out3[4]{};
for (size_t i = 0; i < 4; ++i) {
Out0[i] = FFMul02(In0[0 + i]) ^ FFMul03(In0[4 + i]) ^ In0[8 + i] ^ In0[12 + i];
Out1[i] = In0[0 + i] ^ FFMul02(In0[4 + i]) ^ FFMul03(In0[8 + i]) ^ In0[12 + i];
Out2[i] = In0[0 + i] ^ In0[4 + i] ^ FFMul02(In0[8 + i]) ^ FFMul03(In0[12 + i]);
Out3[i] = FFMul03(In0[0 + i]) ^ In0[4 + i] ^ In0[8 + i] ^ FFMul02(In0[12 + i]);
}
uint8_t OutArray[16] = {
Out0[0], Out1[0], Out2[0], Out3[0],
Out0[1], Out1[1], Out2[1], Out3[1],
Out0[2], Out1[2], Out2[2], Out3[2],
Out0[3], Out1[3], Out2[3], Out3[3],
};
__uint128_t Res{};
memcpy(&Res, OutArray, 16);
return Res;
}
static uint8_t FFMul09(uint8_t in) {
static const uint8_t FFMul09[256] = {
0x00, 0x09, 0x12, 0x1b, 0x24, 0x2d, 0x36, 0x3f, 0x48, 0x41, 0x5a, 0x53, 0x6c, 0x65, 0x7e, 0x77,
0x90, 0x99, 0x82, 0x8b, 0xb4, 0xbd, 0xa6, 0xaf, 0xd8, 0xd1, 0xca, 0xc3, 0xfc, 0xf5, 0xee, 0xe7,
0x3b, 0x32, 0x29, 0x20, 0x1f, 0x16, 0x0d, 0x04, 0x73, 0x7a, 0x61, 0x68, 0x57, 0x5e, 0x45, 0x4c,
0xab, 0xa2, 0xb9, 0xb0, 0x8f, 0x86, 0x9d, 0x94, 0xe3, 0xea, 0xf1, 0xf8, 0xc7, 0xce, 0xd5, 0xdc,
0x76, 0x7f, 0x64, 0x6d, 0x52, 0x5b, 0x40, 0x49, 0x3e, 0x37, 0x2c, 0x25, 0x1a, 0x13, 0x08, 0x01,
0xe6, 0xef, 0xf4, 0xfd, 0xc2, 0xcb, 0xd0, 0xd9, 0xae, 0xa7, 0xbc, 0xb5, 0x8a, 0x83, 0x98, 0x91,
0x4d, 0x44, 0x5f, 0x56, 0x69, 0x60, 0x7b, 0x72, 0x05, 0x0c, 0x17, 0x1e, 0x21, 0x28, 0x33, 0x3a,
0xdd, 0xd4, 0xcf, 0xc6, 0xf9, 0xf0, 0xeb, 0xe2, 0x95, 0x9c, 0x87, 0x8e, 0xb1, 0xb8, 0xa3, 0xaa,
0xec, 0xe5, 0xfe, 0xf7, 0xc8, 0xc1, 0xda, 0xd3, 0xa4, 0xad, 0xb6, 0xbf, 0x80, 0x89, 0x92, 0x9b,
0x7c, 0x75, 0x6e, 0x67, 0x58, 0x51, 0x4a, 0x43, 0x34, 0x3d, 0x26, 0x2f, 0x10, 0x19, 0x02, 0x0b,
0xd7, 0xde, 0xc5, 0xcc, 0xf3, 0xfa, 0xe1, 0xe8, 0x9f, 0x96, 0x8d, 0x84, 0xbb, 0xb2, 0xa9, 0xa0,
0x47, 0x4e, 0x55, 0x5c, 0x63, 0x6a, 0x71, 0x78, 0x0f, 0x06, 0x1d, 0x14, 0x2b, 0x22, 0x39, 0x30,
0x9a, 0x93, 0x88, 0x81, 0xbe, 0xb7, 0xac, 0xa5, 0xd2, 0xdb, 0xc0, 0xc9, 0xf6, 0xff, 0xe4, 0xed,
0x0a, 0x03, 0x18, 0x11, 0x2e, 0x27, 0x3c, 0x35, 0x42, 0x4b, 0x50, 0x59, 0x66, 0x6f, 0x74, 0x7d,
0xa1, 0xa8, 0xb3, 0xba, 0x85, 0x8c, 0x97, 0x9e, 0xe9, 0xe0, 0xfb, 0xf2, 0xcd, 0xc4, 0xdf, 0xd6,
0x31, 0x38, 0x23, 0x2a, 0x15, 0x1c, 0x07, 0x0e, 0x79, 0x70, 0x6b, 0x62, 0x5d, 0x54, 0x4f, 0x46,
};
return FFMul09[in];
}
static uint8_t FFMul0B(uint8_t in) {
static const uint8_t FFMul0B[256] = {
0x00, 0x0b, 0x16, 0x1d, 0x2c, 0x27, 0x3a, 0x31, 0x58, 0x53, 0x4e, 0x45, 0x74, 0x7f, 0x62, 0x69,
0xb0, 0xbb, 0xa6, 0xad, 0x9c, 0x97, 0x8a, 0x81, 0xe8, 0xe3, 0xfe, 0xf5, 0xc4, 0xcf, 0xd2, 0xd9,
0x7b, 0x70, 0x6d, 0x66, 0x57, 0x5c, 0x41, 0x4a, 0x23, 0x28, 0x35, 0x3e, 0x0f, 0x04, 0x19, 0x12,
0xcb, 0xc0, 0xdd, 0xd6, 0xe7, 0xec, 0xf1, 0xfa, 0x93, 0x98, 0x85, 0x8e, 0xbf, 0xb4, 0xa9, 0xa2,
0xf6, 0xfd, 0xe0, 0xeb, 0xda, 0xd1, 0xcc, 0xc7, 0xae, 0xa5, 0xb8, 0xb3, 0x82, 0x89, 0x94, 0x9f,
0x46, 0x4d, 0x50, 0x5b, 0x6a, 0x61, 0x7c, 0x77, 0x1e, 0x15, 0x08, 0x03, 0x32, 0x39, 0x24, 0x2f,
0x8d, 0x86, 0x9b, 0x90, 0xa1, 0xaa, 0xb7, 0xbc, 0xd5, 0xde, 0xc3, 0xc8, 0xf9, 0xf2, 0xef, 0xe4,
0x3d, 0x36, 0x2b, 0x20, 0x11, 0x1a, 0x07, 0x0c, 0x65, 0x6e, 0x73, 0x78, 0x49, 0x42, 0x5f, 0x54,
0xf7, 0xfc, 0xe1, 0xea, 0xdb, 0xd0, 0xcd, 0xc6, 0xaf, 0xa4, 0xb9, 0xb2, 0x83, 0x88, 0x95, 0x9e,
0x47, 0x4c, 0x51, 0x5a, 0x6b, 0x60, 0x7d, 0x76, 0x1f, 0x14, 0x09, 0x02, 0x33, 0x38, 0x25, 0x2e,
0x8c, 0x87, 0x9a, 0x91, 0xa0, 0xab, 0xb6, 0xbd, 0xd4, 0xdf, 0xc2, 0xc9, 0xf8, 0xf3, 0xee, 0xe5,
0x3c, 0x37, 0x2a, 0x21, 0x10, 0x1b, 0x06, 0x0d, 0x64, 0x6f, 0x72, 0x79, 0x48, 0x43, 0x5e, 0x55,
0x01, 0x0a, 0x17, 0x1c, 0x2d, 0x26, 0x3b, 0x30, 0x59, 0x52, 0x4f, 0x44, 0x75, 0x7e, 0x63, 0x68,
0xb1, 0xba, 0xa7, 0xac, 0x9d, 0x96, 0x8b, 0x80, 0xe9, 0xe2, 0xff, 0xf4, 0xc5, 0xce, 0xd3, 0xd8,
0x7a, 0x71, 0x6c, 0x67, 0x56, 0x5d, 0x40, 0x4b, 0x22, 0x29, 0x34, 0x3f, 0x0e, 0x05, 0x18, 0x13,
0xca, 0xc1, 0xdc, 0xd7, 0xe6, 0xed, 0xf0, 0xfb, 0x92, 0x99, 0x84, 0x8f, 0xbe, 0xb5, 0xa8, 0xa3,
};
return FFMul0B[in];
}
static uint8_t FFMul0D(uint8_t in) {
static const uint8_t FFMul0D[256] = {
0x00, 0x0d, 0x1a, 0x17, 0x34, 0x39, 0x2e, 0x23, 0x68, 0x65, 0x72, 0x7f, 0x5c, 0x51, 0x46, 0x4b,
0xd0, 0xdd, 0xca, 0xc7, 0xe4, 0xe9, 0xfe, 0xf3, 0xb8, 0xb5, 0xa2, 0xaf, 0x8c, 0x81, 0x96, 0x9b,
0xbb, 0xb6, 0xa1, 0xac, 0x8f, 0x82, 0x95, 0x98, 0xd3, 0xde, 0xc9, 0xc4, 0xe7, 0xea, 0xfd, 0xf0,
0x6b, 0x66, 0x71, 0x7c, 0x5f, 0x52, 0x45, 0x48, 0x03, 0x0e, 0x19, 0x14, 0x37, 0x3a, 0x2d, 0x20,
0x6d, 0x60, 0x77, 0x7a, 0x59, 0x54, 0x43, 0x4e, 0x05, 0x08, 0x1f, 0x12, 0x31, 0x3c, 0x2b, 0x26,
0xbd, 0xb0, 0xa7, 0xaa, 0x89, 0x84, 0x93, 0x9e, 0xd5, 0xd8, 0xcf, 0xc2, 0xe1, 0xec, 0xfb, 0xf6,
0xd6, 0xdb, 0xcc, 0xc1, 0xe2, 0xef, 0xf8, 0xf5, 0xbe, 0xb3, 0xa4, 0xa9, 0x8a, 0x87, 0x90, 0x9d,
0x06, 0x0b, 0x1c, 0x11, 0x32, 0x3f, 0x28, 0x25, 0x6e, 0x63, 0x74, 0x79, 0x5a, 0x57, 0x40, 0x4d,
0xda, 0xd7, 0xc0, 0xcd, 0xee, 0xe3, 0xf4, 0xf9, 0xb2, 0xbf, 0xa8, 0xa5, 0x86, 0x8b, 0x9c, 0x91,
0x0a, 0x07, 0x10, 0x1d, 0x3e, 0x33, 0x24, 0x29, 0x62, 0x6f, 0x78, 0x75, 0x56, 0x5b, 0x4c, 0x41,
0x61, 0x6c, 0x7b, 0x76, 0x55, 0x58, 0x4f, 0x42, 0x09, 0x04, 0x13, 0x1e, 0x3d, 0x30, 0x27, 0x2a,
0xb1, 0xbc, 0xab, 0xa6, 0x85, 0x88, 0x9f, 0x92, 0xd9, 0xd4, 0xc3, 0xce, 0xed, 0xe0, 0xf7, 0xfa,
0xb7, 0xba, 0xad, 0xa0, 0x83, 0x8e, 0x99, 0x94, 0xdf, 0xd2, 0xc5, 0xc8, 0xeb, 0xe6, 0xf1, 0xfc,
0x67, 0x6a, 0x7d, 0x70, 0x53, 0x5e, 0x49, 0x44, 0x0f, 0x02, 0x15, 0x18, 0x3b, 0x36, 0x21, 0x2c,
0x0c, 0x01, 0x16, 0x1b, 0x38, 0x35, 0x22, 0x2f, 0x64, 0x69, 0x7e, 0x73, 0x50, 0x5d, 0x4a, 0x47,
0xdc, 0xd1, 0xc6, 0xcb, 0xe8, 0xe5, 0xf2, 0xff, 0xb4, 0xb9, 0xae, 0xa3, 0x80, 0x8d, 0x9a, 0x97,
};
return FFMul0D[in];
}
static uint8_t FFMul0E(uint8_t in) {
static const uint8_t FFMul0E[256] = {
0x00, 0x0e, 0x1c, 0x12, 0x38, 0x36, 0x24, 0x2a, 0x70, 0x7e, 0x6c, 0x62, 0x48, 0x46, 0x54, 0x5a,
0xe0, 0xee, 0xfc, 0xf2, 0xd8, 0xd6, 0xc4, 0xca, 0x90, 0x9e, 0x8c, 0x82, 0xa8, 0xa6, 0xb4, 0xba,
0xdb, 0xd5, 0xc7, 0xc9, 0xe3, 0xed, 0xff, 0xf1, 0xab, 0xa5, 0xb7, 0xb9, 0x93, 0x9d, 0x8f, 0x81,
0x3b, 0x35, 0x27, 0x29, 0x03, 0x0d, 0x1f, 0x11, 0x4b, 0x45, 0x57, 0x59, 0x73, 0x7d, 0x6f, 0x61,
0xad, 0xa3, 0xb1, 0xbf, 0x95, 0x9b, 0x89, 0x87, 0xdd, 0xd3, 0xc1, 0xcf, 0xe5, 0xeb, 0xf9, 0xf7,
0x4d, 0x43, 0x51, 0x5f, 0x75, 0x7b, 0x69, 0x67, 0x3d, 0x33, 0x21, 0x2f, 0x05, 0x0b, 0x19, 0x17,
0x76, 0x78, 0x6a, 0x64, 0x4e, 0x40, 0x52, 0x5c, 0x06, 0x08, 0x1a, 0x14, 0x3e, 0x30, 0x22, 0x2c,
0x96, 0x98, 0x8a, 0x84, 0xae, 0xa0, 0xb2, 0xbc, 0xe6, 0xe8, 0xfa, 0xf4, 0xde, 0xd0, 0xc2, 0xcc,
0x41, 0x4f, 0x5d, 0x53, 0x79, 0x77, 0x65, 0x6b, 0x31, 0x3f, 0x2d, 0x23, 0x09, 0x07, 0x15, 0x1b,
0xa1, 0xaf, 0xbd, 0xb3, 0x99, 0x97, 0x85, 0x8b, 0xd1, 0xdf, 0xcd, 0xc3, 0xe9, 0xe7, 0xf5, 0xfb,
0x9a, 0x94, 0x86, 0x88, 0xa2, 0xac, 0xbe, 0xb0, 0xea, 0xe4, 0xf6, 0xf8, 0xd2, 0xdc, 0xce, 0xc0,
0x7a, 0x74, 0x66, 0x68, 0x42, 0x4c, 0x5e, 0x50, 0x0a, 0x04, 0x16, 0x18, 0x32, 0x3c, 0x2e, 0x20,
0xec, 0xe2, 0xf0, 0xfe, 0xd4, 0xda, 0xc8, 0xc6, 0x9c, 0x92, 0x80, 0x8e, 0xa4, 0xaa, 0xb8, 0xb6,
0x0c, 0x02, 0x10, 0x1e, 0x34, 0x3a, 0x28, 0x26, 0x7c, 0x72, 0x60, 0x6e, 0x44, 0x4a, 0x58, 0x56,
0x37, 0x39, 0x2b, 0x25, 0x0f, 0x01, 0x13, 0x1d, 0x47, 0x49, 0x5b, 0x55, 0x7f, 0x71, 0x63, 0x6d,
0xd7, 0xd9, 0xcb, 0xc5, 0xef, 0xe1, 0xf3, 0xfd, 0xa7, 0xa9, 0xbb, 0xb5, 0x9f, 0x91, 0x83, 0x8d,
};
return FFMul0E[in];
}
static __uint128_t InvMixColumns(uint8_t *State) {
uint8_t In0[16] = {
State[0], State[4], State[8], State[12],
State[1], State[5], State[9], State[13],
State[2], State[6], State[10], State[14],
State[3], State[7], State[11], State[15],
};
uint8_t Out0[4]{};
uint8_t Out1[4]{};
uint8_t Out2[4]{};
uint8_t Out3[4]{};
for (size_t i = 0; i < 4; ++i) {
Out0[i] = FFMul0E(In0[0 + i]) ^ FFMul0B(In0[4 + i]) ^ FFMul0D(In0[8 + i]) ^ FFMul09(In0[12 + i]);
Out1[i] = FFMul09(In0[0 + i]) ^ FFMul0E(In0[4 + i]) ^ FFMul0B(In0[8 + i]) ^ FFMul0D(In0[12 + i]);
Out2[i] = FFMul0D(In0[0 + i]) ^ FFMul09(In0[4 + i]) ^ FFMul0E(In0[8 + i]) ^ FFMul0B(In0[12 + i]);
Out3[i] = FFMul0B(In0[0 + i]) ^ FFMul0D(In0[4 + i]) ^ FFMul09(In0[8 + i]) ^ FFMul0E(In0[12 + i]);
}
uint8_t OutArray[16] = {
Out0[0], Out1[0], Out2[0], Out3[0],
Out0[1], Out1[1], Out2[1], Out3[1],
Out0[2], Out1[2], Out2[2], Out3[2],
Out0[3], Out1[3], Out2[3], Out3[3],
};
__uint128_t Res{};
memcpy(&Res, OutArray, 16);
return Res;
}
}
namespace FEXCore::CPU {
#define DEF_OP(x) void InterpreterOps::Op_##x(FEXCore::IR::IROp_Header *IROp, IROpData *Data, uint32_t Node)
DEF_OP(AESImc) {
auto Op = IROp->C<IR::IROp_VAESImc>();
__uint128_t Src1 = *GetSrc<__uint128_t*>(Data->SSAData, Op->Header.Args[0]);
// Pseudo-code
// Dst = InvMixColumns(STATE)
__uint128_t Tmp{};
Tmp = AES::InvMixColumns(reinterpret_cast<uint8_t*>(&Src1));
memcpy(GDP, &Tmp, sizeof(Tmp));
}
DEF_OP(AESEnc) {
auto Op = IROp->C<IR::IROp_VAESEnc>();
__uint128_t Src1 = *GetSrc<__uint128_t*>(Data->SSAData, Op->Header.Args[0]);
__uint128_t Src2 = *GetSrc<__uint128_t*>(Data->SSAData, Op->Header.Args[1]);
// Pseudo-code
// STATE = Src1
// RoundKey = Src2
// STATE = ShiftRows(STATE)
// STATE = SubBytes(STATE)
// STATE = MixColumns(STATE)
// Dst = STATE XOR RoundKey
__uint128_t Tmp{};
Tmp = AES::ShiftRows(reinterpret_cast<uint8_t*>(&Src1));
Tmp = AES::SubBytes(reinterpret_cast<uint8_t*>(&Tmp), 16);
Tmp = AES::MixColumns(reinterpret_cast<uint8_t*>(&Tmp));
Tmp = Tmp ^ Src2;
memcpy(GDP, &Tmp, sizeof(Tmp));
}
DEF_OP(AESEncLast) {
auto Op = IROp->C<IR::IROp_VAESEncLast>();
__uint128_t Src1 = *GetSrc<__uint128_t*>(Data->SSAData, Op->Header.Args[0]);
__uint128_t Src2 = *GetSrc<__uint128_t*>(Data->SSAData, Op->Header.Args[1]);
// Pseudo-code
// STATE = Src1
// RoundKey = Src2
// STATE = ShiftRows(STATE)
// STATE = SubBytes(STATE)
// Dst = STATE XOR RoundKey
__uint128_t Tmp{};
Tmp = AES::ShiftRows(reinterpret_cast<uint8_t*>(&Src1));
Tmp = AES::SubBytes(reinterpret_cast<uint8_t*>(&Tmp), 16);
Tmp = Tmp ^ Src2;
memcpy(GDP, &Tmp, sizeof(Tmp));
}
DEF_OP(AESDec) {
auto Op = IROp->C<IR::IROp_VAESDec>();
__uint128_t Src1 = *GetSrc<__uint128_t*>(Data->SSAData, Op->Header.Args[0]);
__uint128_t Src2 = *GetSrc<__uint128_t*>(Data->SSAData, Op->Header.Args[1]);
// Pseudo-code
// STATE = Src1
// RoundKey = Src2
// STATE = InvShiftRows(STATE)
// STATE = InvSubBytes(STATE)
// STATE = InvMixColumns(STATE)
// Dst = STATE XOR RoundKey
__uint128_t Tmp{};
Tmp = AES::InvShiftRows(reinterpret_cast<uint8_t*>(&Src1));
Tmp = AES::InvSubBytes(reinterpret_cast<uint8_t*>(&Tmp));
Tmp = AES::InvMixColumns(reinterpret_cast<uint8_t*>(&Tmp));
Tmp = Tmp ^ Src2;
memcpy(GDP, &Tmp, sizeof(Tmp));
}
DEF_OP(AESDecLast) {
auto Op = IROp->C<IR::IROp_VAESDecLast>();
__uint128_t Src1 = *GetSrc<__uint128_t*>(Data->SSAData, Op->Header.Args[0]);
__uint128_t Src2 = *GetSrc<__uint128_t*>(Data->SSAData, Op->Header.Args[1]);
// Pseudo-code
// STATE = Src1
// RoundKey = Src2
// STATE = InvShiftRows(STATE)
// STATE = InvSubBytes(STATE)
// Dst = STATE XOR RoundKey
__uint128_t Tmp{};
Tmp = AES::InvShiftRows(reinterpret_cast<uint8_t*>(&Src1));
Tmp = AES::InvSubBytes(reinterpret_cast<uint8_t*>(&Tmp));
Tmp = Tmp ^ Src2;
memcpy(GDP, &Tmp, sizeof(Tmp));
}
DEF_OP(AESKeyGenAssist) {
auto Op = IROp->C<IR::IROp_VAESKeyGenAssist>();
uint8_t *Src1 = GetSrc<uint8_t*>(Data->SSAData, Op->Header.Args[0]);
// Pseudo-code
// X3 = Src1[127:96]
// X2 = Src1[95:64]
// X1 = Src1[63:32]
// X0 = Src1[31:30]
// RCON = (Zext)rcon
// Dest[31:0] = SubWord(X1)
// Dest[63:32] = RotWord(SubWord(X1)) XOR RCON
// Dest[95:64] = SubWord(X3)
// Dest[127:96] = RotWord(SubWord(X3)) XOR RCON
__uint128_t Tmp{};
uint32_t X1{};
uint32_t X3{};
memcpy(&X1, &Src1[4], 4);
memcpy(&X3, &Src1[12], 4);
uint32_t SubWord_X1 = AES::SubBytes(reinterpret_cast<uint8_t*>(&X1), 4);
uint32_t SubWord_X3 = AES::SubBytes(reinterpret_cast<uint8_t*>(&X3), 4);
auto Ror = [] (auto In, auto R) {
auto RotateMask = sizeof(In) * 8 - 1;
R &= RotateMask;
return (In >> R) | (In << (sizeof(In) * 8 - R));
};
uint32_t Rot_X1 = Ror(SubWord_X1, 8);
uint32_t Rot_X3 = Ror(SubWord_X3, 8);
Tmp = Rot_X3 ^ Op->RCON;
Tmp <<= 32;
Tmp |= SubWord_X3;
Tmp <<= 32;
Tmp |= Rot_X1 ^ Op->RCON;
Tmp <<= 32;
Tmp |= SubWord_X1;
memcpy(GDP, &Tmp, sizeof(Tmp));
}
#undef DEF_OP
void InterpreterOps::RegisterEncryptionHandlers() {
#define REGISTER_OP(op, x) OpHandlers[FEXCore::IR::IROps::OP_##op] = &InterpreterOps::Op_##x
REGISTER_OP(VAESIMC, AESImc);
REGISTER_OP(VAESENC, AESEnc);
REGISTER_OP(VAESENCLAST, AESEncLast);
REGISTER_OP(VAESDEC, AESDec);
REGISTER_OP(VAESDECLAST, AESDecLast);
REGISTER_OP(VAESKEYGENASSIST, AESKeyGenAssist);
#undef REGISTER_OP
}
}
@@ -0,0 +1,389 @@
/*
$info$
tags: backend|interpreter
$end_info$
*/
#include "Interface/Core/Interpreter/InterpreterClass.h"
#include "Interface/Core/Interpreter/InterpreterOps.h"
#include "Interface/Core/Interpreter/InterpreterDefines.h"
#include "F80Ops.h"
#include <cstdint>
namespace FEXCore::CPU {
#define DEF_OP(x) void InterpreterOps::Op_##x(FEXCore::IR::IROp_Header *IROp, IROpData *Data, uint32_t Node)
DEF_OP(F80LOADFCW) {
FEXCore::CPU::OpHandlers<IR::OP_F80LOADFCW>::handle(*GetSrc<uint16_t*>(Data->SSAData, IROp->Args[0]));
}
DEF_OP(F80ADD) {
auto Op = IROp->C<IR::IROp_F80Add>();
X80SoftFloat Src1 = *GetSrc<X80SoftFloat*>(Data->SSAData, Op->Header.Args[0]);
X80SoftFloat Src2 = *GetSrc<X80SoftFloat*>(Data->SSAData, Op->Header.Args[1]);
X80SoftFloat Tmp;
Tmp = X80SoftFloat::FADD(Src1, Src2);
memcpy(GDP, &Tmp, sizeof(X80SoftFloat));
}
DEF_OP(F80SUB) {
auto Op = IROp->C<IR::IROp_F80Sub>();
X80SoftFloat Src1 = *GetSrc<X80SoftFloat*>(Data->SSAData, Op->Header.Args[0]);
X80SoftFloat Src2 = *GetSrc<X80SoftFloat*>(Data->SSAData, Op->Header.Args[1]);
X80SoftFloat Tmp;
Tmp = X80SoftFloat::FSUB(Src1, Src2);
memcpy(GDP, &Tmp, sizeof(X80SoftFloat));
}
DEF_OP(F80MUL) {
auto Op = IROp->C<IR::IROp_F80Mul>();
X80SoftFloat Src1 = *GetSrc<X80SoftFloat*>(Data->SSAData, Op->Header.Args[0]);
X80SoftFloat Src2 = *GetSrc<X80SoftFloat*>(Data->SSAData, Op->Header.Args[1]);
X80SoftFloat Tmp;
Tmp = X80SoftFloat::FMUL(Src1, Src2);
memcpy(GDP, &Tmp, sizeof(X80SoftFloat));
}
DEF_OP(F80DIV) {
auto Op = IROp->C<IR::IROp_F80Div>();
X80SoftFloat Src1 = *GetSrc<X80SoftFloat*>(Data->SSAData, Op->Header.Args[0]);
X80SoftFloat Src2 = *GetSrc<X80SoftFloat*>(Data->SSAData, Op->Header.Args[1]);
X80SoftFloat Tmp;
Tmp = X80SoftFloat::FDIV(Src1, Src2);
memcpy(GDP, &Tmp, sizeof(X80SoftFloat));
}
DEF_OP(F80FYL2X) {
auto Op = IROp->C<IR::IROp_F80FYL2X>();
X80SoftFloat Src1 = *GetSrc<X80SoftFloat*>(Data->SSAData, Op->Header.Args[0]);
X80SoftFloat Src2 = *GetSrc<X80SoftFloat*>(Data->SSAData, Op->Header.Args[1]);
X80SoftFloat Tmp;
Tmp = X80SoftFloat::FYL2X(Src1, Src2);
memcpy(GDP, &Tmp, sizeof(X80SoftFloat));
}
DEF_OP(F80ATAN) {
auto Op = IROp->C<IR::IROp_F80ATAN>();
X80SoftFloat Src1 = *GetSrc<X80SoftFloat*>(Data->SSAData, Op->Header.Args[0]);
X80SoftFloat Src2 = *GetSrc<X80SoftFloat*>(Data->SSAData, Op->Header.Args[1]);
X80SoftFloat Tmp;
Tmp = X80SoftFloat::FATAN(Src1, Src2);
memcpy(GDP, &Tmp, sizeof(X80SoftFloat));
}
DEF_OP(F80FPREM1) {
auto Op = IROp->C<IR::IROp_F80FPREM1>();
X80SoftFloat Src1 = *GetSrc<X80SoftFloat*>(Data->SSAData, Op->Header.Args[0]);
X80SoftFloat Src2 = *GetSrc<X80SoftFloat*>(Data->SSAData, Op->Header.Args[1]);
X80SoftFloat Tmp;
Tmp = X80SoftFloat::FREM1(Src1, Src2);
memcpy(GDP, &Tmp, sizeof(X80SoftFloat));
}
DEF_OP(F80FPREM) {
auto Op = IROp->C<IR::IROp_F80FPREM>();
X80SoftFloat Src1 = *GetSrc<X80SoftFloat*>(Data->SSAData, Op->Header.Args[0]);
X80SoftFloat Src2 = *GetSrc<X80SoftFloat*>(Data->SSAData, Op->Header.Args[1]);
X80SoftFloat Tmp;
Tmp = X80SoftFloat::FREM(Src1, Src2);
memcpy(GDP, &Tmp, sizeof(X80SoftFloat));
}
DEF_OP(F80SCALE) {
auto Op = IROp->C<IR::IROp_F80SCALE>();
X80SoftFloat Src1 = *GetSrc<X80SoftFloat*>(Data->SSAData, Op->Header.Args[0]);
X80SoftFloat Src2 = *GetSrc<X80SoftFloat*>(Data->SSAData, Op->Header.Args[1]);
X80SoftFloat Tmp;
Tmp = X80SoftFloat::FSCALE(Src1, Src2);
memcpy(GDP, &Tmp, sizeof(X80SoftFloat));
}
DEF_OP(F80CVT) {
auto Op = IROp->C<IR::IROp_F80CVT>();
uint8_t OpSize = IROp->Size;
X80SoftFloat Src = *GetSrc<X80SoftFloat*>(Data->SSAData, Op->Header.Args[0]);
switch (OpSize) {
case 4: {
float Tmp = Src;
memcpy(GDP, &Tmp, OpSize);
break;
}
case 8: {
double Tmp = Src;
memcpy(GDP, &Tmp, OpSize);
break;
}
default: LogMan::Msg::DFmt("Unhandled size: {}", OpSize);
}
}
DEF_OP(F80CVTINT) {
auto Op = IROp->C<IR::IROp_F80CVTInt>();
uint8_t OpSize = IROp->Size;
X80SoftFloat Src = *GetSrc<X80SoftFloat*>(Data->SSAData, Op->Header.Args[0]);
switch (OpSize) {
case 2: {
int16_t Tmp = (Op->Truncate? FEXCore::CPU::OpHandlers<IR::OP_F80CVTINT>::handle2t : FEXCore::CPU::OpHandlers<IR::OP_F80CVTINT>::handle2)(Src);
memcpy(GDP, &Tmp, sizeof(Tmp));
break;
}
case 4: {
int32_t Tmp = (Op->Truncate? FEXCore::CPU::OpHandlers<IR::OP_F80CVTINT>::handle4t : FEXCore::CPU::OpHandlers<IR::OP_F80CVTINT>::handle4)(Src);
memcpy(GDP, &Tmp, sizeof(Tmp));
break;
}
case 8: {
int64_t Tmp = (Op->Truncate? FEXCore::CPU::OpHandlers<IR::OP_F80CVTINT>::handle8t : FEXCore::CPU::OpHandlers<IR::OP_F80CVTINT>::handle8)(Src);
memcpy(GDP, &Tmp, sizeof(Tmp));
break;
}
default: LogMan::Msg::DFmt("Unhandled size: {}", OpSize);
}
}
DEF_OP(F80CVTTO) {
auto Op = IROp->C<IR::IROp_F80CVTTo>();
switch (Op->Size) {
case 4: {
float Src = *GetSrc<float *>(Data->SSAData, Op->Header.Args[0]);
X80SoftFloat Tmp = Src;
memcpy(GDP, &Tmp, sizeof(X80SoftFloat));
break;
}
case 8: {
double Src = *GetSrc<double *>(Data->SSAData, Op->Header.Args[0]);
X80SoftFloat Tmp = Src;
memcpy(GDP, &Tmp, sizeof(X80SoftFloat));
break;
}
default: LogMan::Msg::DFmt("Unhandled size: {}", Op->Size);
}
}
DEF_OP(F80CVTTOINT) {
auto Op = IROp->C<IR::IROp_F80CVTToInt>();
switch (Op->Size) {
case 2: {
int16_t Src = *GetSrc<int16_t*>(Data->SSAData, Op->Header.Args[0]);
X80SoftFloat Tmp = Src;
memcpy(GDP, &Tmp, sizeof(X80SoftFloat));
break;
}
case 4: {
int32_t Src = *GetSrc<int32_t*>(Data->SSAData, Op->Header.Args[0]);
X80SoftFloat Tmp = Src;
memcpy(GDP, &Tmp, sizeof(X80SoftFloat));
break;
}
default: LogMan::Msg::DFmt("Unhandled size: {}", Op->Size);
}
}
DEF_OP(F80ROUND) {
auto Op = IROp->C<IR::IROp_F80Round>();
X80SoftFloat Src = *GetSrc<X80SoftFloat*>(Data->SSAData, Op->Header.Args[0]);
X80SoftFloat Tmp;
Tmp = X80SoftFloat::FRNDINT(Src);
memcpy(GDP, &Tmp, sizeof(X80SoftFloat));
}
DEF_OP(F80F2XM1) {
auto Op = IROp->C<IR::IROp_F80F2XM1>();
X80SoftFloat Src = *GetSrc<X80SoftFloat*>(Data->SSAData, Op->Header.Args[0]);
X80SoftFloat Tmp;
Tmp = X80SoftFloat::F2XM1(Src);
memcpy(GDP, &Tmp, sizeof(X80SoftFloat));
}
DEF_OP(F80TAN) {
auto Op = IROp->C<IR::IROp_F80TAN>();
X80SoftFloat Src = *GetSrc<X80SoftFloat*>(Data->SSAData, Op->Header.Args[0]);
X80SoftFloat Tmp;
Tmp = X80SoftFloat::FTAN(Src);
memcpy(GDP, &Tmp, sizeof(X80SoftFloat));
}
DEF_OP(F80SQRT) {
auto Op = IROp->C<IR::IROp_F80SQRT>();
X80SoftFloat Src = *GetSrc<X80SoftFloat*>(Data->SSAData, Op->Header.Args[0]);
X80SoftFloat Tmp;
Tmp = X80SoftFloat::FSQRT(Src);
memcpy(GDP, &Tmp, sizeof(X80SoftFloat));
}
DEF_OP(F80SIN) {
auto Op = IROp->C<IR::IROp_F80SIN>();
X80SoftFloat Src = *GetSrc<X80SoftFloat*>(Data->SSAData, Op->Header.Args[0]);
X80SoftFloat Tmp;
Tmp = X80SoftFloat::FSIN(Src);
memcpy(GDP, &Tmp, sizeof(X80SoftFloat));
}
DEF_OP(F80COS) {
auto Op = IROp->C<IR::IROp_F80COS>();
X80SoftFloat Src = *GetSrc<X80SoftFloat*>(Data->SSAData, Op->Header.Args[0]);
X80SoftFloat Tmp;
Tmp = X80SoftFloat::FCOS(Src);
memcpy(GDP, &Tmp, sizeof(X80SoftFloat));
}
DEF_OP(F80XTRACT_EXP) {
auto Op = IROp->C<IR::IROp_F80XTRACT_EXP>();
X80SoftFloat Src = *GetSrc<X80SoftFloat*>(Data->SSAData, Op->Header.Args[0]);
X80SoftFloat Tmp;
Tmp = X80SoftFloat::FXTRACT_EXP(Src);
memcpy(GDP, &Tmp, sizeof(X80SoftFloat));
}
DEF_OP(F80XTRACT_SIG) {
auto Op = IROp->C<IR::IROp_F80XTRACT_SIG>();
X80SoftFloat Src = *GetSrc<X80SoftFloat*>(Data->SSAData, Op->Header.Args[0]);
X80SoftFloat Tmp;
Tmp = X80SoftFloat::FXTRACT_SIG(Src);
memcpy(GDP, &Tmp, sizeof(X80SoftFloat));
}
DEF_OP(F80CMP) {
auto Op = IROp->C<IR::IROp_F80Cmp>();
uint32_t ResultFlags{};
X80SoftFloat Src1 = *GetSrc<X80SoftFloat*>(Data->SSAData, Op->Header.Args[0]);
X80SoftFloat Src2 = *GetSrc<X80SoftFloat*>(Data->SSAData, Op->Header.Args[1]);
bool eq, lt, nan;
X80SoftFloat::FCMP(Src1, Src2, &eq, &lt, &nan);
if (Op->Flags & (1 << IR::FCMP_FLAG_LT) &&
lt) {
ResultFlags |= (1 << IR::FCMP_FLAG_LT);
}
if (Op->Flags & (1 << IR::FCMP_FLAG_UNORDERED) &&
nan) {
ResultFlags |= (1 << IR::FCMP_FLAG_UNORDERED);
}
if (Op->Flags & (1 << IR::FCMP_FLAG_EQ) &&
eq) {
ResultFlags |= (1 << IR::FCMP_FLAG_EQ);
}
GD = ResultFlags;
}
DEF_OP(F80BCDLOAD) {
auto Op = IROp->C<IR::IROp_F80BCDLoad>();
uint8_t *Src1 = GetSrc<uint8_t*>(Data->SSAData, Op->Header.Args[0]);
uint64_t BCD{};
// We walk through each uint8_t and pull out the BCD encoding
// Each 4bit split is a digit
// Only 0-9 is supported, A-F results in undefined data
// | 4 bit | 4 bit |
// | 10s place | 1s place |
// EG 0x48 = 48
// EG 0x4847 = 4847
// This gives us an 18digit value encoded in BCD
// The last byte lets us know if it negative or not
for (size_t i = 0; i < 9; ++i) {
uint8_t Digit = Src1[8 - i];
// First shift our last value over
BCD *= 100;
// Add the tens place digit
BCD += (Digit >> 4) * 10;
// Add the ones place digit
BCD += Digit & 0xF;
}
// Set negative flag once converted to x87
bool Negative = Src1[9] & 0x80;
X80SoftFloat Tmp;
Tmp = BCD;
Tmp.Sign = Negative;
memcpy(GDP, &Tmp, sizeof(X80SoftFloat));
}
DEF_OP(F80BCDSTORE) {
auto Op = IROp->C<IR::IROp_F80BCDStore>();
X80SoftFloat Src1 = *GetSrc<X80SoftFloat*>(Data->SSAData, Op->Header.Args[0]);
bool Negative = Src1.Sign;
// Clear the Sign bit
Src1.Sign = 0;
uint64_t Tmp = Src1;
uint8_t BCD[10]{};
for (size_t i = 0; i < 9; ++i) {
if (Tmp == 0) {
// Nothing left? Just leave
break;
}
// Extract the lower 100 values
uint8_t Digit = Tmp % 100;
// Now divide it for the next iteration
Tmp /= 100;
uint8_t UpperNibble = Digit / 10;
uint8_t LowerNibble = Digit % 10;
// Now store the BCD
BCD[i] = (UpperNibble << 4) | LowerNibble;
}
// Set negative flag once converted to x87
BCD[9] = Negative ? 0x80 : 0;
memcpy(GDP, BCD, 10);
}
#undef DEF_OP
void InterpreterOps::RegisterF80Handlers() {
#define REGISTER_OP(op, x) OpHandlers[FEXCore::IR::IROps::OP_##op] = &InterpreterOps::Op_##x
REGISTER_OP(F80LOADFCW, F80LOADFCW);
REGISTER_OP(F80ADD, F80ADD);
REGISTER_OP(F80SUB, F80SUB);
REGISTER_OP(F80MUL, F80MUL);
REGISTER_OP(F80DIV, F80DIV);
REGISTER_OP(F80FYL2X, F80FYL2X);
REGISTER_OP(F80ATAN, F80ATAN);
REGISTER_OP(F80FPREM1, F80FPREM1);
REGISTER_OP(F80FPREM, F80FPREM);
REGISTER_OP(F80SCALE, F80SCALE);
REGISTER_OP(F80CVT, F80CVT);
REGISTER_OP(F80CVTINT, F80CVTINT);
REGISTER_OP(F80CVTTO, F80CVTTO);
REGISTER_OP(F80CVTTOINT, F80CVTTOINT);
REGISTER_OP(F80ROUND, F80ROUND);
REGISTER_OP(F80F2XM1, F80F2XM1);
REGISTER_OP(F80TAN, F80TAN);
REGISTER_OP(F80SQRT, F80SQRT);
REGISTER_OP(F80SIN, F80SIN);
REGISTER_OP(F80COS, F80COS);
REGISTER_OP(F80XTRACT_EXP, F80XTRACT_EXP);
REGISTER_OP(F80XTRACT_SIG, F80XTRACT_SIG);
REGISTER_OP(F80CMP, F80CMP);
REGISTER_OP(F80BCDLOAD, F80BCDLOAD);
REGISTER_OP(F80BCDSTORE, F80BCDSTORE);
#undef REGISTER_OP
}
}
@@ -0,0 +1,330 @@
#pragma once
#include "Common/SoftFloat.h"
#include "Common/SoftFloat-3e/softfloat.h"
#include <FEXCore/IR/IR.h>
namespace FEXCore::CPU {
template<IR::IROps Op>
struct OpHandlers {
};
template<>
struct OpHandlers<IR::OP_F80CVTTO> {
static X80SoftFloat handle4(float src) {
return src;
}
static X80SoftFloat handle8(double src) {
return src;
}
};
template<>
struct OpHandlers<IR::OP_F80CMP> {
template<uint32_t Flags>
static uint64_t handle(X80SoftFloat Src1, X80SoftFloat Src2) {
bool eq, lt, nan;
uint64_t ResultFlags = 0;
X80SoftFloat::FCMP(Src1, Src2, &eq, &lt, &nan);
if (Flags & (1 << IR::FCMP_FLAG_LT) &&
lt) {
ResultFlags |= (1 << IR::FCMP_FLAG_LT);
}
if (Flags & (1 << IR::FCMP_FLAG_UNORDERED) &&
nan) {
ResultFlags |= (1 << IR::FCMP_FLAG_UNORDERED);
}
if (Flags & (1 << IR::FCMP_FLAG_EQ) &&
eq) {
ResultFlags |= (1 << IR::FCMP_FLAG_EQ);
}
return ResultFlags;
}
};
template<>
struct OpHandlers<IR::OP_F80CVT> {
static float handle4(X80SoftFloat src) {
return src;
}
static double handle8(X80SoftFloat src) {
return src;
}
};
template<>
struct OpHandlers<IR::OP_F80CVTINT> {
static int16_t handle2(X80SoftFloat src) {
return src;
}
static int32_t handle4(X80SoftFloat src) {
return src;
}
static int64_t handle8(X80SoftFloat src) {
return src;
}
static int16_t handle2t(X80SoftFloat src) {
auto rv = extF80_to_i32(src, softfloat_round_minMag, false);
if (rv > INT16_MAX) {
return INT16_MAX;
} else if (rv < INT16_MIN) {
return INT16_MIN;
} else {
return rv;
}
}
static int32_t handle4t(X80SoftFloat src) {
return extF80_to_i32(src, softfloat_round_minMag, false);
}
static int64_t handle8t(X80SoftFloat src) {
return extF80_to_i64(src, softfloat_round_minMag, false);
}
};
template<>
struct OpHandlers<IR::OP_F80CVTTOINT> {
static X80SoftFloat handle2(int16_t src) {
return src;
}
static X80SoftFloat handle4(int32_t src) {
return src;
}
};
template<>
struct OpHandlers<IR::OP_F80ROUND> {
static X80SoftFloat handle(X80SoftFloat Src1) {
return X80SoftFloat::FRNDINT(Src1);
}
};
template<>
struct OpHandlers<IR::OP_F80F2XM1> {
static X80SoftFloat handle(X80SoftFloat Src1) {
return X80SoftFloat::F2XM1(Src1);
}
};
template<>
struct OpHandlers<IR::OP_F80TAN> {
static X80SoftFloat handle(X80SoftFloat Src1) {
return X80SoftFloat::FTAN(Src1);
}
};
template<>
struct OpHandlers<IR::OP_F80SQRT> {
static X80SoftFloat handle(X80SoftFloat Src1) {
return X80SoftFloat::FSQRT(Src1);
}
};
template<>
struct OpHandlers<IR::OP_F80SIN> {
static X80SoftFloat handle(X80SoftFloat Src1) {
return X80SoftFloat::FSIN(Src1);
}
};
template<>
struct OpHandlers<IR::OP_F80COS> {
static X80SoftFloat handle(X80SoftFloat Src1) {
return X80SoftFloat::FCOS(Src1);
}
};
template<>
struct OpHandlers<IR::OP_F80XTRACT_EXP> {
static X80SoftFloat handle(X80SoftFloat Src1) {
return X80SoftFloat::FXTRACT_EXP(Src1);
}
};
template<>
struct OpHandlers<IR::OP_F80XTRACT_SIG> {
static X80SoftFloat handle(X80SoftFloat Src1) {
return X80SoftFloat::FXTRACT_SIG(Src1);
}
};
template<>
struct OpHandlers<IR::OP_F80ADD> {
static X80SoftFloat handle(X80SoftFloat Src1, X80SoftFloat Src2) {
return X80SoftFloat::FADD(Src1, Src2);
}
};
template<>
struct OpHandlers<IR::OP_F80SUB> {
static X80SoftFloat handle(X80SoftFloat Src1, X80SoftFloat Src2) {
return X80SoftFloat::FSUB(Src1, Src2);
}
};
template<>
struct OpHandlers<IR::OP_F80MUL> {
static X80SoftFloat handle(X80SoftFloat Src1, X80SoftFloat Src2) {
return X80SoftFloat::FMUL(Src1, Src2);
}
};
template<>
struct OpHandlers<IR::OP_F80DIV> {
static X80SoftFloat handle(X80SoftFloat Src1, X80SoftFloat Src2) {
return X80SoftFloat::FDIV(Src1, Src2);
}
};
template<>
struct OpHandlers<IR::OP_F80FYL2X> {
static X80SoftFloat handle(X80SoftFloat Src1, X80SoftFloat Src2) {
return X80SoftFloat::FYL2X(Src1, Src2);
}
};
template<>
struct OpHandlers<IR::OP_F80ATAN> {
static X80SoftFloat handle(X80SoftFloat Src1, X80SoftFloat Src2) {
return X80SoftFloat::FATAN(Src1, Src2);
}
};
template<>
struct OpHandlers<IR::OP_F80FPREM1> {
static X80SoftFloat handle(X80SoftFloat Src1, X80SoftFloat Src2) {
return X80SoftFloat::FREM1(Src1, Src2);
}
};
template<>
struct OpHandlers<IR::OP_F80FPREM> {
static X80SoftFloat handle(X80SoftFloat Src1, X80SoftFloat Src2) {
return X80SoftFloat::FREM(Src1, Src2);
}
};
template<>
struct OpHandlers<IR::OP_F80SCALE> {
static X80SoftFloat handle(X80SoftFloat Src1, X80SoftFloat Src2) {
return X80SoftFloat::FSCALE(Src1, Src2);
}
};
template<>
struct OpHandlers<IR::OP_F80BCDSTORE> {
static X80SoftFloat handle(X80SoftFloat Src1) {
bool Negative = Src1.Sign;
// Clear the Sign bit
Src1.Sign = 0;
uint64_t Tmp = Src1;
X80SoftFloat Rv;
uint8_t *BCD = reinterpret_cast<uint8_t*>(&Rv);
memset(BCD, 0, 10);
for (size_t i = 0; i < 9; ++i) {
if (Tmp == 0) {
// Nothing left? Just leave
break;
}
// Extract the lower 100 values
uint8_t Digit = Tmp % 100;
// Now divide it for the next iteration
Tmp /= 100;
uint8_t UpperNibble = Digit / 10;
uint8_t LowerNibble = Digit % 10;
// Now store the BCD
BCD[i] = (UpperNibble << 4) | LowerNibble;
}
// Set negative flag once converted to x87
BCD[9] = Negative ? 0x80 : 0;
return Rv;
}
};
template<>
struct OpHandlers<IR::OP_F80BCDLOAD> {
static X80SoftFloat handle(X80SoftFloat Src) {
uint8_t *Src1 = reinterpret_cast<uint8_t *>(&Src);
uint64_t BCD{};
// We walk through each uint8_t and pull out the BCD encoding
// Each 4bit split is a digit
// Only 0-9 is supported, A-F results in undefined data
// | 4 bit | 4 bit |
// | 10s place | 1s place |
// EG 0x48 = 48
// EG 0x4847 = 4847
// This gives us an 18digit value encoded in BCD
// The last byte lets us know if it negative or not
for (size_t i = 0; i < 9; ++i) {
uint8_t Digit = Src1[8 - i];
// First shift our last value over
BCD *= 100;
// Add the tens place digit
BCD += (Digit >> 4) * 10;
// Add the ones place digit
BCD += Digit & 0xF;
}
// Set negative flag once converted to x87
bool Negative = Src1[9] & 0x80;
X80SoftFloat Tmp;
Tmp = BCD;
Tmp.Sign = Negative;
return Tmp;
}
};
template<>
struct OpHandlers<IR::OP_F80LOADFCW> {
static void handle(uint16_t NewFCW) {
auto PC = (NewFCW >> 8) & 3;
switch(PC) {
case 0: extF80_roundingPrecision = 32; break;
case 2: extF80_roundingPrecision = 64; break;
case 3: extF80_roundingPrecision = 80; break;
case 1: LOGMAN_MSG_A_FMT("Invalid x87 precision mode, {}", PC);
}
auto RC = (NewFCW >> 10) & 3;
switch(RC) {
case 0:
softfloat_roundingMode = softfloat_round_near_even;
break;
case 1:
softfloat_roundingMode = softfloat_round_min;
break;
case 2:
softfloat_roundingMode = softfloat_round_max;
break;
case 3:
softfloat_roundingMode = softfloat_round_minMag;
break;
}
}
};
}
@@ -0,0 +1,27 @@
/*
$info$
tags: backend|interpreter
$end_info$
*/
#include "Interface/Core/Interpreter/InterpreterClass.h"
#include "Interface/Core/Interpreter/InterpreterOps.h"
#include "Interface/Core/Interpreter/InterpreterDefines.h"
#include <cstdint>
namespace FEXCore::CPU {
#define DEF_OP(x) void InterpreterOps::Op_##x(FEXCore::IR::IROp_Header *IROp, IROpData *Data, uint32_t Node)
DEF_OP(GetHostFlag) {
auto Op = IROp->C<IR::IROp_GetHostFlag>();
GD = (*GetSrc<uint64_t*>(Data->SSAData, Op->Header.Args[0]) >> Op->Flag) & 1;
}
#undef DEF_OP
void InterpreterOps::RegisterFlagHandlers() {
#define REGISTER_OP(op, x) OpHandlers[FEXCore::IR::IROps::OP_##op] = &InterpreterOps::Op_##x
REGISTER_OP(GETHOSTFLAG, GetHostFlag);
#undef REGISTER_OP
}
}
@@ -1,6 +1,5 @@
#pragma once
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/InternalThreadState.h"
#include "Interface/Core/Dispatcher/Dispatcher.h"
@@ -21,31 +20,38 @@ using DestMapType = std::vector<uint32_t>;
class InterpreterCore final : public CPUBackend {
public:
explicit InterpreterCore(FEXCore::Context::Context *ctx, FEXCore::Core::InternalThreadState *Thread, bool CompileThread);
std::string GetName() override { return "Interpreter"; }
void *CompileCode(uint64_t Entry, FEXCore::IR::IRListView const *IR, FEXCore::Core::DebugData *DebugData, FEXCore::IR::RegisterAllocationData *RAData) override;
explicit InterpreterCore(FEXCore::Context::Context *ctx,
FEXCore::Core::InternalThreadState *Thread,
bool CompileThread);
void *MapRegion(void* HostPtr, uint64_t, uint64_t) override { return HostPtr; }
[[nodiscard]] std::string GetName() override { return "Interpreter"; }
bool NeedsOpDispatch() override { return true; }
[[nodiscard]] void *CompileCode(uint64_t Entry,
FEXCore::IR::IRListView const *IR,
FEXCore::Core::DebugData *DebugData,
FEXCore::IR::RegisterAllocationData *RAData) override;
[[nodiscard]] void *MapRegion(void* HostPtr, uint64_t, uint64_t) override { return HostPtr; }
[[nodiscard]] bool NeedsOpDispatch() override { return true; }
void CreateAsmDispatch(FEXCore::Context::Context *ctx, FEXCore::Core::InternalThreadState *Thread);
bool HandleSIGBUS(int Signal, void *info, void *ucontext);
static void InitializeInterpreterOpHandlers();
private:
FEXCore::Context::Context *CTX;
FEXCore::Core::InternalThreadState *State;
uint32_t AllocateTmpSpace(size_t Size);
template<typename Res>
Res GetDest(void* SSAData, IR::OrderedNodeWrapper Op);
template<typename Res>
Res GetSrc(void* SSAData, IR::OrderedNodeWrapper Src);
std::unique_ptr<Dispatcher> Dispatcher{};
};
}
template<typename T>
T AtomicCompareAndSwap(T expected, T desired, T *addr);
uint8_t AtomicFetchNeg(uint8_t *Addr);
uint16_t AtomicFetchNeg(uint16_t *Addr);
uint32_t AtomicFetchNeg(uint32_t *Addr);
uint64_t AtomicFetchNeg(uint64_t *Addr);
} // namespace FEXCore::CPU
@@ -1,30 +1,31 @@
#include "Common/MathUtils.h"
#include "Common/SoftFloat.h"
#include "Interface/Context/Context.h"
#include "Interface/Core/ArchHelpers/Arm64.h"
#include "Interface/Core/ArchHelpers/MContext.h"
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/DebugData.h"
#include "Interface/Core/InternalThreadState.h"
#include "Interface/Core/Dispatcher/Dispatcher.h"
#include "Interface/Core/Interpreter/InterpreterClass.h"
#include <FEXCore/Config/Config.h>
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/Core/SignalDelegator.h>
#include <FEXCore/Debug/InternalThreadState.h>
#include <FEXCore/Utils/LogManager.h>
#include <FEXCore/Core/CPUBackend.h>
#include <FEXCore/HLE/SyscallHandler.h>
#include <FEXCore/IR/IR.h>
#include <FEXCore/IR/IntrusiveIRList.h>
#include "Interface/HLE/Thunks/Thunks.h"
#include <atomic>
#include <cmath>
#include <limits>
#include <vector>
#include <memory>
#include <bits/types/stack_t.h>
#include <signal.h>
#include <stdint.h>
#include <unordered_map>
#include <utility>
#include "InterpreterOps.h"
namespace FEXCore::IR {
class IRListView;
class RegisterAllocationData;
}
namespace FEXCore::CPU {
class CPUBackend;
static void InterpreterExecution(FEXCore::Core::CpuStateFrame *Frame) {
auto Thread = Frame->Thread;
@@ -34,58 +35,27 @@ static void InterpreterExecution(FEXCore::Core::CpuStateFrame *Frame) {
InterpreterOps::InterpretIR(Thread, Thread->CurrentFrame->State.rip, LocalEntry->second.IR.get(), LocalEntry->second.DebugData.get());
}
bool InterpreterCore::HandleSIGBUS(int Signal, void *info, void *ucontext) {
#ifdef _M_ARM_64
constexpr bool is_arm64 = true;
#else
constexpr bool is_arm64 = false;
#endif
if constexpr (is_arm64) {
uint32_t *PC = reinterpret_cast<uint32_t*>(ArchHelpers::Context::GetPc(ucontext));
uint32_t Instr = PC[0];
if ((Instr & FEXCore::ArchHelpers::Arm64::CASPAL_MASK) == FEXCore::ArchHelpers::Arm64::CASPAL_INST) { // CASPAL
if (FEXCore::ArchHelpers::Arm64::HandleCASPAL(ucontext, info, Instr)) {
// Skip this instruction now
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) + 4);
return true;
}
else {
LogMan::Msg::E("Unhandled JIT SIGBUS CASPAL: PC: %p Instruction: 0x%08x\n", PC, PC[0]);
return false;
}
}
else if ((Instr & FEXCore::ArchHelpers::Arm64::CASAL_MASK) == FEXCore::ArchHelpers::Arm64::CASAL_INST) { // CASAL
if (FEXCore::ArchHelpers::Arm64::HandleCASAL(ucontext, info, Instr)) {
// Skip this instruction now
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) + 4);
return true;
}
else {
LogMan::Msg::E("Unhandled JIT SIGBUS CASAL: PC: %p Instruction: 0x%08x\n", PC, PC[0]);
return false;
}
}
else if ((Instr & FEXCore::ArchHelpers::Arm64::ATOMIC_MEM_MASK) == FEXCore::ArchHelpers::Arm64::ATOMIC_MEM_INST) { // Atomic memory op
if (FEXCore::ArchHelpers::Arm64::HandleAtomicMemOp(ucontext, info, Instr)) {
// Skip this instruction now
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) + 4);
return true;
}
else {
uint8_t Op = (PC[0] >> 12) & 0xF;
LogMan::Msg::E("Unhandled JIT SIGBUS Atomic mem op 0x%02x: PC: %p Instruction: 0x%08x\n", Op, PC, PC[0]);
return false;
}
}
void InitializeInterpreterOpHandlers() {
for (uint32_t i = 0; i <= FEXCore::IR::IROps::OP_LAST; ++i) {
InterpreterOps::OpHandlers[i] = &InterpreterOps::Op_Unhandled;
}
return false;
InterpreterOps::RegisterALUHandlers();
InterpreterOps::RegisterAtomicHandlers();
InterpreterOps::RegisterBranchHandlers();
InterpreterOps::RegisterConversionHandlers();
InterpreterOps::RegisterFlagHandlers();
InterpreterOps::RegisterMemoryHandlers();
InterpreterOps::RegisterMiscHandlers();
InterpreterOps::RegisterMoveHandlers();
InterpreterOps::RegisterVectorHandlers();
InterpreterOps::RegisterEncryptionHandlers();
InterpreterOps::RegisterF80Handlers();
}
InterpreterCore::InterpreterCore(FEXCore::Context::Context *ctx, FEXCore::Core::InternalThreadState *Thread, bool CompileThread)
: CTX {ctx}
, State {Thread} {
// Grab our space for temporary data
if (!CompileThread &&
CTX->Config.Core == FEXCore::Config::CONFIG_INTERPRETER) {
@@ -95,17 +65,19 @@ InterpreterCore::InterpreterCore(FEXCore::Context::Context *ctx, FEXCore::Core::
return Core->Dispatcher->HandleSignalPause(Signal, info, ucontext);
}, true);
#ifdef _M_ARM_64
CTX->SignalDelegation->RegisterHostSignalHandler(SIGBUS, [](FEXCore::Core::InternalThreadState *Thread, int Signal, void *info, void *ucontext) -> bool {
InterpreterCore *Core = reinterpret_cast<InterpreterCore*>(Thread->CPUBackend.get());
return Core->HandleSIGBUS(Signal, info, ucontext);
return FEXCore::ArchHelpers::Arm64::HandleSIGBUS(true, Signal, info, ucontext);
}, true);
#endif
auto GuestSignalHandler = [](FEXCore::Core::InternalThreadState *Thread, int Signal, void *info, void *ucontext, GuestSigAction *GuestAction, stack_t *GuestStack) -> bool {
InterpreterCore *Core = reinterpret_cast<InterpreterCore*>(Thread->CPUBackend.get());
return Core->Dispatcher->HandleGuestSignal(Signal, info, ucontext, GuestAction, GuestStack);
};
for (uint32_t Signal = 0; Signal < SignalDelegator::MAX_SIGNALS; ++Signal) {
for (uint32_t Signal = 0; Signal <= SignalDelegator::MAX_SIGNALS; ++Signal) {
CTX->SignalDelegation->RegisterHostSignalHandlerForGuest(Signal, GuestSignalHandler);
}
}
@@ -13,6 +13,10 @@ namespace FEXCore::Core {
namespace FEXCore::CPU {
class CPUBackend;
std::unique_ptr<CPUBackend> CreateInterpreterCore(FEXCore::Context::Context *ctx, FEXCore::Core::InternalThreadState *Thread, bool CompileThread);
void InitializeInterpreterOpHandlers();
}
[[nodiscard]] std::unique_ptr<CPUBackend> CreateInterpreterCore(FEXCore::Context::Context *ctx,
FEXCore::Core::InternalThreadState *Thread,
bool CompileThread);
} // namespace FEXCore::CPU
@@ -0,0 +1,179 @@
#pragma once
#include <FEXCore/IR/IR.h>
#define GD *GetDest<uint64_t*>(Data->SSAData, Node)
#define GDP GetDest<void*>(Data->SSAData, Node)
#define DO_OP(size, type, func) \
case size: { \
auto *Dst_d = reinterpret_cast<type*>(GDP); \
auto *Src1_d = reinterpret_cast<type*>(Src1); \
auto *Src2_d = reinterpret_cast<type*>(Src2); \
*Dst_d = func(*Src1_d, *Src2_d); \
break; \
}
#define DO_SCALAR_COMPARE_OP(size, type, type2, func) \
case size: { \
auto *Dst_d = reinterpret_cast<type2*>(Tmp); \
auto *Src1_d = reinterpret_cast<type*>(Src1); \
auto *Src2_d = reinterpret_cast<type*>(Src2); \
Dst_d[0] = func(Src1_d[0], Src2_d[0]); \
break; \
}
#define DO_VECTOR_COMPARE_OP(size, type, type2, func) \
case size: { \
auto *Dst_d = reinterpret_cast<type2*>(Tmp); \
auto *Src1_d = reinterpret_cast<type*>(Src1); \
auto *Src2_d = reinterpret_cast<type*>(Src2); \
for (uint8_t i = 0; i < Elements; ++i) { \
Dst_d[i] = func(Src1_d[i], Src2_d[i]); \
} \
break; \
}
#define DO_VECTOR_OP(size, type, func) \
case size: { \
auto *Dst_d = reinterpret_cast<type*>(Tmp); \
auto *Src1_d = reinterpret_cast<type*>(Src1); \
auto *Src2_d = reinterpret_cast<type*>(Src2); \
for (uint8_t i = 0; i < Elements; ++i) { \
Dst_d[i] = func(Src1_d[i], Src2_d[i]); \
} \
break; \
}
#define DO_VECTOR_PAIR_OP(size, type, func) \
case size: { \
auto *Dst_d = reinterpret_cast<type*>(Tmp); \
auto *Src1_d = reinterpret_cast<type*>(Src1); \
auto *Src2_d = reinterpret_cast<type*>(Src2); \
for (uint8_t i = 0; i < Elements; ++i) { \
Dst_d[i] = func(Src1_d[i*2], Src1_d[i*2 + 1]); \
Dst_d[i+Elements] = func(Src2_d[i*2], Src2_d[i*2 + 1]); \
} \
break; \
}
#define DO_VECTOR_SCALAR_OP(size, type, func)\
case size: { \
auto *Dst_d = reinterpret_cast<type*>(Tmp); \
auto *Src1_d = reinterpret_cast<type*>(Src1); \
auto *Src2_d = reinterpret_cast<type*>(Src2); \
for (uint8_t i = 0; i < Elements; ++i) { \
Dst_d[i] = func(Src1_d[i], *Src2_d); \
} \
break; \
}
#define DO_VECTOR_0SRC_OP(size, type, func) \
case size: { \
auto *Dst_d = reinterpret_cast<type*>(Tmp); \
for (uint8_t i = 0; i < Elements; ++i) { \
Dst_d[i] = func(); \
} \
break; \
}
#define DO_VECTOR_1SRC_OP(size, type, func) \
case size: { \
auto *Dst_d = reinterpret_cast<type*>(Tmp); \
auto *Src_d = reinterpret_cast<type*>(Src); \
for (uint8_t i = 0; i < Elements; ++i) { \
Dst_d[i] = func(Src_d[i]); \
} \
break; \
}
#define DO_VECTOR_REDUCE_1SRC_OP(size, type, func, start_val) \
case size: { \
auto *Dst_d = reinterpret_cast<type*>(Tmp); \
auto *Src_d = reinterpret_cast<type*>(Src); \
type begin = start_val; \
for (uint8_t i = 0; i < Elements; ++i) { \
begin = func(begin, Src_d[i]); \
} \
Dst_d[0] = begin; \
break; \
}
#define DO_VECTOR_SAT_OP(size, type, func, min, max) \
case size: { \
auto *Dst_d = reinterpret_cast<type*>(Tmp); \
auto *Src1_d = reinterpret_cast<type*>(Src1); \
auto *Src2_d = reinterpret_cast<type*>(Src2); \
for (uint8_t i = 0; i < Elements; ++i) { \
Dst_d[i] = func(Src1_d[i], Src2_d[i], min, max); \
} \
break; \
}
#define DO_VECTOR_1SRC_2TYPE_OP(size, type, type2, func, min, max) \
case size: { \
auto *Dst_d = reinterpret_cast<type*>(Tmp); \
auto *Src_d = reinterpret_cast<type2*>(Src); \
for (uint8_t i = 0; i < Elements; ++i) { \
Dst_d[i] = (type)func(Src_d[i], min, max); \
} \
break; \
}
#define DO_VECTOR_1SRC_2TYPE_OP_NOSIZE(type, type2, func, min, max) \
auto *Dst_d = reinterpret_cast<type*>(Tmp); \
auto *Src_d = reinterpret_cast<type2*>(Src); \
for (uint8_t i = 0; i < Elements; ++i) { \
Dst_d[i] = (type)func(Src_d[i], min, max); \
}
#define DO_VECTOR_1SRC_2TYPE_OP_TOP(size, type, type2, func, min, max) \
case size: { \
auto *Dst_d = reinterpret_cast<type*>(Tmp); \
auto *Src_d = reinterpret_cast<type2*>(Src2); \
memcpy(Dst_d, Src1, Elements * sizeof(type2));\
for (uint8_t i = 0; i < Elements; ++i) { \
Dst_d[i+Elements] = (type)func(Src_d[i], min, max); \
} \
break; \
}
#define DO_VECTOR_1SRC_2TYPE_OP_TOP_SRC(size, type, type2, func, min, max) \
case size: { \
auto *Dst_d = reinterpret_cast<type*>(Tmp); \
auto *Src_d = reinterpret_cast<type2*>(Src); \
for (uint8_t i = 0; i < Elements; ++i) { \
Dst_d[i] = (type)func(Src_d[i+Elements], min, max); \
} \
break; \
}
#define DO_VECTOR_2SRC_2TYPE_OP(size, type, type2, func) \
case size: { \
auto *Dst_d = reinterpret_cast<type*>(Tmp); \
auto *Src1_d = reinterpret_cast<type2*>(Src1); \
auto *Src2_d = reinterpret_cast<type2*>(Src2); \
for (uint8_t i = 0; i < Elements; ++i) { \
Dst_d[i] = (type)func((type)Src1_d[i], (type)Src2_d[i]); \
} \
break; \
}
#define DO_VECTOR_2SRC_2TYPE_OP_TOP_SRC(size, type, type2, func) \
case size: { \
auto *Dst_d = reinterpret_cast<type*>(Tmp); \
auto *Src1_d = reinterpret_cast<type2*>(Src1); \
auto *Src2_d = reinterpret_cast<type2*>(Src2); \
for (uint8_t i = 0; i < Elements; ++i) { \
Dst_d[i] = (type)func((type)Src1_d[i+Elements], (type)Src2_d[i+Elements]); \
} \
break; \
}
template<typename Res>
Res GetDest(void* SSAData, FEXCore::IR::OrderedNodeWrapper Op) {
auto DstPtr = &reinterpret_cast<__uint128_t*>(SSAData)[Op.ID()];
return reinterpret_cast<Res>(DstPtr);
}
template<typename Res>
Res GetDest(void* SSAData, uint32_t Op) {
auto DstPtr = &reinterpret_cast<__uint128_t*>(SSAData)[Op];
return reinterpret_cast<Res>(DstPtr);
}
template<typename Res>
Res GetSrc(void* SSAData, FEXCore::IR::OrderedNodeWrapper Src) {
auto DstPtr = &reinterpret_cast<__uint128_t*>(SSAData)[Src.ID()];
return reinterpret_cast<Res>(DstPtr);
}
File diff suppressed because it is too large. Load diff
@@ -1,9 +1,16 @@
#pragma once
#include <stdint.h>
#include <FEXCore/IR/IR.h>
#include <FEXCore/IR/IntrusiveIRList.h>
namespace FEXCore::Core {
struct InternalThreadState;
}
namespace FEXCore::IR {
class IRListView;
struct IROp_Header;
}
namespace FEXCore::Core{
@@ -32,11 +39,372 @@ namespace FEXCore::CPU {
FallbackABI ABI;
void *fn;
};
class InterpreterOps {
public:
static void InterpretIR(FEXCore::Core::InternalThreadState *Thread, uint64_t Entry, FEXCore::IR::IRListView *CurrentIR, FEXCore::Core::DebugData *DebugData);
static bool GetFallbackHandler(IR::IROp_Header *IROp, FallbackInfo *Info);
static void RegisterALUHandlers();
static void RegisterAtomicHandlers();
static void RegisterBranchHandlers();
static void RegisterConversionHandlers();
static void RegisterFlagHandlers();
static void RegisterMemoryHandlers();
static void RegisterMiscHandlers();
static void RegisterMoveHandlers();
static void RegisterVectorHandlers();
static void RegisterEncryptionHandlers();
static void RegisterF80Handlers();
struct IROpData {
FEXCore::Core::InternalThreadState *State{};
uint64_t CurrentEntry{};
FEXCore::IR::IRListView *CurrentIR{};
volatile void *StackEntry{};
void *SSAData{};
struct {
bool Quit;
bool Redo;
} BlockResults{};
IR::NodeIterator BlockIterator{0, 0};
};
using OpHandler = std::function<void(FEXCore::IR::IROp_Header *IROp, IROpData *Data, uint32_t Node)>;
static std::array<OpHandler, FEXCore::IR::IROps::OP_LAST + 1> OpHandlers;
#define DEF_OP(x) static void Op_##x(FEXCore::IR::IROp_Header *IROp, IROpData *Data, uint32_t Node)
///< Unhandled handler
DEF_OP(Unhandled);
///< No-op Handler
DEF_OP(NoOp);
///< ALU Ops
DEF_OP(TruncElementPair);
DEF_OP(Constant);
DEF_OP(EntrypointOffset);
DEF_OP(InlineConstant);
DEF_OP(InlineEntrypointOffset);
DEF_OP(CycleCounter);
DEF_OP(Add);
DEF_OP(Sub);
DEF_OP(Neg);
DEF_OP(Mul);
DEF_OP(UMul);
DEF_OP(Div);
DEF_OP(UDiv);
DEF_OP(Rem);
DEF_OP(URem);
DEF_OP(MulH);
DEF_OP(UMulH);
DEF_OP(Or);
DEF_OP(And);
DEF_OP(Andn);
DEF_OP(Xor);
DEF_OP(Lshl);
DEF_OP(Lshr);
DEF_OP(Ashr);
DEF_OP(Rol);
DEF_OP(Ror);
DEF_OP(Extr);
DEF_OP(LDiv);
DEF_OP(LUDiv);
DEF_OP(LRem);
DEF_OP(LURem);
DEF_OP(Zext);
DEF_OP(Not);
DEF_OP(Popcount);
DEF_OP(FindLSB);
DEF_OP(FindMSB);
DEF_OP(FindTrailingZeros);
DEF_OP(CountLeadingZeroes);
DEF_OP(Rev);
DEF_OP(Bfi);
DEF_OP(Bfe);
DEF_OP(Sbfe);
DEF_OP(Select);
DEF_OP(VExtractToGPR);
DEF_OP(Float_ToGPR_ZU);
DEF_OP(Float_ToGPR_ZS);
DEF_OP(Float_ToGPR_S);
DEF_OP(FCmp);
///< Atomic ops
DEF_OP(CASPair);
DEF_OP(CAS);
DEF_OP(AtomicAdd);
DEF_OP(AtomicSub);
DEF_OP(AtomicAnd);
DEF_OP(AtomicOr);
DEF_OP(AtomicXor);
DEF_OP(AtomicSwap);
DEF_OP(AtomicFetchAdd);
DEF_OP(AtomicFetchSub);
DEF_OP(AtomicFetchAnd);
DEF_OP(AtomicFetchOr);
DEF_OP(AtomicFetchXor);
DEF_OP(AtomicFetchNeg);
///< Branch ops
DEF_OP(GuestCallDirect);
DEF_OP(GuestCallIndirect);
DEF_OP(GuestReturn);
DEF_OP(SignalReturn);
DEF_OP(CallbackReturn);
DEF_OP(ExitFunction);
DEF_OP(Jump);
DEF_OP(CondJump);
DEF_OP(Syscall);
DEF_OP(Thunk);
DEF_OP(ValidateCode);
DEF_OP(RemoveCodeEntry);
DEF_OP(CPUID);
///< Conversion ops
DEF_OP(VInsGPR);
DEF_OP(VCastFromGPR);
DEF_OP(Float_FromGPR_S);
DEF_OP(Float_FToF);
DEF_OP(Vector_SToF);
DEF_OP(Vector_FToZS);
DEF_OP(Vector_FToS);
DEF_OP(Vector_FToF);
DEF_OP(Vector_FToI);
///< Flag ops
DEF_OP(GetHostFlag);
///< Memory ops
DEF_OP(LoadContext);
DEF_OP(StoreContext);
DEF_OP(LoadRegister);
DEF_OP(StoreRegister);
DEF_OP(LoadContextIndexed);
DEF_OP(StoreContextIndexed);
DEF_OP(SpillRegister);
DEF_OP(FillRegister);
DEF_OP(LoadFlag);
DEF_OP(StoreFlag);
DEF_OP(LoadMem);
DEF_OP(StoreMem);
DEF_OP(VLoadMemElement);
DEF_OP(VStoreMemElement);
DEF_OP(CacheLineClear);
///< Misc ops
DEF_OP(EndBlock);
DEF_OP(Fence);
DEF_OP(Break);
DEF_OP(Phi);
DEF_OP(PhiValue);
DEF_OP(Print);
DEF_OP(GetRoundingMode);
DEF_OP(SetRoundingMode);
///< Move ops
DEF_OP(ExtractElementPair);
DEF_OP(CreateElementPair);
DEF_OP(Mov);
///< Vector ops
DEF_OP(VectorZero);
DEF_OP(VectorImm);
DEF_OP(CreateVector2);
DEF_OP(CreateVector4);
DEF_OP(SplatVector);
DEF_OP(VMov);
DEF_OP(VAnd);
DEF_OP(VBic);
DEF_OP(VOr);
DEF_OP(VXor);
DEF_OP(VAdd);
DEF_OP(VSub);
DEF_OP(VUQAdd);
DEF_OP(VUQSub);
DEF_OP(VSQAdd);
DEF_OP(VSQSub);
DEF_OP(VAddP);
DEF_OP(VAddV);
DEF_OP(VUMinV);
DEF_OP(VURAvg);
DEF_OP(VAbs);
DEF_OP(VPopcount);
DEF_OP(VFAdd);
DEF_OP(VFAddP);
DEF_OP(VFSub);
DEF_OP(VFMul);
DEF_OP(VFDiv);
DEF_OP(VFMin);
DEF_OP(VFMax);
DEF_OP(VFRecp);
DEF_OP(VFSqrt);
DEF_OP(VFRSqrt);
DEF_OP(VNeg);
DEF_OP(VFNeg);
DEF_OP(VNot);
DEF_OP(VUMin);
DEF_OP(VSMin);
DEF_OP(VUMax);
DEF_OP(VSMax);
DEF_OP(VZip);
DEF_OP(VUnZip);
DEF_OP(VBSL);
DEF_OP(VCMPEQ);
DEF_OP(VCMPEQZ);
DEF_OP(VCMPGT);
DEF_OP(VCMPGTZ);
DEF_OP(VCMPLTZ);
DEF_OP(VFCMPEQ);
DEF_OP(VFCMPNEQ);
DEF_OP(VFCMPLT);
DEF_OP(VFCMPGT);
DEF_OP(VFCMPLE);
DEF_OP(VFCMPORD);
DEF_OP(VFCMPUNO);
DEF_OP(VUShl);
DEF_OP(VUShr);
DEF_OP(VSShr);
DEF_OP(VUShlS);
DEF_OP(VUShrS);
DEF_OP(VSShrS);
DEF_OP(VInsElement);
DEF_OP(VInsScalarElement);
DEF_OP(VExtractElement);
DEF_OP(VDupElement);
DEF_OP(VExtr);
DEF_OP(VSLI);
DEF_OP(VSRI);
DEF_OP(VUShrI);
DEF_OP(VSShrI);
DEF_OP(VShlI);
DEF_OP(VUShrNI);
DEF_OP(VUShrNI2);
DEF_OP(VBitcast);
DEF_OP(VSXTL);
DEF_OP(VSXTL2);
DEF_OP(VUXTL);
DEF_OP(VUXTL2);
DEF_OP(VSQXTN);
DEF_OP(VSQXTN2);
DEF_OP(VSQXTUN);
DEF_OP(VSQXTUN2);
DEF_OP(VUMul);
DEF_OP(VUMull);
DEF_OP(VSMul);
DEF_OP(VSMull);
DEF_OP(VUMull2);
DEF_OP(VSMull2);
DEF_OP(VUABDL);
DEF_OP(VTBL1);
///< Encryption ops
DEF_OP(AESImc);
DEF_OP(AESEnc);
DEF_OP(AESEncLast);
DEF_OP(AESDec);
DEF_OP(AESDecLast);
DEF_OP(AESKeyGenAssist);
///< F80 ops
DEF_OP(F80LOADFCW);
DEF_OP(F80ADD);
DEF_OP(F80SUB);
DEF_OP(F80MUL);
DEF_OP(F80DIV);
DEF_OP(F80FYL2X);
DEF_OP(F80ATAN);
DEF_OP(F80FPREM1);
DEF_OP(F80FPREM);
DEF_OP(F80SCALE);
DEF_OP(F80CVT);
DEF_OP(F80CVTINT);
DEF_OP(F80CVTTO);
DEF_OP(F80CVTTOINT);
DEF_OP(F80ROUND);
DEF_OP(F80F2XM1);
DEF_OP(F80TAN);
DEF_OP(F80SQRT);
DEF_OP(F80SIN);
DEF_OP(F80COS);
DEF_OP(F80XTRACT_EXP);
DEF_OP(F80XTRACT_SIG);
DEF_OP(F80CMP);
DEF_OP(F80BCDLOAD);
DEF_OP(F80BCDSTORE);
#undef DEF_OP
template<typename unsigned_type, typename signed_type, typename float_type>
[[nodiscard]] static bool IsConditionTrue(uint8_t Cond, uint64_t Src1, uint64_t Src2) {
bool CompResult = false;
switch (Cond) {
case FEXCore::IR::COND_EQ:
CompResult = static_cast<unsigned_type>(Src1) == static_cast<unsigned_type>(Src2);
break;
case FEXCore::IR::COND_NEQ:
CompResult = static_cast<unsigned_type>(Src1) != static_cast<unsigned_type>(Src2);
break;
case FEXCore::IR::COND_SGE:
CompResult = static_cast<signed_type>(Src1) >= static_cast<signed_type>(Src2);
break;
case FEXCore::IR::COND_SLT:
CompResult = static_cast<signed_type>(Src1) < static_cast<signed_type>(Src2);
break;
case FEXCore::IR::COND_SGT:
CompResult = static_cast<signed_type>(Src1) > static_cast<signed_type>(Src2);
break;
case FEXCore::IR::COND_SLE:
CompResult = static_cast<signed_type>(Src1) <= static_cast<signed_type>(Src2);
break;
case FEXCore::IR::COND_UGE:
CompResult = static_cast<unsigned_type>(Src1) >= static_cast<unsigned_type>(Src2);
break;
case FEXCore::IR::COND_ULT:
CompResult = static_cast<unsigned_type>(Src1) < static_cast<unsigned_type>(Src2);
break;
case FEXCore::IR::COND_UGT:
CompResult = static_cast<unsigned_type>(Src1) > static_cast<unsigned_type>(Src2);
break;
case FEXCore::IR::COND_ULE:
CompResult = static_cast<unsigned_type>(Src1) <= static_cast<unsigned_type>(Src2);
break;
case FEXCore::IR::COND_FLU:
CompResult = reinterpret_cast<float_type&>(Src1) < reinterpret_cast<float_type&>(Src2) || (std::isnan(reinterpret_cast<float_type&>(Src1)) || std::isnan(reinterpret_cast<float_type&>(Src2)));
break;
case FEXCore::IR::COND_FGE:
CompResult = reinterpret_cast<float_type&>(Src1) >= reinterpret_cast<float_type&>(Src2) && !(std::isnan(reinterpret_cast<float_type&>(Src1)) || std::isnan(reinterpret_cast<float_type&>(Src2)));
break;
case FEXCore::IR::COND_FLEU:
CompResult = reinterpret_cast<float_type&>(Src1) <= reinterpret_cast<float_type&>(Src2) || (std::isnan(reinterpret_cast<float_type&>(Src1)) || std::isnan(reinterpret_cast<float_type&>(Src2)));
break;
case FEXCore::IR::COND_FGT:
CompResult = reinterpret_cast<float_type&>(Src1) > reinterpret_cast<float_type&>(Src2) && !(std::isnan(reinterpret_cast<float_type&>(Src1)) || std::isnan(reinterpret_cast<float_type&>(Src2)));
break;
case FEXCore::IR::COND_FU:
CompResult = (std::isnan(reinterpret_cast<float_type&>(Src1)) || std::isnan(reinterpret_cast<float_type&>(Src2)));
break;
case FEXCore::IR::COND_FNU:
CompResult = !(std::isnan(reinterpret_cast<float_type&>(Src1)) || std::isnan(reinterpret_cast<float_type&>(Src2)));
break;
case FEXCore::IR::COND_MI:
case FEXCore::IR::COND_PL:
case FEXCore::IR::COND_VS:
case FEXCore::IR::COND_VC:
default:
LOGMAN_MSG_A_FMT("Unsupported compare type");
break;
}
return CompResult;
}
static uint8_t GetOpSize(FEXCore::IR::IRListView *CurrentIR, IR::OrderedNodeWrapper Node) {
auto IROp = CurrentIR->GetOp<FEXCore::IR::IROp_Header>(Node);
return IROp->Size;
}
};
};
};
@@ -0,0 +1,289 @@
/*
$info$
tags: backend|interpreter
$end_info$
*/
#include "Interface/Core/Interpreter/InterpreterClass.h"
#include "Interface/Core/Interpreter/InterpreterOps.h"
#include "Interface/Core/Interpreter/InterpreterDefines.h"
#include <cstdint>
namespace FEXCore::CPU {
static inline void CacheLineFlush(char *Addr) {
#ifdef _M_X86_64
__asm volatile (
"clflush (%[Addr]);"
:: [Addr] "r" (Addr)
: "memory");
#else
__builtin___clear_cache(Addr, Addr+64);
#endif
}
#define DEF_OP(x) void InterpreterOps::Op_##x(FEXCore::IR::IROp_Header *IROp, IROpData *Data, uint32_t Node)
DEF_OP(LoadContext) {
auto Op = IROp->C<IR::IROp_LoadContext>();
uint8_t OpSize = IROp->Size;
uintptr_t ContextPtr = reinterpret_cast<uintptr_t>(Data->State->CurrentFrame);
ContextPtr += Op->Offset;
#define LOAD_CTX(x, y) \
case x: { \
y const *MemData = reinterpret_cast<y const*>(ContextPtr); \
GD = *MemData; \
break; \
}
switch (OpSize) {
LOAD_CTX(1, uint8_t)
LOAD_CTX(2, uint16_t)
LOAD_CTX(4, uint32_t)
LOAD_CTX(8, uint64_t)
case 16: {
void const *MemData = reinterpret_cast<void const*>(ContextPtr);
memcpy(GDP, MemData, OpSize);
break;
}
default: LOGMAN_MSG_A_FMT("Unhandled LoadContext size: {}", OpSize);
}
#undef LOAD_CTX
}
DEF_OP(StoreContext) {
auto Op = IROp->C<IR::IROp_StoreContext>();
uint8_t OpSize = IROp->Size;
uintptr_t ContextPtr = reinterpret_cast<uintptr_t>(Data->State->CurrentFrame);
ContextPtr += Op->Offset;
void *MemData = reinterpret_cast<void*>(ContextPtr);
void *Src = GetSrc<void*>(Data->SSAData, Op->Header.Args[0]);
memcpy(MemData, Src, OpSize);
}
DEF_OP(LoadRegister) {
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(StoreRegister) {
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(LoadContextIndexed) {
auto Op = IROp->C<IR::IROp_LoadContextIndexed>();
uint64_t Index = *GetSrc<uint64_t*>(Data->SSAData, Op->Header.Args[0]);
uintptr_t ContextPtr = reinterpret_cast<uintptr_t>(Data->State->CurrentFrame);
ContextPtr += Op->BaseOffset;
ContextPtr += Index * Op->Stride;
#define LOAD_CTX(x, y) \
case x: { \
y const *MemData = reinterpret_cast<y const*>(ContextPtr); \
GD = *MemData; \
break; \
}
switch (IROp->Size) {
LOAD_CTX(1, uint8_t)
LOAD_CTX(2, uint16_t)
LOAD_CTX(4, uint32_t)
LOAD_CTX(8, uint64_t)
case 16: {
void const *MemData = reinterpret_cast<void const*>(ContextPtr);
memcpy(GDP, MemData, IROp->Size);
break;
}
default: LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed size: {}", IROp->Size);
}
#undef LOAD_CTX
}
DEF_OP(StoreContextIndexed) {
auto Op = IROp->C<IR::IROp_StoreContextIndexed>();
uint64_t Index = *GetSrc<uint64_t*>(Data->SSAData, Op->Header.Args[1]);
uintptr_t ContextPtr = reinterpret_cast<uintptr_t>(Data->State->CurrentFrame);
ContextPtr += Op->BaseOffset;
ContextPtr += Index * Op->Stride;
void *MemData = reinterpret_cast<void*>(ContextPtr);
void *Src = GetSrc<void*>(Data->SSAData, Op->Header.Args[0]);
memcpy(MemData, Src, IROp->Size);
}
DEF_OP(SpillRegister) {
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(FillRegister) {
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(LoadFlag) {
auto Op = IROp->C<IR::IROp_LoadFlag>();
uintptr_t ContextPtr = reinterpret_cast<uintptr_t>(Data->State->CurrentFrame);
ContextPtr += offsetof(FEXCore::Core::CPUState, flags[0]);
ContextPtr += Op->Flag;
uint8_t const *MemData = reinterpret_cast<uint8_t const*>(ContextPtr);
GD = *MemData;
}
DEF_OP(StoreFlag) {
auto Op = IROp->C<IR::IROp_StoreFlag>();
uint8_t Arg = *GetSrc<uint8_t*>(Data->SSAData, Op->Header.Args[0]);
uintptr_t ContextPtr = reinterpret_cast<uintptr_t>(Data->State->CurrentFrame);
ContextPtr += offsetof(FEXCore::Core::CPUState, flags[0]);
ContextPtr += Op->Flag;
uint8_t *MemData = reinterpret_cast<uint8_t*>(ContextPtr);
*MemData = Arg;
}
DEF_OP(LoadMem) {
auto Op = IROp->C<IR::IROp_LoadMem>();
uint8_t OpSize = IROp->Size;
uint8_t const *MemData = *GetSrc<uint8_t const**>(Data->SSAData, Op->Addr);
if (!Op->Offset.IsInvalid()) {
auto Offset = *GetSrc<uintptr_t const*>(Data->SSAData, Op->Offset) * Op->OffsetScale;
switch(Op->OffsetType.Val) {
case IR::MEM_OFFSET_SXTX.Val: MemData += Offset; break;
case IR::MEM_OFFSET_UXTW.Val: MemData += (uint32_t)Offset; break;
case IR::MEM_OFFSET_SXTW.Val: MemData += (int32_t)Offset; break;
}
}
memset(GDP, 0, 16);
switch (OpSize) {
case 1: {
auto D = reinterpret_cast<const std::atomic<uint8_t>*>(MemData);
GD = D->load();
break;
}
case 2: {
auto D = reinterpret_cast<const std::atomic<uint16_t>*>(MemData);
GD = D->load();
break;
}
case 4: {
auto D = reinterpret_cast<const std::atomic<uint32_t>*>(MemData);
GD = D->load();
break;
}
case 8: {
auto D = reinterpret_cast<const std::atomic<uint64_t>*>(MemData);
GD = D->load();
break;
}
default:
memcpy(GDP, MemData, IROp->Size);
break;
}
}
DEF_OP(StoreMem) {
auto Op = IROp->C<IR::IROp_StoreMem>();
uint8_t OpSize = IROp->Size;
uint8_t *MemData = *GetSrc<uint8_t **>(Data->SSAData, Op->Addr);
if (!Op->Offset.IsInvalid()) {
auto Offset = *GetSrc<uintptr_t const*>(Data->SSAData, Op->Offset) * Op->OffsetScale;
switch(Op->OffsetType.Val) {
case IR::MEM_OFFSET_SXTX.Val: MemData += Offset; break;
case IR::MEM_OFFSET_UXTW.Val: MemData += (uint32_t)Offset; break;
case IR::MEM_OFFSET_SXTW.Val: MemData += (int32_t)Offset; break;
}
}
switch (OpSize) {
case 1: {
reinterpret_cast<std::atomic<uint8_t>*>(MemData)->store(*GetSrc<uint8_t*>(Data->SSAData, Op->Value));
break;
}
case 2: {
reinterpret_cast<std::atomic<uint16_t>*>(MemData)->store(*GetSrc<uint16_t*>(Data->SSAData, Op->Value));
break;
}
case 4: {
reinterpret_cast<std::atomic<uint32_t>*>(MemData)->store(*GetSrc<uint32_t*>(Data->SSAData, Op->Value));
break;
}
case 8: {
reinterpret_cast<std::atomic<uint64_t>*>(MemData)->store(*GetSrc<uint64_t*>(Data->SSAData, Op->Value));
break;
}
default:
memcpy(MemData, GetSrc<void*>(Data->SSAData, Op->Value), IROp->Size);
break;
}
}
DEF_OP(VLoadMemElement) {
auto Op = IROp->C<IR::IROp_VLoadMemElement>();
void const *MemData = *GetSrc<void const**>(Data->SSAData, Op->Header.Args[0]);
memcpy(GDP, GetSrc<void*>(Data->SSAData, Op->Header.Args[1]), 16);
memcpy(reinterpret_cast<void*>(reinterpret_cast<uintptr_t>(GDP) + (Op->Header.ElementSize * Op->Index)),
MemData, Op->Header.ElementSize);
}
DEF_OP(VStoreMemElement) {
#define STORE_DATA(x, y) \
case x: { \
y *MemData = *GetSrc<y**>(Data->SSAData, Op->Header.Args[0]); \
memcpy(MemData, &GetSrc<y*>(Data->SSAData, Op->Header.Args[1])[Op->Index], sizeof(y)); \
break; \
}
auto Op = IROp->C<IR::IROp_VStoreMemElement>();
uint8_t OpSize = IROp->Size;
switch (OpSize) {
STORE_DATA(1, uint8_t)
STORE_DATA(2, uint16_t)
STORE_DATA(4, uint32_t)
STORE_DATA(8, uint64_t)
default: LOGMAN_MSG_A_FMT("Unhandled StoreMem size"); break;
}
#undef STORE_DATA
}
DEF_OP(CacheLineClear) {
auto Op = IROp->C<IR::IROp_CacheLineClear>();
char *MemData = *GetSrc<char **>(Data->SSAData, Op->Addr);
// 64-byte cache line clear
CacheLineFlush(MemData);
}
#undef DEF_OP
void InterpreterOps::RegisterMemoryHandlers() {
#define REGISTER_OP(op, x) OpHandlers[FEXCore::IR::IROps::OP_##op] = &InterpreterOps::Op_##x
REGISTER_OP(LOADCONTEXT, LoadContext);
REGISTER_OP(STORECONTEXT, StoreContext);
REGISTER_OP(LOADREGISTER, LoadRegister);
REGISTER_OP(STOREREGISTER, StoreRegister);
REGISTER_OP(LOADCONTEXTINDEXED, LoadContextIndexed);
REGISTER_OP(STORECONTEXTINDEXED, StoreContextIndexed);
REGISTER_OP(SPILLREGISTER, SpillRegister);
REGISTER_OP(FILLREGISTER, FillRegister);
REGISTER_OP(LOADFLAG, LoadFlag);
REGISTER_OP(STOREFLAG, StoreFlag);
REGISTER_OP(LOADMEM, LoadMem);
REGISTER_OP(STOREMEM, StoreMem);
REGISTER_OP(LOADMEMTSO, LoadMem);
REGISTER_OP(STOREMEMTSO, StoreMem);
REGISTER_OP(VLOADMEMELEMENT, VLoadMemElement);
REGISTER_OP(VSTOREMEMELEMENT, VStoreMemElement);
REGISTER_OP(CACHELINECLEAR, CacheLineClear);
#undef REGISTER_OP
}
}
@@ -0,0 +1,158 @@
/*
$info$
tags: backend|interpreter
$end_info$
*/
#include "Interface/Core/Interpreter/InterpreterClass.h"
#include "Interface/Core/Interpreter/InterpreterOps.h"
#include "Interface/Core/Interpreter/InterpreterDefines.h"
#include <cstdint>
#ifdef _M_X86_64
#include <xmmintrin.h>
#endif
namespace FEXCore::CPU {
[[noreturn]]
static void StopThread(FEXCore::Core::InternalThreadState *Thread) {
Thread->CTX->StopThread(Thread);
LOGMAN_MSG_A_FMT("unreachable");
FEX_UNREACHABLE;
}
#define DEF_OP(x) void InterpreterOps::Op_##x(FEXCore::IR::IROp_Header *IROp, IROpData *Data, uint32_t Node)
DEF_OP(Fence) {
auto Op = IROp->C<IR::IROp_Fence>();
switch (Op->Fence) {
case IR::Fence_Load.Val:
std::atomic_thread_fence(std::memory_order_acquire);
break;
case IR::Fence_LoadStore.Val:
std::atomic_thread_fence(std::memory_order_seq_cst);
break;
case IR::Fence_Store.Val:
std::atomic_thread_fence(std::memory_order_release);
break;
default: LOGMAN_MSG_A_FMT("Unknown Fence: {}", Op->Fence); break;
}
}
DEF_OP(Break) {
auto Op = IROp->C<IR::IROp_Break>();
switch (Op->Reason) {
case 4: // HLT
StopThread(Data->State);
break;
default: LOGMAN_MSG_A_FMT("Unknown Break Reason: {}", Op->Reason); break;
}
}
DEF_OP(GetRoundingMode) {
uint32_t GuestRounding{};
#ifdef _M_ARM_64
uint64_t Tmp{};
__asm(R"(
mrs %[Tmp], FPCR;
)"
: [Tmp] "=r" (Tmp));
// Extract the rounding
// On ARM the ordering is different than on x86
GuestRounding |= ((Tmp >> 24) & 1) ? IR::ROUND_MODE_FLUSH_TO_ZERO : 0;
uint8_t RoundingMode = (Tmp >> 22) & 0b11;
if (RoundingMode == 0)
GuestRounding |= IR::ROUND_MODE_NEAREST;
else if (RoundingMode == 1)
GuestRounding |= IR::ROUND_MODE_POSITIVE_INFINITY;
else if (RoundingMode == 2)
GuestRounding |= IR::ROUND_MODE_NEGATIVE_INFINITY;
else if (RoundingMode == 3)
GuestRounding |= IR::ROUND_MODE_TOWARDS_ZERO;
#else
GuestRounding = _mm_getcsr();
// Extract the rounding
GuestRounding = (GuestRounding >> 13) & 0b111;
#endif
memcpy(GDP, &GuestRounding, sizeof(GuestRounding));
}
DEF_OP(SetRoundingMode) {
auto Op = IROp->C<IR::IROp_SetRoundingMode>();
uint8_t GuestRounding = *GetSrc<uint8_t*>(Data->SSAData, Op->Header.Args[0]);
#ifdef _M_ARM_64
uint64_t HostRounding{};
__asm volatile(R"(
mrs %[Tmp], FPCR;
)"
: [Tmp] "=r" (HostRounding));
// Mask out the rounding
HostRounding &= ~(0b111 << 22);
HostRounding |= (GuestRounding & IR::ROUND_MODE_FLUSH_TO_ZERO) ? (1U << 24) : 0;
uint8_t RoundingMode = GuestRounding & 0b11;
if (RoundingMode == IR::ROUND_MODE_NEAREST)
HostRounding |= (0b00U << 22);
else if (RoundingMode == IR::ROUND_MODE_POSITIVE_INFINITY)
HostRounding |= (0b01U << 22);
else if (RoundingMode == IR::ROUND_MODE_NEGATIVE_INFINITY)
HostRounding |= (0b10U << 22);
else if (RoundingMode == IR::ROUND_MODE_TOWARDS_ZERO)
HostRounding |= (0b11U << 22);
__asm volatile(R"(
msr FPCR, %[Tmp];
)"
:: [Tmp] "r" (HostRounding));
#else
uint32_t HostRounding = _mm_getcsr();
// Cut out the host rounding mode
HostRounding &= ~(0b111 << 13);
// Insert our new rounding mode
HostRounding |= GuestRounding << 13;
_mm_setcsr(HostRounding);
#endif
}
DEF_OP(Print) {
auto Op = IROp->C<IR::IROp_Print>();
uint8_t OpSize = IROp->Size;
if (OpSize <= 8) {
uint64_t Src = *GetSrc<uint64_t*>(Data->SSAData, Op->Header.Args[0]);
LogMan::Msg::IFmt(">>>> Value in Arg: 0x{:x}, {}", Src, Src);
}
else if (OpSize == 16) {
__uint128_t Src = *GetSrc<__uint128_t*>(Data->SSAData, Op->Header.Args[0]);
uint64_t Src0 = Src;
uint64_t Src1 = Src >> 64;
LogMan::Msg::IFmt(">>>> Value[0] in Arg: 0x{:x}, {}", Src0, Src0);
LogMan::Msg::IFmt(" Value[1] in Arg: 0x{:x}, {}", Src1, Src1);
}
else
LOGMAN_MSG_A_FMT("Unknown value size: {}", OpSize);
}
#undef DEF_OP
void InterpreterOps::RegisterMiscHandlers() {
#define REGISTER_OP(op, x) OpHandlers[FEXCore::IR::IROps::OP_##op] = &InterpreterOps::Op_##x
REGISTER_OP(DUMMY, NoOp);
REGISTER_OP(IRHEADER, NoOp);
REGISTER_OP(CODEBLOCK, NoOp);
REGISTER_OP(BEGINBLOCK, NoOp);
REGISTER_OP(ENDBLOCK, NoOp);
REGISTER_OP(FENCE, Fence);
REGISTER_OP(BREAK, Break);
REGISTER_OP(PHI, NoOp);
REGISTER_OP(PHIVALUE, NoOp);
REGISTER_OP(PRINT, Print);
REGISTER_OP(GETROUNDINGMODE, GetRoundingMode);
REGISTER_OP(SETROUNDINGMODE, SetRoundingMode);
REGISTER_OP(INVALIDATEFLAGS, NoOp);
#undef REGISTER_OP
}
}
@@ -0,0 +1,50 @@
/*
$info$
tags: backend|interpreter
$end_info$
*/
#include "Interface/Core/Interpreter/InterpreterClass.h"
#include "Interface/Core/Interpreter/InterpreterOps.h"
#include "Interface/Core/Interpreter/InterpreterDefines.h"
#include <cstdint>
namespace FEXCore::CPU {
#define DEF_OP(x) void InterpreterOps::Op_##x(FEXCore::IR::IROp_Header *IROp, IROpData *Data, uint32_t Node)
DEF_OP(ExtractElementPair) {
auto Op = IROp->C<IR::IROp_ExtractElementPair>();
uintptr_t Src = GetSrc<uintptr_t>(Data->SSAData, Op->Header.Args[0]);
memcpy(GDP,
reinterpret_cast<void*>(Src + Op->Header.Size * Op->Element), Op->Header.Size);
}
DEF_OP(CreateElementPair) {
auto Op = IROp->C<IR::IROp_CreateElementPair>();
void *Src_Lower = GetSrc<void*>(Data->SSAData, Op->Header.Args[0]);
void *Src_Upper = GetSrc<void*>(Data->SSAData, Op->Header.Args[1]);
uint8_t *Dst = GetDest<uint8_t*>(Data->SSAData, Node);
memcpy(Dst, Src_Lower, Op->Header.Size);
memcpy(Dst + Op->Header.Size, Src_Upper, Op->Header.Size);
}
DEF_OP(Mov) {
auto Op = IROp->C<IR::IROp_Mov>();
uint8_t OpSize = IROp->Size;
memcpy(GDP, GetSrc<void*>(Data->SSAData, Op->Header.Args[0]), OpSize);
}
#undef DEF_OP
void InterpreterOps::RegisterMoveHandlers() {
#define REGISTER_OP(op, x) OpHandlers[FEXCore::IR::IROps::OP_##op] = &InterpreterOps::Op_##x
REGISTER_OP(EXTRACTELEMENTPAIR, ExtractElementPair);
REGISTER_OP(CREATEELEMENTPAIR, CreateElementPair);
REGISTER_OP(MOV, Mov);
#undef REGISTER_OP
}
}
File diff suppressed because it is too large. Load diff
+50 -36
View File
@@ -46,7 +46,7 @@ DEF_OP(TruncElementPair) {
mov(Dst.second, Src.second);
break;
}
default: LOGMAN_MSG_A("Unhandled Truncation size: %d", Op->Size); break;
default: LOGMAN_MSG_A_FMT("Unhandled Truncation size: {}", Op->Size); break;
}
}
@@ -95,7 +95,7 @@ DEF_OP(Add) {
case 8:
add(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()), Const);
break;
default: LOGMAN_MSG_A("Unsupported Add size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported Add size: {}", OpSize);
}
} else {
switch (OpSize) {
@@ -105,7 +105,7 @@ DEF_OP(Add) {
case 8:
add(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()), GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unsupported Add size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported Add size: {}", OpSize);
}
}
}
@@ -121,7 +121,7 @@ DEF_OP(Sub) {
case 8:
sub(GRS(Node), GRS(Op->Header.Args[0].ID()), Const);
break;
default: LOGMAN_MSG_A("Unsupported Sub size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported Sub size: {}", OpSize);
}
} else {
switch (OpSize) {
@@ -131,7 +131,7 @@ DEF_OP(Sub) {
case 8:
sub(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()), GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unsupported Sub size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported Sub size: {}", OpSize);
}
}
@@ -147,7 +147,7 @@ DEF_OP(Neg) {
case 8:
neg(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unsupported Not size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported Neg size: {}", OpSize);
}
}
@@ -163,7 +163,7 @@ DEF_OP(Mul) {
case 8:
mul(Dst, GetReg<RA_64>(Op->Header.Args[0].ID()), GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unknown Sext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown Mul size: {}", OpSize);
}
}
@@ -179,7 +179,7 @@ DEF_OP(UMul) {
case 8:
mul(Dst, GetReg<RA_64>(Op->Header.Args[0].ID()), GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unknown Sext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown UMul size: {}", OpSize);
}
}
@@ -216,7 +216,7 @@ DEF_OP(Div) {
sdiv(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()), GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown DIV Size: %d", Size); break;
default: LOGMAN_MSG_A_FMT("Unknown DIV Size: {}", Size); break;
}
}
@@ -243,7 +243,7 @@ DEF_OP(UDiv) {
udiv(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()), GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown UDIV Size: %d", Size); break;
default: LOGMAN_MSG_A_FMT("Unknown UDIV Size: {}", Size); break;
}
}
@@ -290,7 +290,7 @@ DEF_OP(Rem) {
msub(GetReg<RA_64>(Node), TMP1, Divisor, Dividend);
break;
}
default: LOGMAN_MSG_A("Unknown REM Size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown REM Size: {}", OpSize); break;
}
}
@@ -332,7 +332,7 @@ DEF_OP(URem) {
msub(GetReg<RA_64>(Node), TMP1, Divisor, Dividend);
break;
}
default: LOGMAN_MSG_A("Unknown UREM Size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown UREM Size: {}", OpSize); break;
}
}
@@ -349,7 +349,7 @@ DEF_OP(MulH) {
case 8:
smulh(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()), GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unknown Sext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown Sext size: {}", OpSize);
}
}
@@ -366,7 +366,7 @@ DEF_OP(UMulH) {
case 8:
umulh(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()), GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unknown Sext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown Sext size: {}", OpSize);
}
}
@@ -390,6 +390,19 @@ DEF_OP(And) {
}
}
DEF_OP(Andn) {
auto Op = IROp->C<IR::IROp_Andn>();
const auto& Lhs = Op->Header.Args[0];
const auto& Rhs = Op->Header.Args[1];
uint64_t Const{};
if (IsInlineConstant(Rhs, &Const)) {
bic(GRS(Node), GRS(Lhs.ID()), Const);
} else {
bic(GRS(Node), GRS(Lhs.ID()), GRS(Rhs.ID()));
}
}
DEF_OP(Xor) {
auto Op = IROp->C<IR::IROp_Xor>();
uint64_t Const;
@@ -462,7 +475,7 @@ DEF_OP(Ror) {
break;
}
default: LOGMAN_MSG_A("Unhandled ROR size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled ROR size: {}", OpSize);
}
} else {
switch (OpSize) {
@@ -475,7 +488,7 @@ DEF_OP(Ror) {
break;
}
default: LOGMAN_MSG_A("Unhandled ROR size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled ROR size: {}", OpSize);
}
}
}
@@ -494,7 +507,7 @@ DEF_OP(Extr) {
break;
}
default: LOGMAN_MSG_A("Unhandled EXTR size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled EXTR size: {}", OpSize);
}
}
@@ -539,7 +552,7 @@ DEF_OP(LDiv) {
mov(GetReg<RA_64>(Node), x0);
break;
}
default: LOGMAN_MSG_A("Unknown LDIV Size: %d", Size); break;
default: LOGMAN_MSG_A_FMT("Unknown LDIV Size: {}", Size); break;
}
}
@@ -582,7 +595,7 @@ DEF_OP(LUDiv) {
mov(GetReg<RA_64>(Node), x0);
break;
}
default: LOGMAN_MSG_A("Unknown LUDIV Size: %d", Size); break;
default: LOGMAN_MSG_A_FMT("Unknown LUDIV Size: {}", Size); break;
}
}
@@ -635,7 +648,7 @@ DEF_OP(LRem) {
mov(GetReg<RA_64>(Node), x0);
break;
}
default: LOGMAN_MSG_A("Unknown LREM Size: %d", Size); break;
default: LOGMAN_MSG_A_FMT("Unknown LREM Size: {}", Size); break;
}
}
@@ -685,7 +698,7 @@ DEF_OP(LURem) {
mov(GetReg<RA_64>(Node), x0);
break;
}
default: LOGMAN_MSG_A("Unknown LUREM Size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown LUREM Size: {}", OpSize); break;
}
}
@@ -699,7 +712,7 @@ DEF_OP(Not) {
case 8:
mvn(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unsupported Not size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported Not size: {}", OpSize);
}
}
@@ -730,7 +743,7 @@ DEF_OP(Popcount) {
// fmov has zero extended, unused bytes are zero
addv(VTMP1.B(), VTMP1.V8B());
break;
default: LOGMAN_MSG_A("Unsupported Popcount size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported Popcount size: {}", OpSize);
}
auto Dst = GetReg<RA_32>(Node);
@@ -779,7 +792,7 @@ DEF_OP(FindMSB) {
clz(Dst, GetReg<RA_64>(Op->Header.Args[0].ID()));
sub(Dst, TMP1, Dst);
break;
default: LOGMAN_MSG_A("Unknown REV size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown FindMSB size: {}", OpSize); break;
}
}
@@ -800,7 +813,7 @@ DEF_OP(FindTrailingZeros) {
rbit(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()));
clz(GetReg<RA_64>(Node), GetReg<RA_64>(Node));
break;
default: LOGMAN_MSG_A("Unknown size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown FindTrailingZeros size: {}", OpSize); break;
}
}
@@ -819,7 +832,7 @@ DEF_OP(CountLeadingZeroes) {
case 8:
clz(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unknown size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown CountLeadingZeroes size: {}", OpSize); break;
}
}
@@ -837,7 +850,7 @@ DEF_OP(Rev) {
case 8:
rev(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unknown REV size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown REV size: {}", OpSize); break;
}
}
@@ -859,14 +872,14 @@ DEF_OP(Bfi) {
bfi(TMP1, GetReg<RA_64>(Op->Header.Args[1].ID()), Op->lsb, Op->Width);
mov(GetReg<RA_64>(Node), TMP1);
break;
default: LOGMAN_MSG_A("Unknown BFI size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown BFI size: {}", OpSize); break;
}
}
DEF_OP(Bfe) {
auto Op = IROp->C<IR::IROp_Bfe>();
LOGMAN_THROW_A(IROp->Size <= 8, "OpSize is too large for BFE: %d", IROp->Size);
LOGMAN_THROW_A(Op->Width != 0, "Invalid BFE width of 0");
LOGMAN_THROW_A_FMT(IROp->Size <= 8, "OpSize is too large for BFE: {}", IROp->Size);
LOGMAN_THROW_A_FMT(Op->Width != 0, "Invalid BFE width of 0");
auto Dst = GetReg<RA_64>(Node);
ubfx(Dst, GetReg<RA_64>(Op->Header.Args[0].ID()), Op->lsb, Op->Width);
@@ -880,7 +893,7 @@ DEF_OP(Sbfe) {
if (OpSize == 8) {
sbfx(Dst, GetReg<RA_64>(Op->Header.Args[0].ID()), Op->lsb, Op->Width);
} else {
LogMan::Msg::D("Unimplemented Sbfe size");
LogMan::Msg::DFmt("Unimplemented Sbfe size");
}
}
@@ -911,7 +924,7 @@ Condition MapSelectCC(IR::CondClassType Cond) {
case FEXCore::IR::COND_MI:
case FEXCore::IR::COND_PL:
default:
LOGMAN_MSG_A("Unsupported compare type");
LOGMAN_MSG_A_FMT("Unsupported compare type");
return Condition::nv;
}
}
@@ -929,7 +942,7 @@ DEF_OP(Select) {
} else if (IsFPR(Op->Cmp1.ID())) {
fcmp(GRFCMP(Op->Cmp1.ID()), GRFCMP(Op->Cmp2.ID()));
} else {
LOGMAN_MSG_A("Select: Expected GPR or FPR");
LOGMAN_MSG_A_FMT("Select: Expected GPR or FPR");
}
auto cc = MapSelectCC(Op->Cond);
@@ -940,7 +953,7 @@ DEF_OP(Select) {
if (is_const_true || is_const_false) {
if (is_const_false != true || is_const_true != true || const_true != 1 || const_false != 0) {
LOGMAN_MSG_A("Select: Unsupported compare inline parameters");
LOGMAN_MSG_A_FMT("Select: Unsupported compare inline parameters");
}
cset(GRS(Node), cc);
} else {
@@ -964,7 +977,7 @@ DEF_OP(VExtractToGPR) {
case 8:
umov(GetReg<RA_64>(Node), GetSrc(Op->Header.Args[0].ID()).V2D(), Op->Idx);
break;
default: LOGMAN_MSG_A("Unhandled ExtractElementSize: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled ExtractElementSize: {}", OpSize);
}
}
@@ -1027,7 +1040,7 @@ DEF_OP(FCmp) {
bool set = false;
if (Op->Flags & (1 << IR::FCMP_FLAG_EQ)) {
LOGMAN_THROW_A(IR::FCMP_FLAG_EQ == 0, "IR::FCMP_FLAG_EQ must equal 0");
LOGMAN_THROW_A_FMT(IR::FCMP_FLAG_EQ == 0, "IR::FCMP_FLAG_EQ must equal 0");
// EQ or unordered
cset(Dst, Condition::eq); // Z = 1
csinc(Dst, Dst, xzr, Condition::vc); // IF !V ? Z : 1
@@ -1079,6 +1092,7 @@ void Arm64JITCore::RegisterALUHandlers() {
REGISTER_OP(UMULH, UMulH);
REGISTER_OP(OR, Or);
REGISTER_OP(AND, And);
REGISTER_OP(ANDN, Andn);
REGISTER_OP(XOR, Xor);
REGISTER_OP(LSHL, Lshl);
REGISTER_OP(LSHR, Lshr);
+107 -55
View File
@@ -34,7 +34,7 @@ DEF_OP(CASPair) {
mov(Dst.first, TMP3);
mov(Dst.second, TMP4);
break;
default: LOGMAN_MSG_A("Unsupported: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported: {}", OpSize);
}
}
else {
@@ -44,6 +44,7 @@ DEF_OP(CASPair) {
aarch64::Label LoopNotExpected;
aarch64::Label LoopExpected;
bind(&LoopTop);
ldaxp(TMP2.W(), TMP3.W(), MemOperand(MemSrc));
cmp(TMP2.W(), Expected.first.W());
ccmp(TMP3.W(), Expected.second.W(), NoFlag, Condition::eq);
@@ -69,6 +70,7 @@ DEF_OP(CASPair) {
aarch64::Label LoopNotExpected;
aarch64::Label LoopExpected;
bind(&LoopTop);
ldaxp(TMP2.X(), TMP3.X(), MemOperand(MemSrc));
cmp(TMP2.X(), Expected.first.X());
ccmp(TMP3.X(), Expected.second.X(), NoFlag, Condition::eq);
@@ -89,7 +91,7 @@ DEF_OP(CASPair) {
bind(&LoopExpected);
break;
}
default: LOGMAN_MSG_A("Unsupported: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported: {}", OpSize);
}
}
}
@@ -115,7 +117,7 @@ DEF_OP(CAS) {
case 2: casalh(TMP2.W(), Desired.W(), MemOperand(MemSrc)); break;
case 4: casal(TMP2.W(), Desired.W(), MemOperand(MemSrc)); break;
case 8: casal(TMP2.X(), Desired.X(), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unsupported: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported: {}", OpSize);
}
mov(GetReg<RA_64>(Node), TMP2);
}
@@ -206,7 +208,7 @@ DEF_OP(CAS) {
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", OpSize);
}
}
}
@@ -217,17 +219,17 @@ DEF_OP(AtomicAdd) {
auto MemSrc = GetReg<RA_64>(Op->Header.Args[0].ID());
if (SupportsAtomics) {
switch (Op->Size) {
switch (IROp->Size) {
case 1: staddlb(GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
case 2: staddlh(GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
case 4: staddl(GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
case 8: staddl(GetReg<RA_64>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
else {
// TMP2-TMP3
switch (Op->Size) {
switch (IROp->Size) {
case 1: {
aarch64::Label LoopTop;
bind(&LoopTop);
@@ -264,7 +266,7 @@ DEF_OP(AtomicAdd) {
cbnz(TMP2, &LoopTop);
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
}
@@ -276,17 +278,17 @@ DEF_OP(AtomicSub) {
if (SupportsAtomics) {
neg(TMP2, GetReg<RA_64>(Op->Header.Args[1].ID()));
switch (Op->Size) {
switch (IROp->Size) {
case 1: staddlb(TMP2.W(), MemOperand(MemSrc)); break;
case 2: staddlh(TMP2.W(), MemOperand(MemSrc)); break;
case 4: staddl(TMP2.W(), MemOperand(MemSrc)); break;
case 8: staddl(TMP2.X(), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
else {
// TMP2-TMP3
switch (Op->Size) {
switch (IROp->Size) {
case 1: {
aarch64::Label LoopTop;
bind(&LoopTop);
@@ -323,7 +325,7 @@ DEF_OP(AtomicSub) {
cbnz(TMP2, &LoopTop);
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
}
@@ -335,17 +337,17 @@ DEF_OP(AtomicAnd) {
if (SupportsAtomics) {
mvn(TMP2, GetReg<RA_64>(Op->Header.Args[1].ID()));
switch (Op->Size) {
switch (IROp->Size) {
case 1: stclrlb(TMP2.W(), MemOperand(MemSrc)); break;
case 2: stclrlh(TMP2.W(), MemOperand(MemSrc)); break;
case 4: stclrl(TMP2.W(), MemOperand(MemSrc)); break;
case 8: stclrl(TMP2.X(), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
else {
// TMP2-TMP3
switch (Op->Size) {
switch (IROp->Size) {
case 1: {
aarch64::Label LoopTop;
bind(&LoopTop);
@@ -382,7 +384,7 @@ DEF_OP(AtomicAnd) {
cbnz(TMP2, &LoopTop);
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
}
@@ -393,17 +395,17 @@ DEF_OP(AtomicOr) {
auto MemSrc = GetReg<RA_64>(Op->Header.Args[0].ID());
if (SupportsAtomics) {
switch (Op->Size) {
switch (IROp->Size) {
case 1: stsetlb(GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
case 2: stsetlh(GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
case 4: stsetl(GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
case 8: stsetl(GetReg<RA_64>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
else {
// TMP2-TMP3
switch (Op->Size) {
switch (IROp->Size) {
case 1: {
aarch64::Label LoopTop;
bind(&LoopTop);
@@ -440,7 +442,7 @@ DEF_OP(AtomicOr) {
cbnz(TMP2, &LoopTop);
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
}
@@ -451,17 +453,17 @@ DEF_OP(AtomicXor) {
auto MemSrc = GetReg<RA_64>(Op->Header.Args[0].ID());
if (SupportsAtomics) {
switch (Op->Size) {
switch (IROp->Size) {
case 1: steorlb(GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
case 2: steorlh(GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
case 4: steorl(GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
case 8: steorl(GetReg<RA_64>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
else {
// TMP2-TMP3
switch (Op->Size) {
switch (IROp->Size) {
case 1: {
aarch64::Label LoopTop;
bind(&LoopTop);
@@ -498,7 +500,7 @@ DEF_OP(AtomicXor) {
cbnz(TMP2, &LoopTop);
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
}
@@ -510,41 +512,40 @@ DEF_OP(AtomicSwap) {
if (SupportsAtomics) {
mov(TMP2, GetReg<RA_64>(Op->Header.Args[1].ID()));
switch (Op->Size) {
switch (IROp->Size) {
case 1: swplb(TMP2.W(), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 2: swplh(TMP2.W(), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 4: swpl(TMP2.W(), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 8: swpl(TMP2.X(), GetReg<RA_64>(Node), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
else {
// TMP2-TMP3
mov(TMP3, GetReg<RA_64>(Op->Header.Args[1].ID()));
switch (Op->Size) {
switch (IROp->Size) {
case 1: {
aarch64::Label LoopTop;
bind(&LoopTop);
ldaxrb(TMP2.W(), MemOperand(MemSrc));
stlxrb(TMP4.W(), TMP3.W(), MemOperand(MemSrc));
stlxrb(TMP4.W(), GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc));
cbnz(TMP4.W(), &LoopTop);
uxtb(GetReg<RA_64>(Node), TMP2.W());
uxtb(GetReg<RA_32>(Node), TMP2.W());
break;
}
case 2: {
aarch64::Label LoopTop;
bind(&LoopTop);
ldaxrh(TMP2.W(), MemOperand(MemSrc));
stlxrh(TMP4.W(), TMP3.W(), MemOperand(MemSrc));
stlxrh(TMP4.W(), GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc));
cbnz(TMP4.W(), &LoopTop);
uxtw(GetReg<RA_64>(Node), TMP2.W());
uxtw(GetReg<RA_32>(Node), TMP2.W());
break;
}
case 4: {
aarch64::Label LoopTop;
bind(&LoopTop);
ldaxr(TMP2.W(), MemOperand(MemSrc));
stlxr(TMP4.W(), TMP3.W(), MemOperand(MemSrc));
stlxr(TMP4.W(), GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc));
cbnz(TMP4.W(), &LoopTop);
mov(GetReg<RA_32>(Node), TMP2.W());
break;
@@ -553,12 +554,12 @@ DEF_OP(AtomicSwap) {
aarch64::Label LoopTop;
bind(&LoopTop);
ldaxr(TMP2, MemOperand(MemSrc));
stlxr(TMP4, TMP3.X(), MemOperand(MemSrc));
stlxr(TMP4, GetReg<RA_64>(Op->Header.Args[1].ID()), MemOperand(MemSrc));
cbnz(TMP4, &LoopTop);
mov(GetReg<RA_64>(Node), TMP2.X());
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
}
@@ -568,17 +569,17 @@ DEF_OP(AtomicFetchAdd) {
auto MemSrc = GetReg<RA_64>(Op->Header.Args[0].ID());
if (SupportsAtomics) {
switch (Op->Size) {
switch (IROp->Size) {
case 1: ldaddalb(GetReg<RA_32>(Op->Header.Args[1].ID()), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 2: ldaddalh(GetReg<RA_32>(Op->Header.Args[1].ID()), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 4: ldaddal(GetReg<RA_32>(Op->Header.Args[1].ID()), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 8: ldaddal(GetReg<RA_64>(Op->Header.Args[1].ID()), GetReg<RA_64>(Node), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
else {
// TMP2-TMP3
switch (Op->Size) {
switch (IROp->Size) {
case 1: {
aarch64::Label LoopTop;
bind(&LoopTop);
@@ -619,7 +620,7 @@ DEF_OP(AtomicFetchAdd) {
mov(GetReg<RA_64>(Node), TMP2);
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
}
@@ -630,17 +631,17 @@ DEF_OP(AtomicFetchSub) {
if (SupportsAtomics) {
neg(TMP2, GetReg<RA_64>(Op->Header.Args[1].ID()));
switch (Op->Size) {
switch (IROp->Size) {
case 1: ldaddalb(TMP2.W(), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 2: ldaddalh(TMP2.W(), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 4: ldaddal(TMP2.W(), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 8: ldaddal(TMP2.X(), GetReg<RA_64>(Node), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
else {
// TMP2-TMP3
switch (Op->Size) {
switch (IROp->Size) {
case 1: {
aarch64::Label LoopTop;
bind(&LoopTop);
@@ -681,7 +682,7 @@ DEF_OP(AtomicFetchSub) {
mov(GetReg<RA_64>(Node), TMP2);
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
}
@@ -692,17 +693,17 @@ DEF_OP(AtomicFetchAnd) {
if (SupportsAtomics) {
mvn(TMP2, GetReg<RA_64>(Op->Header.Args[1].ID()));
switch (Op->Size) {
switch (IROp->Size) {
case 1: ldclralb(TMP2.W(), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 2: ldclralh(TMP2.W(), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 4: ldclral(TMP2.W(), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 8: ldclral(TMP2.X(), GetReg<RA_64>(Node), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
else {
// TMP2-TMP3
switch (Op->Size) {
switch (IROp->Size) {
case 1: {
aarch64::Label LoopTop;
bind(&LoopTop);
@@ -743,7 +744,7 @@ DEF_OP(AtomicFetchAnd) {
mov(GetReg<RA_64>(Node), TMP2);
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
}
@@ -753,17 +754,17 @@ DEF_OP(AtomicFetchOr) {
auto MemSrc = GetReg<RA_64>(Op->Header.Args[0].ID());
if (SupportsAtomics) {
switch (Op->Size) {
switch (IROp->Size) {
case 1: ldsetalb(GetReg<RA_32>(Op->Header.Args[1].ID()), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 2: ldsetalh(GetReg<RA_32>(Op->Header.Args[1].ID()), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 4: ldsetal(GetReg<RA_32>(Op->Header.Args[1].ID()), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 8: ldsetal(GetReg<RA_64>(Op->Header.Args[1].ID()), GetReg<RA_64>(Node), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
else {
// TMP2-TMP3
switch (Op->Size) {
switch (IROp->Size) {
case 1: {
aarch64::Label LoopTop;
bind(&LoopTop);
@@ -804,7 +805,7 @@ DEF_OP(AtomicFetchOr) {
mov(GetReg<RA_64>(Node), TMP2);
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
}
@@ -814,17 +815,17 @@ DEF_OP(AtomicFetchXor) {
auto MemSrc = GetReg<RA_64>(Op->Header.Args[0].ID());
if (SupportsAtomics) {
switch (Op->Size) {
switch (IROp->Size) {
case 1: ldeoralb(GetReg<RA_32>(Op->Header.Args[1].ID()), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 2: ldeoralh(GetReg<RA_32>(Op->Header.Args[1].ID()), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 4: ldeoral(GetReg<RA_32>(Op->Header.Args[1].ID()), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 8: ldeoral(GetReg<RA_64>(Op->Header.Args[1].ID()), GetReg<RA_64>(Node), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
else {
// TMP2-TMP3
switch (Op->Size) {
switch (IROp->Size) {
case 1: {
aarch64::Label LoopTop;
bind(&LoopTop);
@@ -865,11 +866,61 @@ DEF_OP(AtomicFetchXor) {
mov(GetReg<RA_64>(Node), TMP2);
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
}
DEF_OP(AtomicFetchNeg) {
auto Op = IROp->C<IR::IROp_AtomicFetchNeg>();
auto MemSrc = GetReg<RA_64>(Op->Header.Args[0].ID());
// TMP2-TMP3
switch (IROp->Size) {
case 1: {
aarch64::Label LoopTop;
bind(&LoopTop);
ldaxrb(TMP2.W(), MemOperand(MemSrc));
neg(TMP3.W(), TMP2.W());
stlxrb(TMP4.W(), TMP3.W(), MemOperand(MemSrc));
cbnz(TMP4.W(), &LoopTop);
mov(GetReg<RA_32>(Node), TMP2.W());
break;
}
case 2: {
aarch64::Label LoopTop;
bind(&LoopTop);
ldaxrh(TMP2.W(), MemOperand(MemSrc));
neg(TMP3.W(), TMP2.W());
stlxrh(TMP4.W(), TMP3.W(), MemOperand(MemSrc));
cbnz(TMP4.W(), &LoopTop);
mov(GetReg<RA_32>(Node), TMP2.W());
break;
}
case 4: {
aarch64::Label LoopTop;
bind(&LoopTop);
ldaxr(TMP2.W(), MemOperand(MemSrc));
neg(TMP3.W(), TMP2.W());
stlxr(TMP4.W(), TMP3.W(), MemOperand(MemSrc));
cbnz(TMP4.W(), &LoopTop);
mov(GetReg<RA_32>(Node), TMP2.W());
break;
}
case 8: {
aarch64::Label LoopTop;
bind(&LoopTop);
ldaxr(TMP2, MemOperand(MemSrc));
neg(TMP3, TMP2);
stlxr(TMP4, TMP3, MemOperand(MemSrc));
cbnz(TMP4, &LoopTop);
mov(GetReg<RA_64>(Node), TMP2);
break;
}
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", IROp->Size);
}
}
#undef DEF_OP
void Arm64JITCore::RegisterAtomicHandlers() {
#define REGISTER_OP(op, x) OpHandlers[FEXCore::IR::IROps::OP_##op] = &Arm64JITCore::Op_##x
@@ -886,6 +937,7 @@ void Arm64JITCore::RegisterAtomicHandlers() {
REGISTER_OP(ATOMICFETCHAND, AtomicFetchAnd);
REGISTER_OP(ATOMICFETCHOR, AtomicFetchOr);
REGISTER_OP(ATOMICFETCHXOR, AtomicFetchXor);
REGISTER_OP(ATOMICFETCHNEG, AtomicFetchNeg);
#undef REGISTER_OP
}
}
@@ -4,6 +4,8 @@ tags: backend|arm64
$end_info$
*/
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/JIT/Arm64/JITClass.h"
#include "Interface/Core/InternalThreadState.h"
@@ -16,15 +18,15 @@ using namespace vixl;
using namespace vixl::aarch64;
#define DEF_OP(x) void Arm64JITCore::Op_##x(FEXCore::IR::IROp_Header *IROp, uint32_t Node)
DEF_OP(GuestCallDirect) {
LogMan::Msg::D("Unimplemented");
LogMan::Msg::DFmt("Unimplemented");
}
DEF_OP(GuestCallIndirect) {
LogMan::Msg::D("Unimplemented");
LogMan::Msg::DFmt("Unimplemented");
}
DEF_OP(GuestReturn) {
LogMan::Msg::D("Unimplemented");
LogMan::Msg::DFmt("Unimplemented");
}
DEF_OP(SignalReturn) {
@@ -142,7 +144,7 @@ Condition MapBranchCC(IR::CondClassType Cond) {
case FEXCore::IR::COND_MI:
case FEXCore::IR::COND_PL:
default:
LOGMAN_MSG_A("Unsupported compare type");
LOGMAN_MSG_A_FMT("Unsupported compare type");
return Condition::nv;
}
}
@@ -169,10 +171,10 @@ DEF_OP(CondJump) {
bool isConst = IsInlineConstant(Op->Cmp2, &Const);
if (isConst && Const == 0 && Op->Cond.Val == FEXCore::IR::COND_EQ) {
LOGMAN_THROW_A(IsGPR(Op->Cmp1.ID()), "CondJump: Expected GPR");
LOGMAN_THROW_A_FMT(IsGPR(Op->Cmp1.ID()), "CondJump: Expected GPR");
cbz(GRCMP(Op->Cmp1.ID()), TrueTargetLabel);
} else if (isConst && Const == 0 && Op->Cond.Val == FEXCore::IR::COND_NEQ) {
LOGMAN_THROW_A(IsGPR(Op->Cmp1.ID()), "CondJump: Expected GPR");
LOGMAN_THROW_A_FMT(IsGPR(Op->Cmp1.ID()), "CondJump: Expected GPR");
cbnz(GRCMP(Op->Cmp1.ID()), TrueTargetLabel);
} else {
if (IsGPR(Op->Cmp1.ID())) {
@@ -183,7 +185,7 @@ DEF_OP(CondJump) {
} else if (IsFPR(Op->Cmp1.ID())) {
fcmp(GRFCMP(Op->Cmp1.ID()), GRFCMP(Op->Cmp2.ID()));
} else {
LOGMAN_MSG_A("CondJump: Expected GPR or FPR");
LOGMAN_MSG_A_FMT("CondJump: Expected GPR or FPR");
}
b(TrueTargetLabel, MapBranchCC(Op->Cond));
@@ -31,7 +31,7 @@ DEF_OP(VInsGPR) {
ins(GetDst(Node).V2D(), Op->Index, GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -52,7 +52,7 @@ DEF_OP(VCastFromGPR) {
case 8:
fmov(GetDst(Node).D(), GetReg<RA_64>(Op->Header.Args[0].ID()).X());
break;
default: LOGMAN_MSG_A("Unknown castGPR element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown castGPR element size: {}", Op->Header.ElementSize);
}
}
@@ -91,7 +91,7 @@ DEF_OP(Float_FToF) {
fcvt(GetDst(Node).S(), GetSrc(Op->Header.Args[0].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown FCVT sizes: 0x%x", Conv);
default: LOGMAN_MSG_A_FMT("Unknown FCVT sizes: 0x{:x}", Conv);
}
}
@@ -104,7 +104,7 @@ DEF_OP(Vector_SToF) {
case 8:
scvtf(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
default: LOGMAN_MSG_A("Unknown castGPR element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Vector_SToF element size: {}", Op->Header.ElementSize);
}
}
@@ -117,7 +117,7 @@ DEF_OP(Vector_FToZS) {
case 8:
fcvtzs(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
default: LOGMAN_MSG_A("Unknown castGPR element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Vector_FToZS element size: {}", Op->Header.ElementSize);
}
}
@@ -132,7 +132,7 @@ DEF_OP(Vector_FToS) {
frinti(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
fcvtzs(GetDst(Node).V2D(), GetDst(Node).V2D());
break;
default: LOGMAN_MSG_A("Unknown castGPR element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Vector_FToS element size: {}", Op->Header.ElementSize);
}
}
@@ -149,7 +149,7 @@ DEF_OP(Vector_FToF) {
fcvtn(GetDst(Node).V2S(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Conversion Type : 0%04x", Conv); break;
default: LOGMAN_MSG_A_FMT("Unknown Vector_FToF Type : 0x{:04x}", Conv); break;
}
}
+45 -145
View File
@@ -11,6 +11,7 @@ $end_info$
*/
#include "Interface/Context/Context.h"
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/ArchHelpers/Arm64.h"
#include "Interface/Core/ArchHelpers/MContext.h"
@@ -45,8 +46,7 @@ void Arm64JITCore::Op_Unhandled(FEXCore::IR::IROp_Header *IROp, uint32_t Node) {
FallbackInfo Info;
if (!InterpreterOps::GetFallbackHandler(IROp, &Info)) {
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
auto Name = FEXCore::IR::GetName(IROp->Op);
LOGMAN_MSG_A("Unhandled IR Op: %s", std::string(Name).c_str());
LOGMAN_MSG_A_FMT("Unhandled IR Op: {}", FEXCore::IR::GetName(IROp->Op));
#endif
} else {
switch(Info.ABI) {
@@ -55,7 +55,7 @@ void Arm64JITCore::Op_Unhandled(FEXCore::IR::IROp_Header *IROp, uint32_t Node) {
PushDynamicRegsAndLR();
mov(w0, GetReg<RA_32>(IROp->Args[0].ID()));
uxth(w0, GetReg<RA_32>(IROp->Args[0].ID()));
LoadConstant(x1, (uintptr_t)Info.fn);
blr(x1);
@@ -112,7 +112,12 @@ void Arm64JITCore::Op_Unhandled(FEXCore::IR::IROp_Header *IROp, uint32_t Node) {
PushDynamicRegsAndLR();
mov(w0, GetReg<RA_32>(IROp->Args[0].ID()));
if (Info.ABI == FABI_F80_I16) {
uxth(w0, GetReg<RA_32>(IROp->Args[0].ID()));
}
else {
mov(w0, GetReg<RA_32>(IROp->Args[0].ID()));
}
LoadConstant(x1, (uintptr_t)Info.fn);
blr(x1);
@@ -133,7 +138,7 @@ void Arm64JITCore::Op_Unhandled(FEXCore::IR::IROp_Header *IROp, uint32_t Node) {
PushDynamicRegsAndLR();
umov(x0, GetSrc(IROp->Args[0].ID()).V2D(), 0);
umov(x1, GetSrc(IROp->Args[0].ID()).V2D(), 1);
umov(w1, GetSrc(IROp->Args[0].ID()).V8H(), 4);
LoadConstant(x2, (uintptr_t)Info.fn);
@@ -153,7 +158,7 @@ void Arm64JITCore::Op_Unhandled(FEXCore::IR::IROp_Header *IROp, uint32_t Node) {
PushDynamicRegsAndLR();
umov(x0, GetSrc(IROp->Args[0].ID()).V2D(), 0);
umov(x1, GetSrc(IROp->Args[0].ID()).V2D(), 1);
umov(w1, GetSrc(IROp->Args[0].ID()).V8H(), 4);
LoadConstant(x2, (uintptr_t)Info.fn);
@@ -173,7 +178,7 @@ void Arm64JITCore::Op_Unhandled(FEXCore::IR::IROp_Header *IROp, uint32_t Node) {
PushDynamicRegsAndLR();
umov(x0, GetSrc(IROp->Args[0].ID()).V2D(), 0);
umov(x1, GetSrc(IROp->Args[0].ID()).V2D(), 1);
umov(w1, GetSrc(IROp->Args[0].ID()).V8H(), 4);
LoadConstant(x2, (uintptr_t)Info.fn);
@@ -192,7 +197,7 @@ void Arm64JITCore::Op_Unhandled(FEXCore::IR::IROp_Header *IROp, uint32_t Node) {
PushDynamicRegsAndLR();
umov(x0, GetSrc(IROp->Args[0].ID()).V2D(), 0);
umov(x1, GetSrc(IROp->Args[0].ID()).V2D(), 1);
umov(w1, GetSrc(IROp->Args[0].ID()).V8H(), 4);
LoadConstant(x2, (uintptr_t)Info.fn);
@@ -211,7 +216,7 @@ void Arm64JITCore::Op_Unhandled(FEXCore::IR::IROp_Header *IROp, uint32_t Node) {
PushDynamicRegsAndLR();
umov(x0, GetSrc(IROp->Args[0].ID()).V2D(), 0);
umov(x1, GetSrc(IROp->Args[0].ID()).V2D(), 1);
umov(w1, GetSrc(IROp->Args[0].ID()).V8H(), 4);
LoadConstant(x2, (uintptr_t)Info.fn);
@@ -230,10 +235,10 @@ void Arm64JITCore::Op_Unhandled(FEXCore::IR::IROp_Header *IROp, uint32_t Node) {
PushDynamicRegsAndLR();
umov(x0, GetSrc(IROp->Args[0].ID()).V2D(), 0);
umov(x1, GetSrc(IROp->Args[0].ID()).V2D(), 1);
umov(w1, GetSrc(IROp->Args[0].ID()).V8H(), 4);
umov(x2, GetSrc(IROp->Args[1].ID()).V2D(), 0);
umov(x3, GetSrc(IROp->Args[1].ID()).V2D(), 1);
umov(w3, GetSrc(IROp->Args[1].ID()).V8H(), 4);
LoadConstant(x4, (uintptr_t)Info.fn);
@@ -252,7 +257,7 @@ void Arm64JITCore::Op_Unhandled(FEXCore::IR::IROp_Header *IROp, uint32_t Node) {
PushDynamicRegsAndLR();
umov(x0, GetSrc(IROp->Args[0].ID()).V2D(), 0);
umov(x1, GetSrc(IROp->Args[0].ID()).V2D(), 1);
umov(w1, GetSrc(IROp->Args[0].ID()).V8H(), 4);
LoadConstant(x2, (uintptr_t)Info.fn);
@@ -273,10 +278,10 @@ void Arm64JITCore::Op_Unhandled(FEXCore::IR::IROp_Header *IROp, uint32_t Node) {
PushDynamicRegsAndLR();
umov(x0, GetSrc(IROp->Args[0].ID()).V2D(), 0);
umov(x1, GetSrc(IROp->Args[0].ID()).V2D(), 1);
umov(w1, GetSrc(IROp->Args[0].ID()).V8H(), 4);
umov(x2, GetSrc(IROp->Args[1].ID()).V2D(), 0);
umov(x3, GetSrc(IROp->Args[1].ID()).V2D(), 1);
umov(w3, GetSrc(IROp->Args[1].ID()).V8H(), 4);
LoadConstant(x4, (uintptr_t)Info.fn);
@@ -295,8 +300,7 @@ void Arm64JITCore::Op_Unhandled(FEXCore::IR::IROp_Header *IROp, uint32_t Node) {
case FABI_UNKNOWN:
default:
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
auto Name = FEXCore::IR::GetName(IROp->Op);
LOGMAN_MSG_A("Unhandled IR Fallback abi: %s %d", std::string(Name).c_str(), Info.ABI);
LOGMAN_MSG_A_FMT("Unhandled IR Fallback ABI: {} {}", FEXCore::IR::GetName(IROp->Op), Info.ABI);
#endif
break;
}
@@ -315,8 +319,11 @@ Arm64JITCore::CodeBuffer Arm64JITCore::AllocateNewCodeBuffer(size_t Size) {
PROT_READ | PROT_WRITE | PROT_EXEC,
MAP_PRIVATE | MAP_ANONYMOUS,
-1, 0));
LOGMAN_THROW_A(!!Buffer.Ptr, "Couldn't allocate code buffer");
LOGMAN_THROW_A_FMT(!!Buffer.Ptr, "Couldn't allocate code buffer");
Dispatcher->RegisterCodeBuffer(Buffer.Ptr, Buffer.Size);
if (CTX->Config.GlobalJITNaming()) {
CTX->Symbols.RegisterJITSpace(Buffer.Ptr, Buffer.Size);
}
return Buffer;
}
@@ -325,118 +332,6 @@ void Arm64JITCore::FreeCodeBuffer(CodeBuffer Buffer) {
Dispatcher->RemoveCodeBuffer(Buffer.Ptr);
}
bool Arm64JITCore::HandleSIGBUS(int Signal, void *info, void *ucontext) {
uint32_t *PC = (uint32_t*)ArchHelpers::Context::GetPc(ucontext);
uint32_t Instr = PC[0];
if (!Dispatcher->IsAddressInJITCode(ArchHelpers::Context::GetPc(ucontext))) {
// Wasn't a sigbus in JIT code
return false;
}
// 1 = 16bit
// 2 = 32bit
// 3 = 64bit
uint32_t Size = (Instr & 0xC000'0000) >> 30;
uint32_t AddrReg = (Instr >> 5) & 0x1F;
uint32_t DataReg = Instr & 0x1F;
uint32_t DMB = 0b1101'0101'0000'0011'0011'0000'1011'1111 |
0b1011'0000'0000; // Inner shareable all
if ((Instr & 0x3F'FF'FC'00) == 0x08'DF'FC'00 || // LDAR*
(Instr & 0x3F'FF'FC'00) == 0x38'BF'C0'00) { // LDAPR*
uint32_t LDR = 0b0011'1000'0111'1111'0110'1000'0000'0000;
LDR |= Size << 30;
LDR |= AddrReg << 5;
LDR |= DataReg;
PC[-1] = DMB;
PC[0] = LDR;
PC[1] = DMB;
// Back up one instruction and have another go
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) - 4);
}
else if ( (Instr & 0x3F'FF'FC'00) == 0x08'9F'FC'00) { // STLR*
uint32_t STR = 0b0011'1000'0011'1111'0110'1000'0000'0000;
STR |= Size << 30;
STR |= AddrReg << 5;
STR |= DataReg;
PC[-1] = DMB;
PC[0] = STR;
PC[1] = DMB;
// Back up one instruction and have another go
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) - 4);
}
else if ((Instr & FEXCore::ArchHelpers::Arm64::LDAXP_MASK) == FEXCore::ArchHelpers::Arm64::LDAXP_INST) { // LDAXP
uint32_t DataReg2 = (Instr >> 10) & 0x1F;
// Convert to LDP
uint32_t LDP = 0b0010'1001'0100'0000'0000'0000'0000'0000;
LDP |= Size << 31;
LDP |= DataReg2 << 10;
LDP |= AddrReg << 5;
LDP |= DataReg;
PC[-1] = DMB;
PC[0] = LDP;
PC[1] = DMB;
// Back up one instruction and have another go
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) - 4);
}
else if ((Instr & FEXCore::ArchHelpers::Arm64::STLXP_MASK) == FEXCore::ArchHelpers::Arm64::STLXP_INST) { // STLXP
uint32_t DataReg2 = (Instr >> 10) & 0x1F;
// Convert to STP
uint32_t STP = 0b0010'1001'0000'0000'0000'0000'0000'0000;
STP |= Size << 31;
STP |= DataReg2 << 10;
STP |= AddrReg << 5;
STP |= DataReg;
PC[-1] = DMB;
PC[0] = STP;
PC[1] = DMB;
// Back up one instruction and have another go
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) - 4);
}
else if ((Instr & FEXCore::ArchHelpers::Arm64::CASPAL_MASK) == FEXCore::ArchHelpers::Arm64::CASPAL_INST) { // CASPAL
if (FEXCore::ArchHelpers::Arm64::HandleCASPAL(ucontext, info, Instr)) {
// Skip this instruction now
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) + 4);
return true;
}
else {
LogMan::Msg::E("Unhandled JIT SIGBUS CASPAL: PC: %p Instruction: 0x%08x\n", PC, PC[0]);
return false;
}
}
else if ((Instr & FEXCore::ArchHelpers::Arm64::CASAL_MASK) == FEXCore::ArchHelpers::Arm64::CASAL_INST) { // CASAL
if (FEXCore::ArchHelpers::Arm64::HandleCASAL(ucontext, info, Instr)) {
// Skip this instruction now
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) + 4);
return true;
}
else {
LogMan::Msg::E("Unhandled JIT SIGBUS CASAL: PC: %p Instruction: 0x%08x\n", PC, PC[0]);
return false;
}
}
else if ((Instr & FEXCore::ArchHelpers::Arm64::ATOMIC_MEM_MASK) == FEXCore::ArchHelpers::Arm64::ATOMIC_MEM_INST) { // Atomic memory op
if (FEXCore::ArchHelpers::Arm64::HandleAtomicMemOp(ucontext, info, Instr)) {
// Skip this instruction now
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) + 4);
return true;
}
else {
uint8_t Op = (PC[0] >> 12) & 0xF;
LogMan::Msg::E("Unhandled JIT SIGBUS Atomic mem op 0x%02x: PC: %p Instruction: 0x%08x\n", Op, PC, PC[0]);
return false;
}
}
else {
LogMan::Msg::E("Unhandled JIT SIGBUS: PC: %p Instruction: 0x%08x\n", PC, PC[0]);
return false;
}
vixl::aarch64::CPU::EnsureIAndDCacheCoherency(&PC[-1], 16);
return true;
}
Arm64JITCore::Arm64JITCore(FEXCore::Context::Context *ctx, FEXCore::Core::InternalThreadState *Thread, bool CompileThread)
: Arm64Emitter(0)
, CTX {ctx}
@@ -445,7 +340,7 @@ Arm64JITCore::Arm64JITCore(FEXCore::Context::Context *ctx, FEXCore::Core::Intern
DispatcherConfig config;
config.ExitFunctionLink = reinterpret_cast<uintptr_t>(&ExitFunctionLink);
config.ExitFunctionLinkThis = reinterpret_cast<uintptr_t>(this);
config.StaticRegisterAssignment = true;
config.StaticRegisterAssignment = ctx->Config.StaticRegisterAllocation;
Dispatcher = std::make_unique<Arm64Dispatcher>(CTX, ThreadState, config);
DispatchPtr = Dispatcher->DispatchPtr;
@@ -459,7 +354,7 @@ Arm64JITCore::Arm64JITCore(FEXCore::Context::Context *ctx, FEXCore::Core::Intern
CurrentCodeBuffer = &InitialCodeBuffer;
RAPass = Thread->PassManager->GetRAPass();
RAPass = Thread->PassManager->GetPass<IR::RegisterAllocationPass>("RA");
#if DEBUG
Decoder.AppendVisitor(&Disasm)
@@ -511,7 +406,13 @@ Arm64JITCore::Arm64JITCore(FEXCore::Context::Context *ctx, FEXCore::Core::Intern
CTX->SignalDelegation->RegisterHostSignalHandler(SIGBUS, [](FEXCore::Core::InternalThreadState *Thread, int Signal, void *info, void *ucontext) -> bool {
Arm64JITCore *Core = reinterpret_cast<Arm64JITCore*>(Thread->CPUBackend.get());
return Core->HandleSIGBUS(Signal, info, ucontext);
if (!Core->Dispatcher->IsAddressInJITCode(ArchHelpers::Context::GetPc(ucontext))) {
// Wasn't a sigbus in JIT code
return false;
}
return FEXCore::ArchHelpers::Arm64::HandleSIGBUS(Core->CTX->Config.ParanoidTSO(), Signal, info, ucontext);
}, true);
CTX->SignalDelegation->RegisterHostSignalHandler(SignalDelegator::SIGNAL_FOR_PAUSE, [](FEXCore::Core::InternalThreadState *Thread, int Signal, void *info, void *ucontext) -> bool {
@@ -524,7 +425,7 @@ Arm64JITCore::Arm64JITCore(FEXCore::Context::Context *ctx, FEXCore::Core::Intern
return Core->Dispatcher->HandleGuestSignal(Signal, info, ucontext, GuestAction, GuestStack);
};
for (uint32_t Signal = 0; Signal < SignalDelegator::MAX_SIGNALS; ++Signal) {
for (uint32_t Signal = 0; Signal <= SignalDelegator::MAX_SIGNALS; ++Signal) {
CTX->SignalDelegation->RegisterHostSignalHandlerForGuest(Signal, GuestSignalHandler);
}
}
@@ -584,7 +485,7 @@ Arm64JITCore::~Arm64JITCore() {
IR::PhysicalRegister Arm64JITCore::GetPhys(uint32_t Node) const {
auto PhyReg = RAData->GetNodeRegister(Node);
LOGMAN_THROW_A(!PhyReg.IsInvalid(), "Couldn't Allocate register for node: ssa%d. Class: %d", Node, PhyReg.Class);
LOGMAN_THROW_A_FMT(!PhyReg.IsInvalid(), "Couldn't Allocate register for node: ssa{}. Class: {}", Node, PhyReg.Class);
return PhyReg;
}
@@ -598,7 +499,7 @@ aarch64::Register Arm64JITCore::GetReg<Arm64JITCore::RA_32>(uint32_t Node) const
} else if (Reg.Class == IR::GPRClass.Val) {
return RA64[Reg.Reg].W();
} else {
LOGMAN_THROW_A(false, "Unexpected Class: %d", Reg.Class);
LOGMAN_THROW_A_FMT(false, "Unexpected Class: {}", Reg.Class);
}
FEX_UNREACHABLE;
@@ -613,7 +514,7 @@ aarch64::Register Arm64JITCore::GetReg<Arm64JITCore::RA_64>(uint32_t Node) const
} else if (Reg.Class == IR::GPRClass.Val) {
return RA64[Reg.Reg];
} else {
LOGMAN_THROW_A(false, "Unexpected Class: %d", Reg.Class);
LOGMAN_THROW_A_FMT(false, "Unexpected Class: {}", Reg.Class);
}
FEX_UNREACHABLE;
@@ -639,7 +540,7 @@ aarch64::VRegister Arm64JITCore::GetSrc(uint32_t Node) const {
} else if (Reg.Class == IR::FPRClass.Val) {
return RAFPR[Reg.Reg];
} else {
LOGMAN_THROW_A(false, "Unexpected Class: %d", Reg.Class);
LOGMAN_THROW_A_FMT(false, "Unexpected Class: {}", Reg.Class);
}
FEX_UNREACHABLE;
@@ -653,7 +554,7 @@ aarch64::VRegister Arm64JITCore::GetDst(uint32_t Node) const {
} else if (Reg.Class == IR::FPRClass.Val) {
return RAFPR[Reg.Reg];
} else {
LOGMAN_THROW_A(false, "Unexpected Class: %d", Reg.Class);
LOGMAN_THROW_A_FMT(false, "Unexpected Class: {}", Reg.Class);
}
FEX_UNREACHABLE;
@@ -744,8 +645,7 @@ void *Arm64JITCore::CompileCode(uint64_t Entry, [[maybe_unused]] FEXCore::IR::IR
// X1-X3 = Temp
// X4-r18 = RA
auto Buffer = GetBuffer();
auto GuestEntry = Buffer->GetOffsetAddress<uint64_t>(GetCursorOffset());
auto GuestEntry = GetCursorAddress<uint64_t>();
if (CTX->GetGdbServerStatus()) {
aarch64::Label RunBlock;
@@ -791,7 +691,7 @@ void *Arm64JITCore::CompileCode(uint64_t Entry, [[maybe_unused]] FEXCore::IR::IR
using namespace FEXCore::IR;
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
auto BlockIROp = BlockHeader->CW<FEXCore::IR::IROp_CodeBlock>();
LOGMAN_THROW_A(BlockIROp->Header.Op == IR::OP_CODEBLOCK, "IR type failed to be a code block");
LOGMAN_THROW_A_FMT(BlockIROp->Header.Op == IR::OP_CODEBLOCK, "IR type failed to be a code block");
#endif
{
@@ -812,7 +712,7 @@ void *Arm64JITCore::CompileCode(uint64_t Entry, [[maybe_unused]] FEXCore::IR::IR
}
if (DebugData) {
DebugData->Subblocks.push_back({Buffer->GetOffsetAddress<uintptr_t>(GetCursorOffset()), 0, IR->GetID(BlockNode)});
DebugData->Subblocks.push_back({GetCursorAddress<uintptr_t>(), 0, IR->GetID(BlockNode)});
}
for (auto [CodeNode, IROp] : IR->GetCode(BlockNode)) {
@@ -824,7 +724,7 @@ void *Arm64JITCore::CompileCode(uint64_t Entry, [[maybe_unused]] FEXCore::IR::IR
}
if (DebugData) {
DebugData->Subblocks.back().HostCodeSize = Buffer->GetOffsetAddress<uintptr_t>(GetCursorOffset()) - DebugData->Subblocks.back().HostCodeStart;
DebugData->Subblocks.back().HostCodeSize = GetCursorAddress<uintptr_t>() - DebugData->Subblocks.back().HostCodeStart;
}
}
@@ -837,7 +737,7 @@ void *Arm64JITCore::CompileCode(uint64_t Entry, [[maybe_unused]] FEXCore::IR::IR
FinalizeCode();
auto CodeEnd = Buffer->GetOffsetAddress<uint64_t>(GetCursorOffset());
auto CodeEnd = GetCursorAddress<uint64_t>();
CPU.EnsureIAndDCacheCoherency(reinterpret_cast<void*>(GuestEntry), CodeEnd - reinterpret_cast<uint64_t>(GuestEntry));
if (DebugData) {
@@ -856,7 +756,7 @@ uint64_t Arm64JITCore::ExitFunctionLink(Arm64JITCore *core, FEXCore::Core::CpuSt
auto HostCode = Thread->LookupCache->FindBlock(GuestRip);
if (!HostCode) {
//printf("ExitFunctionLink: Aborting, %lX not in cache\n", GuestRip);
//fmt::print("ExitFunctionLink: Aborting, {:X} not in cache\n", GuestRip);
Frame->State.rip = GuestRip;
return core->ThreadSharedData.Dispatcher->AbsoluteLoopTopAddress;
}
+34 -25
View File
@@ -6,7 +6,6 @@ $end_info$
#pragma once
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/ArchHelpers/Arm64Emitter.h"
#include "Interface/Core/Dispatcher/Dispatcher.h"
@@ -43,22 +42,26 @@ public:
size_t Size;
};
explicit Arm64JITCore(FEXCore::Context::Context *ctx, FEXCore::Core::InternalThreadState *Thread, bool CompileThread);
explicit Arm64JITCore(FEXCore::Context::Context *ctx,
FEXCore::Core::InternalThreadState *Thread,
bool CompileThread);
~Arm64JITCore() override;
std::string GetName() override { return "JIT"; }
void *CompileCode(uint64_t Entry, FEXCore::IR::IRListView const *IR, FEXCore::Core::DebugData *DebugData, FEXCore::IR::RegisterAllocationData *RAData) override;
void *MapRegion(void* HostPtr, uint64_t, uint64_t) override { return HostPtr; }
[[nodiscard]] std::string GetName() override { return "JIT"; }
bool NeedsOpDispatch() override { return true; }
[[nodiscard]] void *CompileCode(uint64_t Entry,
FEXCore::IR::IRListView const *IR,
FEXCore::Core::DebugData *DebugData,
FEXCore::IR::RegisterAllocationData *RAData) override;
[[nodiscard]] void *MapRegion(void* HostPtr, uint64_t, uint64_t) override { return HostPtr; }
[[nodiscard]] bool NeedsOpDispatch() override { return true; }
void ClearCache() override;
bool HandleSIGBUS(int Signal, void *info, void *ucontext);
static constexpr size_t INITIAL_CODE_SIZE = 1024 * 1024 * 16;
CodeBuffer AllocateNewCodeBuffer(size_t Size);
[[nodiscard]] CodeBuffer AllocateNewCodeBuffer(size_t Size);
void CopyNecessaryDataForCompileThread(CPUBackend *Original) override;
@@ -96,35 +99,39 @@ private:
constexpr static uint8_t RA_FPR = 2;
template<uint8_t RAType>
aarch64::Register GetReg(uint32_t Node) const;
[[nodiscard]] aarch64::Register GetReg(uint32_t Node) const;
template<>
aarch64::Register GetReg<RA_32>(uint32_t Node) const;
[[nodiscard]] aarch64::Register GetReg<RA_32>(uint32_t Node) const;
template<>
aarch64::Register GetReg<RA_64>(uint32_t Node) const;
[[nodiscard]] aarch64::Register GetReg<RA_64>(uint32_t Node) const;
template<uint8_t RAType>
std::pair<aarch64::Register, aarch64::Register> GetSrcPair(uint32_t Node) const;
[[nodiscard]] std::pair<aarch64::Register, aarch64::Register> GetSrcPair(uint32_t Node) const;
template<>
std::pair<aarch64::Register, aarch64::Register> GetSrcPair<RA_32>(uint32_t Node) const;
[[nodiscard]] std::pair<aarch64::Register, aarch64::Register> GetSrcPair<RA_32>(uint32_t Node) const;
template<>
std::pair<aarch64::Register, aarch64::Register> GetSrcPair<RA_64>(uint32_t Node) const;
[[nodiscard]] std::pair<aarch64::Register, aarch64::Register> GetSrcPair<RA_64>(uint32_t Node) const;
aarch64::VRegister GetSrc(uint32_t Node) const;
aarch64::VRegister GetDst(uint32_t Node) const;
[[nodiscard]] aarch64::VRegister GetSrc(uint32_t Node) const;
[[nodiscard]] aarch64::VRegister GetDst(uint32_t Node) const;
FEXCore::IR::RegisterClassType GetRegClass(uint32_t Node) const;
[[nodiscard]] FEXCore::IR::RegisterClassType GetRegClass(uint32_t Node) const;
IR::PhysicalRegister GetPhys(uint32_t Node) const;
[[nodiscard]] IR::PhysicalRegister GetPhys(uint32_t Node) const;
bool IsFPR(uint32_t Node) const;
bool IsGPR(uint32_t Node) const;
[[nodiscard]] bool IsFPR(uint32_t Node) const;
[[nodiscard]] bool IsGPR(uint32_t Node) const;
MemOperand GenerateMemOperand(uint8_t AccessSize, aarch64::Register Base, IR::OrderedNodeWrapper Offset, IR::MemOffsetType OffsetType, uint8_t OffsetScale);
[[nodiscard]] MemOperand GenerateMemOperand(uint8_t AccessSize,
aarch64::Register Base,
IR::OrderedNodeWrapper Offset,
IR::MemOffsetType OffsetType,
uint8_t OffsetScale);
bool IsInlineConstant(const IR::OrderedNodeWrapper& Node, uint64_t* Value = nullptr) const;
bool IsInlineEntrypointOffset(const IR::OrderedNodeWrapper& WNode, uint64_t* Value) const;
[[nodiscard]] bool IsInlineConstant(const IR::OrderedNodeWrapper& Node, uint64_t* Value = nullptr) const;
[[nodiscard]] bool IsInlineEntrypointOffset(const IR::OrderedNodeWrapper& WNode, uint64_t* Value) const;
struct LiveRange {
uint32_t Begin;
@@ -214,6 +221,7 @@ private:
DEF_OP(UMulH);
DEF_OP(Or);
DEF_OP(And);
DEF_OP(Andn);
DEF_OP(Xor);
DEF_OP(Lshl);
DEF_OP(Lshr);
@@ -257,6 +265,7 @@ private:
DEF_OP(AtomicFetchAnd);
DEF_OP(AtomicFetchOr);
DEF_OP(AtomicFetchXor);
DEF_OP(AtomicFetchNeg);
///< Branch ops
DEF_OP(GuestCallDirect);
+105 -115
View File
@@ -30,7 +30,7 @@ DEF_OP(LoadContext) {
case 8:
ldr(GetReg<RA_64>(Node), MemOperand(STATE, Op->Offset));
break;
default: LOGMAN_MSG_A("Unhandled LoadContext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled LoadContext size: {}", OpSize);
}
}
else {
@@ -51,7 +51,7 @@ DEF_OP(LoadContext) {
case 16:
ldr(Dst, MemOperand(STATE, Op->Offset));
break;
default: LOGMAN_MSG_A("Unhandled LoadContext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled LoadContext size: {}", OpSize);
}
}
}
@@ -73,7 +73,7 @@ DEF_OP(StoreContext) {
case 8:
str(GetReg<RA_64>(Op->Header.Args[0].ID()), MemOperand(STATE, Op->Offset));
break;
default: LOGMAN_MSG_A("Unhandled StoreContext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled StoreContext size: {}", OpSize);
}
}
else {
@@ -94,7 +94,7 @@ DEF_OP(StoreContext) {
case 16:
str(Src, MemOperand(STATE, Op->Offset));
break;
default: LOGMAN_MSG_A("Unhandled LoadContext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled StoreContext size: {}", OpSize);
}
}
}
@@ -107,29 +107,29 @@ DEF_OP(LoadRegister) {
auto regId = (Op->Offset - offsetof(FEXCore::Core::CpuStateFrame, State.gregs[0])) / 8;
auto regOffs = Op->Offset & 7;
LOGMAN_THROW_A(regId < SRA64.size(), "out of range regId");
LOGMAN_THROW_A_FMT(regId < SRA64.size(), "out of range regId");
auto reg = SRA64[regId];
switch(Op->Header.Size) {
case 1:
LOGMAN_THROW_A(regOffs == 0 || regOffs == 1, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0 || regOffs == 1, "unexpected regOffs");
ubfx(GetReg<RA_64>(Node), reg, regOffs * 8, 8);
break;
case 2:
LOGMAN_THROW_A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
ubfx(GetReg<RA_64>(Node), reg, 0, 16);
break;
case 4:
LOGMAN_THROW_A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
if (GetReg<RA_64>(Node).GetCode() != reg.GetCode())
mov(GetReg<RA_32>(Node), reg.W());
break;
case 8:
LOGMAN_THROW_A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
if (GetReg<RA_64>(Node).GetCode() != reg.GetCode())
mov(GetReg<RA_64>(Node), reg);
break;
@@ -138,24 +138,24 @@ DEF_OP(LoadRegister) {
auto regId = (Op->Offset - offsetof(FEXCore::Core::CpuStateFrame, State.xmm[0][0])) / 16;
auto regOffs = Op->Offset & 15;
LOGMAN_THROW_A(regId < SRAFPR.size(), "out of range regId");
LOGMAN_THROW_A_FMT(regId < SRAFPR.size(), "out of range regId");
auto guest = SRAFPR[regId];
auto host = GetSrc(Node);
switch(Op->Header.Size) {
case 1:
LOGMAN_THROW_A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
mov(host.B(), guest.B());
break;
case 2:
LOGMAN_THROW_A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
fmov(host.H(), guest.H());
break;
case 4:
LOGMAN_THROW_A((regOffs & 3) == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT((regOffs & 3) == 0, "unexpected regOffs");
if (regOffs == 0) {
if (host.GetCode() != guest.GetCode())
fmov(host.S(), guest.S());
@@ -165,7 +165,7 @@ DEF_OP(LoadRegister) {
break;
case 8:
LOGMAN_THROW_A((regOffs & 7) == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT((regOffs & 7) == 0, "unexpected regOffs");
if (regOffs == 0) {
if (host.GetCode() != guest.GetCode())
mov(host.D(), guest.D());
@@ -175,13 +175,13 @@ DEF_OP(LoadRegister) {
break;
case 16:
LOGMAN_THROW_A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
if (host.GetCode() != guest.GetCode())
mov(host.Q(), guest.Q());
break;
}
} else {
LOGMAN_THROW_A(false, "Unhandled Op->Class %d", Op->Class);
LOGMAN_THROW_A_FMT(false, "Unhandled Op->Class {}", Op->Class);
}
}
@@ -192,28 +192,28 @@ DEF_OP(StoreRegister) {
auto regId = Op->Offset / 8 - 1;
auto regOffs = Op->Offset & 7;
LOGMAN_THROW_A(regId < SRA64.size(), "out of range regId");
LOGMAN_THROW_A_FMT(regId < SRA64.size(), "out of range regId");
auto reg = SRA64[regId];
switch(Op->Header.Size) {
case 1:
LOGMAN_THROW_A(regOffs == 0 || regOffs == 1, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0 || regOffs == 1, "unexpected regOffs");
bfi(reg, GetReg<RA_64>(Op->Value.ID()), regOffs * 8, 8);
break;
case 2:
LOGMAN_THROW_A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
bfi(reg, GetReg<RA_64>(Op->Value.ID()), 0, 16);
break;
case 4:
LOGMAN_THROW_A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
bfi(reg, GetReg<RA_64>(Op->Value.ID()), 0, 32);
break;
case 8:
LOGMAN_THROW_A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
if (GetReg<RA_64>(Op->Value.ID()).GetCode() != reg.GetCode())
mov(reg, GetReg<RA_64>(Op->Value.ID()));
break;
@@ -222,7 +222,7 @@ DEF_OP(StoreRegister) {
auto regId = (Op->Offset - offsetof(FEXCore::Core::CpuStateFrame, State.xmm[0][0])) / 16;
auto regOffs = Op->Offset & 15;
LOGMAN_THROW_A(regId < SRAFPR.size(), "regId out of range");
LOGMAN_THROW_A_FMT(regId < SRAFPR.size(), "regId out of range");
auto guest = SRAFPR[regId];
auto host = GetSrc(Op->Value.ID());
@@ -233,35 +233,35 @@ DEF_OP(StoreRegister) {
break;
case 2:
LOGMAN_THROW_A((regOffs & 1) == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT((regOffs & 1) == 0, "unexpected regOffs");
ins(guest.V8H(), regOffs/2, host.V8H(), 0);
break;
case 4:
LOGMAN_THROW_A((regOffs & 3) == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT((regOffs & 3) == 0, "unexpected regOffs");
ins(guest.V4S(), regOffs/4, host.V4S(), 0);
break;
case 8:
LOGMAN_THROW_A((regOffs & 7) == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT((regOffs & 7) == 0, "unexpected regOffs");
ins(guest.V2D(), regOffs / 8, host.V2D(), 0);
break;
case 16:
LOGMAN_THROW_A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
if (guest.GetCode() != host.GetCode())
mov(guest.Q(), host.Q());
break;
}
} else {
LOGMAN_THROW_A(false, "Unhandled Op->Class %d", Op->Class);
LOGMAN_THROW_A_FMT(false, "Unhandled Op->Class {}", Op->Class);
}
}
DEF_OP(LoadContextIndexed) {
auto Op = IROp->C<IR::IROp_LoadContextIndexed>();
size_t size = Op->Size;
size_t size = IROp->Size;
auto index = GetReg<RA_64>(Op->Header.Args[0].ID());
if (Op->Class == FEXCore::IR::GPRClass) {
@@ -288,15 +288,17 @@ DEF_OP(LoadContextIndexed) {
ldr(GetReg<RA_64>(Node), MemOperand(TMP1, Op->BaseOffset));
break;
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed size: %d", Op->Size);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed size: {}", IROp->Size);
break;
}
break;
}
case 16:
LOGMAN_MSG_A("Invalid Class load of size 16");
LOGMAN_MSG_A_FMT("Invalid Class load of size 16");
break;
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed stride: %d", Op->Stride);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed stride: {}", Op->Stride);
break;
}
}
else {
@@ -333,19 +335,21 @@ DEF_OP(LoadContextIndexed) {
}
break;
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed size: %d", Op->Size);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed size: {}", IROp->Size);
break;
}
break;
}
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed stride: %d", Op->Stride);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed stride: {}", Op->Stride);
break;
}
}
}
DEF_OP(StoreContextIndexed) {
auto Op = IROp->C<IR::IROp_StoreContextIndexed>();
size_t size = Op->Size;
size_t size = IROp->Size;
auto index = GetReg<RA_64>(Op->Header.Args[1].ID());
if (Op->Class == FEXCore::IR::GPRClass) {
@@ -374,15 +378,17 @@ DEF_OP(StoreContextIndexed) {
str(value, MemOperand(TMP1, Op->BaseOffset));
break;
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed size: %d", Op->Size);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed size: {}", IROp->Size);
break;
}
break;
}
case 16:
LOGMAN_MSG_A("Invalid Class load of size 16");
LOGMAN_MSG_A_FMT("Invalid Class store of size 16");
break;
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed stride: %d", Op->Stride);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed stride: {}", Op->Stride);
break;
}
}
else {
@@ -421,12 +427,14 @@ DEF_OP(StoreContextIndexed) {
}
break;
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed size: %d", Op->Size);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed size: {}", IROp->Size);
break;
}
break;
}
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed stride: %d", Op->Stride);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed stride: {}", Op->Stride);
break;
}
}
}
@@ -434,7 +442,7 @@ DEF_OP(StoreContextIndexed) {
DEF_OP(SpillRegister) {
auto Op = IROp->C<IR::IROp_SpillRegister>();
uint8_t OpSize = IROp->Size;
uint32_t SlotOffset = Op->Slot * 16 + 16;
uint32_t SlotOffset = Op->Slot * 16;
if (Op->Class == FEXCore::IR::GPRClass) {
switch (OpSize) {
@@ -454,7 +462,7 @@ DEF_OP(SpillRegister) {
str(GetReg<RA_64>(Op->Header.Args[0].ID()), MemOperand(sp, SlotOffset));
break;
}
default: LOGMAN_MSG_A("Unhandled SpillRegister size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled SpillRegister size: {}", OpSize);
}
} else if (Op->Class == FEXCore::IR::FPRClass) {
switch (OpSize) {
@@ -470,17 +478,17 @@ DEF_OP(SpillRegister) {
str(GetSrc(Op->Header.Args[0].ID()), MemOperand(sp, SlotOffset));
break;
}
default: LOGMAN_MSG_A("Unhandled SpillRegister size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled SpillRegister size: {}", OpSize);
}
} else {
LOGMAN_MSG_A("Unhandled SpillRegister class: %d", Op->Class.Val);
LOGMAN_MSG_A_FMT("Unhandled SpillRegister class: {}", Op->Class.Val);
}
}
DEF_OP(FillRegister) {
auto Op = IROp->C<IR::IROp_FillRegister>();
uint8_t OpSize = IROp->Size;
uint32_t SlotOffset = Op->Slot * 16 + 16;
uint32_t SlotOffset = Op->Slot * 16;
if (Op->Class == FEXCore::IR::GPRClass) {
switch (OpSize) {
@@ -500,7 +508,7 @@ DEF_OP(FillRegister) {
ldr(GetReg<RA_64>(Node), MemOperand(sp, SlotOffset));
break;
}
default: LOGMAN_MSG_A("Unhandled SpillRegister size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled FillRegister size: {}", OpSize);
}
} else if (Op->Class == FEXCore::IR::FPRClass) {
switch (OpSize) {
@@ -516,10 +524,10 @@ DEF_OP(FillRegister) {
ldr(GetDst(Node), MemOperand(sp, SlotOffset));
break;
}
default: LOGMAN_MSG_A("Unhandled SpillRegister size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled FillRegister size: {}", OpSize);
}
} else {
LOGMAN_MSG_A("Unhandled FillRegister class: %d", Op->Class.Val);
LOGMAN_MSG_A_FMT("Unhandled FillRegister class: {}", Op->Class.Val);
}
}
@@ -539,7 +547,7 @@ MemOperand Arm64JITCore::GenerateMemOperand(uint8_t AccessSize, aarch64::Registe
return MemOperand(Base);
} else {
if (OffsetScale != 1 && OffsetScale != AccessSize) {
LOGMAN_MSG_A("Unhandled GenerateMemOperand OffsetScale: %d", OffsetScale);
LOGMAN_MSG_A_FMT("Unhandled GenerateMemOperand OffsetScale: {}", OffsetScale);
}
uint64_t Const;
if (IsInlineConstant(Offset, &Const)) {
@@ -551,7 +559,7 @@ MemOperand Arm64JITCore::GenerateMemOperand(uint8_t AccessSize, aarch64::Registe
case IR::MEM_OFFSET_UXTW.Val: return MemOperand(Base, RegOffset.W(), Extend::UXTW, (int)std::log2(OffsetScale) );
case IR::MEM_OFFSET_SXTW.Val: return MemOperand(Base, RegOffset.W(), Extend::SXTW, (int)std::log2(OffsetScale) );
default: LOGMAN_MSG_A("Unhandled GenerateMemOperand OffsetType: %d", OffsetType.Val); break;
default: LOGMAN_MSG_A_FMT("Unhandled GenerateMemOperand OffsetType: {}", OffsetType.Val); break;
}
}
}
@@ -563,11 +571,11 @@ DEF_OP(LoadMem) {
auto Op = IROp->C<IR::IROp_LoadMem>();
auto MemReg = GetReg<RA_64>(Op->Header.Args[0].ID());
auto MemSrc = GenerateMemOperand(Op->Size, MemReg, Op->Offset, Op->OffsetType, Op->OffsetScale);
auto MemSrc = GenerateMemOperand(IROp->Size, MemReg, Op->Offset, Op->OffsetType, Op->OffsetScale);
if (Op->Class == FEXCore::IR::GPRClass) {
auto Dst = GetReg<RA_64>(Node);
switch (Op->Size) {
switch (IROp->Size) {
case 1:
ldrb(Dst, MemSrc);
break;
@@ -580,12 +588,12 @@ DEF_OP(LoadMem) {
case 8:
ldr(Dst, MemSrc);
break;
default: LOGMAN_MSG_A("Unhandled LoadMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled LoadMem size: {}", IROp->Size);
}
}
else {
auto Dst = GetDst(Node);
switch (Op->Size) {
switch (IROp->Size) {
case 1:
ldr(Dst.B(), MemSrc);
break;
@@ -601,7 +609,7 @@ DEF_OP(LoadMem) {
case 16:
ldr(Dst, MemSrc);
break;
default: LOGMAN_MSG_A("Unhandled LoadMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled LoadMem size: {}", IROp->Size);
}
}
}
@@ -612,11 +620,11 @@ DEF_OP(LoadMemTSO) {
auto MemSrc = MemOperand(GetReg<RA_64>(Op->Header.Args[0].ID()));
if (!Op->Offset.IsInvalid()) {
LOGMAN_MSG_A("LoadMemTSO: No offset allowed");
LOGMAN_MSG_A_FMT("LoadMemTSO: No offset allowed");
}
if (SupportsRCPC && Op->Class == FEXCore::IR::GPRClass) {
if (Op->Size == 1) {
if (IROp->Size == 1) {
// 8bit load is always aligned to natural alignment
auto Dst = GetReg<RA_64>(Node);
ldaprb(Dst, MemSrc);
@@ -625,7 +633,7 @@ DEF_OP(LoadMemTSO) {
// Aligned
auto Dst = GetReg<RA_64>(Node);
nop();
switch (Op->Size) {
switch (IROp->Size) {
case 2:
ldaprh(Dst, MemSrc);
break;
@@ -635,13 +643,13 @@ DEF_OP(LoadMemTSO) {
case 8:
ldapr(Dst, MemSrc);
break;
default: LOGMAN_MSG_A("Unhandled LoadMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled LoadMemTSO size: {}", IROp->Size);
}
nop();
}
}
else if (Op->Class == FEXCore::IR::GPRClass) {
if (Op->Size == 1) {
if (IROp->Size == 1) {
// 8bit load is always aligned to natural alignment
auto Dst = GetReg<RA_64>(Node);
ldarb(Dst, MemSrc);
@@ -650,7 +658,7 @@ DEF_OP(LoadMemTSO) {
// Aligned
auto Dst = GetReg<RA_64>(Node);
nop();
switch (Op->Size) {
switch (IROp->Size) {
case 2:
ldarh(Dst, MemSrc);
break;
@@ -660,7 +668,7 @@ DEF_OP(LoadMemTSO) {
case 8:
ldar(Dst, MemSrc);
break;
default: LOGMAN_MSG_A("Unhandled LoadMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled LoadMemTSO size: {}", IROp->Size);
}
nop();
}
@@ -668,7 +676,7 @@ DEF_OP(LoadMemTSO) {
else {
dmb(InnerShareable, BarrierAll);
auto Dst = GetDst(Node);
switch (Op->Size) {
switch (IROp->Size) {
case 2:
ldr(Dst.H(), MemSrc);
break;
@@ -681,7 +689,7 @@ DEF_OP(LoadMemTSO) {
case 16:
ldr(Dst, MemSrc);
break;
default: LOGMAN_MSG_A("Unhandled LoadMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled LoadMemTSO size: {}", IROp->Size);
}
dmb(InnerShareable, BarrierAll);
}
@@ -692,10 +700,10 @@ DEF_OP(StoreMem) {
auto MemReg = GetReg<RA_64>(Op->Header.Args[0].ID());
auto MemSrc = GenerateMemOperand(Op->Size, MemReg, Op->Offset, Op->OffsetType, Op->OffsetScale);
auto MemSrc = GenerateMemOperand(IROp->Size, MemReg, Op->Offset, Op->OffsetType, Op->OffsetScale);
if (Op->Class == FEXCore::IR::GPRClass) {
switch (Op->Size) {
switch (IROp->Size) {
case 1:
strb(GetReg<RA_64>(Op->Header.Args[1].ID()), MemSrc);
break;
@@ -708,12 +716,12 @@ DEF_OP(StoreMem) {
case 8:
str(GetReg<RA_64>(Op->Header.Args[1].ID()), MemSrc);
break;
default: LOGMAN_MSG_A("Unhandled StoreMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled StoreMem size: {}", IROp->Size);
}
}
else {
auto Src = GetSrc(Op->Header.Args[1].ID());
switch (Op->Size) {
switch (IROp->Size) {
case 1:
str(Src.B(), MemSrc);
break;
@@ -729,7 +737,7 @@ DEF_OP(StoreMem) {
case 16:
str(Src, MemSrc);
break;
default: LOGMAN_MSG_A("Unhandled StoreMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled StoreMem size: {}", IROp->Size);
}
}
}
@@ -739,17 +747,17 @@ DEF_OP(StoreMemTSO) {
auto MemSrc = MemOperand(GetReg<RA_64>(Op->Header.Args[0].ID()));
if (!Op->Offset.IsInvalid()) {
LOGMAN_MSG_A("StoreMemTSO: No offset allowed");
LOGMAN_MSG_A_FMT("StoreMemTSO: No offset allowed");
}
if (Op->Class == FEXCore::IR::GPRClass) {
if (Op->Size == 1) {
if (IROp->Size == 1) {
// 8bit load is always aligned to natural alignment
stlrb(GetReg<RA_64>(Op->Header.Args[1].ID()), MemSrc);
}
else {
nop();
switch (Op->Size) {
switch (IROp->Size) {
case 2:
stlrh(GetReg<RA_64>(Op->Header.Args[1].ID()), MemSrc);
break;
@@ -759,7 +767,7 @@ DEF_OP(StoreMemTSO) {
case 8:
stlr(GetReg<RA_64>(Op->Header.Args[1].ID()), MemSrc);
break;
default: LOGMAN_MSG_A("Unhandled StoreMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled StoreMemTSO size: {}", IROp->Size);
}
nop();
}
@@ -767,7 +775,7 @@ DEF_OP(StoreMemTSO) {
else {
dmb(InnerShareable, BarrierAll);
auto Src = GetSrc(Op->Header.Args[1].ID());
switch (Op->Size) {
switch (IROp->Size) {
case 1:
str(Src.B(), MemSrc);
break;
@@ -783,7 +791,7 @@ DEF_OP(StoreMemTSO) {
case 16:
str(Src, MemSrc);
break;
default: LOGMAN_MSG_A("Unhandled StoreMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled StoreMemTSO size: {}", IROp->Size);
}
dmb(InnerShareable, BarrierAll);
}
@@ -795,19 +803,18 @@ DEF_OP(ParanoidLoadMemTSO) {
auto MemSrc = MemOperand(GetReg<RA_64>(Op->Header.Args[0].ID()));
if (!Op->Offset.IsInvalid()) {
LOGMAN_MSG_A("LoadMemTSO: No offset allowed");
LOGMAN_MSG_A_FMT("ParanoidLoadMemTSO: No offset allowed");
}
if (Op->Class == FEXCore::IR::GPRClass) {
if (Op->Size == 1) {
if (IROp->Size == 1) {
// 8bit load is always aligned to natural alignment
auto Dst = GetReg<RA_64>(Node);
ldarb(Dst, MemSrc);
}
else {
auto Dst = GetReg<RA_64>(Node);
nop();
switch (Op->Size) {
switch (IROp->Size) {
case 2:
ldarh(Dst, MemSrc);
break;
@@ -817,31 +824,24 @@ DEF_OP(ParanoidLoadMemTSO) {
case 8:
ldar(Dst, MemSrc);
break;
default: LOGMAN_MSG_A("Unhandled LoadMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled ParanoidLoadMemTSO size: {}", IROp->Size);
}
nop();
}
}
else {
auto Dst = GetDst(Node);
switch (Op->Size) {
switch (IROp->Size) {
case 2:
nop();
ldarh(TMP1, MemSrc);
nop();
fmov(Dst, TMP1);
ldarh(TMP1.W(), MemSrc);
fmov(Dst.H(), TMP1.W());
break;
case 4:
nop();
ldar(TMP1.W(), MemSrc);
nop();
fmov(Dst, TMP1);
fmov(Dst.S(), TMP1.W());
break;
case 8:
nop();
ldar(TMP1, MemSrc);
nop();
fmov(Dst, TMP1);
fmov(Dst.D(), TMP1);
break;
case 16:
nop();
@@ -850,7 +850,7 @@ DEF_OP(ParanoidLoadMemTSO) {
mov(Dst.V2D(), 0, TMP1);
mov(Dst.V2D(), 1, TMP2);
break;
default: LOGMAN_MSG_A("Unhandled LoadMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled ParanoidLoadMemTSO size: {}", IROp->Size);
}
}
}
@@ -860,17 +860,16 @@ DEF_OP(ParanoidStoreMemTSO) {
auto MemSrc = MemOperand(GetReg<RA_64>(Op->Header.Args[0].ID()));
if (!Op->Offset.IsInvalid()) {
LOGMAN_MSG_A("StoreMemTSO: No offset allowed");
LOGMAN_MSG_A_FMT("ParanoidStoreMemTSO: No offset allowed");
}
if (Op->Class == FEXCore::IR::GPRClass) {
if (Op->Size == 1) {
if (IROp->Size == 1) {
// 8bit load is always aligned to natural alignment
stlrb(GetReg<RA_64>(Op->Header.Args[1].ID()), MemSrc);
}
else {
nop();
switch (Op->Size) {
switch (IROp->Size) {
case 2:
stlrh(GetReg<RA_64>(Op->Header.Args[1].ID()), MemSrc);
break;
@@ -880,37 +879,30 @@ DEF_OP(ParanoidStoreMemTSO) {
case 8:
stlr(GetReg<RA_64>(Op->Header.Args[1].ID()), MemSrc);
break;
default: LOGMAN_MSG_A("Unhandled StoreMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled ParanoidStoreMemTSO size: {}", IROp->Size);
}
nop();
}
}
else {
auto Src = GetSrc(Op->Header.Args[1].ID());
if (Op->Size == 1) {
if (IROp->Size == 1) {
// 8bit load is always aligned to natural alignment
mov(TMP1, Src.V4S(), 0);
mov(TMP1.W(), Src.V16B(), 0);
stlrb(TMP1, MemSrc);
}
else {
switch (Op->Size) {
switch (IROp->Size) {
case 2:
mov(TMP1, Src.V4S(), 0);
nop();
mov(TMP1.W(), Src.V8H(), 0);
stlrh(TMP1, MemSrc);
nop();
break;
case 4:
mov(TMP1, Src.V4S(), 0);
nop();
mov(TMP1.W(), Src.V4S(), 0);
stlr(TMP1.W(), MemSrc);
nop();
break;
case 8:
mov(TMP1, Src.V2D(), 0);
nop();
stlr(TMP1, MemSrc);
nop();
break;
case 16: {
// Move vector to GPRs
@@ -919,26 +911,24 @@ DEF_OP(ParanoidStoreMemTSO) {
Label B;
bind(&B);
nop(); // < Overwritten with DMB
// ldaxp must not have both the destination registers be the same
ldaxp(xzr, TMP3, MemSrc); // <- Can hit SIGBUS
nop(); // < Overwritten with DMB
ldaxp(xzr, TMP3, MemSrc); // <- Can hit SIGBUS. Overwritten with DMB
stlxp(TMP3, TMP1, TMP2, MemSrc); // <- Can also hit SIGBUS
cbnz(TMP3, &B); // < Overwritten with DMB
break;
}
default: LOGMAN_MSG_A("Unhandled StoreMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled ParanoidStoreMemTSO size: {}", IROp->Size);
}
}
}
}
DEF_OP(VLoadMemElement) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(VStoreMemElement) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(CacheLineClear) {
@@ -8,11 +8,11 @@ $end_info$
namespace FEXCore::CPU {
static void PrintValue(uint64_t Value) {
LogMan::Msg::D("Value: 0x%lx", Value);
LogMan::Msg::DFmt("Value: 0x{:x}", Value);
}
static void PrintVectorValue(uint64_t Value, uint64_t ValueUpper) {
LogMan::Msg::D("Value: 0x%016lx'%016lx", ValueUpper, Value);
LogMan::Msg::DFmt("Value: 0x{:016x}'{:016x}", ValueUpper, Value);
}
using namespace vixl;
@@ -31,7 +31,7 @@ DEF_OP(Fence) {
case IR::Fence_Store.Val:
dmb(FullSystem, BarrierWrites);
break;
default: LOGMAN_MSG_A("Unknown Fence: %d", Op->Fence); break;
default: LOGMAN_MSG_A_FMT("Unknown Fence: {}", Op->Fence); break;
}
}
@@ -69,7 +69,7 @@ DEF_OP(Break) {
br(TMP1);
break;
}
default: LOGMAN_MSG_A("Unknown Break reason: %d", Op->Reason);
default: LOGMAN_MSG_A_FMT("Unknown Break reason: {}", Op->Reason);
}
}
@@ -26,7 +26,7 @@ DEF_OP(ExtractElementPair) {
mov (GetReg<RA_64>(Node), Regs[Op->Element]);
break;
}
default: LOGMAN_MSG_A("Unknown Size"); break;
default: LOGMAN_MSG_A_FMT("Unknown Size"); break;
}
}
@@ -52,7 +52,7 @@ DEF_OP(CreateElementPair) {
RegTmp = x0;
break;
}
default: LOGMAN_MSG_A("Unknown Size"); break;
default: LOGMAN_MSG_A_FMT("Unknown Size"); break;
}
if (Dst.first.GetCode() != RegSecond.GetCode()) {
+132 -132
View File
@@ -22,7 +22,7 @@ DEF_OP(VectorZero) {
eor(GetDst(Node).V16B(), GetDst(Node).V16B(), GetDst(Node).V16B());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", OpSize); break;
}
}
@@ -43,47 +43,47 @@ DEF_OP(VectorImm) {
}
DEF_OP(CreateVector2) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(CreateVector4) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(SplatVector2) {
auto Op = IROp->C<IR::IROp_SplatVector2>();
auto Op = IROp->C<IR::IROp_SplatVector2>();
uint8_t OpSize = IROp->Size;
LOGMAN_THROW_A(OpSize <= 16, "Can't handle a vector of size: %d", OpSize);
LOGMAN_THROW_A_FMT(OpSize <= 16, "Can't handle a vector of size: {}", OpSize);
uint8_t ElementSize = OpSize / 2;
uint8_t ElementSize = OpSize / 2;
switch (ElementSize) {
case 4:
dup(GetDst(Node).V4S(), GetSrc(Op->Header.Args[0].ID()).V4S(), 0);
break;
case 8:
dup(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), 0);
break;
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.Size); break;
}
switch (ElementSize) {
case 4:
dup(GetDst(Node).V4S(), GetSrc(Op->Header.Args[0].ID()).V4S(), 0);
break;
case 8:
dup(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), 0);
break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.Size); break;
}
}
DEF_OP(SplatVector4) {
auto Op = IROp->C<IR::IROp_SplatVector4>();
auto Op = IROp->C<IR::IROp_SplatVector4>();
uint8_t OpSize = IROp->Size;
LOGMAN_THROW_A(OpSize <= 16, "Can't handle a vector of size: %d", OpSize);
LOGMAN_THROW_A_FMT(OpSize <= 16, "Can't handle a vector of size: {}", OpSize);
uint8_t ElementSize = OpSize / 4;
uint8_t ElementSize = OpSize / 4;
switch (ElementSize) {
case 4:
dup(GetDst(Node).V4S(), GetSrc(Op->Header.Args[0].ID()).V4S(), 0);
break;
case 8:
dup(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), 0);
break;
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.Size); break;
}
switch (ElementSize) {
case 4:
dup(GetDst(Node).V4S(), GetSrc(Op->Header.Args[0].ID()).V4S(), 0);
break;
case 8:
dup(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), 0);
break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.Size); break;
}
}
DEF_OP(VMov) {
@@ -118,7 +118,7 @@ DEF_OP(VMov) {
mov(GetDst(Node).V16B(), GetSrc(Op->Header.Args[0].ID()).V16B());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", OpSize); break;
}
}
@@ -161,7 +161,7 @@ DEF_OP(VAdd) {
add(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -184,7 +184,7 @@ DEF_OP(VSub) {
sub(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -207,7 +207,7 @@ DEF_OP(VUQAdd) {
uqadd(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -230,7 +230,7 @@ DEF_OP(VUQSub) {
uqsub(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -253,7 +253,7 @@ DEF_OP(VSQAdd) {
sqadd(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -276,7 +276,7 @@ DEF_OP(VSQSub) {
sqsub(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -298,7 +298,7 @@ DEF_OP(VAddP) {
addp(GetDst(Node).V2S(), GetSrc(Op->Header.Args[0].ID()).V2S(), GetSrc(Op->Header.Args[1].ID()).V2S());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -319,7 +319,7 @@ DEF_OP(VAddP) {
addp(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -338,7 +338,7 @@ DEF_OP(VAddV) {
case 8:
addp(GetDst(Node).VCast(OpSize * 8, 1), GetSrc(Op->Header.Args[0].ID()).VCast(OpSize * 8, Elements));
break;
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -353,7 +353,7 @@ DEF_OP(VUMinV) {
case 4:
uminv(GetDst(Node).VCast(Op->Header.ElementSize * 8, 1), GetSrc(Op->Header.Args[0].ID()).VCast(OpSize * 8, Elements));
break;
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -368,7 +368,7 @@ DEF_OP(VURAvg) {
urhadd(GetDst(Node).V8H(), GetSrc(Op->Header.Args[0].ID()).V8H(), GetSrc(Op->Header.Args[1].ID()).V8H());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -383,7 +383,7 @@ DEF_OP(VAbs) {
abs(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -395,7 +395,7 @@ DEF_OP(VAbs) {
case 8:
abs(GetDst(Node).VCast(OpSize * 8, Elements), GetSrc(Op->Header.Args[0].ID()).VCast(OpSize * 8, Elements));
break;
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -410,7 +410,7 @@ DEF_OP(VPopcount) {
cnt(GetDst(Node).V8B(), GetSrc(Op->Header.Args[0].ID()).V8B());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -419,7 +419,7 @@ DEF_OP(VPopcount) {
case 1:
cnt(GetDst(Node).V16B(), GetSrc(Op->Header.Args[0].ID()).V16B());
break;
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -438,7 +438,7 @@ DEF_OP(VFAdd) {
fadd(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), GetSrc(Op->Header.Args[1].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -452,7 +452,7 @@ DEF_OP(VFAdd) {
fadd(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -468,7 +468,7 @@ DEF_OP(VFAddP) {
faddp(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -486,7 +486,7 @@ DEF_OP(VFSub) {
fsub(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), GetSrc(Op->Header.Args[1].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -500,7 +500,7 @@ DEF_OP(VFSub) {
fsub(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -519,7 +519,7 @@ DEF_OP(VFMul) {
fmul(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), GetSrc(Op->Header.Args[1].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -533,7 +533,7 @@ DEF_OP(VFMul) {
fmul(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -552,7 +552,7 @@ DEF_OP(VFDiv) {
fdiv(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), GetSrc(Op->Header.Args[1].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -566,7 +566,7 @@ DEF_OP(VFDiv) {
fdiv(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -587,7 +587,7 @@ DEF_OP(VFMin) {
fcsel(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), GetSrc(Op->Header.Args[1].ID()).D(), Condition::mi);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -607,7 +607,7 @@ DEF_OP(VFMin) {
mov(GetDst(Node).V2D(), VTMP2.V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -628,7 +628,7 @@ DEF_OP(VFMax) {
fcsel(GetDst(Node).D(), GetSrc(Op->Header.Args[1].ID()).D(), GetSrc(Op->Header.Args[0].ID()).D(), Condition::mi);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -648,7 +648,7 @@ DEF_OP(VFMax) {
mov(GetDst(Node).V2D(), VTMP2.V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -669,7 +669,7 @@ DEF_OP(VFRecp) {
fdiv(GetDst(Node).D(), VTMP1.D(), GetSrc(Op->Header.Args[0].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -685,7 +685,7 @@ DEF_OP(VFRecp) {
fdiv(GetDst(Node).V2D(), VTMP1.V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -704,7 +704,7 @@ DEF_OP(VFSqrt) {
fsqrt(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -718,7 +718,7 @@ DEF_OP(VFSqrt) {
fsqrt(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -741,7 +741,7 @@ DEF_OP(VFRSqrt) {
fdiv(GetDst(Node).D(), VTMP1.D(), VTMP2.D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -759,7 +759,7 @@ DEF_OP(VFRSqrt) {
fdiv(GetDst(Node).V2D(), VTMP1.V2D(), VTMP2.V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -779,7 +779,7 @@ DEF_OP(VNeg) {
case 8:
neg(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
default: LOGMAN_MSG_A("Unsupported Not size: %d", IROp->Size);
default: LOGMAN_MSG_A_FMT("Unsupported VNeg size: {}", IROp->Size);
}
}
@@ -792,7 +792,7 @@ DEF_OP(VFNeg) {
case 8:
fneg(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
default: LOGMAN_MSG_A("Unsupported Not size: %d", IROp->Size);
default: LOGMAN_MSG_A_FMT("Unsupported VFNeg size: {}", IROp->Size);
}
}
@@ -823,7 +823,7 @@ DEF_OP(VUMin) {
mov(GetDst(Node).V2D(), VTMP2.V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -849,7 +849,7 @@ DEF_OP(VSMin) {
mov(GetDst(Node).V2D(), VTMP2.V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -875,7 +875,7 @@ DEF_OP(VUMax) {
mov(GetDst(Node).V2D(), VTMP2.V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -901,7 +901,7 @@ DEF_OP(VSMax) {
mov(GetDst(Node).V2D(), VTMP2.V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -922,7 +922,7 @@ DEF_OP(VZip) {
zip1(GetDst(Node).V2S(), GetSrc(Op->Header.Args[0].ID()).V2S(), GetSrc(Op->Header.Args[1].ID()).V2S());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -943,7 +943,7 @@ DEF_OP(VZip) {
zip1(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -965,7 +965,7 @@ DEF_OP(VZip2) {
zip2(GetDst(Node).V2S(), GetSrc(Op->Header.Args[0].ID()).V2S(), GetSrc(Op->Header.Args[1].ID()).V2S());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -986,7 +986,7 @@ DEF_OP(VZip2) {
zip2(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1008,7 +1008,7 @@ DEF_OP(VUnZip) {
uzp1(GetDst(Node).V2S(), GetSrc(Op->Header.Args[0].ID()).V2S(), GetSrc(Op->Header.Args[1].ID()).V2S());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1029,7 +1029,7 @@ DEF_OP(VUnZip) {
uzp1(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1051,7 +1051,7 @@ DEF_OP(VUnZip2) {
uzp2(GetDst(Node).V2S(), GetSrc(Op->Header.Args[0].ID()).V2S(), GetSrc(Op->Header.Args[1].ID()).V2S());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1072,7 +1072,7 @@ DEF_OP(VUnZip2) {
uzp2(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1105,7 +1105,7 @@ DEF_OP(VCMPEQ) {
cmeq(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), GetSrc(Op->Header.Args[1].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1127,7 +1127,7 @@ DEF_OP(VCMPEQ) {
cmeq(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1146,7 +1146,7 @@ DEF_OP(VCMPEQZ) {
cmeq(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), 0);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1168,7 +1168,7 @@ DEF_OP(VCMPEQZ) {
cmeq(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), 0);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1187,7 +1187,7 @@ DEF_OP(VCMPGT) {
cmgt(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), GetSrc(Op->Header.Args[1].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1209,7 +1209,7 @@ DEF_OP(VCMPGT) {
cmgt(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1228,7 +1228,7 @@ DEF_OP(VCMPGTZ) {
cmgt(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), 0);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1250,7 +1250,7 @@ DEF_OP(VCMPGTZ) {
cmgt(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), 0);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1269,7 +1269,7 @@ DEF_OP(VCMPLTZ) {
cmlt(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), 0);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1291,7 +1291,7 @@ DEF_OP(VCMPLTZ) {
cmlt(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), 0);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1310,7 +1310,7 @@ DEF_OP(VFCMPEQ) {
fcmeq(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), GetSrc(Op->Header.Args[1].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1328,7 +1328,7 @@ DEF_OP(VFCMPEQ) {
fcmeq(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1347,7 +1347,7 @@ DEF_OP(VFCMPNEQ) {
fcmeq(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), GetSrc(Op->Header.Args[1].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
mvn(GetDst(Node).V8B(), GetDst(Node).V8B());
}
@@ -1366,7 +1366,7 @@ DEF_OP(VFCMPNEQ) {
fcmeq(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
mvn(GetDst(Node).V16B(), GetDst(Node).V16B());
}
@@ -1386,7 +1386,7 @@ DEF_OP(VFCMPLT) {
fcmgt(GetDst(Node).D(), GetSrc(Op->Header.Args[1].ID()).D(), GetSrc(Op->Header.Args[0].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1404,7 +1404,7 @@ DEF_OP(VFCMPLT) {
fcmgt(GetDst(Node).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1423,7 +1423,7 @@ DEF_OP(VFCMPGT) {
fcmgt(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), GetSrc(Op->Header.Args[1].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1441,7 +1441,7 @@ DEF_OP(VFCMPGT) {
fcmgt(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1460,7 +1460,7 @@ DEF_OP(VFCMPLE) {
fcmge(GetDst(Node).D(), GetSrc(Op->Header.Args[1].ID()).D(), GetSrc(Op->Header.Args[0].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1478,7 +1478,7 @@ DEF_OP(VFCMPLE) {
fcmge(GetDst(Node).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1501,7 +1501,7 @@ DEF_OP(VFCMPORD) {
orr(GetDst(Node).V8B(), VTMP1.V8B(), VTMP2.V8B());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1525,7 +1525,7 @@ DEF_OP(VFCMPORD) {
orr(GetDst(Node).V16B(), VTMP1.V16B(), VTMP2.V16B());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1550,7 +1550,7 @@ DEF_OP(VFCMPUNO) {
mvn(GetDst(Node).V8B(), GetDst(Node).V8B());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1577,21 +1577,21 @@ DEF_OP(VFCMPUNO) {
mvn(GetDst(Node).V16B(), GetDst(Node).V16B());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
DEF_OP(VUShl) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(VUShr) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(VSShr) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(VUShlS) {
@@ -1618,7 +1618,7 @@ DEF_OP(VUShlS) {
ushl(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), VTMP1.V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1650,7 +1650,7 @@ DEF_OP(VUShrS) {
ushl(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), VTMP1.V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1682,7 +1682,7 @@ DEF_OP(VSShrS) {
sshl(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), VTMP1.V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1713,7 +1713,7 @@ DEF_OP(VInsElement) {
mov(reg.V2D(), Op->DestIdx, GetSrc(Op->Header.Args[1].ID()).V2D(), Op->SrcIdx);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
if (GetDst(Node).GetCode() != reg.GetCode()) {
@@ -1748,7 +1748,7 @@ DEF_OP(VInsScalarElement) {
mov(reg.V2D(), Op->DestIdx, GetSrc(Op->Header.Args[1].ID()).V2D(), 0);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
if (GetDst(Node).GetCode() != reg.GetCode()) {
@@ -1772,7 +1772,7 @@ DEF_OP(VExtractElement) {
case 8:
mov(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).V2D(), Op->Index);
break;
default: LOGMAN_MSG_A("Unhandled ExtractElementSize: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled VExtractElement element size: {}", OpSize);
}
}
@@ -1791,7 +1791,7 @@ DEF_OP(VDupElement) {
case 8:
dup(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), Op->Index);
break;
default: LOGMAN_MSG_A("Unhandled DupElementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unhandled VDupElement element size: {}", Op->Header.ElementSize);
}
}
@@ -1906,7 +1906,7 @@ DEF_OP(VUShrI) {
ushr(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), Op->BitShift);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1931,7 +1931,7 @@ DEF_OP(VSShrI) {
sshr(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), std::min((uint8_t)(Op->Header.ElementSize * 8 - 1), Op->BitShift));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1959,7 +1959,7 @@ DEF_OP(VShlI) {
shl(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), Op->BitShift);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1980,7 +1980,7 @@ DEF_OP(VUShrNI) {
shrn(GetDst(Node).V2S(), GetSrc(Op->Header.Args[0].ID()).V2D(), Op->BitShift);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -2000,7 +2000,7 @@ DEF_OP(VUShrNI2) {
shrn2(VTMP1.V4S(), GetSrc(Op->Header.Args[1].ID()).V2D(), Op->BitShift);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
mov(GetDst(Node), VTMP1);
@@ -2023,7 +2023,7 @@ DEF_OP(VSXTL) {
case 8:
sxtl(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2S());
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
@@ -2039,7 +2039,7 @@ DEF_OP(VSXTL2) {
case 8:
sxtl2(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V4S());
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
@@ -2055,7 +2055,7 @@ DEF_OP(VUXTL) {
case 8:
uxtl(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2S());
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
@@ -2071,7 +2071,7 @@ DEF_OP(VUXTL2) {
case 8:
uxtl2(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V4S());
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
@@ -2087,7 +2087,7 @@ DEF_OP(VSQXTN) {
case 4:
sqxtn(GetDst(Node).V2S(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
@@ -2109,7 +2109,7 @@ DEF_OP(VSQXTN2) {
sqxtn(VTMP2.V2S(), GetSrc(Op->Header.Args[1].ID()).V2D());
ins(VTMP1.V4S(), 1, VTMP2.V4S(), 0);
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
else {
@@ -2123,7 +2123,7 @@ DEF_OP(VSQXTN2) {
case 4:
sqxtn2(VTMP1.V4S(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
mov(GetDst(Node), VTMP1);
@@ -2141,7 +2141,7 @@ DEF_OP(VSQXTUN) {
case 4:
sqxtun(GetDst(Node).V2S(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
@@ -2163,7 +2163,7 @@ DEF_OP(VSQXTUN2) {
sqxtun(VTMP2.V2S(), GetSrc(Op->Header.Args[1].ID()).V2D());
ins(VTMP1.V4S(), 1, VTMP2.V4S(), 0);
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
else {
@@ -2177,7 +2177,7 @@ DEF_OP(VSQXTUN2) {
case 4:
sqxtun2(VTMP1.V4S(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
mov(GetDst(Node), VTMP1);
@@ -2202,7 +2202,7 @@ DEF_OP(VMul) {
mul(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -2221,7 +2221,7 @@ DEF_OP(VUMull) {
umull(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2S(), GetSrc(Op->Header.Args[1].ID()).V2S());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize >> 1); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize >> 1); break;
}
}
@@ -2240,7 +2240,7 @@ DEF_OP(VSMull) {
smull(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2S(), GetSrc(Op->Header.Args[1].ID()).V2S());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize >> 1); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize >> 1); break;
}
}
@@ -2259,7 +2259,7 @@ DEF_OP(VUMull2) {
umull2(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V4S(), GetSrc(Op->Header.Args[1].ID()).V4S());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize >> 1); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize >> 1); break;
}
}
@@ -2278,7 +2278,7 @@ DEF_OP(VSMull2) {
smull2(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V4S(), GetSrc(Op->Header.Args[1].ID()).V4S());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize >> 1); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize >> 1); break;
}
}
@@ -2297,7 +2297,7 @@ DEF_OP(VUABDL) {
uabdl(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2S(), GetSrc(Op->Header.Args[1].ID()).V2S());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize >> 1); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize >> 1); break;
}
}
@@ -2314,7 +2314,7 @@ DEF_OP(VTBL1) {
tbl(GetDst(Node).V16B(), GetSrc(Op->Header.Args[0].ID()).V16B(), GetSrc(Op->Header.Args[1].ID()).V16B());
break;
}
default: LOGMAN_MSG_A("Unknown OpSize: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown OpSize: {}", OpSize); break;
}
}
+8 -3
View File
@@ -13,6 +13,11 @@ struct InternalThreadState;
namespace FEXCore::CPU {
class CPUBackend;
std::unique_ptr<CPUBackend> CreateX86JITCore(FEXCore::Context::Context *ctx, FEXCore::Core::InternalThreadState *Thread, bool CompileThread);
std::unique_ptr<CPUBackend> CreateArm64JITCore(FEXCore::Context::Context *ctx, FEXCore::Core::InternalThreadState *Thread, bool CompileThread);
}
[[nodiscard]] std::unique_ptr<CPUBackend> CreateX86JITCore(FEXCore::Context::Context *ctx,
FEXCore::Core::InternalThreadState *Thread,
bool CompileThread);
[[nodiscard]] std::unique_ptr<CPUBackend> CreateArm64JITCore(FEXCore::Context::Context *ctx,
FEXCore::Core::InternalThreadState *Thread,
bool CompileThread);
} // namespace FEXCore::CPU
+67 -39
View File
@@ -5,9 +5,21 @@ $end_info$
*/
#include "Interface/Core/JIT/x86_64/JITClass.h"
#include "Interface/IR/Passes/RegisterAllocationPass.h"
#include <FEXCore/IR/IR.h>
#include <FEXCore/Utils/LogManager.h>
#include <array>
#include <stdint.h>
#include <utility>
#include <xbyak/xbyak.h>
namespace FEXCore::CPU {
#define GRS(Node) (IROp->Size <= 4 ? GetSrc<RA_32>(Node) : GetSrc<RA_64>(Node))
#define GRD(Node) (IROp->Size <= 4 ? GetDst<RA_32>(Node) : GetDst<RA_64>(Node))
#define GRCMP(Node) (Op->CompareSize == 4 ? GetSrc<RA_32>(Node) : GetSrc<RA_64>(Node))
#define DEF_OP(x) void X86JITCore::Op_##x(FEXCore::IR::IROp_Header *IROp, uint32_t Node)
DEF_OP(TruncElementPair) {
auto Op = IROp->C<IR::IROp_TruncElementPair>();
@@ -20,7 +32,7 @@ DEF_OP(TruncElementPair) {
mov(Dst.second, Src.second);
break;
}
default: LOGMAN_MSG_A("Unhandled Truncation size: %d", Op->Size); break;
default: LOGMAN_MSG_A_FMT("Unhandled Truncation size: {}", Op->Size); break;
}
}
@@ -70,7 +82,7 @@ DEF_OP(Add) {
case 8:
add(rax, Const);
break;
default: LOGMAN_MSG_A("Unhandled Add size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled Add size: {}", OpSize);
break;
}
} else {
@@ -81,7 +93,7 @@ DEF_OP(Add) {
case 8:
add(rax, GetSrc<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unhandled Add size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled Add size: {}", OpSize);
break;
}
}
@@ -103,7 +115,7 @@ DEF_OP(Sub) {
case 8:
sub(rax, Const);
break;
default: LOGMAN_MSG_A("Unhandled Sub size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled Sub size: {}", OpSize);
break;
}
} else {
@@ -114,7 +126,7 @@ DEF_OP(Sub) {
case 8:
sub(rax, GetSrc<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unhandled Sub size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled Sub size: {}", OpSize);
break;
}
}
@@ -136,7 +148,7 @@ DEF_OP(Neg) {
Src = GetSrc<RA_64>(Op->Header.Args[0].ID());
Dst = GetDst<RA_64>(Node);
break;
default: LOGMAN_MSG_A("Unhandled Neg size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled Neg size: {}", OpSize);
break;
}
mov(Dst, Src);
@@ -160,7 +172,7 @@ DEF_OP(Mul) {
imul(rax, GetSrc<RA_64>(Op->Header.Args[1].ID()));
mov(Dst, rax);
break;
default: LOGMAN_MSG_A("Unknown Sext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown Mul size: {}", OpSize);
}
}
@@ -179,7 +191,7 @@ DEF_OP(UMul) {
mul(GetSrc<RA_64>(Op->Header.Args[1].ID()));
mov(GetDst<RA_64>(Node), rax);
break;
default: LOGMAN_MSG_A("Unknown Sext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown UMul size: {}", OpSize);
}
}
@@ -218,7 +230,7 @@ DEF_OP(Div) {
mov(GetDst<RA_64>(Node), rax);
break;
}
default: LOGMAN_MSG_A("Unknown UDIV Size: %d", Size); break;
default: LOGMAN_MSG_A_FMT("Unknown DIV Size: {}", Size); break;
}
}
@@ -261,7 +273,7 @@ DEF_OP(UDiv) {
mov(GetDst<RA_64>(Node), rax);
break;
}
default: LOGMAN_MSG_A("Unknown UDIV OpSize: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown UDIV OpSize: {}", OpSize); break;
}
}
@@ -298,7 +310,7 @@ DEF_OP(Rem) {
mov(GetDst<RA_64>(Node), rdx);
break;
}
default: LOGMAN_MSG_A("Unknown UDIV Size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Rem Size: {}", OpSize); break;
}
}
@@ -341,7 +353,7 @@ DEF_OP(URem) {
mov(GetDst<RA_64>(Node), rdx);
break;
}
default: LOGMAN_MSG_A("Unknown UDIV OpSize: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown URem OpSize: {}", OpSize); break;
}
}
@@ -360,7 +372,7 @@ DEF_OP(MulH) {
imul(GetSrc<RA_64>(Op->Header.Args[1].ID()));
mov(GetDst<RA_64>(Node), rdx);
break;
default: LOGMAN_MSG_A("Unknown Sext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown MulH size: {}", OpSize);
}
}
@@ -379,7 +391,7 @@ DEF_OP(UMulH) {
mul(GetSrc<RA_64>(Op->Header.Args[1].ID()));
mov(GetDst<RA_64>(Node), rdx);
break;
default: LOGMAN_MSG_A("Unknown Sext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown UMulH size: {}", OpSize);
}
}
@@ -410,6 +422,25 @@ DEF_OP(And) {
mov(Dst, rax);
}
DEF_OP(Andn) {
auto Op = IROp->C<IR::IROp_Andn>();
const auto& Lhs = Op->Header.Args[0];
const auto& Rhs = Op->Header.Args[1];
auto Dst = GRD(Node);
uint64_t Const{};
if (IsInlineConstant(Rhs, &Const)) {
mov(Dst, GRS(Lhs.ID()));
and_(Dst, ~Const);
} else {
const auto Temp = IROp->Size <= 4 ? Xbyak::Reg{rax.cvt32()} : Xbyak::Reg{rax};
mov(Temp, GRS(Rhs.ID()));
not_(Temp);
and_(Temp, GRS(Lhs.ID()));
mov(Dst, Temp);
}
}
DEF_OP(Xor) {
auto Op = IROp->C<IR::IROp_Xor>();
auto Dst = GetDst<RA_64>(Node);
@@ -441,7 +472,7 @@ DEF_OP(Lshl) {
mov(GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
shl(GetDst<RA_64>(Node), Const);
break;
default: LOGMAN_MSG_A("Unknown LSHL Size: %d\n", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown LSHL Size: {}\n", OpSize); break;
};
} else {
mov(rcx, GetSrc<RA_64>(Op->Header.Args[1].ID()));
@@ -456,7 +487,7 @@ DEF_OP(Lshl) {
mov(GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
shl(GetDst<RA_64>(Node), cl);
break;
default: LOGMAN_MSG_A("Unknown LSHL Size: %d\n", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown LSHL Size: {}\n", OpSize); break;
};
}
}
@@ -488,7 +519,7 @@ DEF_OP(Lshr) {
mov(GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
shr(GetDst<RA_64>(Node), Const);
break;
default: LOGMAN_MSG_A("Unknown Size: %d\n", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Size: {}\n", OpSize); break;
};
} else {
@@ -512,7 +543,7 @@ DEF_OP(Lshr) {
mov(GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
shr(GetDst<RA_64>(Node), cl);
break;
default: LOGMAN_MSG_A("Unknown Size: %d\n", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Size: {}\n", OpSize); break;
};
}
}
@@ -546,7 +577,7 @@ DEF_OP(Ashr) {
mov(GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
sar(GetDst<RA_64>(Node), Const);
break;
default: LOGMAN_MSG_A("Unknown ASHR Size: %d\n", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown ASHR Size: {}\n", OpSize); break;
};
} else {
@@ -571,7 +602,7 @@ DEF_OP(Ashr) {
mov(GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
sar(GetDst<RA_64>(Node), cl);
break;
default: LOGMAN_MSG_A("Unknown ASHR Size: %d\n", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown ASHR Size: {}\n", OpSize); break;
};
}
}
@@ -596,7 +627,7 @@ DEF_OP(Ror) {
ror(rax, Const);
break;
}
default: LOGMAN_MSG_A("Unknown ROR Size: %d\n", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown ROR Size: {}\n", OpSize); break;
}
} else {
mov (rcx, GetSrc<RA_64>(Op->Header.Args[1].ID()));
@@ -612,7 +643,7 @@ DEF_OP(Ror) {
ror(rax, cl);
break;
}
default: LOGMAN_MSG_A("Unknown ROR Size: %d\n", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown ROR Size: {}\n", OpSize); break;
}
}
mov(GetDst<RA_64>(Node), rax);
@@ -668,7 +699,7 @@ DEF_OP(LDiv) {
mov(GetDst<RA_64>(Node), rax);
break;
}
default: LOGMAN_MSG_A("Unknown LDIV OpSize: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown LDIV OpSize: {}", OpSize); break;
}
}
@@ -700,7 +731,7 @@ DEF_OP(LUDiv) {
mov(GetDst<RA_64>(Node), rax);
break;
}
default: LOGMAN_MSG_A("Unknown LUDIV OpSize: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown LUDIV OpSize: {}", OpSize); break;
}
}
@@ -732,7 +763,7 @@ DEF_OP(LRem) {
mov(GetDst<RA_64>(Node), rdx);
break;
}
default: LOGMAN_MSG_A("Unknown LREM OpSize: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown LREM OpSize: {}", OpSize); break;
}
}
@@ -764,7 +795,7 @@ DEF_OP(LURem) {
mov(GetDst<RA_64>(Node), rdx);
break;
}
default: LOGMAN_MSG_A("Unknown LUDIV OpSize: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown LUREM OpSize: {}", OpSize); break;
}
}
@@ -829,7 +860,7 @@ DEF_OP(FindMSB) {
case 8:
bsr(GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unknown OpSize: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown FindMSB OpSize: {}", OpSize);
}
}
@@ -853,7 +884,7 @@ DEF_OP(FindTrailingZeros) {
mov(rax, 0x40);
cmovz(GetDst<RA_64>(Node), rax);
break;
default: LOGMAN_MSG_A("Unknown size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown FindTrailingZeros size: {}", OpSize); break;
}
}
@@ -876,7 +907,7 @@ DEF_OP(CountLeadingZeroes) {
lzcnt(GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown CountLeadingZeros size: {}", OpSize); break;
}
}
else {
@@ -915,7 +946,7 @@ DEF_OP(CountLeadingZeroes) {
mov(GetDst<RA_64>(Node), rax);
break;
}
default: LOGMAN_MSG_A("Unknown size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown CountLeadingZeros size: {}", OpSize); break;
}
}
}
@@ -937,7 +968,7 @@ DEF_OP(Rev) {
mov (GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
bswap(GetDst<RA_64>(Node).cvt64());
break;
default: LOGMAN_MSG_A("Unknown REV size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown REV size: {}", OpSize); break;
}
}
@@ -970,7 +1001,7 @@ DEF_OP(Bfi) {
DEF_OP(Bfe) {
auto Op = IROp->C<IR::IROp_Bfe>();
LOGMAN_THROW_A(IROp->Size <= 8, "OpSize is too large for BFE: %d", IROp->Size);
LOGMAN_THROW_A_FMT(IROp->Size <= 8, "OpSize is too large for BFE: {}", IROp->Size);
auto Dst = GetDst<RA_64>(Node);
@@ -1041,10 +1072,6 @@ DEF_OP(Sbfe) {
}
}
#define GRS(Node) (IROp->Size <= 4 ? GetSrc<RA_32>(Node) : GetSrc<RA_64>(Node))
#define GRD(Node) (IROp->Size <= 4 ? GetDst<RA_32>(Node) : GetDst<RA_64>(Node))
#define GRCMP(Node) (Op->CompareSize == 4 ? GetSrc<RA_32>(Node) : GetSrc<RA_64>(Node))
DEF_OP(Select) {
auto Op = IROp->C<IR::IROp_Select>();
auto Dst = GRD(Node);
@@ -1071,7 +1098,7 @@ DEF_OP(Select) {
if (is_const_true || is_const_false) {
if (is_const_false != true || is_const_true != true || const_true != 1 || const_false != 0) {
LOGMAN_MSG_A("Select: Unsupported compare inline parameters");
LOGMAN_MSG_A_FMT("Select: Unsupported compare inline parameters");
}
(this->*SetCC)(al);
movzx(Dst, al);
@@ -1102,7 +1129,7 @@ DEF_OP(VExtractToGPR) {
pextrq(GetDst<RA_64>(Node), GetSrc(Op->Header.Args[0].ID()), Op->Idx);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1214,6 +1241,7 @@ void X86JITCore::RegisterALUHandlers() {
REGISTER_OP(UMULH, UMulH);
REGISTER_OP(OR, Or);
REGISTER_OP(AND, And);
REGISTER_OP(ANDN, Andn);
REGISTER_OP(XOR, Xor);
REGISTER_OP(LSHL, Lshl);
REGISTER_OP(LSHR, Lshr);
@@ -5,7 +5,14 @@ $end_info$
*/
#include "Interface/Core/JIT/x86_64/JITClass.h"
#include "Interface/IR/Passes/RegisterAllocationPass.h"
#include <FEXCore/IR/IR.h>
#include <FEXCore/Utils/LogManager.h>
#include <array>
#include <stdint.h>
#include <utility>
#include <xbyak/xbyak.h>
namespace FEXCore::CPU {
#define DEF_OP(x) void X86JITCore::Op_##x(FEXCore::IR::IROp_Header *IROp, uint32_t Node)
@@ -55,7 +62,7 @@ DEF_OP(CASPair) {
mov(Dst.second, rdx);
break;
}
default: LOGMAN_MSG_A("Unsupported: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported: {}", OpSize);
}
}
@@ -104,7 +111,7 @@ DEF_OP(CAS) {
mov (GetDst<RA_64>(Node), rax);
break;
}
default: LOGMAN_MSG_A("Unsupported: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported: {}", OpSize);
}
}
@@ -114,7 +121,7 @@ DEF_OP(AtomicAdd) {
Xbyak::Reg MemReg = GetSrc<RA_64>(Op->Header.Args[0].ID());
lock();
switch (Op->Size) {
switch (IROp->Size) {
case 1:
add(byte [MemReg], GetSrc<RA_8>(Op->Header.Args[1].ID()));
break;
@@ -127,7 +134,7 @@ DEF_OP(AtomicAdd) {
case 8:
add(qword [MemReg], GetSrc<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unhandled AtomicAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicAdd size: {}", IROp->Size);
}
}
@@ -136,7 +143,7 @@ DEF_OP(AtomicSub) {
Xbyak::Reg MemReg = GetSrc<RA_64>(Op->Header.Args[0].ID());
lock();
switch (Op->Size) {
switch (IROp->Size) {
case 1:
sub(byte [MemReg], GetSrc<RA_8>(Op->Header.Args[1].ID()));
break;
@@ -149,7 +156,7 @@ DEF_OP(AtomicSub) {
case 8:
sub(qword [MemReg], GetSrc<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unhandled AtomicAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicAdd size: {}", IROp->Size);
}
}
@@ -158,7 +165,7 @@ DEF_OP(AtomicAnd) {
Xbyak::Reg MemReg = GetSrc<RA_64>(Op->Header.Args[0].ID());
lock();
switch (Op->Size) {
switch (IROp->Size) {
case 1:
and_(byte [MemReg], GetSrc<RA_8>(Op->Header.Args[1].ID()));
break;
@@ -171,7 +178,7 @@ DEF_OP(AtomicAnd) {
case 8:
and_(qword [MemReg], GetSrc<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unhandled AtomicAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicAdd size: {}", IROp->Size);
}
}
@@ -180,7 +187,7 @@ DEF_OP(AtomicOr) {
Xbyak::Reg MemReg = GetSrc<RA_64>(Op->Header.Args[0].ID());
lock();
switch (Op->Size) {
switch (IROp->Size) {
case 1:
or_(byte [MemReg], GetSrc<RA_8>(Op->Header.Args[1].ID()));
break;
@@ -193,7 +200,7 @@ DEF_OP(AtomicOr) {
case 8:
or_(qword [MemReg], GetSrc<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unhandled AtomicAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicAdd size: {}", IROp->Size);
}
}
@@ -202,7 +209,7 @@ DEF_OP(AtomicXor) {
Xbyak::Reg MemReg = GetSrc<RA_64>(Op->Header.Args[0].ID());
lock();
switch (Op->Size) {
switch (IROp->Size) {
case 1:
xor_(byte [MemReg], GetSrc<RA_8>(Op->Header.Args[1].ID()));
break;
@@ -215,7 +222,7 @@ DEF_OP(AtomicXor) {
case 8:
xor_(qword [MemReg], GetSrc<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unhandled AtomicAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicAdd size: {}", IROp->Size);
}
}
@@ -225,7 +232,7 @@ DEF_OP(AtomicSwap) {
Xbyak::Reg MemReg = rax;
mov(MemReg, GetSrc<RA_64>(Op->Header.Args[0].ID()));
switch (Op->Size) {
switch (IROp->Size) {
case 1:
movzx(GetDst<RA_64>(Node), GetSrc<RA_8>(Op->Header.Args[1].ID()));
lock();
@@ -246,7 +253,7 @@ DEF_OP(AtomicSwap) {
lock();
xchg(qword [MemReg], GetDst<RA_64>(Node));
break;
default: LOGMAN_MSG_A("Unhandled AtomicAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicSwap size: {}", IROp->Size);
}
}
@@ -254,7 +261,7 @@ DEF_OP(AtomicFetchAdd) {
auto Op = IROp->C<IR::IROp_AtomicFetchAdd>();
Xbyak::Reg MemReg = GetSrc<RA_64>(Op->Header.Args[0].ID());
switch (Op->Size) {
switch (IROp->Size) {
case 1:
movzx(rcx, GetSrc<RA_8>(Op->Header.Args[1].ID()));
lock();
@@ -279,7 +286,7 @@ DEF_OP(AtomicFetchAdd) {
xadd(qword [MemReg], rcx);
mov(GetDst<RA_64>(Node), rcx);
break;
default: LOGMAN_MSG_A("Unhandled AtomicFetchAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicFetchAdd size: {}", IROp->Size);
}
}
@@ -287,7 +294,7 @@ DEF_OP(AtomicFetchSub) {
auto Op = IROp->C<IR::IROp_AtomicFetchSub>();
Xbyak::Reg MemReg = GetSrc<RA_64>(Op->Header.Args[0].ID());
switch (Op->Size) {
switch (IROp->Size) {
case 1:
mov(cl, GetSrc<RA_8>(Op->Header.Args[1].ID()));
neg(cl);
@@ -316,7 +323,7 @@ DEF_OP(AtomicFetchSub) {
xadd(qword [MemReg], rcx);
mov(GetDst<RA_64>(Node), rcx);
break;
default: LOGMAN_MSG_A("Unhandled AtomicFetchAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicFetchSub size: {}", IROp->Size);
}
}
@@ -326,7 +333,7 @@ DEF_OP(AtomicFetchAnd) {
// TMP1 = rax
Xbyak::Reg MemReg = GetSrc<RA_64>(Op->Header.Args[0].ID());
switch (Op->Size) {
switch (IROp->Size) {
case 1: {
mov(TMP1.cvt8(), byte [MemReg]);
@@ -394,7 +401,7 @@ DEF_OP(AtomicFetchAnd) {
mov(GetDst<RA_64>(Node), TMP3.cvt64());
break;
}
default: LOGMAN_MSG_A("Unhandled AtomicFetchAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicFetchAnd size: {}", IROp->Size);
}
}
@@ -403,7 +410,7 @@ DEF_OP(AtomicFetchOr) {
// TMP1 = rax
Xbyak::Reg MemReg = GetSrc<RA_64>(Op->Header.Args[0].ID());
switch (Op->Size) {
switch (IROp->Size) {
case 1: {
mov(TMP1.cvt8(), byte [MemReg]);
@@ -471,7 +478,7 @@ DEF_OP(AtomicFetchOr) {
mov(GetDst<RA_64>(Node), TMP3.cvt64());
break;
}
default: LOGMAN_MSG_A("Unhandled AtomicFetchAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicFetchOr size: {}", IROp->Size);
}
}
@@ -480,7 +487,7 @@ DEF_OP(AtomicFetchXor) {
// TMP1 = rax
Xbyak::Reg MemReg = GetSrc<RA_64>(Op->Header.Args[0].ID());
switch (Op->Size) {
switch (IROp->Size) {
case 1: {
mov(TMP1.cvt8(), byte [MemReg]);
@@ -548,7 +555,83 @@ DEF_OP(AtomicFetchXor) {
mov(GetDst<RA_64>(Node), TMP3.cvt64());
break;
}
default: LOGMAN_MSG_A("Unhandled AtomicFetchAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicFetchXor size: {}", IROp->Size);
}
}
DEF_OP(AtomicFetchNeg) {
auto Op = IROp->C<IR::IROp_AtomicFetchNeg>();
Xbyak::Reg MemReg = GetSrc<RA_64>(Op->Header.Args[0].ID());
switch (IROp->Size) {
case 1: {
mov(TMP1.cvt8(), byte [MemReg]);
Label Loop;
L(Loop);
mov(TMP2.cvt8(), TMP1.cvt8());
mov(TMP3.cvt8(), TMP1.cvt8());
neg(TMP2.cvt8());
// Updates RAX with the value from memory
lock(); cmpxchg(byte [MemReg], TMP2.cvt8());
jne(Loop);
// Result is the previous value from memory, which is currently in TMP3
movzx(GetDst<RA_64>(Node), TMP3.cvt8());
break;
}
case 2: {
mov(TMP1.cvt16(), word [MemReg]);
Label Loop;
L(Loop);
mov(TMP2.cvt16(), TMP1.cvt16());
mov(TMP3.cvt16(), TMP1.cvt16());
neg(TMP2.cvt16());
// Updates RAX with the value from memory
lock(); cmpxchg(word [MemReg], TMP2.cvt16());
jne(Loop);
// Result is the previous value from memory, which is currently in TMP3
movzx(GetDst<RA_64>(Node), TMP3.cvt16());
break;
}
case 4: {
mov(TMP1.cvt32(), dword [MemReg]);
Label Loop;
L(Loop);
mov(TMP2.cvt32(), TMP1.cvt32());
mov(TMP3.cvt32(), TMP1.cvt32());
neg(TMP2.cvt32());
// Updates RAX with the value from memory
lock(); cmpxchg(dword [MemReg], TMP2.cvt32());
jne(Loop);
// Result is the previous value from memory, which is currently in TMP3
mov(GetDst<RA_32>(Node), TMP3.cvt32());
break;
}
case 8: {
mov(TMP1.cvt64(), qword [MemReg]);
Label Loop;
L(Loop);
mov(TMP2.cvt64(), TMP1.cvt64());
mov(TMP3.cvt64(), TMP1.cvt64());
neg(TMP2.cvt64());
// Updates RAX with the value from memory
lock(); cmpxchg(qword [MemReg], TMP2.cvt64());
jne(Loop);
// Result is the previous value from memory, which is currently in TMP3
mov(GetDst<RA_64>(Node), TMP3.cvt64());
break;
}
default: LOGMAN_MSG_A_FMT("Unhandled AtomicFetchNeg size: {}", IROp->Size);
}
}
@@ -568,6 +651,7 @@ void X86JITCore::RegisterAtomicHandlers() {
REGISTER_OP(ATOMICFETCHAND, AtomicFetchAnd);
REGISTER_OP(ATOMICFETCHOR, AtomicFetchOr);
REGISTER_OP(ATOMICFETCHXOR, AtomicFetchXor);
REGISTER_OP(ATOMICFETCHNEG, AtomicFetchNeg);
#undef REGISTER_OP
}
}
@@ -4,25 +4,41 @@ tags: backend|x86-64
$end_info$
*/
#include "Interface/Context/Context.h"
#include "Interface/Core/CPUID.h"
#include "Interface/Core/Dispatcher/Dispatcher.h"
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/JIT/x86_64/JITClass.h"
#include "Interface/IR/Passes/RegisterAllocationPass.h"
#include "Interface/HLE/Thunks/Thunks.h"
#include <FEXCore/Core/CPUID.h>
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/Core/X86Enums.h>
#include <FEXCore/Debug/InternalThreadState.h>
#include <FEXCore/HLE/SyscallHandler.h>
#include <Interface/HLE/Thunks/Thunks.h>
#include <FEXCore/IR/IR.h>
#include <FEXCore/Utils/LogManager.h>
#include <array>
#include <memory>
#include <stddef.h>
#include <stdint.h>
#include <unordered_map>
#include <utility>
#include <xbyak/xbyak.h>
namespace FEXCore::CPU {
#define DEF_OP(x) void X86JITCore::Op_##x(FEXCore::IR::IROp_Header *IROp, uint32_t Node)
DEF_OP(GuestCallDirect) {
LogMan::Msg::D("Unimplemented");
LogMan::Msg::DFmt("Unimplemented");
}
DEF_OP(GuestCallIndirect) {
LogMan::Msg::D("Unimplemented");
LogMan::Msg::DFmt("Unimplemented");
}
DEF_OP(GuestReturn) {
LogMan::Msg::D("Unimplemented");
LogMan::Msg::DFmt("Unimplemented");
}
DEF_OP(SignalReturn) {
@@ -5,7 +5,13 @@ $end_info$
*/
#include "Interface/Core/JIT/x86_64/JITClass.h"
#include "Interface/IR/Passes/RegisterAllocationPass.h"
#include <FEXCore/IR/IR.h>
#include <FEXCore/Utils/LogManager.h>
#include <array>
#include <stdint.h>
#include <xbyak/xbyak.h>
namespace FEXCore::CPU {
@@ -31,7 +37,7 @@ DEF_OP(VInsGPR) {
pinsrq(GetDst(Node), GetSrc<RA_64>(Op->Header.Args[1].ID()), Op->Index);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -52,7 +58,7 @@ DEF_OP(VCastFromGPR) {
case 8:
vmovq(GetDst(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()).cvt64());
break;
default: LOGMAN_MSG_A("Unknown castGPR element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown VCastFromGPR element size: {}", Op->Header.ElementSize);
}
}
@@ -91,7 +97,7 @@ DEF_OP(Float_FToF) {
cvtsd2ss(GetDst(Node), GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown FCVT sizes: 0x%x", Conv);
default: LOGMAN_MSG_A_FMT("Unknown Float_FToF sizes: 0x{:x}", Conv);
}
}
@@ -113,7 +119,7 @@ DEF_OP(Vector_SToF) {
cvtsi2sd(xmm15, rax);
movlhps(GetDst(Node), xmm15);
break;
default: LOGMAN_MSG_A("Unknown castGPR element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Vector_SToF element size: {}", Op->Header.ElementSize);
}
}
@@ -126,7 +132,7 @@ DEF_OP(Vector_FToZS) {
case 8:
cvttpd2dq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unknown castGPR element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Vector_FToZS element size: {}", Op->Header.ElementSize);
}
}
@@ -139,7 +145,7 @@ DEF_OP(Vector_FToS) {
case 8:
cvtpd2dq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unknown castGPR element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Vector_FToS element size: {}", Op->Header.ElementSize);
}
}
@@ -156,7 +162,7 @@ DEF_OP(Vector_FToF) {
cvtpd2ps(GetDst(Node), GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Conversion Type : 0%04x", Conv); break;
default: LOGMAN_MSG_A_FMT("Unknown Vector_FToF conversion type : 0x{:04x}", Conv); break;
}
}
@@ -5,7 +5,12 @@ $end_info$
*/
#include "Interface/Core/JIT/x86_64/JITClass.h"
#include "Interface/IR/Passes/RegisterAllocationPass.h"
#include <FEXCore/IR/IR.h>
#include <array>
#include <stdint.h>
#include <xbyak/xbyak.h>
namespace FEXCore::CPU {
#define DEF_OP(x) void X86JITCore::Op_##x(FEXCore::IR::IROp_Header *IROp, uint32_t Node)
@@ -5,7 +5,12 @@ $end_info$
*/
#include "Interface/Core/JIT/x86_64/JITClass.h"
#include "Interface/IR/Passes/RegisterAllocationPass.h"
#include <FEXCore/IR/IR.h>
#include <array>
#include <stdint.h>
#include <xbyak/xbyak.h>
namespace FEXCore::CPU {
+48 -33
View File
@@ -6,28 +6,45 @@ $end_info$
*/
#include "Interface/Context/Context.h"
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/Dispatcher/Dispatcher.h"
#include "Interface/Core/Dispatcher/X86Dispatcher.h"
#include "Interface/Core/Interpreter/InterpreterOps.h"
#include "Interface/Core/JIT/x86_64/JITClass.h"
#include "Interface/Core/InternalThreadState.h"
#include "Interface/IR/PassManager.h"
#include "Interface/IR/Passes/RegisterAllocationPass.h"
#include <FEXCore/Core/X86Enums.h>
#include <FEXCore/Core/UContext.h>
#include <FEXCore/Core/CPUBackend.h>
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/Core/SignalDelegator.h>
#include <FEXCore/Debug/InternalThreadState.h>
#include <FEXCore/IR/IR.h>
#include <FEXCore/IR/IntrusiveIRList.h>
#include <FEXCore/IR/RegisterAllocationData.h>
#include <FEXCore/Utils/Allocator.h>
#include <FEXCore/Utils/LogManager.h>
#include <cmath>
#include <algorithm>
#include <array>
#include <bits/types/stack_t.h>
#include <memory>
#include <stddef.h>
#include <stdint.h>
#include <signal.h>
#include "Interface/Core/Interpreter/InterpreterOps.h"
#include <sys/mman.h>
#include <tuple>
#include <unordered_map>
#include <utility>
#include <vector>
#include <xbyak/xbyak.h>
// #define DEBUG_RA 1
// #define DEBUG_CYCLES
namespace FEXCore::CPU {
CodeBuffer AllocateNewCodeBuffer(size_t Size) {
CodeBuffer AllocateNewCodeBuffer(FEXCore::Context::Context *CTX, size_t Size) {
CodeBuffer Buffer;
Buffer.Size = Size;
Buffer.Ptr = static_cast<uint8_t*>(
@@ -36,7 +53,10 @@ CodeBuffer AllocateNewCodeBuffer(size_t Size) {
PROT_READ | PROT_WRITE | PROT_EXEC,
MAP_PRIVATE | MAP_ANONYMOUS,
-1, 0));
LOGMAN_THROW_A(Buffer.Ptr != reinterpret_cast<uint8_t*>(~0ULL), "Couldn't allocate code buffer");
LOGMAN_THROW_A_FMT(Buffer.Ptr != reinterpret_cast<uint8_t*>(~0ULL), "Couldn't allocate code buffer");
if (CTX->Config.GlobalJITNaming()) {
CTX->Symbols.RegisterJITSpace(Buffer.Ptr, Buffer.Size);
}
return Buffer;
}
@@ -44,19 +64,15 @@ void FreeCodeBuffer(CodeBuffer Buffer) {
FEXCore::Allocator::munmap(Buffer.Ptr, Buffer.Size);
}
}
namespace FEXCore::CPU {
void X86JITCore::CopyNecessaryDataForCompileThread(CPUBackend *Original) {
X86JITCore *Core = reinterpret_cast<X86JITCore*>(Original);
ThreadSharedData = Core->ThreadSharedData;
}
void X86JITCore::PushRegs() {
for (auto &Xmm : RAXMM_x) {
sub(rsp, 16);
movaps(ptr[rsp], Xmm);
sub(rsp, 16 * RAXMM_x.size());
for (size_t i = 0; i < RAXMM_x.size(); ++i) {
movaps(ptr[rsp + i * 16], RAXMM_x[i]);
}
for (auto &Reg : RA64)
@@ -75,18 +91,18 @@ void X86JITCore::PopRegs() {
for (uint32_t i = RA64.size(); i > 0; --i)
pop(RA64[i - 1]);
for (uint32_t i = RAXMM_x.size(); i > 0; --i) {
movaps(RAXMM_x[i - 1], ptr[rsp]);
add(rsp, 16);
for (size_t i = 0; i < RAXMM_x.size(); ++i) {
movaps(RAXMM_x[i], ptr[rsp + i * 16]);
}
add(rsp, 16 * RAXMM_x.size());
}
void X86JITCore::Op_Unhandled(FEXCore::IR::IROp_Header *IROp, uint32_t Node) {
FallbackInfo Info;
if (!InterpreterOps::GetFallbackHandler(IROp, &Info)) {
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
auto Name = FEXCore::IR::GetName(IROp->Op);
LOGMAN_MSG_A("Unhandled IR Op: %s", std::string(Name).c_str());
LOGMAN_MSG_A_FMT("Unhandled IR Op: {}", FEXCore::IR::GetName(IROp->Op));
#endif
} else {
switch(Info.ABI) {
@@ -285,8 +301,7 @@ void X86JITCore::Op_Unhandled(FEXCore::IR::IROp_Header *IROp, uint32_t Node) {
case FABI_UNKNOWN:
default:
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
auto Name = FEXCore::IR::GetName(IROp->Op);
LOGMAN_MSG_A("Unhandled IR Fallback abi: %s %d", std::string(Name).c_str(), Info.ABI);
LOGMAN_MSG_A_FMT("Unhandled IR Fallback ABI: {} {}", FEXCore::IR::GetName(IROp->Op), Info.ABI);
#endif
break;
}
@@ -304,7 +319,7 @@ X86JITCore::X86JITCore(FEXCore::Context::Context *ctx, FEXCore::Core::InternalTh
{
CurrentCodeBuffer = &InitialCodeBuffer;
RAPass = Thread->PassManager->GetRAPass();
RAPass = Thread->PassManager->GetPass<IR::RegisterAllocationPass>("RA");
RAPass->AllocateRegisterSet(RegisterCount, RegisterClasses);
RAPass->AddRegisters(FEXCore::IR::GPRClass, NumGPRs);
@@ -360,7 +375,7 @@ X86JITCore::X86JITCore(FEXCore::Context::Context *ctx, FEXCore::Core::InternalTh
return Core->Dispatcher->HandleGuestSignal(Signal, info, ucontext, GuestAction, GuestStack);
};
for (uint32_t Signal = 0; Signal < SignalDelegator::MAX_SIGNALS; ++Signal) {
for (uint32_t Signal = 0; Signal <= SignalDelegator::MAX_SIGNALS; ++Signal) {
CTX->SignalDelegation->RegisterHostSignalHandlerForGuest(Signal, GuestSignalHandler);
}
}
@@ -402,7 +417,7 @@ void X86JITCore::ClearCache() {
CurrentCodeBuffer->Size *= 1.5;
CurrentCodeBuffer->Size = std::min(CurrentCodeBuffer->Size, MAX_CODE_SIZE);
InitialCodeBuffer = AllocateNewCodeBuffer(CurrentCodeBuffer->Size);
InitialCodeBuffer = AllocateNewCodeBuffer(CTX, CurrentCodeBuffer->Size);
setNewBuffer(InitialCodeBuffer.Ptr, InitialCodeBuffer.Size);
}
}
@@ -410,7 +425,7 @@ void X86JITCore::ClearCache() {
// We have signal handlers that have generated code
// This means that we can not safely clear the code at this point in time
// Allocate some new code buffers that we can switch over to instead
auto NewCodeBuffer = AllocateNewCodeBuffer(X86JITCore::INITIAL_CODE_SIZE);
auto NewCodeBuffer = AllocateNewCodeBuffer(CTX, X86JITCore::INITIAL_CODE_SIZE);
EmplaceNewCodeBuffer(NewCodeBuffer);
setNewBuffer(NewCodeBuffer.Ptr, NewCodeBuffer.Size);
}
@@ -419,7 +434,7 @@ void X86JITCore::ClearCache() {
IR::PhysicalRegister X86JITCore::GetPhys(uint32_t Node) const {
auto PhyReg = RAData->GetNodeRegister(Node);
LOGMAN_THROW_A(PhyReg.Raw != 255, "Couldn't Allocate register for node: ssa%d. Class: %d", Node, PhyReg.Class);
LOGMAN_THROW_A_FMT(PhyReg.Raw != 255, "Couldn't Allocate register for node: ssa{}. Class: {}", Node, PhyReg.Class);
return PhyReg;
}
@@ -568,7 +583,7 @@ std::tuple<X86JITCore::SetCC, X86JITCore::CMovCC, X86JITCore::JCC> X86JITCore::G
case FEXCore::IR::COND_VS:
case FEXCore::IR::COND_VC:
default:
LOGMAN_MSG_A("Unsupported compare type");
LOGMAN_MSG_A_FMT("Unsupported compare type");
break;
}
@@ -611,7 +626,7 @@ void *X86JITCore::CompileCode(uint64_t Entry, [[maybe_unused]] FEXCore::IR::IRLi
L(RunBlock);
}
LOGMAN_THROW_A(RAData != nullptr, "Needs RA");
LOGMAN_THROW_A_FMT(RAData != nullptr, "Needs RA");
SpillSlots = RAData->SpillSlots();
@@ -669,7 +684,7 @@ void *X86JITCore::CompileCode(uint64_t Entry, [[maybe_unused]] FEXCore::IR::IRLi
{
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
auto BlockIROp = BlockHeader->CW<IROp_CodeBlock>();
LOGMAN_THROW_A(BlockIROp->Header.Op == IR::OP_CODEBLOCK, "IR type failed to be a code block");
LOGMAN_THROW_A_FMT(BlockIROp->Header.Op == IR::OP_CODEBLOCK, "IR type failed to be a code block");
#endif
uint32_t Node = IR->GetID(BlockNode);
@@ -721,7 +736,7 @@ void *X86JITCore::CompileCode(uint64_t Entry, [[maybe_unused]] FEXCore::IR::IRLi
Inst << "Reg" << GetPhys(ArgNode) << (i + 1 == NumArgs ? "" : ", ");
}
LogMan::Msg::D("%s", Inst.str().c_str());
LogMan::Msg::DFmt("{}", Inst.str());
}
#endif
uint32_t ID = IR->GetID(CodeNode);
@@ -772,6 +787,6 @@ uint64_t X86JITCore::ExitFunctionLink(X86JITCore *core, FEXCore::Core::CpuStateF
}
std::unique_ptr<CPUBackend> CreateX86JITCore(FEXCore::Context::Context *ctx, FEXCore::Core::InternalThreadState *Thread, bool CompileThread) {
return std::make_unique<X86JITCore>(ctx, Thread, AllocateNewCodeBuffer(CompileThread ? X86JITCore::MAX_CODE_SIZE : X86JITCore::INITIAL_CODE_SIZE), CompileThread);
return std::make_unique<X86JITCore>(ctx, Thread, AllocateNewCodeBuffer(ctx, CompileThread ? X86JITCore::MAX_CODE_SIZE : X86JITCore::INITIAL_CODE_SIZE), CompileThread);
}
}
+28 -23
View File
@@ -6,7 +6,6 @@ $end_info$
#pragma once
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/BlockSamplingData.h"
#include "Interface/Core/Dispatcher/Dispatcher.h"
@@ -31,14 +30,9 @@ struct CodeBuffer {
size_t Size;
};
CodeBuffer AllocateNewCodeBuffer(size_t Size);
[[nodiscard]] CodeBuffer AllocateNewCodeBuffer(size_t Size);
void FreeCodeBuffer(CodeBuffer Buffer);
}
namespace FEXCore::CPU {
// Temp registers
// rax, rcx, rdx, rsi, r8, r9,
// r10, r11
@@ -63,14 +57,22 @@ const std::array<Xbyak::Xmm, 11> RAXMM_x = { xmm1, xmm2, xmm3, xmm4, xmm5, xmm6
class X86JITCore final : public CPUBackend, public Xbyak::CodeGenerator {
public:
explicit X86JITCore(FEXCore::Context::Context *ctx, FEXCore::Core::InternalThreadState *Thread, CodeBuffer Buffer, bool CompileThread);
explicit X86JITCore(FEXCore::Context::Context *ctx,
FEXCore::Core::InternalThreadState *Thread,
CodeBuffer Buffer,
bool CompileThread);
~X86JITCore() override;
std::string GetName() override { return "JIT"; }
void *CompileCode(uint64_t Entry, FEXCore::IR::IRListView const *IR, FEXCore::Core::DebugData *DebugData, FEXCore::IR::RegisterAllocationData *RAData) override;
void *MapRegion(void* HostPtr, uint64_t, uint64_t) override { return HostPtr; }
[[nodiscard]] std::string GetName() override { return "JIT"; }
bool NeedsOpDispatch() override { return true; }
[[nodiscard]] void *CompileCode(uint64_t Entry,
FEXCore::IR::IRListView const *IR,
FEXCore::Core::DebugData *DebugData,
FEXCore::IR::RegisterAllocationData *RAData) override;
[[nodiscard]] void *MapRegion(void* HostPtr, uint64_t, uint64_t) override { return HostPtr; }
[[nodiscard]] bool NeedsOpDispatch() override { return true; }
void ClearCache() override;
@@ -112,26 +114,27 @@ private:
constexpr static uint8_t RA_64 = 3;
constexpr static uint8_t RA_XMM = 4;
IR::PhysicalRegister GetPhys(uint32_t Node) const;
[[nodiscard]] IR::PhysicalRegister GetPhys(uint32_t Node) const;
bool IsFPR(uint32_t Node) const;
bool IsGPR(uint32_t Node) const;
[[nodiscard]] bool IsFPR(uint32_t Node) const;
[[nodiscard]] bool IsGPR(uint32_t Node) const;
template<uint8_t RAType>
Xbyak::Reg GetSrc(uint32_t Node) const;
[[nodiscard]] Xbyak::Reg GetSrc(uint32_t Node) const;
template<uint8_t RAType>
std::pair<Xbyak::Reg, Xbyak::Reg> GetSrcPair(uint32_t Node) const;
[[nodiscard]] std::pair<Xbyak::Reg, Xbyak::Reg> GetSrcPair(uint32_t Node) const;
template<uint8_t RAType>
Xbyak::Reg GetDst(uint32_t Node) const;
[[nodiscard]] Xbyak::Reg GetDst(uint32_t Node) const;
Xbyak::Xmm GetSrc(uint32_t Node) const;
Xbyak::Xmm GetDst(uint32_t Node) const;
[[nodiscard]] Xbyak::Xmm GetSrc(uint32_t Node) const;
[[nodiscard]] Xbyak::Xmm GetDst(uint32_t Node) const;
Xbyak::RegExp GenerateModRM(Xbyak::Reg Base, IR::OrderedNodeWrapper Offset, IR::MemOffsetType OffsetType, uint8_t OffsetScale) const;
[[nodiscard]] Xbyak::RegExp GenerateModRM(Xbyak::Reg Base, IR::OrderedNodeWrapper Offset,
IR::MemOffsetType OffsetType, uint8_t OffsetScale) const;
bool IsInlineConstant(const IR::OrderedNodeWrapper& Node, uint64_t* Value = nullptr) const;
bool IsInlineEntrypointOffset(const IR::OrderedNodeWrapper& WNode, uint64_t* Value) const;
[[nodiscard]] bool IsInlineConstant(const IR::OrderedNodeWrapper& Node, uint64_t* Value = nullptr) const;
[[nodiscard]] bool IsInlineEntrypointOffset(const IR::OrderedNodeWrapper& WNode, uint64_t* Value) const;
IR::RegisterAllocationPass *RAPass;
FEXCore::IR::RegisterAllocationData *RAData;
@@ -217,6 +220,7 @@ private:
DEF_OP(UMulH);
DEF_OP(Or);
DEF_OP(And);
DEF_OP(Andn);
DEF_OP(Xor);
DEF_OP(Lshl);
DEF_OP(Lshr);
@@ -260,6 +264,7 @@ private:
DEF_OP(AtomicFetchAnd);
DEF_OP(AtomicFetchOr);
DEF_OP(AtomicFetchXor);
DEF_OP(AtomicFetchNeg);
///< Branch ops
DEF_OP(GuestCallDirect);
@@ -5,9 +5,15 @@ $end_info$
*/
#include "Interface/Core/JIT/x86_64/JITClass.h"
#include "Interface/IR/Passes/RegisterAllocationPass.h"
#include <cmath>
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/IR/IR.h>
#include <FEXCore/Utils/LogManager.h>
#include <array>
#include <stddef.h>
#include <stdint.h>
#include <xbyak/xbyak.h>
namespace FEXCore::CPU {
@@ -36,10 +42,10 @@ DEF_OP(LoadContext) {
}
break;
case 16: {
LOGMAN_MSG_A("Invalid GPR load of size 16");
LOGMAN_MSG_A_FMT("Invalid GPR load of size 16");
}
break;
default: LOGMAN_MSG_A("Unhandled LoadContext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled LoadContext size: {}", OpSize);
}
}
else {
@@ -69,7 +75,7 @@ DEF_OP(LoadContext) {
movups(GetDst(Node), xword [STATE + Op->Offset]);
}
break;
default: LOGMAN_MSG_A("Unhandled LoadContext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled LoadContext size: {}", OpSize);
}
}
}
@@ -98,9 +104,9 @@ DEF_OP(StoreContext) {
}
break;
case 16:
LogMan::Msg::D("Invalid store size of 16");
LogMan::Msg::DFmt("Invalid store size of 16");
break;
default: LOGMAN_MSG_A("Unhandled StoreContext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled StoreContext size: {}", OpSize);
}
}
else {
@@ -129,14 +135,14 @@ DEF_OP(StoreContext) {
movups(xword [STATE + Op->Offset], GetSrc(Op->Header.Args[0].ID()));
}
break;
default: LOGMAN_MSG_A("Unhandled StoreContext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled StoreContext size: {}", OpSize);
}
}
}
DEF_OP(LoadContextIndexed) {
auto Op = IROp->C<IR::IROp_LoadContextIndexed>();
size_t size = Op->Size;
size_t size = IROp->Size;
Reg index = GetSrc<RA_64>(Op->Header.Args[0].ID());
if (Op->Class.Val == 0) {
@@ -160,17 +166,18 @@ DEF_OP(LoadContextIndexed) {
mov(GetDst<RA_64>(Node), qword [rax + index * Op->Stride]);
break;
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed size: %d", Op->Size);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed size: {}", IROp->Size);
break;
}
break;
}
case 16:
LOGMAN_MSG_A("Invalid Class load of size 16");
LOGMAN_MSG_A_FMT("Invalid Class load of size 16");
break;
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed stride: %d", Op->Stride);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed stride: {}", Op->Stride);
break;
}
}
else {
switch (Op->Stride) {
@@ -195,7 +202,8 @@ DEF_OP(LoadContextIndexed) {
vmovq(GetDst(Node), qword [rax + index * Op->Stride]);
break;
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed size: %d", Op->Size);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed size: {}", IROp->Size);
break;
}
break;
}
@@ -223,12 +231,14 @@ DEF_OP(LoadContextIndexed) {
movups(GetDst(Node), xword [STATE + rax]);
break;
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed size: %d", Op->Size);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed size: {}", IROp->Size);
break;
}
break;
}
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed stride: %d", Op->Stride);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed stride: {}", Op->Stride);
break;
}
}
}
@@ -236,7 +246,7 @@ DEF_OP(LoadContextIndexed) {
DEF_OP(StoreContextIndexed) {
auto Op = IROp->C<IR::IROp_StoreContextIndexed>();
Reg index = GetSrc<RA_64>(Op->Header.Args[1].ID());
size_t size = Op->Size;
size_t size = IROp->Size;
if (Op->Class.Val == 0) {
auto value = GetSrc<RA_64>(Op->Header.Args[0].ID());
@@ -248,13 +258,14 @@ DEF_OP(StoreContextIndexed) {
case 4:
case 8: {
if (!(size == 1 || size == 2 || size == 4 || size == 8)) {
LOGMAN_MSG_A("Unhandled StoreContextIndexed size: %d", Op->Size);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed size: {}", IROp->Size);
}
mov(AddressFrame(Op->Size * 8) [rax + index * Op->Stride], value);
mov(AddressFrame(IROp->Size * 8) [rax + index * Op->Stride], value);
break;
}
default:
LOGMAN_MSG_A("Unhandled StoreContextIndexed stride: %d", Op->Stride);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed stride: {}", Op->Stride);
break;
}
}
else {
@@ -267,19 +278,20 @@ DEF_OP(StoreContextIndexed) {
lea(rax, dword [STATE + Op->BaseOffset]);
switch (size) {
case 1:
pextrb(AddressFrame(Op->Size * 8) [rax + index * Op->Stride], value, 0);
pextrb(AddressFrame(IROp->Size * 8) [rax + index * Op->Stride], value, 0);
break;
case 2:
pextrw(AddressFrame(Op->Size * 8) [rax + index * Op->Stride], value, 0);
pextrw(AddressFrame(IROp->Size * 8) [rax + index * Op->Stride], value, 0);
break;
case 4:
vmovd(AddressFrame(Op->Size * 8) [rax + index * Op->Stride], value);
vmovd(AddressFrame(IROp->Size * 8) [rax + index * Op->Stride], value);
break;
case 8:
vmovq(AddressFrame(Op->Size * 8) [rax + index * Op->Stride], value);
vmovq(AddressFrame(IROp->Size * 8) [rax + index * Op->Stride], value);
break;
default:
LOGMAN_MSG_A("Unhandled StoreContextIndexed size: %d", size);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed size: {}", size);
break;
}
break;
}
@@ -289,16 +301,16 @@ DEF_OP(StoreContextIndexed) {
lea(rax, dword [rax + Op->BaseOffset]);
switch (size) {
case 1:
pextrb(AddressFrame(Op->Size * 8) [STATE + rax], value, 0);
pextrb(AddressFrame(IROp->Size * 8) [STATE + rax], value, 0);
break;
case 2:
pextrw(AddressFrame(Op->Size * 8) [STATE + rax], value, 0);
pextrw(AddressFrame(IROp->Size * 8) [STATE + rax], value, 0);
break;
case 4:
vmovd(AddressFrame(Op->Size * 8) [STATE + rax], value);
vmovd(AddressFrame(IROp->Size * 8) [STATE + rax], value);
break;
case 8:
vmovq(AddressFrame(Op->Size * 8) [STATE + rax], value);
vmovq(AddressFrame(IROp->Size * 8) [STATE + rax], value);
break;
case 16:
if (Op->BaseOffset % 16 == 0)
@@ -307,12 +319,14 @@ DEF_OP(StoreContextIndexed) {
movups(xword [STATE + rax], value);
break;
default:
LOGMAN_MSG_A("Unhandled StoreContextIndexed size: %d", size);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed size: {}", size);
break;
}
break;
}
default:
LOGMAN_MSG_A("Unhandled StoreContextIndexed stride: %d", Op->Stride);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed stride: {}", Op->Stride);
break;
}
}
}
@@ -340,7 +354,7 @@ DEF_OP(SpillRegister) {
mov(qword [rsp + SlotOffset], GetSrc<RA_64>(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unhandled SpillRegister size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled SpillRegister size: {}", OpSize);
}
} else if (Op->Class == FEXCore::IR::FPRClass) {
switch (OpSize) {
@@ -356,10 +370,10 @@ DEF_OP(SpillRegister) {
movaps(xword [rsp + SlotOffset], GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unhandled SpillRegister size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled SpillRegister size: {}", OpSize);
}
} else {
LOGMAN_MSG_A("Unhandled SpillRegister class: %d", Op->Class.Val);
LOGMAN_MSG_A_FMT("Unhandled SpillRegister class: {}", Op->Class.Val);
}
@@ -388,7 +402,7 @@ DEF_OP(FillRegister) {
mov(GetDst<RA_64>(Node), qword [rsp + SlotOffset]);
break;
}
default: LOGMAN_MSG_A("Unhandled FillRegister size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled FillRegister size: {}", OpSize);
}
} else if (Op->Class == FEXCore::IR::FPRClass) {
switch (OpSize) {
@@ -404,10 +418,10 @@ DEF_OP(FillRegister) {
movaps(GetDst(Node), xword [rsp + SlotOffset]);
break;
}
default: LOGMAN_MSG_A("Unhandled FillRegister size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled FillRegister size: {}", OpSize);
}
} else {
LOGMAN_MSG_A("Unhandled FillRegister class: %d", Op->Class.Val);
LOGMAN_MSG_A_FMT("Unhandled FillRegister class: {}", Op->Class.Val);
}
}
@@ -430,11 +444,11 @@ Xbyak::RegExp X86JITCore::GenerateModRM(Xbyak::Reg Base, IR::OrderedNodeWrapper
return Base;
} else {
if (OffsetScale != 1 && OffsetScale != 2 && OffsetScale != 4 && OffsetScale != 8) {
LOGMAN_MSG_A("Unhandled GenerateModRM OffsetScale: %d", OffsetScale);
LOGMAN_MSG_A_FMT("Unhandled GenerateModRM OffsetScale: {}", OffsetScale);
}
if (OffsetType != IR::MEM_OFFSET_SXTX) {
LOGMAN_MSG_A("Unhandled GenerateModRM OffsetType: %d", OffsetType.Val);
LOGMAN_MSG_A_FMT("Unhandled GenerateModRM OffsetType: {}", OffsetType.Val);
}
uint64_t Const;
@@ -458,7 +472,7 @@ DEF_OP(LoadMem) {
if (Op->Class.Val == 0) {
auto Dst = GetDst<RA_64>(Node);
switch (Op->Size) {
switch (IROp->Size) {
case 1: {
movzx (Dst, byte [MemPtr]);
}
@@ -475,14 +489,14 @@ DEF_OP(LoadMem) {
mov(Dst, qword [MemPtr]);
}
break;
default: LOGMAN_MSG_A("Unhandled LoadMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled LoadMem size: {}", IROp->Size);
}
}
else
{
auto Dst = GetDst(Node);
switch (Op->Size) {
switch (IROp->Size) {
case 1: {
movzx(eax, byte [MemPtr]);
vmovd(Dst, eax);
@@ -502,7 +516,7 @@ DEF_OP(LoadMem) {
}
break;
case 16: {
if (Op->Size == Op->Align)
if (IROp->Size == Op->Align)
movups(GetDst(Node), xword [MemPtr]);
else
movups(GetDst(Node), xword [MemPtr]);
@@ -511,7 +525,7 @@ DEF_OP(LoadMem) {
}
}
break;
default: LOGMAN_MSG_A("Unhandled LoadMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled LoadMem size: {}", IROp->Size);
}
}
}
@@ -524,7 +538,7 @@ DEF_OP(StoreMem) {
auto MemPtr = GenerateModRM(MemReg, Op->Offset, Op->OffsetType, Op->OffsetScale);
if (Op->Class.Val == 0) {
switch (Op->Size) {
switch (IROp->Size) {
case 1:
mov(byte [MemPtr], GetSrc<RA_8>(Op->Header.Args[1].ID()));
break;
@@ -537,11 +551,11 @@ DEF_OP(StoreMem) {
case 8:
mov(qword [MemPtr], GetSrc<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unhandled StoreMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled StoreMem size: {}", IROp->Size);
}
}
else {
switch (Op->Size) {
switch (IROp->Size) {
case 1:
pextrb(byte [MemPtr], GetSrc(Op->Header.Args[1].ID()), 0);
break;
@@ -555,22 +569,22 @@ DEF_OP(StoreMem) {
vmovq(qword [MemPtr], GetSrc(Op->Header.Args[1].ID()));
break;
case 16:
if (Op->Size == Op->Align)
if (IROp->Size == Op->Align)
movups(xword [MemPtr], GetSrc(Op->Header.Args[1].ID()));
else
movups(xword [MemPtr], GetSrc(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unhandled StoreMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled StoreMem size: {}", IROp->Size);
}
}
}
DEF_OP(VLoadMemElement) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(VStoreMemElement) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(CacheLineClear) {
@@ -4,16 +4,26 @@ tags: backend|x86-64
$end_info$
*/
#include "Interface/Context/Context.h"
#include "Interface/Core/Dispatcher/Dispatcher.h"
#include "Interface/Core/JIT/x86_64/JITClass.h"
#include "Interface/IR/Passes/RegisterAllocationPass.h"
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/Utils/LogManager.h>
#include <FEXCore/IR/IR.h>
#include <array>
#include <stddef.h>
#include <stdint.h>
#include <xbyak/xbyak.h>
namespace FEXCore::CPU {
static void PrintValue(uint64_t Value) {
LogMan::Msg::D("Value: 0x%lx", Value);
LogMan::Msg::DFmt("Value: 0x{:x}", Value);
}
static void PrintVectorValue(uint64_t Value, uint64_t ValueUpper) {
LogMan::Msg::D("Value: 0x%016lx'%016lx", ValueUpper, Value);
LogMan::Msg::DFmt("Value: 0x{:016x}'{:016x}", ValueUpper, Value);
}
#define DEF_OP(x) void X86JITCore::Op_##x(FEXCore::IR::IROp_Header *IROp, uint32_t Node)
@@ -30,7 +40,7 @@ DEF_OP(Fence) {
case IR::Fence_Store.Val:
sfence();
break;
default: LOGMAN_MSG_A("Unknown Fence: %d", Op->Fence); break;
default: LOGMAN_MSG_A_FMT("Unknown Fence: {}", Op->Fence); break;
}
}
@@ -83,7 +93,7 @@ DEF_OP(Break) {
}
break;
}
default: LOGMAN_MSG_A("Unknown Break reason: %d", Op->Reason);
default: LOGMAN_MSG_A_FMT("Unknown Break reason: {}", Op->Reason);
}
}
@@ -5,7 +5,13 @@ $end_info$
*/
#include "Interface/Core/JIT/x86_64/JITClass.h"
#include "Interface/IR/Passes/RegisterAllocationPass.h"
#include <FEXCore/Utils/LogManager.h>
#include <FEXCore/IR/IR.h>
#include <array>
#include <stdint.h>
#include <utility>
namespace FEXCore::CPU {
@@ -25,7 +31,7 @@ DEF_OP(ExtractElementPair) {
mov (GetDst<RA_64>(Node), Regs[Op->Element]);
break;
}
default: LOGMAN_MSG_A("Unknown Size"); break;
default: LOGMAN_MSG_A_FMT("Unknown Size"); break;
}
}
@@ -51,7 +57,7 @@ DEF_OP(CreateElementPair) {
RegTmp = rax;
break;
}
default: LOGMAN_MSG_A("Unknown Size"); break;
default: LOGMAN_MSG_A_FMT("Unknown Size"); break;
}
if (Dst.first != RegSecond) {
+108 -102
View File
@@ -5,8 +5,14 @@ $end_info$
*/
#include "Interface/Core/JIT/x86_64/JITClass.h"
#include "Interface/IR/Passes/RegisterAllocationPass.h"
#include <FEXCore/IR/IR.h>
#include <FEXCore/Utils/LogManager.h>
#include <array>
#include <stddef.h>
#include <stdint.h>
#include <xbyak/xbyak.h>
namespace FEXCore::CPU {
@@ -62,24 +68,24 @@ DEF_OP(VectorImm) {
}
DEF_OP(CreateVector2) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(CreateVector4) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(SplatVector) {
auto Op = IROp->C<IR::IROp_SplatVector2>();
uint8_t OpSize = IROp->Size;
LOGMAN_THROW_A(OpSize <= 16, "Can't handle a vector of size: %d", OpSize);
LOGMAN_THROW_A_FMT(OpSize <= 16, "Can't handle a vector of size: {}", OpSize);
uint8_t Elements = 0;
switch (Op->Header.Op) {
case IR::OP_SPLATVECTOR4: Elements = 4; break;
case IR::OP_SPLATVECTOR2: Elements = 2; break;
default: LOGMAN_MSG_A("Uknown Splat size"); break;
default: LOGMAN_MSG_A_FMT("Unknown Splat size"); break;
}
uint8_t ElementSize = OpSize / Elements;
@@ -92,7 +98,7 @@ DEF_OP(SplatVector) {
case 8:
movddup(GetDst(Node), GetSrc(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.Size); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.Size); break;
}
}
@@ -130,7 +136,7 @@ DEF_OP(VMov) {
movaps(GetDst(Node), GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", OpSize); break;
}
}
@@ -176,7 +182,7 @@ DEF_OP(VAdd) {
vpaddq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -199,7 +205,7 @@ DEF_OP(VSub) {
vpsubq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -214,7 +220,7 @@ DEF_OP(VUQAdd) {
vpaddusw(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -229,7 +235,7 @@ DEF_OP(VUQSub) {
vpsubusw(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -244,7 +250,7 @@ DEF_OP(VSQAdd) {
vpaddsw(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -259,7 +265,7 @@ DEF_OP(VSQSub) {
vpsubsw(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -296,7 +302,7 @@ DEF_OP(VAddP) {
case 4:
vphaddd(GetDst(Node), xmm15, xmm14);
break;
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -325,7 +331,7 @@ DEF_OP(VAddP) {
case 4:
vphaddd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -357,7 +363,7 @@ DEF_OP(VAddV) {
pinsrd(xmm15, eax, 0);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
movaps(Dest, xmm15);
@@ -376,7 +382,7 @@ DEF_OP(VUMinV) {
pinsrw(Dest, eax, 1);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -391,7 +397,7 @@ DEF_OP(VURAvg) {
vpavgw(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -414,7 +420,7 @@ DEF_OP(VAbs) {
vpabsq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -438,7 +444,7 @@ DEF_OP(VPopcount) {
}
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
movaps(Dest, xmm15);
@@ -459,7 +465,7 @@ DEF_OP(VFAdd) {
vaddsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -473,7 +479,7 @@ DEF_OP(VFAdd) {
vaddpd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -487,7 +493,7 @@ DEF_OP(VFAddP) {
case 8:
vhaddpd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -506,7 +512,7 @@ DEF_OP(VFSub) {
vsubsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -520,7 +526,7 @@ DEF_OP(VFSub) {
vsubpd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -540,7 +546,7 @@ DEF_OP(VFMul) {
vmulsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -554,7 +560,7 @@ DEF_OP(VFMul) {
vmulpd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -574,7 +580,7 @@ DEF_OP(VFDiv) {
vdivsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -588,7 +594,7 @@ DEF_OP(VFDiv) {
vdivpd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -608,7 +614,7 @@ DEF_OP(VFMin) {
vminsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -622,7 +628,7 @@ DEF_OP(VFMin) {
vminpd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -642,7 +648,7 @@ DEF_OP(VFMax) {
vmaxsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -656,7 +662,7 @@ DEF_OP(VFMax) {
vmaxpd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -674,7 +680,7 @@ DEF_OP(VFRecp) {
vdivss(GetDst(Node), xmm15, GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -687,7 +693,7 @@ DEF_OP(VFRecp) {
vdivps(GetDst(Node), xmm15, GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -707,7 +713,7 @@ DEF_OP(VFSqrt) {
vsqrtsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -721,7 +727,7 @@ DEF_OP(VFSqrt) {
vsqrtpd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -747,7 +753,7 @@ DEF_OP(VFRSqrt) {
divsd(GetDst(Node), xmm15);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -761,7 +767,7 @@ DEF_OP(VFRSqrt) {
divps(GetDst(Node), xmm15);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -786,7 +792,7 @@ DEF_OP(VNeg) {
vpsubq(GetDst(Node), xmm15, GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -807,7 +813,7 @@ DEF_OP(VFNeg) {
vxorpd(GetDst(Node), xmm15, GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -830,7 +836,7 @@ DEF_OP(VUMin) {
pinsrq(GetDst(Node), TMP2, 0);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -847,7 +853,7 @@ DEF_OP(VUMin) {
vpminud(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -867,7 +873,7 @@ DEF_OP(VSMin) {
vpminsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -886,7 +892,7 @@ DEF_OP(VUMax) {
vpmaxud(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -905,7 +911,7 @@ DEF_OP(VSMax) {
vpmaxsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -930,7 +936,7 @@ DEF_OP(VZip) {
punpcklqdq(xmm15, GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
movapd(GetDst(Node), xmm15);
}
@@ -957,7 +963,7 @@ DEF_OP(VZip2) {
vpunpckhdq(GetDst(Node), xmm15, xmm14);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -978,7 +984,7 @@ DEF_OP(VZip2) {
punpckhqdq(xmm15, GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
movapd(GetDst(Node), xmm15);
}
@@ -989,7 +995,7 @@ DEF_OP(VUnZip) {
uint8_t OpSize = IROp->Size;
if (OpSize == 8) {
LOGMAN_MSG_A("Unsupported registersize on VunZip");
LOGMAN_MSG_A_FMT("Unsupported register size on VUnZip");
}
else {
switch (Op->Header.ElementSize) {
@@ -1031,7 +1037,7 @@ DEF_OP(VUnZip) {
0b0'0);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1042,7 +1048,7 @@ DEF_OP(VUnZip2) {
if (OpSize == 8) {
LOGMAN_MSG_A("Unsupported registersize on VunZip");
LOGMAN_MSG_A_FMT("Unsupported register size on VUnZip2");
}
else {
switch (Op->Header.ElementSize) {
@@ -1084,7 +1090,7 @@ DEF_OP(VUnZip2) {
0b1'1);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1113,7 +1119,7 @@ DEF_OP(VCMPEQ) {
case 8:
vpcmpeqq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
@@ -1134,7 +1140,7 @@ DEF_OP(VCMPEQZ) {
case 8:
vpcmpeqq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), xmm15);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
@@ -1154,7 +1160,7 @@ DEF_OP(VCMPGT) {
case 8:
vpcmpgtq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
@@ -1175,7 +1181,7 @@ DEF_OP(VCMPGTZ) {
case 8:
vpcmpgtq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), xmm15);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
@@ -1196,7 +1202,7 @@ DEF_OP(VCMPLTZ) {
case 8:
vpcmpgtq(GetDst(Node), xmm15, GetSrc(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
@@ -1212,7 +1218,7 @@ DEF_OP(VFCMPEQ) {
case 8:
vcmpsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 0);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
else {
@@ -1223,7 +1229,7 @@ DEF_OP(VFCMPEQ) {
case 8:
vcmppd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 0);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
}
@@ -1240,7 +1246,7 @@ DEF_OP(VFCMPNEQ) {
case 8:
vcmpsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 4);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
@@ -1252,7 +1258,7 @@ DEF_OP(VFCMPNEQ) {
case 8:
vcmppd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 4);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
}
@@ -1269,7 +1275,7 @@ DEF_OP(VFCMPLT) {
case 8:
vcmpsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 1);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
else {
@@ -1280,7 +1286,7 @@ DEF_OP(VFCMPLT) {
case 8:
vcmppd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 1);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
}
@@ -1297,7 +1303,7 @@ DEF_OP(VFCMPGT) {
case 8:
vcmpsd(GetDst(Node), GetSrc(Op->Header.Args[1].ID()), GetSrc(Op->Header.Args[0].ID()), 1);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
else {
@@ -1308,7 +1314,7 @@ DEF_OP(VFCMPGT) {
case 8:
vcmppd(GetDst(Node), GetSrc(Op->Header.Args[1].ID()), GetSrc(Op->Header.Args[0].ID()), 1);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
}
@@ -1325,7 +1331,7 @@ DEF_OP(VFCMPLE) {
case 8:
vcmpsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 2);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
else {
@@ -1336,7 +1342,7 @@ DEF_OP(VFCMPLE) {
case 8:
vcmppd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 2);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
}
@@ -1353,7 +1359,7 @@ DEF_OP(VFCMPORD) {
case 8:
vcmpsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 7);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
else {
@@ -1364,7 +1370,7 @@ DEF_OP(VFCMPORD) {
case 8:
vcmppd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 7);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
}
@@ -1381,7 +1387,7 @@ DEF_OP(VFCMPUNO) {
case 8:
vcmpsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 3);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
else {
@@ -1392,21 +1398,21 @@ DEF_OP(VFCMPUNO) {
case 8:
vcmppd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 3);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
}
DEF_OP(VUShl) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(VUShr) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(VSShr) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(VUShlS) {
@@ -1425,7 +1431,7 @@ DEF_OP(VUShlS) {
vpsllq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1445,7 +1451,7 @@ DEF_OP(VUShrS) {
vpsrlq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1462,7 +1468,7 @@ DEF_OP(VSShrS) {
break;
}
case 8: // Doesn't exist on x86
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1495,7 +1501,7 @@ DEF_OP(VInsElement) {
pinsrq(xmm15, rax, Op->DestIdx);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
movapd(GetDst(Node), xmm15);
@@ -1530,7 +1536,7 @@ DEF_OP(VInsScalarElement) {
pinsrq(xmm15, rax, Op->DestIdx);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
movapd(GetDst(Node), xmm15);
@@ -1560,7 +1566,7 @@ DEF_OP(VExtractElement) {
pinsrq(GetDst(Node), rax, 0);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1613,7 +1619,7 @@ DEF_OP(VDupElement) {
(Op->Index << 1));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1666,7 +1672,7 @@ DEF_OP(VUShrI) {
psrlq(GetDst(Node), Op->BitShift);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1704,7 +1710,7 @@ DEF_OP(VSShrI) {
}
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1724,7 +1730,7 @@ DEF_OP(VShlI) {
psllq(GetDst(Node), Op->BitShift);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1754,7 +1760,7 @@ DEF_OP(VUShrNI) {
mov(rcx, 0x80'80'80'80'80'80'80'80); // Upper
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
vmovq(xmm15, rax);
@@ -1790,7 +1796,7 @@ DEF_OP(VUShrNI2) {
mov(rcx, 0x0B'0A'09'08'03'02'01'00); // Upper
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
vmovq(xmm15, rax);
@@ -1817,7 +1823,7 @@ DEF_OP(VSXTL) {
case 8:
pmovsxdq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
@@ -1836,7 +1842,7 @@ DEF_OP(VSXTL2) {
case 8:
pmovsxdq(GetDst(Node), GetDst(Node));
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
@@ -1852,7 +1858,7 @@ DEF_OP(VUXTL) {
case 8:
pmovzxdq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
@@ -1871,7 +1877,7 @@ DEF_OP(VUXTL2) {
case 8:
pmovzxdq(GetDst(Node), GetDst(Node));
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
@@ -1884,7 +1890,7 @@ DEF_OP(VSQXTN) {
case 2:
packssdw(xmm15, GetSrc(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
psrldq(xmm15, 8);
movaps(GetDst(Node), xmm15);
@@ -1903,7 +1909,7 @@ DEF_OP(VSQXTN2) {
case 2:
packssdw(xmm15, GetSrc(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
if (OpSize == 8) {
@@ -1921,7 +1927,7 @@ DEF_OP(VSQXTUN) {
case 2:
packusdw(xmm15, GetSrc(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
psrldq(xmm15, 8);
movaps(GetDst(Node), xmm15);
@@ -1940,7 +1946,7 @@ DEF_OP(VSQXTUN2) {
case 2:
packusdw(xmm15, GetSrc(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
if (OpSize == 8) {
psrldq(xmm15, OpSize / 2);
@@ -1960,7 +1966,7 @@ DEF_OP(VMul) {
vpmulld(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1991,7 +1997,7 @@ DEF_OP(VUMull) {
vpmuludq(GetDst(Node), xmm14, xmm15);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -2026,7 +2032,7 @@ DEF_OP(VSMull) {
vpmuldq(GetDst(Node), xmm14, xmm15);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -2062,7 +2068,7 @@ DEF_OP(VUMull2) {
vpmuludq(GetDst(Node), xmm14, xmm15);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -2102,7 +2108,7 @@ DEF_OP(VSMull2) {
vpmuldq(GetDst(Node), xmm14, xmm15);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -2123,7 +2129,7 @@ DEF_OP(VUABDL) {
vpabsd(GetDst(Node), GetDst(Node));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -2141,7 +2147,7 @@ DEF_OP(VTBL1) {
vpshufb(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown OpSize: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown OpSize: {}", OpSize); break;
}
}
+4 -3
View File
@@ -5,10 +5,11 @@ desc: Stores information about blocks, and provides C++ implementations to looku
$end_info$
*/
#include "Interface/Context/Context.h"
#include "Interface/Core/Core.h"
#include "Interface/Core/LookupCache.h"
#include <FEXCore/Utils/Allocator.h>
#include <FEXCore/Utils/LogManager.h>
#include "Interface/Context/Context.h"
#include "Interface/Core/LookupCache.h"
#include <sys/mman.h>
+9 -1
View File
@@ -1,10 +1,18 @@
#pragma once
#include "Interface/Context/Context.h"
#include <FEXCore/Utils/LogManager.h>
#include <cstdint>
#include <functional>
#include <map>
#include <stddef.h>
#include <utility>
#include <vector>
namespace FEXCore {
namespace Context {
struct Context;
}
class LookupCache {
public:
File diff suppressed because it is too large. Load diff
+44 -15
View File
@@ -3,7 +3,8 @@
#include "Interface/Core/Frontend.h"
#include "Interface/Context/Context.h"
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/Config/Config.h>
#include <FEXCore/Core/Context.h>
#include <FEXCore/Debug/X86Tables.h>
#include <FEXCore/IR/IntrusiveIRList.h>
#include <FEXCore/IR/IR.h>
@@ -12,9 +13,10 @@
#include <FEXCore/Utils/LogManager.h>
#include <cstdint>
#include <functional>
#include <map>
#include <set>
#include <stddef.h>
#include <utility>
#include <vector>
namespace FEXCore::IR {
class Pass;
@@ -270,15 +272,15 @@ public:
template<size_t ElementSize, bool HalfSize, bool Low>
void PSHUFDOp(OpcodeArgs);
void MOVDOp(OpcodeArgs);
template<size_t ElementSize, bool Scalar, uint32_t SrcIndex>
template<size_t ElementSize>
void PSRLDOp(OpcodeArgs);
template<size_t ElementSize>
void PSRLI(OpcodeArgs);
template<size_t ElementSize>
void PSLLI(OpcodeArgs);
template<size_t ElementSize, bool Scalar, uint32_t SrcIndex>
template<size_t ElementSize>
void PSLL(OpcodeArgs);
template<size_t ElementSize, bool Scalar, uint32_t SrcIndex>
template<size_t ElementSize>
void PSRAOp(OpcodeArgs);
void PSRLDQ(OpcodeArgs);
void PSLLDQ(OpcodeArgs);
@@ -299,9 +301,9 @@ public:
void Vector_CVT_Float_To_Float(OpcodeArgs);
template<size_t SrcElementSize, bool Narrow, bool HostRoundingMode>
void Vector_CVT_Float_To_Int(OpcodeArgs);
template<size_t SrcElementSize, bool Signed, bool Widen>
template<size_t SrcElementSize, bool Widen>
void MMX_To_XMM_Vector_CVT_Int_To_Float(OpcodeArgs);
template<size_t SrcElementSize, bool Narrow, bool HostRoundingMode>
template<size_t SrcElementSize, bool HostRoundingMode>
void XMM_To_MMX_Vector_CVT_Float_To_Int(OpcodeArgs);
void MASKMOVOp(OpcodeArgs);
void MOVBetweenGPR_FPR(OpcodeArgs);
@@ -322,6 +324,13 @@ public:
template<size_t ElementSize>
void PSIGN(OpcodeArgs);
// BMI Ops
void ANDNBMIOp(OpcodeArgs);
void BEXTRBMIOp(OpcodeArgs);
void BLSIBMIOp(OpcodeArgs);
void BLSMSKBMIOp(OpcodeArgs);
void BLSRBMIOp(OpcodeArgs);
// X87 Ops
template<size_t width>
void FLD(OpcodeArgs);
@@ -380,6 +389,8 @@ public:
void X87FRSTOR(OpcodeArgs);
void X87FXAM(OpcodeArgs);
void X87FCMOV(OpcodeArgs);
void X87EMMS(OpcodeArgs);
void X87FFREE(OpcodeArgs);
void FXCH(OpcodeArgs);
@@ -501,9 +512,19 @@ private:
uint8_t GetSrcSize(FEXCore::X86Tables::DecodedOp Op) const;
template<unsigned BitOffset>
void SetRFLAG(OrderedNode *Value);
void SetRFLAG(OrderedNode *Value, unsigned BitOffset);
OrderedNode *GetRFLAG(unsigned BitOffset);
void SetRFLAG(OrderedNode *Value) {
flagsOp = FLAGS_OP_NONE;
_StoreFlag(_Bfe(1, 0, Value), BitOffset);
}
void SetRFLAG(OrderedNode *Value, unsigned BitOffset) {
flagsOp = FLAGS_OP_NONE;
_StoreFlag(_Bfe(1, 0, Value), BitOffset);
}
OrderedNode *GetRFLAG(unsigned BitOffset) {
return _LoadFlag(BitOffset);
}
OrderedNode *SelectCC(uint8_t OP, OrderedNode *TrueValue, OrderedNode *FalseValue);
@@ -526,6 +547,14 @@ private:
void GenerateFlags_RotateLeftImmediate(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, uint64_t Shift);
OrderedNode * GetX87Top();
enum X87Tag {
TAG_VALID = 0b00,
TAG_ZERO = 0b01,
TAG_SPECIAL = 0b10,
TAG_EMPTY = 0b11
};
void SetX87TopTag(OrderedNode *Value, uint32_t Tag);
OrderedNode *GetX87FTW(OrderedNode *Value);
void SetX87Top(OrderedNode *Value);
bool DestIsLockedMem(FEXCore::X86Tables::DecodedOp Op) const {
@@ -544,16 +573,16 @@ private:
OrderedNode* _StoreMemAutoTSO(FEXCore::IR::RegisterClassType Class, uint8_t Size, OrderedNode *ssa0, OrderedNode *ssa1, uint8_t Align = 1) {
if (CTX->Config.TSOEnabled)
return _StoreMemTSO(ssa0, ssa1, Invalid(), Size, Align, Class, MEM_OFFSET_SXTX, 1);
return _StoreMemTSO(ssa0, ssa1, Invalid(), Align, Class, MEM_OFFSET_SXTX, 1, Size);
else
return _StoreMem(ssa0, ssa1, Invalid(), Size, Align, Class, MEM_OFFSET_SXTX, 1);
return _StoreMem(ssa0, ssa1, Invalid(), Align, Class, MEM_OFFSET_SXTX, 1, Size);
}
OrderedNode* _LoadMemAutoTSO(FEXCore::IR::RegisterClassType Class, uint8_t Size, OrderedNode *ssa0, uint8_t Align = 1) {
if (CTX->Config.TSOEnabled)
return _LoadMemTSO(ssa0, Invalid(), Size, Align, Class, MEM_OFFSET_SXTX, 1);
return _LoadMemTSO(ssa0, Invalid(), Align, Class, MEM_OFFSET_SXTX, 1, Size);
else
return _LoadMem(ssa0, Invalid(), Size, Align, Class, MEM_OFFSET_SXTX, 1);
return _LoadMem(ssa0, Invalid(), Align, Class, MEM_OFFSET_SXTX, 1, Size);
}
@@ -0,0 +1,63 @@
/*
$info$
tags: frontend|x86-to-ir, opcodes|dispatcher-implementations
desc: Handles x86/64 Crypto instructions to IR
$end_info$
*/
#include <FEXCore/Debug/X86Tables.h>
#include <FEXCore/IR/IREmitter.h>
#include <FEXCore/Utils/LogManager.h>
#include "Interface/Core/OpcodeDispatcher.h"
#include <stdint.h>
namespace FEXCore::IR {
class OrderedNode;
#define OpcodeArgs [[maybe_unused]] FEXCore::X86Tables::DecodedOp Op
void OpDispatchBuilder::AESImcOp(OpcodeArgs) {
OrderedNode *Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
auto Res = _VAESImc(Src);
StoreResult(FPRClass, Op, Res, -1);
}
void OpDispatchBuilder::AESEncOp(OpcodeArgs) {
OrderedNode *Dest = LoadSource(FPRClass, Op, Op->Dest, Op->Flags, -1);
OrderedNode *Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
auto Res = _VAESEnc(Dest, Src);
StoreResult(FPRClass, Op, Res, -1);
}
void OpDispatchBuilder::AESEncLastOp(OpcodeArgs) {
OrderedNode *Dest = LoadSource(FPRClass, Op, Op->Dest, Op->Flags, -1);
OrderedNode *Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
auto Res = _VAESEncLast(Dest, Src);
StoreResult(FPRClass, Op, Res, -1);
}
void OpDispatchBuilder::AESDecOp(OpcodeArgs) {
OrderedNode *Dest = LoadSource(FPRClass, Op, Op->Dest, Op->Flags, -1);
OrderedNode *Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
auto Res = _VAESDec(Dest, Src);
StoreResult(FPRClass, Op, Res, -1);
}
void OpDispatchBuilder::AESDecLastOp(OpcodeArgs) {
OrderedNode *Dest = LoadSource(FPRClass, Op, Op->Dest, Op->Flags, -1);
OrderedNode *Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
auto Res = _VAESDecLast(Dest, Src);
StoreResult(FPRClass, Op, Res, -1);
}
void OpDispatchBuilder::AESKeyGenAssist(OpcodeArgs) {
OrderedNode *Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
LOGMAN_THROW_A(Op->Src[1].IsLiteral(), "Src1 needs to be literal here");
uint64_t RCON = Op->Src[1].Data.Literal.Value;
auto Res = _VAESKeyGenAssist(Src, RCON);
StoreResult(FPRClass, Op, Res, -1);
}
}
@@ -0,0 +1,818 @@
/*
$info$
tags: frontend|x86-to-ir, opcodes|dispatcher-implementations
desc: Handles x86/64 flag generation
$end_info$
*/
#include "Interface/Context/Context.h"
#include "Interface/Core/OpcodeDispatcher.h"
#include <FEXCore/Core/X86Enums.h>
#include <FEXCore/Config/Config.h>
#include <FEXCore/Debug/X86Tables.h>
#include <FEXCore/Utils/LogManager.h>
#include <FEXCore/IR/IR.h>
#include <array>
#include <cstdint>
namespace FEXCore::IR {
constexpr std::array<uint32_t, 17> FlagOffsets = {
FEXCore::X86State::RFLAG_CF_LOC,
FEXCore::X86State::RFLAG_PF_LOC,
FEXCore::X86State::RFLAG_AF_LOC,
FEXCore::X86State::RFLAG_ZF_LOC,
FEXCore::X86State::RFLAG_SF_LOC,
FEXCore::X86State::RFLAG_TF_LOC,
FEXCore::X86State::RFLAG_IF_LOC,
FEXCore::X86State::RFLAG_DF_LOC,
FEXCore::X86State::RFLAG_OF_LOC,
FEXCore::X86State::RFLAG_IOPL_LOC,
FEXCore::X86State::RFLAG_NT_LOC,
FEXCore::X86State::RFLAG_RF_LOC,
FEXCore::X86State::RFLAG_VM_LOC,
FEXCore::X86State::RFLAG_AC_LOC,
FEXCore::X86State::RFLAG_VIF_LOC,
FEXCore::X86State::RFLAG_VIP_LOC,
FEXCore::X86State::RFLAG_ID_LOC,
};
void OpDispatchBuilder::SetPackedRFLAG(bool Lower8, OrderedNode *Src) {
uint8_t NumFlags = FlagOffsets.size();
if (Lower8) {
NumFlags = 5;
}
auto OneConst = _Constant(1);
for (int i = 0; i < NumFlags; ++i) {
auto Tmp = _And(_Lshr(Src, _Constant(FlagOffsets[i])), OneConst);
SetRFLAG(Tmp, FlagOffsets[i]);
}
}
OrderedNode *OpDispatchBuilder::GetPackedRFLAG(bool Lower8) {
OrderedNode *Original = _Constant(2);
uint8_t NumFlags = FlagOffsets.size();
if (Lower8) {
NumFlags = 5;
}
for (int i = 0; i < NumFlags; ++i) {
OrderedNode *Flag = _LoadFlag(FlagOffsets[i]);
Flag = _Bfe(4, 32, 0, Flag);
Flag = _Lshl(Flag, _Constant(FlagOffsets[i]));
Original = _Or(Original, Flag);
}
return Original;
}
void OpDispatchBuilder::GenerateFlags_ADC(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2, OrderedNode *CF) {
auto Size = GetSrcSize(Op) * 8;
// AF
{
OrderedNode *AFRes = _Xor(_Xor(Src1, Src2), Res);
AFRes = _Bfe(1, 4, AFRes);
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(AFRes);
}
// SF
{
auto SignBitConst = _Constant(GetSrcSize(Op) * 8 - 1);
auto LshrOp = _Lshr(Res, SignBitConst);
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(LshrOp);
}
// PF
if (!CTX->Config.ABINoPF) {
auto PopCountOp = _Popcount(_And(Res, _Constant(0xFF)));
auto XorOp = _Xor(PopCountOp, _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(SelectOp);
}
// CF
// Unsigned
{
auto SelectOpLT = _Select(FEXCore::IR::COND_ULT, Res, Src2, _Constant(1), _Constant(0));
auto SelectOpLE = _Select(FEXCore::IR::COND_ULE, Res, Src2, _Constant(1), _Constant(0));
auto SelectCF = _Select(FEXCore::IR::COND_EQ, CF, _Constant(1), SelectOpLE, SelectOpLT);
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(SelectCF);
}
// OF
// Signed
{
auto NegOne = _Constant(~0ULL);
auto XorOp1 = _Xor(_Xor(Src1, Src2), NegOne);
auto XorOp2 = _Xor(Res, Src1);
OrderedNode *AndOp1 = _And(XorOp1, XorOp2);
switch (Size) {
case 8:
AndOp1 = _Bfe(1, 7, AndOp1);
break;
case 16:
AndOp1 = _Bfe(1, 15, AndOp1);
break;
case 32:
AndOp1 = _Bfe(1, 31, AndOp1);
break;
case 64:
AndOp1 = _Bfe(1, 63, AndOp1);
break;
default: LOGMAN_MSG_A("Unknown BFESize: %d", Size); break;
}
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(AndOp1);
}
}
void OpDispatchBuilder::GenerateFlags_SBB(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2, OrderedNode *CF) {
// AF
{
OrderedNode *AFRes = _Xor(_Xor(Src1, Src2), Res);
AFRes = _Bfe(1, 4, AFRes);
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(AFRes);
}
// SF
{
auto SignBitConst = _Constant(GetSrcSize(Op) * 8 - 1);
auto LshrOp = _Lshr(Res, SignBitConst);
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(LshrOp);
}
// PF
if (!CTX->Config.ABINoPF) {
auto PopCountOp = _Popcount(_And(Res, _Constant(0xFF)));
auto XorOp = _Xor(PopCountOp, _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(SelectOp);
}
// CF
// Unsigned
{
auto SelectOpLT = _Select(FEXCore::IR::COND_UGT, Res, Src1, _Constant(1), _Constant(0));
auto SelectOpLE = _Select(FEXCore::IR::COND_UGE, Res, Src1, _Constant(1), _Constant(0));
auto SelectCF = _Select(FEXCore::IR::COND_EQ, CF, _Constant(1), SelectOpLE, SelectOpLT);
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(SelectCF);
}
// OF
// Signed
{
auto XorOp1 = _Xor(Src1, Src2);
auto XorOp2 = _Xor(Res, Src1);
OrderedNode *AndOp1 = _And(XorOp1, XorOp2);
switch (GetSrcSize(Op)) {
case 1:
AndOp1 = _Bfe(1, 7, AndOp1);
break;
case 2:
AndOp1 = _Bfe(1, 15, AndOp1);
break;
case 4:
AndOp1 = _Bfe(1, 31, AndOp1);
break;
case 8:
AndOp1 = _Bfe(1, 63, AndOp1);
break;
default: LOGMAN_MSG_A("Unknown BFESize: %d", GetSrcSize(Op)); break;
}
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(AndOp1);
}
}
void OpDispatchBuilder::GenerateFlags_SUB(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2, bool UpdateCF) {
// AF
{
OrderedNode *AFRes = _Xor(_Xor(Src1, Src2), Res);
AFRes = _Bfe(1, 4, AFRes);
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(AFRes);
}
// SF
{
auto SignBitConst = _Constant(GetSrcSize(Op) * 8 - 1);
auto LshrOp = _Lshr(Res, SignBitConst);
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(LshrOp);
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// ZF
{
auto ZeroConst = _Constant(0);
auto OneConst = _Constant(1);
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, ZeroConst, OneConst, ZeroConst);
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(SelectOp);
}
// CF
if (UpdateCF) {
auto ZeroConst = _Constant(0);
auto OneConst = _Constant(1);
auto SelectOp = _Select(FEXCore::IR::COND_ULT,
Src1, Src2, OneConst, ZeroConst);
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(SelectOp);
}
// OF
{
auto XorOp1 = _Xor(Src1, Src2);
auto XorOp2 = _Xor(Res, Src1);
OrderedNode *FinalAnd = _And(XorOp1, XorOp2);
FinalAnd = _Bfe(1, GetSrcSize(Op) * 8 - 1, FinalAnd);
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(FinalAnd);
}
}
void OpDispatchBuilder::GenerateFlags_ADD(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2, bool UpdateCF) {
// AF
{
OrderedNode *AFRes = _Xor(_Xor(Src1, Src2), Res);
AFRes = _Bfe(1, 4, AFRes);
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(AFRes);
}
// SF
{
auto SignBitConst = _Constant(GetSrcSize(Op) * 8 - 1);
auto LshrOp = _Lshr(Res, SignBitConst);
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(LshrOp);
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(SelectOp);
}
// CF
if (UpdateCF) {
auto SelectOp = _Select(FEXCore::IR::COND_ULT, Res, Src2, _Constant(1), _Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(SelectOp);
}
// OF
{
auto NegOne = _Constant(~0ULL);
auto XorOp1 = _Xor(_Xor(Src1, Src2), NegOne);
auto XorOp2 = _Xor(Res, Src1);
OrderedNode *AndOp1 = _And(XorOp1, XorOp2);
switch (GetSrcSize(Op)) {
case 1:
AndOp1 = _Bfe(1, 7, AndOp1);
break;
case 2:
AndOp1 = _Bfe(1, 15, AndOp1);
break;
case 4:
AndOp1 = _Bfe(1, 31, AndOp1);
break;
case 8:
AndOp1 = _Bfe(1, 63, AndOp1);
break;
default: LOGMAN_MSG_A("Unknown BFESize: %d", GetSrcSize(Op)); break;
}
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(AndOp1);
}
}
void OpDispatchBuilder::GenerateFlags_MUL(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *High) {
// PF/AF/ZF/SF
// Undefined
{
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(_Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(_Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(_Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(_Constant(0));
}
// CF/OF
{
// CF and OF are set if the result of the operation can't be fit in to the destination register
// If the value can fit then the top bits will be zero
auto SignBit = _Sbfe(1, GetSrcSize(Op) * 8 - 1, Res);
auto SelectOp = _Select(FEXCore::IR::COND_EQ, High, SignBit, _Constant(0), _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(SelectOp);
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(SelectOp);
}
}
void OpDispatchBuilder::GenerateFlags_UMUL(FEXCore::X86Tables::DecodedOp Op, OrderedNode *High) {
// AF/SF/PF/ZF
// Undefined
{
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(_Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(_Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(_Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(_Constant(0));
}
// CF/OF
{
// CF and OF are set if the result of the operation can't be fit in to the destination register
// The result register will be all zero if it can't fit due to how multiplication behaves
auto SelectOp = _Select(FEXCore::IR::COND_EQ, High, _Constant(0), _Constant(0), _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(SelectOp);
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(SelectOp);
}
}
void OpDispatchBuilder::GenerateFlags_Logical(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2) {
// AF
{
// Undefined
// Set to zero anyway
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(_Constant(0));
}
// SF
{
auto SignBitConst = _Constant(GetSrcSize(Op) * 8 - 1);
auto LshrOp = _Lshr(Res, SignBitConst);
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(LshrOp);
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(SelectOp);
}
// CF/OF
{
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(_Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(_Constant(0));
}
}
#define COND_FLAG_SET(cond, flag, newflag) \
auto oldflag = GetRFLAG(FEXCore::X86State::flag);\
auto newval = _Select(FEXCore::IR::COND_EQ, cond, _Constant(0), oldflag, newflag);\
SetRFLAG<FEXCore::X86State::flag>(newval);
void OpDispatchBuilder::GenerateFlags_ShiftLeft(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2) {
// CF
{
// Extract the last bit shifted in to CF
auto Size = _Constant(GetSrcSize(Op) * 8);
auto ShiftAmt = _Sub(Size, Src2);
auto LastBit = _And(_Lshr(Src1, ShiftAmt), _Constant(1));
COND_FLAG_SET(Src2, RFLAG_CF_LOC, LastBit);
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
COND_FLAG_SET(Src2, RFLAG_PF_LOC, XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// AF
{
// Undefined
// Set to zero anyway
COND_FLAG_SET(Src2, RFLAG_AF_LOC, _Constant(0));
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
COND_FLAG_SET(Src2, RFLAG_ZF_LOC, SelectOp);
}
// SF
{
auto val = _Bfe(1, GetSrcSize(Op) * 8 - 1, Res);
COND_FLAG_SET(Src2, RFLAG_SF_LOC, val);
}
// OF
{
// In the case of left shift. OF is only set from the result of <Top Source Bit> XOR <Top Result Bit>
// When Shift > 1 then OF is undefined
auto val = _Bfe(1, GetSrcSize(Op) * 8 - 1, _Xor(Src1, Res));
COND_FLAG_SET(Src2, RFLAG_OF_LOC, val);
}
}
void OpDispatchBuilder::GenerateFlags_ShiftRight(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2) {
// CF
{
// Extract the last bit shifted in to CF
auto ShiftAmt = _Sub(Src2, _Constant(1));
auto LastBit = _And(_Lshr(Src1, ShiftAmt), _Constant(1));
COND_FLAG_SET(Src2, RFLAG_CF_LOC, LastBit);
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
COND_FLAG_SET(Src2, RFLAG_PF_LOC, XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// AF
{
// Undefined
// Set to zero anyway
COND_FLAG_SET(Src2, RFLAG_AF_LOC, _Constant(0));
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
COND_FLAG_SET(Src2, RFLAG_ZF_LOC, SelectOp);
}
// SF
{
auto val =_Bfe(1, GetSrcSize(Op) * 8 - 1, Res);
COND_FLAG_SET(Src2, RFLAG_SF_LOC, val);
}
// OF
{
// Only defined when Shift is 1 else undefined
// OF flag is set if a sign change occurred
auto val = _Bfe(1, GetSrcSize(Op) * 8 - 1, _Xor(Src1, Res));
COND_FLAG_SET(Src2, RFLAG_OF_LOC, val);
}
}
void OpDispatchBuilder::GenerateFlags_SignShiftRight(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2) {
// CF
{
// Extract the last bit shifted in to CF
auto ShiftAmt = _Sub(Src2, _Constant(1));
auto LastBit = _And(_Lshr(Src1, ShiftAmt), _Constant(1));
COND_FLAG_SET(Src2, RFLAG_CF_LOC, LastBit);
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
COND_FLAG_SET(Src2, RFLAG_PF_LOC, XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// AF
{
// Undefined
// Set to zero anyway
COND_FLAG_SET(Src2, RFLAG_AF_LOC, _Constant(0));
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
COND_FLAG_SET(Src2, RFLAG_ZF_LOC, SelectOp);
}
// SF
{
auto SignBitConst = _Constant(GetSrcSize(Op) * 8 - 1);
auto LshrOp = _Lshr(Res, SignBitConst);
COND_FLAG_SET(Src2, RFLAG_SF_LOC, LshrOp);
}
// OF
{
COND_FLAG_SET(Src2, RFLAG_OF_LOC, _Constant(0));
}
}
void OpDispatchBuilder::GenerateFlags_ShiftLeftImmediate(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, uint64_t Shift) {
// No flags changed if shift is zero
if (Shift == 0) return;
// CF
{
// Extract the last bit shifted in to CF
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(_Bfe(1, GetSrcSize(Op) * 8 - Shift, Src1));
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// AF
{
// Undefined
// Set to zero anyway
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(_Constant(0));
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(SelectOp);
}
// SF
{
auto LshrOp = _Bfe(1, GetSrcSize(Op) * 8 - 1, Res);
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(LshrOp);
// OF
// In the case of left shift. OF is only set from the result of <Top Source Bit> XOR <Top Result Bit>
if (Shift == 1) {
auto SourceBit = _Bfe(1, GetSrcSize(Op) * 8 - 1, Src1);
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(_Xor(SourceBit, LshrOp));
}
}
}
void OpDispatchBuilder::GenerateFlags_SignShiftRightImmediate(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, uint64_t Shift) {
// No flags changed if shift is zero
if (Shift == 0) return;
// CF
{
// Extract the last bit shifted in to CF
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(_Bfe(1, Shift-1, Src1));
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// AF
{
// Undefined
// Set to zero anyway
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(_Constant(0));
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(SelectOp);
}
// SF
{
auto SignBitConst = _Constant(GetSrcSize(Op) * 8 - 1);
auto LshrOp = _Lshr(Res, SignBitConst);
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(LshrOp);
// OF
// Only defined when Shift is 1 else undefined
// Only is set if the top bit was set to 1 when shifted
// So it is set to same value as SF
if (Shift == 1) {
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(_Constant(0));
}
}
}
void OpDispatchBuilder::GenerateFlags_ShiftRightImmediate(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, uint64_t Shift) {
// No flags changed if shift is zero
if (Shift == 0) return;
// CF
{
// Extract the last bit shifted in to CF
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(_Bfe(1, Shift-1, Src1));
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// AF
{
// Undefined
// Set to zero anyway
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(_Constant(0));
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(SelectOp);
}
// SF
{
auto SignBitConst = _Constant(GetSrcSize(Op) * 8 - 1);
auto LshrOp = _Lshr(Res, SignBitConst);
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(LshrOp);
}
// OF
{
// Only defined when Shift is 1 else undefined
// Is set to the MSB of the original value
if (Shift == 1) {
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(_Bfe(1, GetSrcSize(Op) * 8 - 1, Src1));
}
}
}
void OpDispatchBuilder::GenerateFlags_RotateRight(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2) {
auto OpSize = GetSrcSize(Op) * 8;
// Extract the last bit shifted in to CF
auto NewCF = _Bfe(1, OpSize - 1, Res);
// CF
{
auto OldCF = GetRFLAG(FEXCore::X86State::RFLAG_CF_LOC);
auto CF = _Select(FEXCore::IR::COND_EQ, Src2, _Constant(0), OldCF, NewCF);
// Extract the last bit shifted in to CF
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(CF);
}
// OF
{
auto OldOF = GetRFLAG(FEXCore::X86State::RFLAG_OF_LOC);
// OF is set to the XOR of the new CF bit and the most significant bit of the result
auto NewOF = _Xor(_Bfe(1, OpSize - 2, Res), NewCF);
// If shift == 0, don't update flags
auto OF = _Select(FEXCore::IR::COND_EQ, Src2, _Constant(0), OldOF, NewOF);
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(OF);
}
}
void OpDispatchBuilder::GenerateFlags_RotateLeft(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2) {
auto OpSize = GetSrcSize(Op) * 8;
// Extract the last bit shifted in to CF
//auto Size = _Constant(GetSrcSize(Res) * 8);
//auto ShiftAmt = _Sub(Size, Src2);
auto NewCF = _Bfe(1, 0, Res);
// CF
{
auto OldCF = GetRFLAG(FEXCore::X86State::RFLAG_CF_LOC);
auto CF = _Select(FEXCore::IR::COND_EQ, Src2, _Constant(0), OldCF, NewCF);
// Extract the last bit shifted in to CF
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(CF);
}
// OF
{
auto OldOF = GetRFLAG(FEXCore::X86State::RFLAG_OF_LOC);
// OF is set to the XOR of the new CF bit and the most significant bit of the result
auto NewOF = _Xor(_Bfe(1, OpSize - 1, Res), NewCF);
auto OF = _Select(FEXCore::IR::COND_EQ, Src2, _Constant(0), OldOF, NewOF);
// If shift == 0, don't update flags
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(OF);
}
}
void OpDispatchBuilder::GenerateFlags_RotateRightImmediate(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, uint64_t Shift) {
if (Shift == 0) return;
auto OpSize = GetSrcSize(Op) * 8;
auto NewCF = _Bfe(1, OpSize - Shift, Src1);
// CF
{
// Extract the last bit shifted in to CF
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(NewCF);
}
// OF
{
if (Shift == 1) {
// OF is set to the XOR of the new CF bit and the most significant bit of the result
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(_Xor(_Bfe(1, OpSize - 1, Res), NewCF));
}
}
}
void OpDispatchBuilder::GenerateFlags_RotateLeftImmediate(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, uint64_t Shift) {
if (Shift == 0) return;
auto OpSize = GetSrcSize(Op) * 8;
// CF
{
// Extract the last bit shifted in to CF
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(_Bfe(1, Shift, Src1));
}
// OF
{
if (Shift == 1) {
// OF is the top two MSBs XOR'd together
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(_Xor(_Bfe(1, OpSize - 1, Src1), _Bfe(1, OpSize - 2, Src1)));
}
}
}
}
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,113 @@
#include <FEXCore/Core/SignalDelegator.h>
#include <FEXCore/Utils/LogManager.h>
#include <unistd.h>
#include <signal.h>
namespace FEXCore {
struct ThreadState {
FEXCore::Core::InternalThreadState *Thread{};
};
thread_local ThreadState ThreadData{};
static bool IsSynchronous(int Signal) {
switch (Signal) {
case SIGBUS:
case SIGFPE:
case SIGILL:
case SIGSEGV:
case SIGTRAP:
return true;
default: break;
};
return false;
}
/**
* @brief Masks signals from the signal mask
*
* @param how Argument to sigmask. SIG_{BLOCK, SETMASK, UNBLOCK}
* @param Signal Which signal to set or -1 to sweep through them all
*/
static void MaskSignals(int how, int Signal = -1) {
// If we have a helper thread, we need to mask a significant amount of signals so the an errant thread doesn't receive a signal that it shouldn't
sigset_t SignalSet{};
sigemptyset(&SignalSet);
if (Signal == -1) {
for (int i = 0; i <= SignalDelegator::MAX_SIGNALS; ++i) {
// If it is a synchronous signal then don't ignore it
if (IsSynchronous(i)) {
continue;
}
// Add this signal to the ignore list
sigaddset(&SignalSet, i);
}
}
else {
sigaddset(&SignalSet, Signal);
}
// Be warned, a thread will inherit the signal mask if created from this thread
int Result = pthread_sigmask(how, &SignalSet, nullptr);
if (Result != 0) {
LogMan::Msg::E("Couldn't register thread to mask signals");
}
}
void SignalDelegator::MaskThreadSignals() {
MaskSignals(SIG_BLOCK);
}
FEXCore::Core::InternalThreadState *SignalDelegator::GetTLSThread() {
return ThreadData.Thread;
}
void SignalDelegator::RegisterTLSState(FEXCore::Core::InternalThreadState *Thread) {
ThreadData.Thread = Thread;
RegisterFrontendTLSState(Thread);
}
void SignalDelegator::UninstallTLSState(FEXCore::Core::InternalThreadState *Thread) {
UninstallFrontendTLSState(Thread);
ThreadData.Thread = nullptr;
}
void SignalDelegator::RegisterHostSignalHandler(int Signal, HostSignalDelegatorFunction Func, bool Required) {
SetHostSignalHandler(Signal, Func, Required);
FrontendRegisterHostSignalHandler(Signal, Func, Required);
}
void SignalDelegator::RegisterFrontendHostSignalHandler(int Signal, HostSignalDelegatorFunction Func, bool Required) {
SetFrontendHostSignalHandler(Signal, Func, Required);
FrontendRegisterFrontendHostSignalHandler(Signal, Func, Required);
}
void SignalDelegator::HandleSignal(int Signal, void *Info, void *UContext) {
// Let the host take first stab at handling the signal
auto Thread = GetTLSThread();
HostSignalHandler &Handler = HostHandlers[Signal];
if (!Thread) {
LogMan::Msg::E("[%d] Thread has received a signal and hasn't registered itself with the delegate! Programming error!", ::gettid());
}
else {
if (Handler.Handler &&
Handler.Handler(Thread, Signal, Info, UContext)) {
// If the host handler handled the fault then we can continue now
return;
}
if (Handler.FrontendHandler &&
Handler.FrontendHandler(Thread, Signal, Info, UContext)) {
return;
}
// Now let the frontend handle the signal
// It's clearly a guest signal and this ends up being an OS specific issue
HandleGuestSignal(Thread, Signal, Info, UContext);
}
}
}
@@ -0,0 +1,285 @@
// This is the vsyscall page for x86_64 guest code
// This was compiled with nasm with the following source then exported to binary
//BITS 64;
//
//align 4096, db 0xcc
// ; __NR_gettimeofday
// mov rax, 96
// syscall
// ret
//
//align 1024, db 0xcc
// ; __NR_time
// mov rax, 201
// syscall
// ret
//
//align 1024, db 0xcc
// ; __NR_getcpu
// mov rax, 309
// syscall
// ret
//
//align 4096, db 0xcc
// We only want one of these pages per FEX process
// One page
const static uint8_t VSyscallData[0x1000] = {
0xB8, 0x60, 0x00, 0x00, 0x00, 0x0F, 0x05, 0xC3, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xB8, 0xC9, 0x00, 0x00, 0x00, 0x0F, 0x05, 0xC3, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xB8, 0x35, 0x01, 0x00, 0x00, 0x0F, 0x05, 0xC3, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC
};
@@ -114,8 +114,6 @@ void InstallDebugInfo() {
GenerateDebugTable(PrimaryInstGroupOps, PrimaryGroupOpTable);
GenerateDebugTable(SecondInstGroupOps, SecondaryExtensionOpTable);
LogMan::Msg::D("Installing debug info");
}
}
#endif
+6 -3
View File
@@ -6,12 +6,15 @@ $end_info$
*/
#include "Interface/Core/X86HelperGen.h"
#include <FEXCore/Config/Config.h>
#include <FEXCore/Utils/Allocator.h>
#include <cstdint>
#include <cstring>
#include <stdlib.h>
#include <vector>
#include <sys/mman.h>
#include <bits/mman-map-flags-generic.h>
namespace FEXCore {
constexpr size_t CODE_SIZE = 0x1000;
@@ -50,14 +53,14 @@ void* X86GeneratedCode::AllocateGuestCodeSpace(size_t Size) {
// We need to have the sigret handler in the lower 32bits of memory space
// Scan top down and try to allocate a location
for (size_t Location = 0xFFFF'E000; Location != 0x0; Location -= 0x1000) {
void *Ptr = FEXCore::Allocator::mmap(reinterpret_cast<void*>(Location), Size, PROT_READ | PROT_WRITE, MAP_FIXED_NOREPLACE | MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
void *Ptr = ::mmap(reinterpret_cast<void*>(Location), Size, PROT_READ | PROT_WRITE, MAP_FIXED_NOREPLACE | MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
if (Ptr != MAP_FAILED &&
reinterpret_cast<uintptr_t>(Ptr) >= LOCATION_MAX) {
// Failed to map in the lower 32bits
// Try again
// Can happen in the case that host kernel ignores MAP_FIXED_NOREPLACE
FEXCore::Allocator::munmap(Ptr, Size);
::munmap(Ptr, Size);
continue;
}
+1 -1
View File
@@ -5,8 +5,8 @@ $end_info$
*/
#pragma once
#include <FEXCore/Config/Config.h>
#include <stddef.h>
#include <stdint.h>
namespace FEXCore {
-7
View File
@@ -5,14 +5,8 @@ tags: frontend|x86-tables
$end_info$
*/
#include <FEXCore/Utils/LogManager.h>
#include <FEXCore/Core/Context.h>
#include <FEXCore/Debug/X86Tables.h>
#include <array>
#include <cstdint>
#include <tuple>
#include <vector>
namespace FEXCore::X86Tables {
@@ -108,7 +102,6 @@ void InitializeInfoTables(Context::OperatingMode Mode) {
#ifndef NDEBUG
X86InstDebugInfo::InstallDebugInfo();
LogMan::Msg::D("X86Tables had %ld total insts, and %ld labeled as understood", Total, NumInsts);
#endif
}
@@ -7,12 +7,15 @@ $end_info$
#include "Interface/Core/X86Tables/X86Tables.h"
#include <FEXCore/Core/Context.h>
#include <FEXCore/Debug/X86Tables.h>
#include <iterator>
namespace FEXCore::X86Tables {
using namespace InstFlags;
void InitializeBaseTables(Context::OperatingMode Mode) {
const U8U8InfoStruct BaseOpTable[] = {
static constexpr U8U8InfoStruct BaseOpTable[] = {
// Prefixes
// Operand size overide
{0x66, 1, X86InstInfo{"", TYPE_PREFIX, FLAGS_NONE, 0, nullptr}},
@@ -231,7 +234,7 @@ void InitializeBaseTables(Context::OperatingMode Mode) {
{0xC4, 2, X86InstInfo{"", TYPE_VEX_TABLE_PREFIX, FLAGS_NONE, 0, nullptr}},
};
const U8U8InfoStruct BaseOpTable_64[] = {
static constexpr U8U8InfoStruct BaseOpTable_64[] = {
{0x06, 2, X86InstInfo{"[INV]", TYPE_INVALID, FLAGS_NONE, 0, nullptr}},
{0x0E, 1, X86InstInfo{"[INV]", TYPE_INVALID, FLAGS_NONE, 0, nullptr}},
{0x16, 2, X86InstInfo{"[INV]", TYPE_INVALID, FLAGS_NONE, 0, nullptr}},
@@ -255,7 +258,7 @@ void InitializeBaseTables(Context::OperatingMode Mode) {
{0xEA, 1, X86InstInfo{"[INV]", TYPE_INVALID, FLAGS_NONE, 0, nullptr}},
};
const U8U8InfoStruct BaseOpTable_32[] = {
static constexpr U8U8InfoStruct BaseOpTable_32[] = {
{0x06, 1, X86InstInfo{"PUSH ES", TYPE_INST, GenFlagsSrcSize(SIZE_16BIT) | FLAGS_DEBUG_MEM_ACCESS, 0, nullptr}},
{0x07, 1, X86InstInfo{"POP ES", TYPE_INST, GenFlagsSizes(SIZE_16BIT, SIZE_DEF) | FLAGS_DEBUG_MEM_ACCESS, 0, nullptr}},
{0x0E, 1, X86InstInfo{"PUSH CS", TYPE_INST, GenFlagsSrcSize(SIZE_16BIT) | FLAGS_DEBUG_MEM_ACCESS, 0, nullptr}},
Loaded 100 of 393 files, more files were not shown because too many files have changed in this diff. Show more