Compare commits

..
290 Commits
Author SHA1 Message Date
Ryan Houdek 1f65bf9e47 Docs: Update for release FEX-2108 2021-08-02 23:21:04 -07:00
Ryan Houdek 4c431518b0 Merge pull request #1192 from Sonicadvance1/workaround_static_pie
AArch64: Workaround static-pie crashing
2021-08-02 23:19:45 -07:00
Ryan Houdek b586592643 Make sure our std::filesystem users use the std::error_code versions
Otherwise these crash out.
2021-08-02 23:10:23 -07:00
Ryan Houdek 83bdbc8392 AArch64: Check if we launched with FD argument
This will give us another way to check if the interpreter is installed rather than file checks.
The binfmt_misc files won't exist inside of a rootfs
2021-08-02 22:47:49 -07:00
Ryan Houdek c49fa5a64f Safely check if we can create paths rather than crashing on failure. 2021-08-02 22:47:42 -07:00
Ryan Houdek d052e87d47 AArch64: Workaround static-pie crashing
Static-pie can only work with GNU ld instead of lld on AArch64 hosts.
This is a known problem on the llvm side: https://bugs.llvm.org/show_bug.cgi?id=49672
2021-08-02 22:02:12 -07:00
Ryan Houdek ff98a43f70 Merge pull request #1191 from Sonicadvance1/improve_fexmountdaemon_stability
FEXMountDaemon: Make squashfs mounting more robust
2021-07-30 16:49:20 -07:00
Ryan Houdek a6c4ad889b FEXMountDaemon: Make squashfs mounting more robust
Instead of watching to ensure our parent process is still alive. Mount the
squashfs once and use a combination of file leases and inotify to
ref count how many processes are using the rootfs.

This makes it so in the common case, the FEXMountDaemon only ever executes once
and runs until all FEX processes stop running.
In the rare edge case there is a race condition where multiple FEXMountDaemon
applications will start, but only one will end up mounting the squashfs.
In this case, one application wins and the one that failed to grab the lease
will wait until the other one completes.

With this change, squashfs should be reasonable to use now.
2021-07-29 21:45:30 -07:00
Ryan Houdek 7e8577a50a Merge pull request #1180 from Sonicadvance1/support_setxid
Linux: Implements support for signals 32 and 33
2021-07-28 00:22:30 -07:00
Ryan Houdek a21cead9dc Fixes memcpy of siginfo_t data
_sifields are offset by a uint32_t between a 32-bit and 64-bit arch.

Fixes thread cancelling on 32-bit processes
2021-07-28 00:10:15 -07:00
Ryan Houdek 555cdb558b Linux: Implements support for signals 32 and 33
When a guest tries to use the setxid syscalls, the guest glibc has
a mechanism in place to ensure that the process wide setxid is handled.
The mechanism is fairly complex but it uses signal 33 and sends the signal to all
active threads to ensure every thread sets the correct state here.

We need to intercept this and pass the context information correctly to the guest instead.
Otherwise FEX just crashes when the glibc HOST handler tries handling the guest applications signal.

Fixes the game SOMA https://store.steampowered.com/app/282140/SOMA/
2021-07-28 00:10:08 -07:00
Ryan Houdek ceaaac3b7a Linux: Pass setxid syscalls directly to the kernel
We don't want these to be captured by the host glibc handlers around these
2021-07-27 18:55:30 -07:00
Ryan Houdek de0337db47 Merge pull request #1188 from Sonicadvance1/wine_fixes
Wine fixes
2021-07-27 18:07:53 -07:00
Ryan Houdek a48e41e4a9 Merge pull request #1187 from Sonicadvance1/cleanup_compile_service_on_fork
Cleanup compile service on fork
2021-07-27 18:07:42 -07:00
Ryan Houdek 896aa52851 Merge pull request #1186 from Sonicadvance1/fix_partial_rclse
RCLSE: Fixes an assumption in RCLSE
2021-07-27 18:07:32 -07:00
Ryan Houdek 0315964848 Merge pull request #1185 from Sonicadvance1/allow_stall_config
Adds an option to stall processes on launch
2021-07-27 18:07:19 -07:00
Ryan Houdek dec822192a Merge pull request #1182 from Sonicadvance1/disable_jemalloc_option
Adds an option to disable jemalloc from cmake
2021-07-27 18:07:09 -07:00
Ryan Houdek a5b142cf02 Merge pull request #1178 from Sonicadvance1/implement_vsyscall
Linux: Implements support for vsyscall
2021-07-27 18:06:54 -07:00
Ryan Houdek 007b1cbe9d Merge pull request #1174 from Sonicadvance1/binfmt_support_preserve_and_fd
binfmt_misc: Support loading ELFs from FD and support preserve
2021-07-27 18:06:39 -07:00
Ryan Houdek b0a28e595f Merge pull request #1181 from Sonicadvance1/fix_rapass
Fixes bug in RAPass
2021-07-27 18:06:27 -07:00
Ryan Houdek b8c5cf2736 Adds FXAM unit tests for ABI conformance
Wine was doing a fun x87 stack clean by checking tag bits for empty.
We hadn't previously supported anything but valid. Which cause wine to overflow the stack
trying to save all of the x87 values.

We now support valid and empty well enough that this doesn't break and we can unit test it
2021-07-27 08:25:40 -07:00
Ryan Houdek 2803113e35 Improves 32-bit signal handling
A decent amount of this was broken.
To the point that the ABI was incorrect even, causing 32-bit signals to completely crash in most instances.
This is enough to get my 32-bit signal handler test application to work and basic SIGUSR behaviour
2021-07-27 08:23:25 -07:00
Ryan Houdek 7a84ab8cfc Implements partial support for x87 FTW
This only implements the Empty and Valid tags.
Anything that is not empty is considered valid currently.
2021-07-27 08:22:11 -07:00
Ryan Houdek 77a7ecbae0 Add x87 FTW to CoreState
This will be needed for x87 FTW emulation
2021-07-27 07:57:41 -07:00
Ryan Houdek 2310ea83bb Cleanup compile service on fork
On fork this thread will be gone. Make sure to delete the shared_ptr.
2021-07-27 07:53:02 -07:00
Ryan Houdek 109df27bba Amd64: Very minor optimization in Push/Pop Regs
Just generates a bit nicer code to look at. Doesn't really change much.
2021-07-24 08:45:38 -07:00
Ryan Houdek 2afa14d1f3 ConstProp: Removes VMOV assumption
ConstProp has an assumption that a VMOV of the same size arguments is a no-op.
This isn't quite 100% true as this could be behaving as a zext and the pass can't see through the op correctly.

Something to be said that the vector ops should be looked over to see which op is leaving dirty bits in the upper bits
of the register. But that's a battle for a different time
2021-07-24 08:43:35 -07:00
Ryan Houdek 744b778013 RCLSE: Fixes an assumption in RCLSE
There is an assumption in the RCLSE pass that if a StoreContext + LoadContext pair of the same size
with a vector register that it is safe to remove the LoadContext.
This isn't quite safe since it can't entirely see through the IR to determine if the store op
had already zext the upper bits of the register.

This was causing 8byte loadcontext operations to "load" with garbage data in the upper
bits when it was expecting a zext.

Which granted it is likely a bug that not all vector ops zero their upper bits but that's a battle for a different day.

Additionally:
- Removes a couple dead optimizations in the GPR path that was never hit.
- Cleans up how access types are checked
- Uses ReplaceAllUsesWithRange to reinforce that this is purely a per block optimization still
2021-07-24 08:37:03 -07:00
Ryan Houdek b3965ab9b4 Adds an option to stall processes on launch
For misbehaving applications before we can fully diagnose, allow
a config that hangs a process on load.
2021-07-23 15:18:52 -07:00
Ryan Houdek ec6cc5cb4f Merge pull request #1184 from lioncash/vec-alloc
64BitAllocator: Convert std::vector table into std::array
2021-07-23 14:29:03 -07:00
Ryan Houdek 7931dc2c09 Merge pull request #1183 from lioncash/size
FlexBitSet: Use non-template type parameter for index parameters
2021-07-23 14:17:13 -07:00
Lioncash 22590806a0 64BitAllocator: Convert std::vector table into std::array
This is small enough that it can be converted over to a std::array so
the heap use isn't necessary
2021-07-23 17:12:37 -04:00
Lioncash f9313f4000 FlexBitSet: Use non-template type parameter for index parameters
These parameters are used as indexes into the tracked memory, so sizing
the index variable relative to the type being stored is kind of sketchy.

e.g. If the tracked types were uint8_t for example, we'd still want the
indexing parameters to be regularly sized so that we aren't implicitly
truncating values all the time when passing values to a uint8_t
parameter (and while all usages are currently using uint64_t as type
T, we may as well address this).

While we're at it, we can make both Get() and operator[] const member
functions, since they don't directly modify any underlying data, they
only read it.
2021-07-23 17:07:41 -04:00
Ryan Houdek ddcc18277b Check for valgrind for memory region check
If running under valgrind then ignore memory region check.
Otherwise you'd need to modify source to get past this check
2021-07-22 18:17:16 -07:00
Ryan Houdek 19de33f9ca Adds an option to disable jemalloc from cmake
While not recommended. It is necessary to allow disabling jemalloc if you want to run asan or tsan
2021-07-22 18:16:24 -07:00
Ryan Houdek a9ee4123ef Fixes bug in RAPass
RegisterNode isn't a POD so we can't memset it.
This was overwriting some unique_ptrs which were memory leaking and causing crashes in other PRs
2021-07-22 18:13:44 -07:00
Ryan Houdek 138e88bc80 Linux: Implements support for vsyscall
vsyscall is a legacy interface to expose three syscalls in a faster fashion.
Most applications should have moved to vDSO but this isn't entirely the case.

golang specifically still used vsyscall in the case that vdso isn't available.
Which that isn't completely safe since vsyscall can be completely disabled.
This is fixed in a newer version of their runtime but we must support legacy behaviour.

This makes it so FEX checks if the incoming RIP is a "special" region and adjusts where it
loads instructions from accordingly. Lying to the guest application about the instruction stream
while maintaining the illusion that its RIP is in the correct location.
This was fairly straightforward since we already decoupled the instruction stream from the RIP location.

This fixes golang execution as long as you also disable their preempt signaling.
environment variable to do so is `GODEBUG=asyncpreemptoff=1`
2021-07-22 16:58:17 -07:00
Ryan Houdek 62775161fe Merge pull request #1176 from Sonicadvance1/fix_static_pie_crashing
Works around static-pie crashing
2021-07-20 19:37:56 -07:00
Ryan Houdek 310cfcbcc4 Works around static-pie crashing
static-pie is crashing early due to an issue with pthread symbols being exposed as weak.
For some reason the weak symbols never get resolved and jump to zero or ELF base.

When linking static-pie, force search for the pthread symbols using an undefined glob.

With this, static-pie works as long as you link with a new enough version of lld (version 13.0)
2021-07-20 05:24:57 -07:00
Ryan Houdek 2fd41de5e0 Merge pull request #1153 from Sonicadvance1/split_opdispatcher
OpcodeDispatcher: Split the opcode handling to multiple files
2021-07-20 04:43:53 -07:00
Ryan Houdek c2a97d4fc1 Minor header shuffling to improve compilation time
Shaves a few seconds off compile time
2021-07-20 04:27:28 -07:00
Ryan Houdek ebb21e86b0 OpcodeDispatcher: Split the opcode handling to multiple files
These are some fairly large separate files still.
Don't want these to be too terribly small but helps significantly with compile time when working
in the OpcodeDispatcher
2021-07-20 04:27:28 -07:00
Ryan Houdek d8da4ce2a5 binfmt_misc: Support loading ELFs from FD and support preserve
This lets us support the final two flags in binfmt_misc that we needed.
1) Support open_binary
2) Support preserve

1)
We already supported the credentials flag, which implied open_binary, but we weren't handling half of it.
With the open_binary flag, the Linux kernel passes us the executable in an FD instead of as a pathname.
This can be found inside of auxv on startup, inside of AT_EXECFD.
If AT_EXECFD is available then we prioritize using that instead of the pathname passed in.
This fixes a potential permissions issue where an executable is executed without read permissions.

2)
The preserve flag has the Linux kernel preserve the original argv[0] that was passed to the application.
Prior to supporting this flag, the kernel would provide us with a resolved program path.
This can happen in the instance where something like `blah` resolves to `/usr/loca/bin/blah` which
isn't what the user originally typed.
This works around this problem by handing the interpreter both the resolved path and the original typed path.

Alongside open_binary, we can just use the FD passed in instead of the resolved path, this means
we can just drop the argv[0] for the guest (which is the kernel resolved path) and pass through arguments
unmangled.
We do have to make a minor assumption here that if we are using EXECFD that we assume preserve.
It isn't until kernel v5.12 that we can actually check AT_FLAGS to see if that was true.
2021-07-20 01:35:23 -07:00
Ryan Houdek aa767b3922 Merge pull request #1166 from Sonicadvance1/static_pie_support
Adds cmake option ENABLE_STATIC_PIE
2021-07-19 23:33:46 -07:00
Ryan Houdek 14480f3ee0 Merge pull request #1167 from Sonicadvance1/tsan_problems
Fixes some issues I found when running under tsan and asan
2021-07-19 23:30:25 -07:00
Ryan Houdek 6ac23003e1 Merge pull request #1168 from Sonicadvance1/fix_spill_slot
Arm64: Fixes fill and spill slot offset calculation
2021-07-19 23:29:01 -07:00
Ryan Houdek 04aacea58b Merge pull request #1173 from Sonicadvance1/fix_binfmt_misc_not_usr
Fixes binfmt_misc install when not installed to /usr
2021-07-19 23:26:33 -07:00
Ryan Houdek 3d46c40aaf Fixes binfmt_misc install when not installed to /usr
We had hardcoded paths in our binfmt_misc files. Change it to a generated
file instead.

Additionally for the binfmt_misc install targets, since we aren't installing to the
global binfmt_misc folder, we need to pass in the import directory

Fixes #1169
2021-07-18 13:05:12 -07:00
Ryan Houdek 7052d022c1 Merge pull request #1172 from lioncash/jits
Jits: Migrate logs over to fmt where applicable
2021-07-16 11:10:27 -07:00
Lioncash 9c7ebca235 JIT/x86_64: Migrate logs over to fmt 2021-07-16 13:57:15 -04:00
Lioncash 22656aedb2 JIT/Arm64: Migrate logs over to fmt 2021-07-16 13:57:12 -04:00
Ryan Houdek 80caf9038f Merge pull request #1171 from lioncash/string
General: Remove redundant string constructions in log calls related to IR op name retrieval
2021-07-16 10:09:43 -07:00
Lioncash 7fddac7b22 General: Remove redundant string constructions in log calls
Prior to the introduction of fmt, there wasn't a nice and easy way to
format std::string_view, but now that we have it in place, we can modify
relevant log calls to make use of it and get rid of the need to
construct a string around the view.

While we're in the same area, we can make the rest of the modified file
use fmt where appropriate.
2021-07-16 09:53:13 -04:00
Ryan Houdek 7f5b352453 Merge pull request #1170 from lioncash/dead
Validation Passes: Remove unused variables
2021-07-16 06:13:35 -07:00
Lioncash f8968b8296 IRValidation: Move stringstream int HadError conditonal body
Same behavior, but now we only construct it when we need it.
2021-07-16 08:54:09 -04:00
Lioncash 914861ba42 PhiValidation: Move stringstream int HadError conditonal body
Same behavior, but now we don't construct it unless we need to.
2021-07-16 08:52:51 -04:00
Lioncash 5d4efa8caa ValueDominanceValidation: Remove unused variables
No warnings are ever appended to the warning string stream, and
HadWarning is never set to anything other than false, so we can remove
it to bring it in line with the PhiValidation pass.
2021-07-16 08:50:33 -04:00
Ryan Houdek 0075e4ed19 Arm64: Fixes fill and spill slot offset calculation
+16 doesn't matter anymore. This can cause stack corruption
2021-07-15 06:47:28 -07:00
Scott Mansell 3a0da68b94 Merge pull request #1165 from Sonicadvance1/fix_lock_flags_ops
Fixes flag setting for 8bit and 16bit LOCK ALU ops
2021-07-15 14:00:33 +12:00
Ryan Houdek d0edd04e58 Amd64: Fixes X86Dispatcher allocator
The entire object didn't need to be an Allocator.
Additionally there was an overwrite happening on the Size variable.
This was causing the shutdown to free a size of zero.
2021-07-14 13:25:29 -07:00
Ryan Houdek 4332c22797 Linux: Fixes BRK deallocation
The SyscallHandler takes over ownership of BRK from the ELFLoader.
We were failing to deallocate the full range.
Additionally if the guest changed BRK size to something smaller than initial load then the size would underflow.
2021-07-14 13:23:58 -07:00
Ryan Houdek 5e12c38298 SignalDelegator: Switch Required flag to atomic bool
This is shared between multiple threads but it isn't never changed beyond first set.
Change it to atomic bool with relaxed loads to quiet tsan
2021-07-14 13:22:38 -07:00
Ryan Houdek bf8e28f935 Fixes TLS for softfloat
softfloat uses TLS for setting the rounding mode per thread.
We had never set the `THREAD_LOCAL` environment variable, so this
rounding mode was being set globally rather than per thread.
We are expecting per thread state here.

Noticed this while running through tsan
2021-07-14 13:21:02 -07:00
Ryan Houdek 6b90663be3 Adds cmake option ENABLE_STATIC_PIE
This option does cmake checks to determine if your system can handle static-pie.
With upstream projects static-pie only works if you use the binutils linker.
Using lld doesn't currently work because it defines __rela_iplt_{start,end} symbols.

Our cmake file will now compile a test application and check for these symbols.
Either the symbols will not exist at all or they will exist but be a null address

Once your system passes the checks then it will allow you to enable static-pie
2021-07-14 09:56:47 -07:00
Ryan Houdek c362bd883d Update jemalloc for more symbols 2021-07-14 08:21:29 -07:00
Ryan Houdek 0ade57614a Fixes flag setting for 8bit and 16bit LOCK ALU ops
MUL is handled in a different function, so this isn't needed there anymore.
This was causing a problem where if you force enabled LOCK on all capable ops it was causing a crash.

Issue #1164 is set to track getting unit tests for these.
2021-07-13 01:37:17 -07:00
Ryan Houdek f4e044a53c Merge pull request #1163 from Sonicadvance1/add_named_thunk_configs
Adds a convenience of named thunk configs
2021-07-13 00:26:36 -07:00
Ryan Houdek 108081e2cd Merge pull request #1162 from Sonicadvance1/update_man_page_with_json_key
Update man page to show JSON key for config options
2021-07-13 00:26:31 -07:00
Ryan Houdek 6ff9b9664d Merge pull request #1161 from Sonicadvance1/fix_thunks_finding_gl
Fixes Host thunks lib not being able to GL
2021-07-13 00:26:26 -07:00
Ryan Houdek 0a0d8f8885 Adds a convenience of named thunk configs
If the thunk config is not a path then search for the filename in $XDG_DATA_DIR/.fex-emu/ThunkConfigs/
for the file.

Just makes it easier to use rather than having full paths
2021-07-12 19:30:03 -07:00
Ryan Houdek bc09d73b9d Update man page to show JSON key for config options
Allows users to reference the man page for the json key instead of code diving
2021-07-12 19:22:57 -07:00
Ryan Houdek ed891f09f1 Fixes Host thunks lib not being able to GL 2021-07-12 19:21:58 -07:00
Ryan Houdek a24d63b5f9 Merge pull request #1160 from Sonicadvance1/allocator_fixes
Allocator fixes
2021-07-12 16:38:22 -07:00
Ryan Houdek 9434a59902 Remove some logs that aren't worth seeing anymore 2021-07-12 00:25:09 -07:00
Ryan Houdek 786a341652 BRKHandler: Don't rely on 64Bit Allocator 32-bit hack.
Use the actual 32-bit allocator if we are running a 32-bit process.
This hack in the 64bit allocator is likely to go away in the future
2021-07-12 00:22:59 -07:00
Ryan Houdek 74bff6c3a5 64BitAllocator: Add a backward scan path
First thing, do a backwards scan to try and fill any holes in the allocator
This reduces the number of holes we have in our VA space
2021-07-12 00:22:44 -07:00
Ryan Houdek 8cfda9c86d RAPass: Mostly revert the PR switching this over to mmap.
1) We needed to switch it to the FEXCore allocator so it didn't live in 32-bit space
2) This generates holes in the VA space due to how the allocations line up.
This makes us almost immediate hit the 65k VMA region.
Switching over to jemalloc allocations means we don't hit that.

Not worth creating an mmap based allocator that can work around that.
2021-07-12 00:16:20 -07:00
Ryan Houdek f79c832f88 X86HelperGen: Use System mmap for 32-bit fixed location
This needs to end up 32-bit space which means we shouldn't use the FEXCore Allocator.
FEXCore allocator has a hack in it for fixed offset in 32-bit space, but we shouldn't rely on it
2021-07-12 00:13:32 -07:00
Ryan Houdek 1fa51f7f74 Frontend: Use mmap helper for DecodeBuffer
This ensures that on 32-bit applications we don't consume 416KB per thread inside of 32-bit space
2021-07-12 00:12:30 -07:00
Ryan Houdek 53417e5322 Merge pull request #1158 from Sonicadvance1/atomic_improvements
Atomic improvements
2021-07-11 18:17:16 -07:00
Ryan Houdek 94e38bb556 Arm64: Fixes duplicated symbols problem in the interpreter
Uses local symbol numbers so the compiler doesn't get upset if this is inlined.
2021-07-10 23:51:02 -07:00
Ryan Houdek 59a4481e5b unittests: Adds explicit 16bit, 32bit, 64bit unaligned unit tests 2021-07-10 23:40:18 -07:00
Ryan Houdek 31a672b1d5 unittests: Adds explicit 128bit vector unaligned load test 2021-07-10 23:39:04 -07:00
Ryan Houdek e03ece0ee6 unittests: Adds LOCK NEG unit tests 2021-07-10 23:38:19 -07:00
Ryan Houdek b1151428e8 Arm64: Stop backpatching loadstores in paranoid TSO mode
When paroid tso is enabled, stop backpatching almost entirely.
Only code still backpatching is 128bit loadstores
2021-07-10 23:37:10 -07:00
Ryan Houdek 818808c3e1 Arm64: Implements unaligned atomic loads store handler
In the case that paranoid TSO is enabled, we can force all unaligned loadstores
down this path.

Instead of backpatching, just do the unaligned loadstore in the handler
2021-07-10 23:32:18 -07:00
Ryan Houdek ee40300eb5 OpcodeDispatcher: Implements support for LOCK NEG
Previously this was unimplemented since it isn't really used on x86.
Fixes #844
2021-07-10 23:30:42 -07:00
Ryan Houdek 484d756a26 Arm64: Implements AtomicFetchNeg IR op 2021-07-10 23:29:49 -07:00
Ryan Houdek d0bcdeda6b amd64: Implements AtomicFetchNeg IR op 2021-07-10 23:29:04 -07:00
Ryan Houdek ea218f8758 Interpreter: Implements AtomicFetchNeg
Does a two's complement negation of what is in memory atomically

Requires a loadstore exclusive loop to implement correctly
2021-07-10 23:27:45 -07:00
Ryan Houdek 821634e300 Arm64: Make sure ARMv8.0 Atomic Swap follows our pattern
The most minor of optimizations as well
2021-07-10 23:26:26 -07:00
Ryan Houdek 5df8354339 Arm64: Allow capturing loadstore exclusive pairing for atomic memory ops
This allows us to capture LDAXR* instructions which appear in ARMv8.0 memory ops.
These scan the memory space to see what the loop is doing and then make an atomic decision to handle
the loadstore exclusive loop with our signal handler based CAS atomics.

This will be necessary to support `LOCK NEG` on ARMv8.1+
2021-07-10 23:23:16 -07:00
Ryan Houdek fe10db9396 Adds an assert if we tried restoring a signal without known frames
Noticed this happen while tinkering. Best to get a debug only message in that case.
2021-07-10 23:19:46 -07:00
Ryan Houdek 3b4119dd31 Arm64: Add some helpers to unaligned atomic load
If we want to do atomic memory loads that are unaligned then we need some helpers to do these.

LoadAcquire128 needs to specifically avoid the std::atomic helper because it can try storing to read only memory
It does this to clear the exclusive lock from its ldaxp but we can just use clrex.
2021-07-10 23:18:46 -07:00
Ryan Houdek 60da70b07e ARMv8.0: Adds some NOPs around CASPair loadstore exclusive pair
The unaligned signal handler will backpatch these with DMBs.
Previously generating corrupted code. This will fix that corruption but not the atomicity problem
2021-07-10 23:15:49 -07:00
Ryan Houdek 35a4f52d7e Merge pull request #1156 from Sonicadvance1/actually_fix_jemalloc
Actually fix jemalloc for FEXConfig
2021-07-09 04:58:51 -07:00
Ryan Houdek 78a9dcf505 Actually fix jemalloc for FEXConfig 2021-07-09 03:48:29 -07:00
Ryan Houdek 1eee2dd004 Merge pull request #1155 from Sonicadvance1/improve_startup_time
Switch some memory allocations over to mmap
2021-07-09 03:47:05 -07:00
Ryan Houdek 25866a00ed Disable rtsignal gvisor test
This test is getting frustrating and is becoming MORE flakey as threads become faster
2021-07-09 01:57:48 -07:00
Ryan Houdek a27c05f577 Switch some memory allocations over to mmap
This cuts start up time from 8.962ms to 3.949ms and lets us use less physical memory.
We pay a small amortized cost from the faulting to populate pages which will be fairly low
2021-07-09 01:57:47 -07:00
Ryan Houdek 0d9ab9ca42 Merge pull request #1154 from Sonicadvance1/fix_jemalloc_override
Update jemalloc to fix glibc override.
2021-07-08 14:48:43 -07:00
Ryan Houdek 9c4c9ea566 Update jemalloc to fix glibc override.
For some reason jemalloc was missing an override
2021-07-08 14:33:35 -07:00
Stefanos Kornilios Mitsis Poiitidis a9bb33b8c2 Merge pull request #1152 from Sonicadvance1/fix_jemalloc_malloc_replace
Fix jemalloc malloc replace
2021-07-08 18:13:20 +03:00
Ryan Houdek faf09fec5f Removes glibc symbol replacement inside FEX
FEX_jemalloc now handles this entirely by itself.
It now replaces glibc during startup rather than when we call for it.
2021-07-06 21:52:57 -07:00
Ryan Houdek 4cd2b1a959 Update Externals jemalloc 2021-07-06 21:20:17 -07:00
Stefanos Kornilios Mitsis Poiitidis 8f09bd9215 Merge pull request #1150 from Sonicadvance1/fix_old_kernel_defines
Fixes old kernel defines for sockios
2021-07-06 15:39:54 +03:00
Stefanos Kornilios Mitsis Poiitidis 84dee03a16 Merge pull request #1151 from Sonicadvance1/error_on_immediate_failure
FEXLoader: stderr on immediate failure
2021-07-06 14:36:11 +03:00
Stefanos Kornilios Mitsis Poiitidis 0a8ee43081 Merge pull request #1149 from Sonicadvance1/enable_local_xxhash
Use externals xxhash if not found installed
2021-07-06 14:35:18 +03:00
Ryan Houdek 9fbeba3b79 FEXLoader: stderr on immediate failure
In the case that there is an immediate configuration failure. Use stderr specifically for outputting.
These errors won't be output typically because silent logging is enabled by default.
In the case of executable missing or rootfs configuration failure, print directly to stderr.
Previously it looked like FEX just exited for no reason.
We had multiple users encounter this and be confused
2021-07-05 18:52:06 -07:00
Ryan Houdek b3bab814de Fixes old kernel defines for sockios
These defines were changed at in kernel v5.2 but the debian defines didn't seem to change?
Redefine them if missing
2021-07-05 18:26:44 -07:00
Ryan Houdek 728e9fc187 Use externals xxhash if not found installed
Also in the case that you don't have v0.8.0 minimum installed
2021-07-05 17:42:35 -07:00
Ryan Houdek e01e6d9293 Adds xxhash external submodule 2021-07-05 17:39:19 -07:00
Stefanos Kornilios Mitsis Poiitidis 1a64c908e5 Merge pull request #1146 from Sonicadvance1/implement_sigqueueinfo
Linux: Implements rt_{tg,}sigqueueinfo
2021-07-05 16:55:34 +03:00
Stefanos Kornilios Mitsis Poiitidis 7dd700394e Merge pull request #1147 from Sonicadvance1/fix_imul_flags
OpDispatcher: Fixes imul flags calculations
2021-07-05 16:55:16 +03:00
Ryan Houdek b7b9d4089f unittests: Adds more imul unit tests for flags 2021-07-03 21:48:18 -07:00
Ryan Houdek 5a7dd844b0 OpDispatcher: Fixes imul flags calculations
We were calculating the high bits incorrectly in a couple variants
2021-07-03 21:45:15 -07:00
Ryan Houdek 80da08b15b Fixes Signal handler check for queue signals
Any signals sent with sigqueue or kill/sigsend are safe on the siginfo path.
2021-07-03 17:12:42 -07:00
Ryan Houdek 20a519a250 unittests: Update posix tests that have been fixed 2021-07-02 23:00:48 -07:00
Ryan Houdek b9efbd6159 Linux: Implements rt_{tg,}sigqueueinfo
We already support receiving user signals in our signal handlers for this.
Just need to push through syscalls.

This fixes Dead Island Definitive Edition so it runs.
2021-07-02 22:53:27 -07:00
Ryan Houdek 749f1eba51 Merge pull request #1145 from lioncash/stub
GdbStub: Fix memory leak in GdbServerLoop()
2021-07-02 03:21:00 -07:00
Lioncash bc52a16fde GdbStub: Fix memory leak in GdbServerLoop()
Previously the opened socket stream would be leaked.
2021-07-02 05:53:59 -04:00
Stefanos Kornilios Mitsis Poiitidis 2db44cf2ed Merge pull request #1142 from Sonicadvance1/implement_base_signalfd
Linux: Implements a base implementation of signalfd{4,}
2021-07-02 10:04:26 +03:00
Ryan Houdek 336eb9a8b4 Merge pull request #1143 from Sonicadvance1/remove_fork_log
Linux: Remove logs about parent or child stack usage
2021-07-01 12:46:18 -07:00
Ryan Houdek b923a82c39 unittests: fallocate gvisor test no longer fails 2021-07-01 07:11:00 -07:00
Ryan Houdek 1304279eb5 Linux: Remove logs about parent or child stack usage
We don't need to care about what the guest stacks are doing now
2021-07-01 07:06:14 -07:00
Ryan Houdek 1f6926a245 Linux: Implements a base implementation of signalfd{4,}
This is a base implementation of signalfd.
Signalfd allows the application to receive siginfo_t information through an FD.
The FD is either provided by the application or created by the kernel depending.
This specifically doesn't pick up *true* synchronous signals. tgkill of the number
should theoretically go through this interface.

This very specifically skips our internal required signals for now.
This means it won't pick up SIGILL, SIGBUS, or SIG63.

This is enough to capture an application that just wants to poll for SIGCHLD.
Anything more complex has the same problems of the guest handling a siginfo_t.
2021-07-01 06:51:39 -07:00
Stefanos Kornilios Mitsis Poiitidis 2d11df03cb Merge pull request #1141 from Sonicadvance1/modify_cpack_name
CPack: Update package name to remove conflict
2021-07-01 16:44:44 +03:00
Stefanos Kornilios Mitsis Poiitidis d142d7c363 Merge pull request #1140 from Sonicadvance1/cpuid_tm
CPUID: Update TM and TM2 CPUID bits
2021-07-01 16:44:09 +03:00
Ryan Houdek b546869d9d CPack: Update package name to remove conflict
'fex' is already taken by another projects. Use 'fex-emu' instead; Similar to other emulator package names.
2021-07-01 05:08:57 -07:00
Ryan Houdek 2a49ad941f CPUID: Update TM and TM2 CPUID bits
These are in place to let applications know that we do our own thermal management

TM was introduced with Pentium M, it would dispatch idle uops to counter thermals.
TM2 was later and is what allows the CPU to dynamically change its multiplier and frequency.
All in an effort to not cook themselves.

Our ARM CPUs also do all this. Claim it as well.
2021-07-01 04:26:39 -07:00
Ryan Houdek 8170307aef Docs: Update for release FEX-2107 2021-07-01 04:20:44 -07:00
Ryan Houdek 12cc979ec6 Merge pull request #1139 from Sonicadvance1/more_unaligned_atomics_fixes
ARM64: Fixes bugs in unaligned atomic signal handlers
2021-06-30 23:21:55 -07:00
Ryan Houdek d6ecd6364c Disables the new tests on ARMv8.0
Still aren't supported correctly there.
2021-06-30 21:48:36 -07:00
Ryan Houdek 7d6dafe25d Adds cmpxchg unit tests for new bugs encountered
Ensures we are testing when Desired != Memory && Memory == Expected to ensure it fails on all alignments
2021-06-30 21:43:51 -07:00
Ryan Houdek db6f4786d2 ARM64: Fixes bugs in unaligned atomic signal handlers
There were two bugs in here.

The first bug here is with with the CMPXCHG emulation.
1) If the *Expected* value did *NOT* match what was in memory
2) *AND* The *Desired* value matched the memory value
3) The CAS would incorrectly return success for this CMPXCHG
4) Thus setting ZF incorrectly

The second bug comes from atomic memory operations (Add, CLR, EOR, SET, SWAP).
This operation is a Load + <Op> + CAS
1) If the memory backing between the Load and CAS changes
2) The CAS would then fail
3) On Atomic memory operations this should then retry to ensure it completes successfully
4) We were not retrying on failure in this case
5) Thus something like `LOCK INC` would have never atomically incremented correctly
6) We can't use our ASM unit tests to test this, since it needs thread contention.
2021-06-30 21:43:44 -07:00
Ryan Houdek 49fa69d4e2 Merge pull request #1138 from lioncash/casfn
Arm64: Use regular function pointers with CAS handling functions
2021-06-30 19:46:38 -07:00
Lioncash 448cbc8e3d Arm64: Use regular function pointers with CAS handling functions
Given these are called in a loop repeatedly, where we know we'll always
have a set function to call, std::function adds a little bit of
overhead.
2021-06-30 22:03:08 -04:00
Ryan Houdek 81ce7b3f16 Merge pull request #1137 from lioncash/cexpr
OpcodeDispatcher: Make use of if constexpr
2021-06-30 18:21:20 -07:00
Lioncash c9615032a9 OpcodeDispatcher: Make use of if constexpr
Given these arguments are template arguments we can use if constexpr
with these branches to guarantee their elision.
2021-06-30 21:11:28 -04:00
Ryan Houdek 52e2c4abc5 Merge pull request #1136 from lioncash/handler
Context: Add alias for exit handler
2021-06-30 17:12:24 -07:00
Lioncash c4216755ab Context: Add alias for exit handler
Places the definition in one place so it doesn't need to be written
several times.
2021-06-30 19:50:57 -04:00
Ryan Houdek 09071e4279 Merge pull request #1135 from lioncash/ctx
Context: Place return value from GenerateIR into a struct
2021-06-30 16:39:24 -07:00
Lioncash 46a6e79233 Context: Place return value from GenerateIR into a struct
This makes it a little more straightforward to see what these return
values mean without needing to look at the implementation.

These tuples were also getting a little bit large.
2021-06-30 19:30:03 -04:00
Ryan Houdek 09363f1b1b Merge pull request #1134 from lioncash/lut
OpcodeDispatcher: Mark lookup tables as static in Get{Src,Dst}Size
2021-06-30 15:51:34 -07:00
Lioncash 9d5f9d6783 OpcodeDispatcher: Mark lookup tables as static in Get{Src,Dst}Size
Same behavior, but allows clang to elide pushing all of these values on
and off the stack, particularly given these are called quite frequently
throughout the opcode dispatcher.
2021-06-30 18:22:14 -04:00
Stefanos Kornilios Mitsis Poiitidis 7d0535e0d9 Merge pull request #1133 from Sonicadvance1/fix_zero_size_pt_load
Fixes PT_LOAD with zero file size
2021-06-29 09:39:22 +03:00
Stefanos Kornilios Mitsis Poiitidis 176b3292ed Merge pull request #1132 from Sonicadvance1/move_elfutils_frontend
Moves ELF handlers from FEXCore to frontend
2021-06-29 09:38:30 +03:00
Stefanos Kornilios Mitsis Poiitidis 3d25f594a9 Merge pull request #1131 from Sonicadvance1/disallow_disabling_cpuid
syscalls: Disallow disabling CPUID from arch_prctl
2021-06-29 09:37:02 +03:00
Stefanos Kornilios Mitsis Poiitidis 861d89e8a8 Merge pull request #1130 from Sonicadvance1/fix_fcntl
x32: Fixes fcntl OP_GETLK64_32
2021-06-29 09:36:19 +03:00
Stefanos Kornilios Mitsis Poiitidis 59c36ebc36 Merge pull request #1129 from Sonicadvance1/fix_msg_alignment
x32: Fixes sendmmsg cmsg alignment
2021-06-29 09:35:00 +03:00
Stefanos Kornilios Mitsis Poiitidis 2d8304547e Merge pull request #1128 from Sonicadvance1/remove_logs
Remove logs that are just noise at this point
2021-06-29 09:34:12 +03:00
Stefanos Kornilios Mitsis Poiitidis 4880097f89 Merge pull request #1122 from Sonicadvance1/CreateAppConfig
FEXConfig: Have it create AppConfig folder on save
2021-06-29 09:29:59 +03:00
Stefanos Kornilios Mitsis Poiitidis 3a11b88d30 Merge pull request #1120 from Sonicadvance1/fix_a_few_32bit_syscalls
x32: Fixes shmdt, sendmsg, recvmsg syscalls
2021-06-29 09:28:58 +03:00
Stefanos Kornilios Mitsis Poiitidis 76a054e50a Merge pull request #1113 from Sonicadvance1/fix_argument_epoll_ctl
Wrap 32-bit epoll_ctl argument in compat_ptr
2021-06-29 09:26:23 +03:00
Ryan Houdek 279d9219a6 Fixes PT_LOAD with zero file size
gzip ships a PT_LOAD program section without a file size and only a memory size.
In the case of a zero file size PT_LOAD then return success immediately in this section loader.
This fixes Steam using gzip to package up crash logs
2021-06-27 02:10:21 -07:00
Ryan Houdek ed4c5d9704 Remove logs that are just noise at this point
Don't print how many instructions are installed in the tables.
  This isn't useful anymore

Not installing signal 32 and 33 are something we don't support right now. Stop complaining in that case.

Stop printing when a thread is starting up and shutting down. If you want to see this then gdb shows it well.

Don't print clone flags unless we are hitting a case where we are printing another log message.
2021-06-27 02:09:27 -07:00
Ryan Houdek 069e279a18 Moves ELF handlers from FEXCore to frontend
Only the frontends need to deal with ELF files specifically.
The backend doesn't need to be aware of them at all.
Since the ELF handling is the frontend's responsibility, move all the code to the frontend.
2021-06-27 02:07:56 -07:00
Ryan Houdek 9ba1948375 syscalls: Disallow disabling CPUID from arch_prctl
In a newer version of the kernel there was a feature to disallow cpuid.
We can emulate this by saying it is always enabled and disallow the ability to disable it.
2021-06-27 02:03:21 -07:00
Ryan Houdek 835ca9cdf9 x32: Fixes fcntl OP_GETLK64_32
This was overwriting the cmd argument and then being checked in the switch statement
after the call.
Since it was overwritten, it wasn't falling down the correct path, returning a flock_32 instead of a flock64_32
2021-06-27 02:01:37 -07:00
Ryan Houdek 59a429ec19 x32: Fixes sendmmsg cmsg alignment
This wasn't aligning like the other recvmsg and sendmsg variants
2021-06-27 02:00:39 -07:00
Ryan Houdek 36b5863c7c Merge pull request #1125 from lioncash/ptr
FEXLoader: Make use of unique_ptr for syscall handlers
2021-06-25 18:36:32 -07:00
Ryan Houdek bdfbebe24d Merge pull request #1126 from lioncash/socket
x32/Socket: Amend std::vector initialization bug in sendmmsg
2021-06-25 17:24:13 -07:00
Ryan Houdek 336ecb8cc7 Merge pull request #1112 from Sonicadvance1/deferred_signal_installation
Implements support for deferred signal handler installation
2021-06-25 16:44:42 -07:00
Lioncash ea9f08e481 x32/Socket: Amend std::vector initialization in sendmmsg
Since the vector was being initialized with braces, this caused the
arrays to only ever have one element within it instead of the size of
the array.

While we're at it, we can also construct a vector in place with the
necessary size.
2021-06-25 10:00:37 -04:00
Lioncash da8c5dc460 FEXLoader: Make use of unique_ptr for syscall handlers
Makes the ownership requirements explicit in the interface.

Also makes it harder to unintentionally/accidentally leak memory.
2021-06-25 09:19:48 -04:00
Ryan Houdek fd8dcda67e Merge pull request #1124 from lioncash/timeofday
x32/Time: Implement settimeofday
2021-06-25 05:59:00 -07:00
Lioncash b2fb48d711 x32/Time: Implement settimeofday 2021-06-25 08:48:05 -04:00
Ryan Houdek 8e6328e946 Merge pull request #1123 from lioncash/time
x32/Time: Fix a few edge cases in time related syscalls
2021-06-25 05:45:23 -07:00
Lioncash 9262264525 x32/Time: Fix potential null pointer dereference in nanosleep
Prevents potential null dereferences that could occur from user code.
2021-06-25 08:30:45 -04:00
Lioncash 4553564c9f x32/Time: Fix potential null pointer dereference in clock_nanosleep
Prevents a potential null dereference in our code that could result from
user code.
2021-06-25 08:30:36 -04:00
Lioncash 2a86421172 x32/Time: Fix potential null pointer dereference in clock_settime
The kernel handles the case where a null address is passed in for tp.

Instead we can handle this case so we don't have a dereference inside
our code.
2021-06-25 08:19:04 -04:00
Ryan Houdek bb85f90f78 FEXConfig: Have it create AppConfig folder on save
Otherwise it will claim that the AppConfig was saved and won't actually be
2021-06-25 04:03:16 -07:00
Ryan Houdek 4a1da85214 Merge pull request #1121 from lioncash/utimesat
x32/Time: Implement futimesat
2021-06-25 03:36:32 -07:00
Lioncash ff9268d6c7 x32/Time: Implement futimesat 2021-06-25 06:20:51 -04:00
Ryan Houdek c4f8e6c934 x32: Fixes non-ipc sendmsg/recvmsg syscalls
We were sending these non-ipc variants of these syscalls through
the generic path. This is broken but luckily glibc from Ubuntu isn't shipping
a version using these yet.
2021-06-25 03:02:31 -07:00
Ryan Houdek 52c50735d5 x32: Fixes shmdt not going through allocator
This was missed. Which means we were leaking SHM allocations if the 32-bit allocator was used
2021-06-25 03:01:38 -07:00
Ryan Houdek aca8903b56 GVisor: Update gvisor tests that have now changed behaviour 2021-06-24 21:08:59 -07:00
Ryan Houdek 2e9b22e042 POSIX: Updates known failures
Now that signal behaviours have changed a little bit, update the tests that have changed
2021-06-24 20:42:39 -07:00
Ryan Houdek d4559ea0c6 Update host signal mask on guest update
Now that are aren't consuming all signals we need to more aggressively handle the host signal mask.
Now more signals are getting masked and blocked how they should be.
2021-06-24 20:42:39 -07:00
Ryan Houdek d4dd4d0972 Copy over siginfo_t to the guest more correctly
si_addr will still be incorrect. What matters more here is that SIGCHLD gets correct information.
The guest needs SIGCHLD ifnromation to be filled out correctly, otherwise TTY handoff hangs
with the child process stopped.
2021-06-24 20:42:39 -07:00
Ryan Houdek 79db23b7d3 Define some of 32-bit x86 siginfo_t 2021-06-24 20:42:39 -07:00
Ryan Houdek fbfc774446 Implements support for deferred signal handler installation
I saw a red herring that I thought the high cpu usage in steamwebhelper could come from signal handlers.
This turned out to not be the case, but now I've got this implemented.

Installs the few signal handlers that we need upfront but for everything that isn't a mandatory signal
we instead now wait until the guest also installs that signal handler.
This fixes #1107
2021-06-24 20:42:39 -07:00
Ryan Houdek 35157c1251 Merge pull request #1118 from lioncash/utimes
x32/Time: Implement utimes
2021-06-24 17:02:02 -07:00
Lioncash 17d836a178 x32/Time: Fix edge case in utimensat
utimensat allows passing null to signify that timestamps should be set
to the current time.

Check for this to avoid dereferencing null.
2021-06-24 18:49:55 -04:00
Lioncash 28d754b466 x32/Time: Implement utimes 2021-06-24 18:45:03 -04:00
Ryan Houdek 01667bdfb2 Merge pull request #1117 from lioncash/fsuid
x32/Thread: Implement setfsuid32/setfsgid32
2021-06-24 09:36:00 -07:00
Lioncash 2ddcaef525 x32/Thread: Implement setfsuid32/setfsgid32
Fairly straightforward to implement.
2021-06-24 12:22:48 -04:00
Ryan Houdek c7cd2241c8 Merge pull request #1116 from lioncash/groups
x32/Thread: Implement getgroups32/setgroups32
2021-06-24 09:11:39 -07:00
Lioncash f67f3fe4de x32/Thread: Implement getgroups32/setgroups32
Implements two other straightforward syscalls for the 32-bit side of
things.
2021-06-24 08:15:11 -04:00
Ryan Houdek 5d7822c987 Merge pull request #1115 from lioncash/chown
x32: Implement chown32/fchown32/lchown32
2021-06-24 04:47:25 -07:00
Lioncash 738354a52d x32/FD: Implement chown32/fchown32/lchown32
Implements three basic hooks for the chown32 family of syscalls.
2021-06-24 07:08:46 -04:00
Stefanos Kornilios Mitsis Poiitidis 26531e9b96 Merge pull request #1111 from Sonicadvance1/32bit_sigpending
Implements 32-bit sigpending
2021-06-24 13:24:38 +03:00
Stefanos Kornilios Mitsis Poiitidis 282402a80c Merge pull request #1110 from Sonicadvance1/cpack_setup
Implements support for cpack debian package building
2021-06-24 13:24:23 +03:00
Stefanos Kornilios Mitsis Poiitidis 28254a3163 Merge pull request #1105 from Sonicadvance1/fix_32bit_syscall_checks
Fixes a couple of 32-bit checks
2021-06-24 13:24:00 +03:00
Ryan Houdek 0dd568f7fb Wrap 32-bit epoll_ctl argument in compat_ptr
Doesn't change behaviour, just makes sure we know it is a compat_ptr
2021-06-24 00:43:14 -07:00
Ryan Houdek 12adc5bc6e Implements 32-bit sigpending
This is a early version of the syscall that only returns the lower 32 signals.
Easy enough to support
2021-06-24 00:35:23 -07:00
Ryan Houdek 6ac942266f Implements support for cpack debian package building
This doesn't currently install thunks which can come a bit later.
We require a postinst and prerm step for importing and unimporting the binfmt_misc files.
Easy enough
2021-06-23 18:47:04 -07:00
Ryan Houdek 24a2a0c4eb Switches binfmt_misc install step to use registration files
This is easier to represent than the raw files. Once we do a debian file install then
this becomes more important
2021-06-22 20:22:09 -07:00
Stefanos Kornilios Mitsis Poiitidis 910c624a8b Merge pull request #1104 from Sonicadvance1/implement_32bit_iret
Implements 32-bit iret instruction
2021-06-22 09:22:00 +03:00
Stefanos Kornilios Mitsis Poiitidis 767ea0fc9e Merge pull request #1103 from Sonicadvance1/deprioritize_aot
Lower priority of AOT compilation threads.
2021-06-22 09:20:23 +03:00
Stefanos Kornilios Mitsis Poiitidis 087e5a9576 Merge pull request #1102 from Sonicadvance1/implement_truncate64
Implements 32-bit truncate64 syscall
2021-06-22 09:20:04 +03:00
Stefanos Kornilios Mitsis Poiitidis 67e6ffbc93 Merge pull request #1101 from Sonicadvance1/fixed_stack_fix
Fixes fixed stack offset for 32-bit applications
2021-06-22 09:19:36 +03:00
Stefanos Kornilios Mitsis Poiitidis 6734d745c6 Merge pull request #1099 from Sonicadvance1/fix_strace_32
Fixes debug strace output
2021-06-22 09:17:14 +03:00
Stefanos Kornilios Mitsis Poiitidis 332123a38c Merge pull request #1098 from Sonicadvance1/fix_more_gdt
Fixes a 32-bit processes wanting another TLS space
2021-06-22 09:16:40 +03:00
Stefanos Kornilios Mitsis Poiitidis 24fdbe4e6d Merge pull request #1096 from Sonicadvance1/fix_kotor2
Fix Star Wars Knights of the Old Republic 2
2021-06-22 09:14:02 +03:00
Ryan Houdek ea48f36511 Fixes a couple of 32-bit checks
The allocator functions don't return -1 on error, instead return the actual error.
This was setup to match the syscall behaviour rather than the glibc/errno behaviour
2021-06-21 00:25:01 -07:00
Ryan Houdek d587485383 Implements 32-bit iret instruction
This is necessary for wine and some wine-like emulation layers.

With this implemented then some Saint's Row games start running.

Fixes #1100
2021-06-21 00:20:44 -07:00
Ryan Houdek c9211ae77a Lower priority of AOT compilation threads.
Set these threads to minimum priority to not complete starve the system if there are other tasks running.
Fixes #1075
2021-06-20 17:24:36 -07:00
Ryan Houdek 7cbe9579ba Fixes 32-bit allocator returning 64-bit pointers on old kernels
In the case of a kernel older than 4.17 then MAP_FIXED_NOREPLACE doesn't exist.
This adds a check in to see if the pointer returned was valid but not what we asked for
2021-06-20 17:22:26 -07:00
Ryan Houdek ca482e3e96 Implements 32-bit truncate64 syscall
A 32-bit game was using this syscall and now it gets farther in game
2021-06-20 01:10:19 -07:00
Ryan Houdek eaa3df9cba Fixes fixed stack offset for 32-bit applications
Instead of forcing a fixed offset for the stack. Allow it to get placed automatically.
64-bit was already doing this; Now we can also do it in 32-bit.
This is possible because the mapper that is mapping the code will always map in the 32-bit space
for a 32-bit guest.
2021-06-20 01:07:58 -07:00
Ryan Houdek c252dc99fc Fixes debug strace output
This custom definition was missing
2021-06-19 22:01:47 -07:00
Ryan Houdek 70dc5217ec Fixes a 32-bit processes wanting another TLS space
Currently our 32-bit code only gave one TLS slot and crashed
if you needed more.

First switch over to the same initial slot as the Linux kernel.
Then allow searching the slots for a free spot.
This allows us to more closely match the behaviour of the Linux kernel just in case
anything has hardcoded the TLS slots

This makes it so Saints Row: The Third stops crashing at boot, plays a few intro videos, then hangs instead.
2021-06-19 21:58:49 -07:00
Ryan Houdek 1745bcceb6 Fixes 32bit statfs and fstatfs
These 32bit syscalls use a compat statfs which is only 64bytes in size.
This was overwriting data on the guest stack and causing crashes.

Describe a 32-bit statfs struct and ensure with struct verifier that it matches.

Fixes a crash in KOTOR2 that would happen just before the main menu.
2021-06-19 07:26:41 -07:00
Ryan Houdek cc8fa9d934 Fixes 32bit sysinfo syscall
This was altered in kernel 2.3.23 to include a mem_unit variable.
mem_unit states what each of the memory units is scaled by.

The kernel will always have this be 1 or page size on x86, so follow that behaviour.
Instead of maxing out the values to what uint32_t can handle, scale all of these by page size instead.

This fixes a crash in KOTOR2's boot sequence where it would divide by zero if everything was maxed out.
2021-06-19 07:23:25 -07:00
Ryan Houdek 424b93b5f8 Merge pull request #1095 from Sonicadvance1/disable_flake2
Disable flaky posix test
2021-06-17 15:12:59 -07:00
Ryan Houdek 5fccf5f741 Disable flaky posix test
Test sleeps for 1 second and expects to come back within 10ms of the time.
When the CPU is doing other things then it can end up missing that timeframe. Thus flake.

Just disable it
2021-06-17 07:51:37 -07:00
Ryan Houdek 42f1dfeede Merge pull request #1094 from Sonicadvance1/fix_fexbash_squashfs
Fixes FEXBash with squashfs
2021-06-17 07:47:04 -07:00
Ryan Houdek 21fa93f3b3 Fixes FEXBash with squashfs
FEXBash wasn't setting up a squashfs on its end. This meant that it couldn't find any
x86-64 libraries and would fail
2021-06-17 06:41:07 -07:00
Ryan Houdek f088d97060 Merge pull request #1093 from lioncash/cast
General: Resolve -Wcast-qual warnings
2021-06-16 23:28:52 -07:00
Lioncash e369626929 BitUtils: Add BitCast
libc++ doesn't implement std::bit_cast, so we can provide our own for
the time being.
2021-06-17 01:30:07 -04:00
Lioncash cba4ca7d01 General: Resolve -Wcast-qual warnings
Ensures that qualifiers are preserved on references and pointers to
prevent undefined behavior.
2021-06-17 01:25:39 -04:00
Ryan Houdek fb7964e6b1 Merge pull request #1092 from lioncash/ignored-qual
General: Resolve -Wignored-qualifiers warnings
2021-06-16 19:52:39 -07:00
Lioncash ef7aff796f General: Resolve -Wignored-qualifiers warnings
Removes const qualifiers that don't do anything to the interface.
2021-06-16 21:51:45 -04:00
Ryan Houdek d2771669e6 Merge pull request #1091 from lioncash/unused
Core: Remove unused DefaultFallbackCore
2021-06-16 17:32:56 -07:00
Lioncash e6e170805e Core: Remove unused DefaultFallbackCore
This doesn't seem to be hooked up to anything.
2021-06-16 19:58:05 -04:00
Ryan Houdek 1f89ca7218 Merge pull request #1090 from lioncash/array
Frontend: Make lookup tables static in MapModRMToReg
2021-06-16 14:44:35 -07:00
Ryan Houdek adcee99625 Merge pull request #1089 from lioncash/gprsize
Context: Move GPR size retrieval to its own function
2021-06-16 14:03:20 -07:00
Lioncash a9623f0e2a Frontend: Shrink MapModRMToReg array size from uint64_t to uint32_t
This function only returns a 32-bit value, so we can save some space by
using uint32_t instead.
2021-06-16 16:59:56 -04:00
Lioncash c40ca14b5b Frontend: Make lookup tables static in MapModRMToReg
Allows clang to emit better code, since it doesn't need to push all the
values onto the stack and off again for every invocation.
2021-06-16 16:47:12 -04:00
Lioncash 100bc4c833 Context: Move GPR size retrieval to its own function
This is repeated in quite a few spots, so we can place it in a utility
function and just call it instead.
2021-06-16 16:14:26 -04:00
Ryan Houdek a8855a330a Merge pull request #1088 from lioncash/table
OpcodeDispatcher: Make GPR indices in SyscallOp constexpr
2021-06-16 12:17:18 -07:00
Lioncash 72a625da4a OpcodeDispatcher: Make gpr indices in SyscallOp constexpr
Places them in RO where they can't be modified.

While we're in the area, we can use an alias to prevent duplicated array
types, and also add a static assert to ensure the arrays are always the
same size.

This allows us to avoid needing to bounds check several accesses in a
row that we know will always be successful.
2021-06-16 14:24:45 -04:00
Stefanos Kornilios Mitsis Poiitidis a67bdddbdc Merge pull request #1087 from Sonicadvance1/fix_typoe
Fixes typo on extension check for squashfs
2021-06-16 10:03:48 +03:00
Ryan Houdek 8274058895 Fixes typo on extension check for squashfs
Accidentally deleted the period on this
2021-06-15 23:03:46 -07:00
Stefanos Kornilios Mitsis Poiitidis 52426ae9f3 Merge pull request #1086 from Sonicadvance1/support_squashfs
Support Squashfs based rootfs files
2021-06-16 08:50:29 +03:00
Ryan Houdek 4ff3705ff0 Update fusermount location
fuse3 package installs this to /usr/fusermount
Try both locations on failure
2021-06-15 19:50:39 -07:00
Ryan Houdek e5d3699b26 FEXConfig: Adds support for squashfs files in FEXConfig
Searches the RootFS path in the fex-emu config folder for squashfs files and displays them
Allows easy configuration of squashfs
2021-06-15 19:34:42 -07:00
Ryan Houdek a5f07f1d01 FEXLoader: Set up the squashfs rootfs in FEXLoader 2021-06-15 19:34:42 -07:00
Ryan Houdek d4789895a0 Adds a helper function for setting up a squashfs rootfs
This function does everything required for setting up a squashfs as a rootfs.
- Checks if the file is a valid squashfs
- Executes the FEXMountDaemon
- Error checks to ensure it was mounted correctly
- Updates CONFIG_ROOTFS to point to the mounted location
- Takes 200-400ms more startup time
2021-06-15 19:34:42 -07:00
Ryan Houdek 86cbf06778 Adds a FileFormatCheck for checking if a file format is squashfs
This will be used to ensure a squashfs file that is selected is sane
2021-06-15 19:20:19 -07:00
Ryan Houdek d77a503510 Adds new FEXMountDaemon tool
This is a tool that communicates with FEXLoader/FEXInterpreter to automatically mount
squashfs based rootfs files on execve.

This tool will launch automatically if you have a squashfs based rootfs selected.
Once the rootfs is mounted, it will watch for the parent FEX processe to completely exit.
Once the parent FEX exits it will unmount and cleanup after itself.

This tool has a dependency on your host having FUSE, fusermount, and squashfuse applications.
If anything goes wrong in the bringup process then it propagates the erro up the chain and will
let FEX know that it couldn't mount.
2021-06-15 19:08:54 -07:00
Ryan Houdek 9b4b136121 Move Config.h to ConfigDefines.h
We already have a Config.h which conflicts with this file.
In issue cropped up if you need to include Config.h (the generated one)
from inside Common/ then it would only pull the Common/Config.h file.

Just change the name to not be confusing
2021-06-15 19:06:49 -07:00
Stefanos Kornilios Mitsis Poiitidis 779aca7e95 Merge pull request #1085 from Sonicadvance1/transparent_huge_pages
Enables transparent huge pages in our 64-bit VA allocator
2021-06-14 17:04:40 +03:00
Stefanos Kornilios Mitsis Poiitidis 3252f793df Merge pull request #1084 from Sonicadvance1/cleanup_stacks_after_fork
Core: After fork make sure to cleanup stacks
2021-06-14 17:03:49 +03:00
Stefanos Kornilios Mitsis Poiitidis 9635b34450 Merge pull request #1083 from Sonicadvance1/more_cpuid_fixes
CPUID: Improvements to have a more sane configuration
2021-06-14 10:14:34 +03:00
Stefanos Kornilios Mitsis Poiitidis afd35be91f Merge pull request #1082 from Sonicadvance1/remove_numa_really
Removes more libnuma references
2021-06-14 10:12:35 +03:00
Stefanos Kornilios Mitsis Poiitidis 50b5ad5762 Merge pull request #1070 from Sonicadvance1/remove_warnings
Remove most warnings in FEX again
2021-06-14 10:12:03 +03:00
Ryan Houdek 10ac1518e1 Enables transparent huge pages in our 64-bit VA allocator
Transparent huge pages is a feature that the linux kernel opportunistically uses.
Depending on kernel configuration this feature is either enabled always, or when you madvise the region.

To ensure we hit both cases, madvise the regions we allocate in the 64-bit VMA allocator always

Can reduce kernel bookkeeping memory usage for our abusive allocator
2021-06-12 18:52:51 -07:00
Ryan Houdek ff1c59b6af CPUID: Improvements to have a more sane configuration
When running the cpuid application (http://www.etallen.com/cpuid.html) I noticed
that we were returning some garbage data here.

After initially implementing support for leaf functions, it still didn't resolve the issue.
So I had to fix those in the x86-64 JIT.

I then went through and solved more issues with the function results.

- We now return a more sane CPU family that is near the feature set we support
- APICID now understands how to fill out the data correctly depending on emulated core counts
- Disabled some CPU features that we don't actually support
- Found two more cache functions that we weren't populating
  - Filled with generic data cache size data
  - Only thing that matters is that we ensure that cacheline size is reported as 64bytes
  - L1D: 32KB, L1I: 32KB, L2: 512KB, L3: 8MB claimed for caches
- Implemented Leafs for functions
  - 7h - Only has leaf 0
  - Dh - Extended CPU features support
    - Another register that lets you claim support for x87, SSE, and AVX
    - Leaf 1 & 2 has some additional data
  - 4h & 8000'0001Dh - Extended cache properties
    - Almost the same as each other. One reports slightly less data though
2021-06-12 18:49:43 -07:00
Ryan Houdek 085fca01bc Core: After fork make sure to cleanup stacks
After FEX has forked, there aren't any other threads in the process but their stacks remain.
We need to have some book keeping in place to have the stack ranges available to clean up
after fork.

We now keep both live stacks and dead stacks in a dequeue and on fork we will walk both to
clean up all stack objects that aren't our current thread.
2021-06-12 18:45:13 -07:00
Ryan Houdek 3d759a91ca Jit64: Fixes register overwrite in CPUID
rsi is a SSA argument, so we need to make sure to move the leaf argument first.
the leaf argument was getting corrupted when moving to the ABI.

Will be necessary once CPUID supports leafs
2021-06-12 18:29:59 -07:00
Ryan Houdek b54385162e Removes more libnuma references
We don't need libnuma, stop trying to link to it
2021-06-12 18:28:08 -07:00
Ryan Houdek 98714a4971 Merge pull request #1079 from lioncash/iostream
Passes: Replace <iostream> header with narrower equivalents
2021-06-11 21:00:29 -07:00
Lioncash d08189c3ed Passes: Log out errors and warnings through a format specifier
Passing in the string directly through the format string input can
unintentionally cause the output string to be interpreted as a format
string.

We can specify a separate format string to ensure it always prints
without any potential mangling.
2021-06-11 22:27:02 -04:00
Lioncash 2ad9ced80a Passes: Replace <iostream> header with narrower equivalents
<iostream> injects a static constructor in translation units that
include it, even if its facilities aren't used.

We can make use of <sstream> to avoid needing to execute those on
startup.
2021-06-11 22:24:45 -04:00
Ryan Houdek 12efcdc98d Merge pull request #1078 from lioncash/passes
Passes: Return by unique_ptr where applicable
2021-06-11 19:15:22 -07:00
Lioncash 120ba3d171 PassManager: std::move exit handler function
Avoids allocations if the internal std::function buffer ever happens to
be large enough to warrant it.
2021-06-11 22:05:17 -04:00
Lioncash 6084bdf982 Passes: Return by unique_ptr where applicable
Same behavior, but makes the ownership intentions explicit in the
interface.
2021-06-11 22:05:15 -04:00
Ryan Houdek 38d25d97f5 Merge pull request #1077 from lioncash/vtab
HostAllocator: Make Create64BitAllocator() return a unique_ptr
2021-06-11 16:36:30 -07:00
Lioncash 6fc117a67c HostAllocator: Make Create64BitAllocator() return a unique_ptr
Communicates the ownership intent in the interface.

Also removes the prototype for CreateBasicAllocator, as it isn't
implemented anywhere.
2021-06-11 19:11:46 -04:00
Lioncash 1b72f73224 GlobalAllocator: Add virtual destructor to interface
Prevents any potential destruction issues from occurring in the
interface.
2021-06-11 19:04:25 -04:00
Ryan Houdek 74f63c904e Merge pull request #1076 from lioncash/leak
RegisterAllocationPass: Prevent leaks in BucketList
2021-06-11 15:08:01 -07:00
Lioncash b6edc8dc53 RegisterAllocationPass: Prevent leaks in BucketList
release() relinquishes control of the allocated memory, but doesn't
deallocate it. We need reset() for this.
2021-06-11 17:49:56 -04:00
Ryan Houdek af112f4c47 Merge pull request #1074 from lioncash/args
Syscalls/Signals: Add missing arguments to pidfd_send_signal syscall
2021-06-11 09:28:59 -07:00
Ryan Houdek 58c66b736d Merge pull request #1073 from lioncash/version
Syscalls: Add helper for version testing
2021-06-11 09:17:57 -07:00
Lioncash 061f393186 Syscalls/Signals: Add missing arguments to pidfd_send_signal syscall
These were accidentally omitted from the call.
2021-06-11 12:14:44 -04:00
Lioncash f1fd197096 Syscalls: Add helper for version testing
Shortens up the version testing code in a few places to make for quicker
reading.
2021-06-11 11:11:59 -04:00
Ryan Houdek dc664b1e2e Merge pull request #1072 from lioncash/fm
FileManager: Minor changes
2021-06-10 19:08:33 -07:00
Lioncash c971d3e56b FileManager: Construct fstream in place
While we're at it, we can make use of .data() to avoid a bounds check
that we know will never fail.
2021-06-10 21:56:08 -04:00
Ryan Houdek 0a03fbdfc6 Merge pull request #1071 from Sonicadvance1/fix_tmpfile_crash
Fixes a crash in 32-bit applications with tmpfile
2021-06-10 18:49:12 -07:00
Lioncash 10ec1d5d15 FileManager: Make use of insert_or_assign
Avoids some default constructions that get assigned over immediately.
2021-06-10 21:38:39 -04:00
Lioncash 340919d0fb FileManager: Make use of heterogenous lookup
Allows lookups with const char* and other non-allocating string types to
be performed without constructing a std::string instance.

Reduces heap usage a little.
2021-06-10 21:04:52 -04:00
Ryan Houdek 999443b898 Fixes a crash in 32-bit applications with tmpfile
If a 32-bit application leaves a temporary file dangling from one of the EmulatedFiles
then glibc tries to clean up the FILE object on shutdown.
This results in a crash on 32-bit applications because our memory allocator will have already cleaned up
at that point

Use raw FDs in this instance which matches better with the syscall hooking expecting raw FDs anyway.
2021-06-10 17:16:45 -07:00
Ryan Houdek adc5e4d6b9 JitArm64: Remove warnings in JIT.cpp 2021-06-10 15:06:53 -07:00
Ryan Houdek 903704ad48 JitArm64: Remove warning in ALUOps 2021-06-10 15:06:53 -07:00
Ryan Houdek 1d2f6c3ba8 JitArm64: Remove warnings in VectorOps.cpp 2021-06-10 15:06:53 -07:00
Ryan Houdek a07c57c52c SignalDelegator: Remove warning 2021-06-10 14:58:09 -07:00
Ryan Houdek 9118285b60 Syscalls: Remove warnings 2021-06-10 14:57:58 -07:00
Ryan Houdek 64acc9ed3c x32 FD Syscalls: Remove warning 2021-06-10 14:57:42 -07:00
Ryan Houdek d10fd4ddd4 Config: Remove warnings 2021-06-10 14:57:14 -07:00
Ryan Houdek a67b0c151e IREmitter: Remove warning 2021-06-10 14:57:02 -07:00
Ryan Houdek ba18b5dca9 ELFSymbolDatabase: Remove warning 2021-06-10 14:56:51 -07:00
Ryan Houdek 0f1a41154d RAPass: Remove warning 2021-06-10 14:56:39 -07:00
Ryan Houdek 97dfe9b26e IRValidation: Remove warning 2021-06-10 14:56:29 -07:00
Ryan Houdek f94ce4c95b IRCompaction: Remove warning 2021-06-10 14:56:17 -07:00
Ryan Houdek 0e1892622c OpcodeDispatcher: Remove warning 2021-06-10 14:56:06 -07:00
Ryan Houdek 17977ab4ac LookupCache: Remove warning 2021-06-10 14:55:53 -07:00
Ryan Houdek 78e207f837 Jit64: Remove warnings in JIT.cpp 2021-06-10 14:55:39 -07:00
Ryan Houdek e244142665 Jit64: Remove warning in ALUOps.cpp 2021-06-10 14:55:25 -07:00
189 changed files with 11792 additions and 6326 deletions

No files matched your search

+3
View File
@@ -39,3 +39,6 @@
[submodule "External/drm-headers"]
path = External/drm-headers
url = https://github.com/FEX-Emu/drm-headers.git
[submodule "External/xxhash"]
path = External/xxhash
url = https://github.com/FEX-Emu/xxHash.git
+242 -34
View File
@@ -14,6 +14,8 @@ option(ENABLE_ASSERTIONS "Enables assertions in build" FALSE)
option(ENABLE_VISUAL_DEBUGGER "Enables the visual debugger for compiling" FALSE)
option(ENABLE_STRICT_WERROR "Enables stricter -Werror for CI" FALSE)
option(ENABLE_WERROR "Enables -Werror" FALSE)
option(ENABLE_STATIC_PIE "Enables static-pie build" FALSE)
option(ENABLE_JEMALLOC "Enables jemalloc allocator" TRUE)
set (X86_C_COMPILER "x86_64-linux-gnu-gcc" CACHE STRING "c compiler for compiling x86 guest libs")
set (X86_CXX_COMPILER "x86_64-linux-gnu-g++" CACHE STRING "c++ compiler for compiling x86 guest libs")
@@ -44,38 +46,6 @@ else()
set(CMAKE_INTERPROCEDURAL_OPTIMIZATION FALSE)
endif()
find_program(CCACHE_PROGRAM ccache)
if(CCACHE_PROGRAM)
message(STATUS "CCache enabled")
set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE "${CCACHE_PROGRAM}")
endif()
if (ENABLE_XRAY)
add_compile_options(-fxray-instrument)
link_libraries(-fxray-instrument)
endif()
if (ENABLE_LLD)
link_libraries(-fuse-ld=lld)
endif()
if (ENABLE_ASAN)
add_definitions(-DENABLE_ASAN=1)
add_compile_options(-fno-omit-frame-pointer -fsanitize=address -fsanitize-address-use-after-scope)
link_libraries(-fno-omit-frame-pointer -fsanitize=address -fsanitize-address-use-after-scope)
endif()
if (ENABLE_TSAN)
add_compile_options(-fno-omit-frame-pointer -fsanitize=thread)
link_libraries(-fno-omit-frame-pointer -fsanitize=thread)
endif()
set (CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} -fno-omit-frame-pointer")
set (CMAKE_LINKER_FLAGS_RELWITHDEBINFO "${CMAKE_LINKER_FLAGS_RELWITHDEBINFO} -fno-omit-frame-pointer")
set (CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -fomit-frame-pointer")
set (CMAKE_LINKER_FLAGS_RELEASE "${CMAKE_LINKER_FLAGS_RELEASE} -fomit-frame-pointer")
if (CMAKE_SYSTEM_PROCESSOR MATCHES "x86_64")
option(ENABLE_X86_HOST_DEBUG "Enables compiling on x86_64 host" FALSE)
if (NOT ENABLE_X86_HOST_DEBUG)
@@ -95,6 +65,174 @@ if (CMAKE_SYSTEM_PROCESSOR MATCHES "aarch64")
add_definitions(-D_M_ARM_64=1)
endif()
find_program(CCACHE_PROGRAM ccache)
if(CCACHE_PROGRAM)
message(STATUS "CCache enabled")
set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE "${CCACHE_PROGRAM}")
endif()
if (ENABLE_XRAY)
add_compile_options(-fxray-instrument)
link_libraries(-fxray-instrument)
endif()
set (PTHREAD_LIB pthread)
if (ENABLE_LLD)
set (LD_OVERRIDE "-fuse-ld=lld")
link_libraries(${LD_OVERRIDE})
endif()
if (ENABLE_STATIC_PIE)
if (_M_ARM_64 AND ENABLE_LLD)
message (FATAL_ERROR "Static linking does not currently work with AArch64+LLD. Use GNU ld for now.")
endif()
file(WRITE ${PROJECT_BINARY_DIR}/CMakeFiles/CMakeTmp/Determine_iplt.c
"int main(int argc, char* argv[])
{
return 0;
}")
# Compile the test application with our LD_OVERRIDE and static-pie options
try_compile(
COMPILE_RESULT
${PROJECT_BINARY_DIR}/CMakeFiles/CMakeTmp
${PROJECT_BINARY_DIR}/CMakeFiles/CMakeTmp/Determine_iplt.c
COMPILE_DEFINITIONS "-fPIE ${LD_OVERRIDE}"
LINK_LIBRARIES "-static-pie ${LD_OVERRIDE}"
COPY_FILE ${PROJECT_BINARY_DIR}/CMakeFiles/CMakeTmp/Determine_iplt
)
if (${COMPILE_RESULT})
# Read the symbols from the elf
execute_process(COMMAND
readelf -s ${PROJECT_BINARY_DIR}/CMakeFiles/CMakeTmp/Determine_iplt
OUTPUT_FILE ${PROJECT_BINARY_DIR}/CMakeFiles/CMakeTmp/plt_out.txt
OUTPUT_VARIABLE PLT_SYMBOLS)
# Pull out the __rela_iplt_{start,end} symbols if they exist
execute_process(COMMAND
"grep" "__rela_iplt" ${PROJECT_BINARY_DIR}/CMakeFiles/CMakeTmp/plt_out.txt
OUTPUT_VARIABLE PLT_SYMBOLS)
set (SYMBOLS_FINE TRUE)
set (HAS_IPLT -1)
# Check if we have any symbols in our grep output
# The symbols must either not exist at all OR the symbols are zero
if (PLT_SYMBOLS)
string(FIND ${PLT_SYMBOLS} "__rela_iplt_start" HAS_IPLT)
endif()
if (NOT HAS_IPLT EQUAL -1)
# We have some symbols from readelf. Let's parse the results to check if they are zero
# Format: '35: 0000000000000000 0 NOTYPE LOCAL HIDDEN UND __rela_iplt_start'
string(REPLACE "\n" ";" SYMBOL_LIST ${PLT_SYMBOLS})
foreach (SYMBOL ${SYMBOL_LIST})
# strip any leading and trailing whitespace
string (STRIP ${SYMBOL} SYMBOL)
# Convert string to a list
string(REPLACE " " ";" SYMBOL_VALUES ${SYMBOL}})
# Pull out the address argument
list(GET SYMBOL_VALUES 1 OFFSET)
# Check against integer zero
if (NOT ${OFFSET} EQUAL 0)
# Symbol wasn't zero, this now fails
set (SYMBOLS_FINE FALSE)
endif()
endforeach()
endif()
if (SYMBOLS_FINE)
# We can now exnable static-pie
set (STATIC_PIE_OPTIONS "-static-pie")
# Pthreads has an issue with exposing symbols
# We need to make some concessions to the pthread gods
if (ENABLE_LLD)
set (PTHREAD_LIB
-Wl,--undefined-glob=pthread_*
-Wl,--undefined=__cxa_finalize
-Wl,--undefined=_pthread_cleanup_push_defer
-Wl,--undefined=_pthread_cleanup_pop_restore
-Wl,--undefined=__pthread_cleanup_upto
pthread)
else()
set (PTHREAD_LIB
-Wl,--undefined=pthread_join
-Wl,--undefined=pthread_attr_getdetachstate
-Wl,--undefined=pthread_sigmask
-Wl,--undefined=pthread_mutex_lock
-Wl,--undefined=pthread_cond_init
-Wl,--undefined=pthread_attr_init
-Wl,--undefined=pthread_mutex_unlock
-Wl,--undefined=pthread_mutexattr_destroy
-Wl,--undefined=pthread_detach
-Wl,--undefined=pthread_mutex_init
-Wl,--undefined=pthread_getattr_np
-Wl,--undefined=pthread_cond_timedwait
-Wl,--undefined=pthread_attr_destroy
-Wl,--undefined=pthread_mutexattr_settype
-Wl,--undefined=pthread_rwlock_unlock
-Wl,--undefined=pthread_rwlock_wrlock
-Wl,--undefined=pthread_setspecific
-Wl,--undefined=pthread_create
-Wl,--undefined=pthread_cond_clockwait
-Wl,--undefined=pthread_key_create
-Wl,--undefined=pthread_rwlock_rdlock
-Wl,--undefined=pthread_setname_np
-Wl,--undefined=pthread_cond_signal
-Wl,--undefined=pthread_mutexattr_init
-Wl,--undefined=pthread_attr_setstack
-Wl,--undefined=pthread_self
-Wl,--undefined=pthread_getaffinity_np
-Wl,--undefined=pthread_cond_wait
-Wl,--undefined=pthread_mutex_trylock
-Wl,--undefined=pthread_cond_broadcast
-Wl,--undefined=pthread_cond_destroy
-Wl,--undefined=pthread_getspecific
-Wl,--undefined=pthread_key_delete
-Wl,--undefined=pthread_once
-Wl,--undefined=__cxa_finalize
-Wl,--undefined=_pthread_cleanup_push_defer
-Wl,--undefined=_pthread_cleanup_pop_restore
-Wl,--undefined=__pthread_cleanup_upto
pthread)
endif()
else()
message (FATAL_ERROR "Application has __rela_iplt_{start,end} symbols. Which means static-pie can't be enabled")
endif()
else()
message (FATAL_ERROR "Couldn't compile static-pie test. Static-pie can't be enabled!")
endif()
endif()
if (ENABLE_ASAN)
add_definitions(-DENABLE_ASAN=1)
add_compile_options(-fno-omit-frame-pointer -fsanitize=address -fsanitize-address-use-after-scope)
link_libraries(-fno-omit-frame-pointer -fsanitize=address -fsanitize-address-use-after-scope)
endif()
if (ENABLE_TSAN)
add_compile_options(-fno-omit-frame-pointer -fsanitize=thread)
link_libraries(-fno-omit-frame-pointer -fsanitize=thread)
endif()
if (ENABLE_JEMALLOC)
add_definitions(-DENABLE_JEMALLOC=1)
else()
message (STATUS
" jemalloc disabled!\n"
" This is not a recommended configuration!\n"
" This will very explicitly break 32-bit application execution!\n"
" Use at your own risk!")
endif()
set (CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} -fno-omit-frame-pointer")
set (CMAKE_LINKER_FLAGS_RELWITHDEBINFO "${CMAKE_LINKER_FLAGS_RELWITHDEBINFO} -fno-omit-frame-pointer")
set (CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -fomit-frame-pointer")
set (CMAKE_LINKER_FLAGS_RELEASE "${CMAKE_LINKER_FLAGS_RELEASE} -fomit-frame-pointer")
add_subdirectory(External/vixl/)
include_directories(External/vixl/src/)
@@ -105,7 +243,13 @@ endif()
find_package(PkgConfig REQUIRED)
find_package(Python 3.0 REQUIRED COMPONENTS Interpreter)
pkg_check_modules(XXHASH libxxhash REQUIRED)
pkg_check_modules(XXHASH libxxhash>=0.8.0 QUIET)
if (NOT XXHASH_FOUND)
message(STATUS "xxHash not found. Using Externals")
add_subdirectory(External/xxhash/)
include_directories(External/xxhash/)
endif()
add_definitions(-Wno-trigraphs)
add_definitions(-DGLOBAL_DATA_DIRECTORY="${DATA_DIRECTORY}/")
@@ -252,7 +396,7 @@ add_compile_options(-Wall)
configure_file(
${CMAKE_CURRENT_SOURCE_DIR}/include/Config.h.in
${CMAKE_BINARY_DIR}/generated/Config.h)
${CMAKE_BINARY_DIR}/generated/ConfigDefines.h)
if (BUILD_TESTS)
include(CTest)
@@ -261,6 +405,9 @@ if (BUILD_TESTS)
endif()
add_subdirectory(External/FEXCore)
# Binfmt_misc files must be installed prior to Source/ installs
add_subdirectory(Data/binfmts/)
add_subdirectory(Source/)
add_subdirectory(Data/AppConfig/)
@@ -307,3 +454,64 @@ if (BUILD_THUNKS)
DEPENDS guest-libs
)
endif()
set(FEX_VERSION_MAJOR "0")
set(FEX_VERSION_MINOR "0")
set(FEX_VERSION_PATCH "0")
find_package(Git)
if (GIT_FOUND)
execute_process(
COMMAND ${GIT_EXECUTABLE} describe --abbrev=0
WORKING_DIRECTORY "${CMAKE_SOURCE_DIR}"
OUTPUT_VARIABLE GIT_DESCRIBE_STRING
RESULT_VARIABLE GIT_ERROR
ERROR_QUIET
OUTPUT_STRIP_TRAILING_WHITESPACE
)
if (NOT ${GIT_ERROR} EQUAL 0)
# Likely built in a way that doesn't have tags
# Setup a version tag that is unknown
set(GIT_DESCRIBE_STRING "FEX-0000")
endif()
# Change something like `FEX-2106.1-76-<hash>` in to a list
string(REPLACE "-" ";" DESCRIBE_LIST ${GIT_DESCRIBE_STRING})
# Extract the `2106.1` element
list(GET DESCRIBE_LIST 1 DESCRIBE_LIST)
# Change `2106.1` in to a list
string(REPLACE "." ";" DESCRIBE_LIST ${DESCRIBE_LIST})
# Calculate list size
list(LENGTH DESCRIBE_LIST LIST_SIZE)
# Pull out the major version
list(GET DESCRIBE_LIST 0 FEX_VERSION_MAJOR)
# Minor version only exists if there is a .1 at the end
# eg: 2106 versus 2106.1
if (LIST_SIZE GREATER 1)
list(GET DESCRIBE_LIST 1 FEX_VERSION_MINOR)
endif()
endif()
# Package creation
set (CPACK_GENERATOR "DEB")
set (CPACK_PACKAGE_NAME fex-emu)
set (CPACK_PACKAGE_CONTACT "team@fex-emu.org")
set (CPACK_PACKAGE_VERSION_MAJOR "${FEX_VERSION_MAJOR}")
set (CPACK_PACKAGE_VERSION_MINOR "${FEX_VERSION_MINOR}")
set (CPACK_PACKAGE_VERSION_PATCH "${FEX_VERSION_PATCH}")
# Debian defines
set (CPACK_DEBIAN_PACKAGE_DEPENDS "libstdc++6")
set (CPACK_DEBIAN_PACKAGE_CONTROL_EXTRA "${CMAKE_CURRENT_SOURCE_DIR}/CPack/postinst;${CMAKE_CURRENT_SOURCE_DIR}/CPack/prerm")
if (CMAKE_SYSTEM_PROCESSOR MATCHES "aarch64")
# binfmt_misc conflicts with qemu-user-static
# We also only install binfmt_misc on aarch64 hosts
set (CPACK_DEBIAN_PACKAGE_CONFLICTS "qemu-user-static")
endif()
include (CPack)
Executable
+18
View File
@@ -0,0 +1,18 @@
#!/bin/sh
set -e
update_binfmt() {
# Check for update-binfmts
command -v update-binfmts >/dev/null || return 0
# Setup binfmt_misc
update-binfmts --import FEX-x86
update-binfmts --import FEX-x86_64
}
# Install FEXInterpreter hardlink
# Needs to be done before setting up binfmt_misc
ln -f /usr/bin/FEXLoader /usr/bin/FEXInterpreter
if [ $(uname -m) = 'aarch64' ]; then
update_binfmt
fi
Executable
+17
View File
@@ -0,0 +1,17 @@
#!/bin/sh
set -e
update_binfmt() {
# Check for update-binfmts
command -v update-binfmts >/dev/null || return 0
# Uninstall
update-binfmts --unimport FEX-x86
update-binfmts --unimport FEX-x86_64
}
if [ $(uname -m) = 'aarch64' ]; then
update_binfmt
fi
# Remove FEXInterpreter hardlink
unlink /usr/bin/FEXInterpreter
+5
View File
@@ -0,0 +1,5 @@
{
"Config": {
"StallProcess": "1"
}
}
+17
View File
@@ -0,0 +1,17 @@
function(GenBinFmt Name)
# Get the filename only component
get_filename_component(FMT_NAME ${Name} NAME_WE)
# Configure it
configure_file(
${Name}
${CMAKE_BINARY_DIR}/Data/binfmts/${FMT_NAME})
# Then install the configured binfmt
install(
FILES ${CMAKE_BINARY_DIR}/Data/binfmts/${FMT_NAME}
DESTINATION ${CMAKE_INSTALL_PREFIX}/share/binfmts/)
endfunction()
GenBinFmt(FEX-x86.in)
GenBinFmt(FEX-x86_64.in)
+8
View File
@@ -0,0 +1,8 @@
package fex
interpreter @CMAKE_INSTALL_PREFIX@/bin/FEXInterpreter
magic \x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x03\x00
offset 0
mask \xff\xff\xff\xff\xff\xfe\xfe\x00\x00\x00\x00\xff\xff\xff\xff\xff\xfe\xff\xff\xff
credentials yes
fix_binary yes
preserve yes
+8
View File
@@ -0,0 +1,8 @@
package fex
interpreter @CMAKE_INSTALL_PREFIX@/bin/FEXInterpreter
magic \x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x3e\x00
offset 0
mask \xff\xff\xff\xff\xff\xfe\xfe\x00\x00\x00\x00\xff\xff\xff\xff\xff\xfe\xff\xff\xff
credentials yes
fix_binary yes
preserve yes
+2 -2
View File
@@ -3,7 +3,7 @@ FROM ubuntu:20.04 as builder
RUN DEBIAN_FRONTEND="noninteractive" apt-get update
RUN DEBIAN_FRONTEND="noninteractive" apt install -y cmake \
clang-10 llvm-10 nasm ninja-build libnuma-dev \
clang-10 llvm-10 nasm ninja-build \
libcap-dev libglfw3-dev libepoxy-dev python3-dev \
python3 linux-headers-generic
@@ -23,7 +23,7 @@ FROM ubuntu:20.04
RUN DEBIAN_FRONTEND="noninteractive" apt-get update
RUN DEBIAN_FRONTEND="noninteractive" apt install -y \
libnuma-dev libcap-dev libglfw3-dev libepoxy-dev
libcap-dev libglfw3-dev libepoxy-dev
COPY --from=builder /opt/FEX/build/Bin/* /usr/bin/
+13 -7
View File
@@ -98,14 +98,19 @@ def print_man_option(short, long, desc, default):
output_man.write("\\fBdefault:\\fR {0}\n".format(default))
output_man.write(".Pp\n\n")
def print_man_env_option(name, desc, default):
output_man.write("\\fBFEX_{0}\\fR\n".format(name))
def print_man_env_option(name, desc, default, no_json_key):
output_man.write("\\fBFEX_{0}\\fR\n".format(name.upper()))
# Print description
for line in desc:
output_man.write(".Pp\n")
output_man.write("{0}\n".format(line))
if (not no_json_key):
output_man.write(".Pp\n")
output_man.write("\\fBJSON key:\\fR '{0}'\n".format(name))
output_man.write(".Pp\n\n")
output_man.write(".Pp\n")
output_man.write("\\fBdefault:\\fR {0}\n".format(default))
output_man.write(".Pp\n\n")
@@ -154,9 +159,10 @@ def print_man_environment(options):
# Wrap the string argument in quotes
default = "'" + default + "'"
print_man_env_option(
op_key.upper(),
op_key,
op_vals["Desc"],
default
default,
False
)
print_man_environment_tail()
@@ -172,7 +178,7 @@ def print_man_environment_tail():
"By default FEX will look in {$HOME, $XDG_CONFIG_HOME}/.fex-emu/",
"This will override the full path",
],
"''")
"''", True)
print_man_env_option(
"FEX_APP_CONFIG",
@@ -183,7 +189,7 @@ def print_man_environment_tail():
"One must be careful with this option as it will override any applications that load with execve as well"
"If you need to support applications that execve then use FEX_APP_CONFIG_LOCATION instead"
],
"''")
"''", True)
print_man_env_option(
"FEX_APP_DATA_LOCATION",
@@ -193,7 +199,7 @@ def print_man_environment_tail():
"This will override the full path",
"This is the folder where FEX stores generated files like IR cache"
],
"''")
"''", True)
def print_man_header():
header ='''.Dd {0}
+9 -5
View File
@@ -81,6 +81,10 @@ set (SRCS
Interface/Core/Frontend.cpp
Interface/Core/GdbServer.cpp
Interface/Core/HostFeatures.cpp
Interface/Core/OpcodeDispatcher/Crypto.cpp
Interface/Core/OpcodeDispatcher/Flags.cpp
Interface/Core/OpcodeDispatcher/Vector.cpp
Interface/Core/OpcodeDispatcher/X87.cpp
Interface/Core/OpcodeDispatcher.cpp
Interface/Core/X86Tables.cpp
Interface/Core/X86DebugInfo.cpp
@@ -124,8 +128,6 @@ set (SRCS
Interface/IR/Passes/SyscallOptimization.cpp
Utils/Allocator.cpp
Utils/Allocator/64BitAllocator.cpp
Utils/ELFContainer.cpp
Utils/ELFSymbolDatabase.cpp
Utils/LogManager.cpp
Utils/Threads.cpp
)
@@ -135,7 +137,7 @@ if(_M_ARM_64)
Interface/Core/ArchHelpers/Arm64.cpp)
endif()
set(DEFINES )
set(DEFINES -DTHREAD_LOCAL=_Thread_local)
if (_M_X86_64)
list(APPEND DEFINES -D_M_X86_64=1)
@@ -267,7 +269,7 @@ function(AddObject Name Type)
add_dependencies(${Name} IR_INC)
add_dependencies(${Name} CONFIG_INC)
target_link_libraries(${Name} pthread vixl dl fmt::fmt xxhash FEX_jemalloc)
target_link_libraries(${Name} vixl dl fmt::fmt xxhash)
set_target_properties(${Name} PROPERTIES OUTPUT_NAME FEXCore)
set_target_properties(${Name} PROPERTIES C_VISIBILITY_PRESET hidden)
set_target_properties(${Name} PROPERTIES CXX_VISIBILITY_PRESET hidden)
@@ -286,6 +288,8 @@ function(AddObject Name Type)
target_compile_options(${Name}
PRIVATE
-Wall
-Werror=cast-qual
-Werror=ignored-qualifiers
-Werror=implicit-fallthrough
-Wno-trigraphs
@@ -306,7 +310,7 @@ endfunction()
function(AddLibrary Name Type)
add_library(${Name} ${Type} $<TARGET_OBJECTS:${PROJECT_NAME}_object>)
target_link_libraries(${Name} pthread vixl dl fmt::fmt xxhash FEX_jemalloc)
target_link_libraries(${Name} vixl dl fmt::fmt xxhash)
set_target_properties(${Name} PROPERTIES OUTPUT_NAME FEXCore)
set_target_properties(${Name} PROPERTIES C_VISIBILITY_PRESET hidden)
set_target_properties(${Name} PROPERTIES CXX_VISIBILITY_PRESET hidden)
+3 -2
View File
@@ -36,9 +36,10 @@ namespace FEXCore::Paths {
*CachePath += "/.fex-emu/";
*EntryCache = *CachePath + "/EntryCache/";
std::error_code ec{};
// Ensure the folder structure is created for our Data
if (!std::filesystem::exists(*EntryCache) &&
!std::filesystem::create_directories(*EntryCache)) {
if (!std::filesystem::exists(*EntryCache, ec) &&
!std::filesystem::create_directories(*EntryCache, ec)) {
LogMan::Msg::D("Couldn't create EntryCache directory: '%s'", EntryCache->c_str());
}
}
+13 -11
View File
@@ -1,4 +1,6 @@
#pragma once
#include <FEXCore/Utils/BitUtils.h>
#include <FEXCore/Utils/LogManager.h>
#include <cmath>
@@ -158,18 +160,18 @@ struct X80SoftFloat {
}
operator float() const {
float32_t Result = extF80_to_f32(*this);
return *(float*)&Result;
const float32_t Result = extF80_to_f32(*this);
return FEXCore::BitCast<float>(Result);
}
operator double() const {
float64_t Result = extF80_to_f64(*this);
return *(double*)&Result;
const float64_t Result = extF80_to_f64(*this);
return FEXCore::BitCast<double>(Result);
}
operator BIGFLOAT() const {
float128_t Result = extF80_to_f128(*this);
return *(BIGFLOAT*)&Result;
const float128_t Result = extF80_to_f128(*this);
return FEXCore::BitCast<BIGFLOAT>(Result);
}
operator int16_t() const {
@@ -196,11 +198,11 @@ struct X80SoftFloat {
}
void operator=(const float rhs) {
*this = f32_to_extF80(*(float32_t*)&rhs);
*this = f32_to_extF80(FEXCore::BitCast<float32_t>(rhs));
}
void operator=(const double rhs) {
*this = f64_to_extF80(*(float64_t*)&rhs);
*this = f64_to_extF80(FEXCore::BitCast<float64_t>(rhs));
}
void operator=(const int16_t rhs) {
@@ -226,15 +228,15 @@ struct X80SoftFloat {
}
X80SoftFloat(const float rhs) {
*this = f32_to_extF80(*(float32_t*)&rhs);
*this = f32_to_extF80(FEXCore::BitCast<float32_t>(rhs));
}
X80SoftFloat(const double rhs) {
*this = f64_to_extF80(*(float64_t*)&rhs);
*this = f64_to_extF80(FEXCore::BitCast<float64_t>(rhs));
}
X80SoftFloat(BIGFLOAT rhs) {
*this = f128_to_extF80(*(float128_t*)&rhs);
*this = f128_to_extF80(FEXCore::BitCast<float128_t>(rhs));
}
X80SoftFloat(const int16_t rhs) {
+36 -10
View File
@@ -59,9 +59,9 @@ namespace FEXCore::Config {
}
// Ensure the folder structure is created for our configuration
if (!std::filesystem::exists(ConfigDir) &&
!std::filesystem::create_directories(ConfigDir)) {
LogMan::Msg::D("Couldn't create config directory: '%s'", ConfigDir.c_str());
std::error_code ec{};
if (!std::filesystem::exists(ConfigDir, ec) &&
!std::filesystem::create_directories(ConfigDir, ec)) {
// Let's go local in this case
return "./";
}
@@ -85,15 +85,27 @@ namespace FEXCore::Config {
std::string GetApplicationConfig(const std::string &Filename, bool Global) {
std::string ConfigFile = GetConfigDirectory(Global);
std::error_code ec{};
if (!Global &&
!std::filesystem::exists(ConfigFile) &&
!std::filesystem::create_directories(ConfigFile)) {
!std::filesystem::exists(ConfigFile, ec) &&
!std::filesystem::create_directories(ConfigFile, ec)) {
LogMan::Msg::D("Couldn't create config directory: '%s'", ConfigFile.c_str());
// Let's go local in this case
return "./";
return "./" + Filename + ".json";
}
ConfigFile += "AppConfig/" + Filename + ".json";
ConfigFile += "AppConfig/";
// Attempt to create the local folder if it doesn't exist
if (!Global &&
!std::filesystem::exists(ConfigFile, ec) &&
!std::filesystem::create_directories(ConfigFile, ec)) {
// Let's go local in this case
return "./" + Filename + ".json";
}
ConfigFile += Filename + ".json";
return ConfigFile;
}
@@ -262,7 +274,8 @@ namespace FEXCore::Config {
Path = std::filesystem::absolute(Path);
// Only return if it exists
if (std::filesystem::exists(Path)) {
std::error_code ec{};
if (std::filesystem::exists(Path, ec)) {
return Path;
}
}
@@ -298,7 +311,8 @@ namespace FEXCore::Config {
else if (!PathName().empty()) {
// If the filesystem doesn't exist then let's see if it exists in the fex-emu folder
std::string NamedRootFS = GetDataDirectory() + "RootFS/" + PathName();
if (std::filesystem::exists(NamedRootFS)) {
std::error_code ec{};
if (std::filesystem::exists(NamedRootFS, ec)) {
FEXCore::Config::EraseSet(FEXCore::Config::CONFIG_ROOTFS, NamedRootFS);
}
}
@@ -313,7 +327,19 @@ namespace FEXCore::Config {
}
if (FEXCore::Config::Exists(FEXCore::Config::CONFIG_THUNKCONFIG)) {
FEX_CONFIG_OPT(PathName, THUNKCONFIG);
ExpandPathIfExists(FEXCore::Config::CONFIG_THUNKCONFIG, PathName());
auto ExpandedString = ExpandPath(PathName());
if (!ExpandedString.empty()) {
// Adjust the path if it ended up being relative
FEXCore::Config::EraseSet(FEXCore::Config::CONFIG_THUNKCONFIG, ExpandedString);
}
else if (!PathName().empty()) {
// If the filesystem doesn't exist then let's see if it exists in the fex-emu folder
std::string NamedConfig = GetDataDirectory() + "ThunkConfigs/" + PathName();
std::error_code ec{};
if (std::filesystem::exists(NamedConfig, ec)) {
FEXCore::Config::EraseSet(FEXCore::Config::CONFIG_THUNKCONFIG, NamedConfig);
}
}
}
if (FEXCore::Config::Exists(FEXCore::Config::CONFIG_OUTPUTLOG)) {
FEX_CONFIG_OPT(PathName, OUTPUTLOG);
+16 -1
View File
@@ -77,7 +77,14 @@
"Default": "",
"ShortArg": "k",
"Desc": [
"A json file specifying where to overlay the thunks."
"A json file specifying where to overlay the thunks.",
"This can be a filesystem path",
"\teg: ~/MyThunkConfig.json",
"Or this can be a named of a Thunk config file",
"If the named config file exists in the FEX data folder folder the it will use that one",
"\teg: $HOME/.fex-emu/ThunkConfigs/<ThunkConfig name>",
"Or if you have XDG_DATA_HOME the config will search in that directory",
"\teg: $XDG_DATA_HOME/.fex-emu/ThunkConfigs/<ThunkConfig name>"
]
},
"Env": {
@@ -207,6 +214,14 @@
"Makes TSO operations even more strict.",
"Forces vector loadstores to also become atomic."
]
},
"StallProcess": {
"Type": "bool",
"Default": "false",
"Desc": [
"Forces a process to stall out on initialization",
"Useful for a process that keeps restarting and doesn't work"
]
}
},
"Misc": {
+8 -8
View File
@@ -2,6 +2,7 @@
#include "Interface/Context/Context.h"
#include "Interface/Core/Core.h"
#include "Interface/Core/OpcodeDispatcher.h"
#include "Interface/Core/X86Tables/X86Tables.h"
#include <FEXCore/Config/Config.h>
#include <FEXCore/Core/CoreState.h>
@@ -37,12 +38,11 @@ namespace FEXCore::Context {
return CTX->InitCore(Loader);
}
void SetExitHandler(FEXCore::Context::Context *CTX,
std::function<void(uint64_t ThreadId, FEXCore::Context::ExitReason)> handler) {
CTX->CustomExitHandler = handler;
void SetExitHandler(FEXCore::Context::Context *CTX, ExitHandler handler) {
CTX->CustomExitHandler = std::move(handler);
}
std::function<void(uint64_t ThreadId, FEXCore::Context::ExitReason)> GetExitHandler(FEXCore::Context::Context *CTX) {
ExitHandler GetExitHandler(FEXCore::Context::Context *CTX) {
return CTX->CustomExitHandler;
}
@@ -105,12 +105,12 @@ namespace FEXCore::Context {
CTX->HandleCallback(RIP);
}
void RegisterHostSignalHandler(FEXCore::Context::Context *CTX, int Signal, HostSignalDelegatorFunction Func) {
CTX->RegisterHostSignalHandler(Signal, Func);
void RegisterHostSignalHandler(FEXCore::Context::Context *CTX, int Signal, HostSignalDelegatorFunction Func, bool Required) {
CTX->RegisterHostSignalHandler(Signal, Func, Required);
}
void RegisterFrontendHostSignalHandler(FEXCore::Context::Context *CTX, int Signal, HostSignalDelegatorFunction Func) {
CTX->RegisterFrontendHostSignalHandler(Signal, Func);
void RegisterFrontendHostSignalHandler(FEXCore::Context::Context *CTX, int Signal, HostSignalDelegatorFunction Func, bool Required) {
CTX->RegisterFrontendHostSignalHandler(Signal, Func, Required);
}
FEXCore::Core::InternalThreadState* CreateThread(FEXCore::Context::Context *CTX, FEXCore::Core::CPUState *NewThreadState, uint64_t ParentTID) {
+27 -6
View File
@@ -147,7 +147,7 @@ namespace FEXCore::Context {
std::unique_ptr<FEXCore::ThunkHandler> ThunkHandler;
CustomCPUFactoryType CustomCPUFactory;
std::function<void(uint64_t ThreadId, FEXCore::Context::ExitReason)> CustomExitHandler;
FEXCore::Context::ExitHandler CustomExitHandler;
struct AOTIRCacheEntry {
AOTIRInlineIndex *Array;
@@ -200,8 +200,8 @@ namespace FEXCore::Context {
void StartGdbServer();
void StopGdbServer();
void HandleCallback(uint64_t RIP);
void RegisterHostSignalHandler(int Signal, HostSignalDelegatorFunction Func);
void RegisterFrontendHostSignalHandler(int Signal, HostSignalDelegatorFunction Func);
void RegisterHostSignalHandler(int Signal, HostSignalDelegatorFunction Func, bool Required);
void RegisterFrontendHostSignalHandler(int Signal, HostSignalDelegatorFunction Func, bool Required);
static void RemoveCodeEntry(FEXCore::Core::InternalThreadState *Thread, uint64_t GuestRIP);
@@ -217,9 +217,28 @@ namespace FEXCore::Context {
bool GetDebugDataForRIP(uint64_t RIP, FEXCore::Core::DebugData *Data);
bool FindHostCodeForRIP(uint64_t RIP, uint8_t **Code);
std::tuple<FEXCore::IR::IRListView *, FEXCore::IR::RegisterAllocationData *, uint64_t, uint64_t, uint64_t, uint64_t> GenerateIR(FEXCore::Core::InternalThreadState *Thread, uint64_t GuestRIP);
struct GenerateIRResult {
FEXCore::IR::IRListView* IRList;
// User's responsibility to deallocate this.
FEXCore::IR::RegisterAllocationData* RAData;
uint64_t TotalInstructions;
uint64_t TotalInstructionsLength;
uint64_t StartAddr;
uint64_t Length;
};
[[nodiscard]] GenerateIRResult GenerateIR(FEXCore::Core::InternalThreadState *Thread, uint64_t GuestRIP);
std::tuple<void *, FEXCore::IR::IRListView *, FEXCore::Core::DebugData *, FEXCore::IR::RegisterAllocationData *, bool, uint64_t, uint64_t> CompileCode(FEXCore::Core::InternalThreadState *Thread, uint64_t GuestRIP);
struct CompileCodeResult {
void* CompiledCode;
FEXCore::IR::IRListView* IRData;
FEXCore::Core::DebugData* DebugData;
// User's responsibility to deallocate this.
FEXCore::IR::RegisterAllocationData* RAData;
bool GeneratedIR;
uint64_t StartAddr;
uint64_t Length;
};
[[nodiscard]] CompileCodeResult CompileCode(FEXCore::Core::InternalThreadState *Thread, uint64_t GuestRIP);
uintptr_t CompileBlock(FEXCore::Core::CpuStateFrame *Frame, uint64_t GuestRIP);
// same as CompileBlock, but aborts on failure
@@ -240,7 +259,9 @@ namespace FEXCore::Context {
void DestroyThread(FEXCore::Core::InternalThreadState *Thread);
void CleanupAfterFork(FEXCore::Core::InternalThreadState *ExceptForThread);
std::vector<FEXCore::Core::InternalThreadState*> *const GetThreads() { return &Threads; }
std::vector<FEXCore::Core::InternalThreadState*>* GetThreads() { return &Threads; }
uint8_t GetGPRSize() const { return Config.Is64BitMode ? 8 : 4; }
void AddNamedRegion(uintptr_t Base, uintptr_t Size, uintptr_t Offset, const std::string &filename);
void RemoveNamedRegion(uintptr_t Base, uintptr_t Size);
File diff suppressed because it is too large. Load diff
@@ -18,6 +18,31 @@ namespace FEXCore::ArchHelpers::Arm64 {
constexpr uint32_t STLXP_MASK = 0xBF'E0'80'00;
constexpr uint32_t STLXP_INST = 0x88'20'80'00;
constexpr uint32_t LDAXR_MASK = 0x3F'FF'FC'00;
constexpr uint32_t LDAXR_INST = 0x08'5F'FC'00;
constexpr uint32_t STLXR_MASK = 0x3F'E0'FC'00;
constexpr uint32_t STLXR_INST = 0x08'00'FC'00;
constexpr uint32_t CBNZ_MASK = 0x7F'00'00'00;
constexpr uint32_t CBNZ_INST = 0x35'00'00'00;
constexpr uint32_t ALU_OP_MASK = 0x7F'00'00'00;
constexpr uint32_t ADD_INST = 0x0B'00'00'00;
constexpr uint32_t SUB_INST = 0x4B'00'00'00;
constexpr uint32_t AND_INST = 0x0A'00'00'00;
constexpr uint32_t OR_INST = 0x2A'00'00'00;
constexpr uint32_t EOR_INST = 0x4A'00'00'00;
enum ExclusiveAtomicPairType {
TYPE_SWAP,
TYPE_ADD,
TYPE_SUB,
TYPE_AND,
TYPE_OR,
TYPE_EOR,
TYPE_NEG, // This is just a sub with zero. Need to know the differences
};
// Load ops are 4 bits
// Acquire and release bits are independent on the instruction
constexpr uint32_t ATOMIC_ADD_OP = 0b0000;
@@ -30,6 +55,27 @@ namespace FEXCore::ArchHelpers::Arm64 {
constexpr uint32_t ATOMIC_UMIN_OP = 0b0111;
constexpr uint32_t ATOMIC_SWAP_OP = 0b1000;
constexpr uint32_t REGISTER_MASK = 0b11111;
constexpr uint32_t RD_OFFSET = 0;
constexpr uint32_t RN_OFFSET = 5;
constexpr uint32_t RM_OFFSET = 16;
inline uint32_t GetRdReg(uint32_t Instr) {
return (Instr >> RD_OFFSET) & REGISTER_MASK;
}
inline uint32_t GetRnReg(uint32_t Instr) {
return (Instr >> RN_OFFSET) & REGISTER_MASK;
}
inline uint32_t GetRmReg(uint32_t Instr) {
return (Instr >> RM_OFFSET) & REGISTER_MASK;
}
bool HandleAtomicLoad(void *_ucontext, void *_info, uint32_t Instr);
bool HandleAtomicStore(void *_ucontext, void *_info, uint32_t Instr);
bool HandleAtomicLoad128(void *_ucontext, void *_info, uint32_t Instr);
uint64_t HandleAtomicLoadstoreExclusive(void *_ucontext, void *_info);
bool HandleCASPAL(void *_ucontext, void *_info, uint32_t Instr);
bool HandleCASAL(void *_ucontext, void *_info, uint32_t Instr);
bool HandleAtomicMemOp(void *_ucontext, void *_info, uint32_t Instr);
+443 -164
View File
@@ -15,7 +15,26 @@ $end_info$
#endif
namespace FEXCore {
//#define CPUID_AMD
constexpr uint32_t SUPPORTS_AVX = 0;
// #define CPUID_AMD
#ifdef CPUID_AMD
constexpr uint32_t FAMILY_IDENTIFIER =
0 | // Stepping
(0xA << 4) | // Model
(0xF << 8) | // Family ID
(0 << 12) | // Processor type
(0 << 16) | // Extended model ID
(1 << 20); // Extended family ID
#else
constexpr uint32_t FAMILY_IDENTIFIER =
0 | // Stepping
(0x7 << 4) | // Model
(0x6 << 8) | // Family ID
(0 << 12) | // Processor type
(1 << 16) | // Extended model ID
(0x0 << 20); // Extended family ID
#endif
#ifdef _M_ARM_64
static uint32_t GetCycleCounterFrequency() {
uint64_t Result{};
@@ -38,7 +57,7 @@ static uint32_t GetCycleCounterFrequency() {
}
#endif
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_0h() {
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_0h(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
// EBX, EDX, ECX become the manufacturer id string
@@ -57,29 +76,27 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_0h() {
}
// Processor Info and Features bits
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_01h() {
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_01h(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
uint32_t CoreCount = Cores();
Res.eax = FAMILY_IDENTIFIER;
Res.eax = 0 | // Stepping
(0 << 4) | // Model
(0xF << 8) | // Family ID
(0 << 12) | // Processor type
(0 << 16) | // Extended model ID
(0 << 20); // Extended family ID
Res.ebx = 0 | // Brand index
(8 << 8) | // Cache line size in bytes
(8 << 16) | // Number of addressable IDs for the logical cores in the physical CPU
(CoreCount << 16) | // Number of addressable IDs for the logical cores in the physical CPU
(0 << 24); // Local APIC ID
Res.ecx =
(1 << 0) | // SSE3
(0 << 1) | // PCLMULQDQ
(1 << 2) | // DS area supports 64bit layout
(1 << 3) | // MWait
(1 << 4) | // DS-CPL
(0 << 4) | // DS-CPL
(0 << 5) | // VMX
(0 << 6) | // SMX
(0 << 7) | // Intel SpeedStep
(0 << 8) | // Thermal Monitor 2
(1 << 8) | // Thermal Monitor 2
(1 << 9) | // SSSE3
(0 << 10) | // L1 context ID
(0 << 11) | // Silicon debug
@@ -89,7 +106,7 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_01h() {
(0 << 15) | // Perfmon and debug capability
(0 << 16) | // Reserved
(0 << 17) | // Process-context identifiers
(1 << 18) | // Prefetching from memory mapped device
(0 << 18) | // Prefetching from memory mapped device
(1 << 19) | // SSE4.1
(0 << 20) | // SSE4.2
(0 << 21) | // X2APIC
@@ -99,7 +116,7 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_01h() {
(CTX->HostFeatures.SupportsAES << 25) | // AES
(0 << 26) | // XSAVE
(0 << 27) | // OSXSAVE
(0 << 28) | // AVX
(SUPPORTS_AVX << 28) | // AVX
(0 << 29) | // F16C
(0 << 30) | // RDRAND
(0 << 31); // Hypervisor always returns zero
@@ -132,16 +149,16 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_01h() {
(1 << 24) | // FXSAVE/FXRSTOR
(1 << 25) | // SSE
(1 << 26) | // SSE2
(1 << 27) | // Self Snoop
(0 << 27) | // Self Snoop
(1 << 28) | // Max APIC IDs reserved field is valid
(1 << 29) | // Thermal monitor
(0 << 30) | // Reserved
(1 << 31); // Pending break enable
(0 << 31); // Pending break enable
return Res;
}
// 2: Cache and TLB information
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_02h() {
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_02h(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
// returns default values from i7 model 1Ah
@@ -165,124 +182,286 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_02h() {
return Res;
}
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_06h() {
// 4: Deterministic cache parameters for each level
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_04h(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
constexpr uint32_t CacheType_Data = 1;
constexpr uint32_t CacheType_Instruction = 2;
constexpr uint32_t CacheType_Unified = 3;
if (Leaf == 0) {
// Report L1D
uint32_t CoreCount = Cores() - 1;
Res.eax = CacheType_Data | // Cache type
(0b001 << 5) | // Cache level
(1 << 8) | // Self initializing cache level
(0 << 9) | // Fully associative
(0 << 14) | // Maximum number of addressable IDs for logical processors sharing this cache (With SMT this would be 1)
(CoreCount << 26); // Maximum number of addressable IDs for processor cores in the physical package
Res.ebx =
(63 << 0) | // Line Size - 1 : Claiming 64 byte
(0 << 12) | // Physical Line partitions
(7 << 22); // Associativity - 1 : Claiming 8 way
// 32KB
Res.ecx = 63; // Number of sets - 1 : Claiming 64 sets
Res.edx =
(0 << 0) | // Write-back invalidate
(0 << 1) | // Cache inclusiveness - Includes lower caches
(0 << 2); // Complex cache indexing - 0: Direct, 1: Complex
}
else if (Leaf == 1) {
// Report L1I
uint32_t CoreCount = Cores() - 1;
Res.eax = CacheType_Instruction | // Cache type
(0b001 << 5) | // Cache level
(1 << 8) | // Self initializing cache level
(0 << 9) | // Fully associative
(0 << 14) | // Maximum number of addressable IDs for logical processors sharing this cache (With SMT this would be 1)
(CoreCount << 26); // Maximum number of addressable IDs for processor cores in the physical package
Res.ebx =
(63 << 0) | // Line Size - 1 : Claiming 64 byte
(0 << 12) | // Physical Line partitions
(7 << 22); // Associativity - 1 : Claiming 8 way
// 32KB
Res.ecx = 63; // Number of sets - 1 : Claiming 64 sets
Res.edx =
(0 << 0) | // Write-back invalidate
(0 << 1) | // Cache inclusiveness - Includes lower caches
(0 << 2); // Complex cache indexing - 0: Direct, 1: Complex
}
else if (Leaf == 2) {
// Report L2
uint32_t CoreCount = Cores() - 1;
Res.eax = CacheType_Unified | // Cache type
(0b010 << 5) | // Cache level
(1 << 8) | // Self initializing cache level
(0 << 9) | // Fully associative
(0 << 14) | // Maximum number of addressable IDs for logical processors sharing this cache
(CoreCount << 26); // Maximum number of addressable IDs for processor cores in the physical package
Res.ebx =
(63 << 0) | // Line Size - 1 : Claiming 64 byte
(0 << 12) | // Physical Line partitions
(7 << 22); // Associativity - 1 : Claiming 8 way
// 512KB
Res.ecx = 0x3FF; // Number of sets - 1 : Claiming 1024 sets
Res.edx =
(0 << 0) | // Write-back invalidate
(0 << 1) | // Cache inclusiveness - Includes lower caches
(0 << 2); // Complex cache indexing - 0: Direct, 1: Complex
}
else if (Leaf == 3) {
// Report L3
uint32_t CoreCount = Cores() - 1;
Res.eax = CacheType_Unified | // Cache type
(0b011 << 5) | // Cache level
(1 << 8) | // Self initializing cache level
(0 << 9) | // Fully associative
(CoreCount << 14) | // Maximum number of addressable IDs for logical processors sharing this cache
(CoreCount << 26); // Maximum number of addressable IDs for processor cores in the physical package
Res.ebx =
(63 << 0) | // Line Size - 1 : Claiming 64 byte
(0 << 12) | // Physical Line partitions
(7 << 22); // Associativity - 1 : Claiming 8 way
// 8MB
Res.ecx = 0x4000; // Number of sets - 1 : Claiming 16384 sets
Res.edx =
(0 << 0) | // Write-back invalidate
(0 << 1) | // Cache inclusiveness - Includes lower caches
(1 << 2); // Complex cache indexing - 0: Direct, 1: Complex
}
return Res;
}
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_06h(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
Res.eax = (1 << 2); // Always running APIC
Res.ecx = (0 << 3); // Intel performance energy bias preference (EPB)
return Res;
}
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_07h() {
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_07h(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
if (Leaf == 0) {
// Number of subfunctions
Res.eax = 0x0;
Res.ebx =
(1 << 0) | // FS/GS support
(0 << 1) | // TSC adjust MSR
(0 << 2) | // SGX
(0 << 3) | // BMI1
(0 << 4) | // Intel Hardware Lock Elison
(0 << 5) | // AVX2 support
(1 << 6) | // FPU data pointer updated only on exception
(1 << 7) | // SMEP support
(0 << 8) | // BMI2
(0 << 9) | // Enhanced REP MOVSB/STOSB
(1 << 10) | // INVPCID for system software control of process-context
(0 << 11) | // Restricted transactional memory
(0 << 12) | // Intel resource directory technology Monitoring
(1 << 13) | // Deprecates FPU CS and DS
(0 << 14) | // Intel MPX
(0 << 15) | // Intel Resource Directory Technology Allocation
(0 << 16) | // Reserved
(0 << 17) | // Reserved
(0 << 18) | // RDSEED
(0 << 19) | // ADCX and ADOX instructions
(0 << 20) | // SMAP Supervisor mode access prevention and CLAC/STAC instructions
(0 << 21) | // Reserved
(0 << 22) | // Reserved
(0 << 23) | // CLFLUSHOPT instruction
(0 << 24) | // CLWB instruction
(0 << 25) | // Intel processor trace
(0 << 26) | // Reserved
(0 << 27) | // Reserved
(0 << 28) | // Reserved
(0 << 29) | // SHA instructions
(0 << 30) | // Reserved
(0 << 31); // Reserved
// Number of subfunctions
Res.eax = 0x0;
Res.ebx =
(1 << 0) | // FS/GS support
(0 << 1) | // TSC adjust MSR
(0 << 2) | // SGX
(0 << 3) | // BMI1
(0 << 4) | // Intel Hardware Lock Elison
(0 << 5) | // AVX2 support
(1 << 6) | // FPU data pointer updated only on exception
(1 << 7) | // SMEP support
(0 << 8) | // BMI2
(0 << 9) | // Enhanced REP MOVSB/STOSB
(1 << 10) | // INVPCID for system software control of process-context
(0 << 11) | // Restricted transactional memory
(0 << 12) | // Intel resource directory technology Monitoring
(1 << 13) | // Deprecates FPU CS and DS
(0 << 14) | // Intel MPX
(0 << 15) | // Intel Resource Directory Technology Allocation
(0 << 16) | // Reserved
(0 << 17) | // Reserved
(0 << 18) | // RDSEED
(0 << 19) | // ADCX and ADOX instructions
(0 << 20) | // SMAP Supervisor mode access prevention and CLAC/STAC instructions
(0 << 21) | // Reserved
(0 << 22) | // Reserved
(0 << 23) | // CLFLUSHOPT instruction
(0 << 24) | // CLWB instruction
(0 << 25) | // Intel processor trace
(0 << 26) | // Reserved
(0 << 27) | // Reserved
(0 << 28) | // Reserved
(0 << 29) | // SHA instructions
(0 << 30) | // Reserved
(0 << 31); // Reserved
Res.ecx =
(1 << 0) | // PREFETCHWT1
(0 << 1) | // AVX512VBMI
(0 << 2) | // Usermode instruction prevention
(0 << 3) | // Protection keys for user mode pages
(0 << 4) | // OS protection keys
(0 << 5) | // waitpkg
(0 << 6) | // AVX512_VBMI2
(0 << 7) | // CET shadow stack
(0 << 8) | // GFNI
(0 << 9) | // VAES
(0 << 10) | // VPCLMULQDQ
(0 << 11) | // AVX512_VNNI
(0 << 12) | // AVX512_BITALG
(0 << 13) | // Intel Total Memory Encryption
(0 << 14) | // AVX512_VPOPCNTDQ
(0 << 15) | // Reserved
(0 << 16) | // 5 Level page tables
(0 << 17) | // MPX MAWAU
(0 << 18) | // MPX MAWAU
(0 << 19) | // MPX MAWAU
(0 << 20) | // MPX MAWAU
(0 << 21) | // MPX MAWAU
(0 << 22) | // RDPID Read Processor ID
(0 << 23) | // Reserved
(0 << 24) | // Reserved
(0 << 25) | // CLDEMOTE
(0 << 26) | // Reserved
(0 << 27) | // MOVDIRI
(0 << 28) | // MOVDIR64B
(0 << 29) | // Reserved
(0 << 30) | // SGX Launch configuration
(0 << 31); // Reserved
Res.ecx =
(1 << 0) | // PREFETCHWT1
(0 << 1) | // AVX512VBMI
(0 << 2) | // Usermode instruction prevention
(0 << 3) | // Protection keys for user mode pages
(1 << 4) | // OS protection keys
(0 << 5) | // waitpkg
(0 << 6) | // AVX512_VBMI2
(0 << 7) | // CET shadow stack
(0 << 8) | // GFNI
(0 << 9) | // VAES
(0 << 10) | // VPCLMULQDQ
(0 << 11) | // AVX512_VNNI
(0 << 12) | // AVX512_BITALG
(0 << 13) | // Intel Total Memory Encryption
(0 << 14) | // AVX512_VPOPCNTDQ
(0 << 15) | // Reserved
(0 << 16) | // 5 Level page tables
(0 << 17) | // MPX MAWAU
(0 << 18) | // MPX MAWAU
(0 << 19) | // MPX MAWAU
(0 << 20) | // MPX MAWAU
(0 << 21) | // MPX MAWAU
(0 << 22) | // RDPID Read Processor ID
(0 << 23) | // Reserved
(0 << 24) | // Reserved
(0 << 25) | // CLDEMOTE
(0 << 26) | // Reserved
(0 << 27) | // MOVDIRI
(0 << 28) | // MOVDIR64B
(0 << 29) | // Reserved
(0 << 30) | // SGX Launch configuration
(0 << 31); // Reserved
Res.edx =
(0 << 0) | // Reserved
(0 << 1) | // Reserved
(0 << 2) | // AVX512_4VNNIW
(0 << 3) | // AVX512_4FMAPS
(0 << 4) | // Fast Short Rep Mov
(0 << 5) | // Reserved
(0 << 6) | // Reserved
(0 << 7) | // Reserved
(0 << 8) | // AVX512_VP2INTERSECT
(0 << 9) | // Reserved
(0 << 10) | // VERW clears CPU buffers
(0 << 11) | // Reserved
(0 << 12) | // Reserved
(0 << 13) | // Reserved
(0 << 14) | // SERIALIZE instruction
(0 << 15) | // Reserved
(0 << 16) | // Reserved
(0 << 17) | // Reserved
(0 << 18) | // Intel PCONFIG
(0 << 19) | // Intel Architectural LBR
(0 << 20) | // Intel CET
(0 << 21) | // Reserved
(0 << 22) | // Reserved
(0 << 23) | // Reserved
(0 << 24) | // Reserved
(0 << 25) | // Reserved
(0 << 26) | // Reserved
(0 << 27) | // Reserved
(0 << 28) | // L1D Flush
(0 << 29) | // Arch capabilities
(0 << 30) | // Reserved
(0 << 31); // Reserved
Res.edx =
(0 << 0) | // Reserved
(0 << 1) | // Reserved
(0 << 2) | // AVX512_4VNNIW
(0 << 3) | // AVX512_4FMAPS
(0 << 4) | // Fast Short Rep Mov
(0 << 5) | // Reserved
(0 << 6) | // Reserved
(0 << 7) | // Reserved
(0 << 8) | // AVX512_VP2INTERSECT
(0 << 9) | // Reserved
(0 << 10) | // VERW clears CPU buffers
(0 << 11) | // Reserved
(0 << 12) | // Reserved
(0 << 13) | // Reserved
(0 << 14) | // SERIALIZE instruction
(0 << 15) | // Reserved
(0 << 16) | // Reserved
(0 << 17) | // Reserved
(0 << 18) | // Intel PCONFIG
(0 << 19) | // Intel Architectural LBR
(0 << 20) | // Intel CET
(0 << 21) | // Reserved
(0 << 22) | // Reserved
(0 << 23) | // Reserved
(0 << 24) | // Reserved
(0 << 25) | // Reserved
(0 << 26) | // Reserved
(0 << 27) | // Reserved
(0 << 28) | // L1D Flush
(0 << 29) | // Arch capabilities
(0 << 30) | // Reserved
(0 << 31); // Reserved
}
return Res;
}
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_15h() {
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_0Dh(uint32_t Leaf) {
// Leaf 0
FEXCore::CPUID::FunctionResults Res{};
uint32_t XFeatureSupportedSizeMax = SUPPORTS_AVX ? 0x0000'0340 : 0x0000'0240; // XFeatureEnabledSizeMax: Legacy Header + FPU/SSE + AVX
if (Leaf == 0) {
// XFeatureSupportedMask[31:0]
Res.eax =
(1 << 0) | // X87 support
(1 << 1) | // 128-bit SSE support
(SUPPORTS_AVX << 2) | // 256-bit AVX support
(0b00 << 3) | // MPX State
(0b000 << 5) | // AVX-512 state
(0 << 8) | // "Used for IA32_XSS" ... Used for what?
(0 << 9); // PKRU state
// EBX and ECX doesn't need to match if a feature is supported but not enabled
Res.ebx = XFeatureSupportedSizeMax;
Res.ecx = XFeatureSupportedSizeMax; // XFeatureSupportedSizeMax: Size in bytes of XSAVE/XRSTOR area
// XFeatureSupportedMask[63:32]
Res.edx = 0; // Upper 32-bits of XFeatureSupportedMask
}
else if (Leaf == 1) {
Res.eax =
(0 << 0) | // XSAVEOPT
(0 << 1) | // XSAVEC (and XRSTOR)
(0 << 2) | // XGETBV - XGETBV with ECX=1 supported
(0 << 3); // XSAVES - XSAVES, XRSTORS, and IA32_XSS supported
// Same information as Leaf 0 for ebx
Res.ebx = XFeatureSupportedSizeMax;
// Lower supported 32bits of IA32_XSS MSR. IA32_XSS[n] can only be set to 1 if ECX[n] is 1
Res.ecx =
(0b0000'0000 << 0) | // Used for XCR0
(0 << 8) | // PT state
(0 << 9); // Used for XCR0
// Upper supported 32bits of IA32_XSS MSR. IA32_XSS[n+32] can only be set to 1 if EDX[n] is 1
// Entirely reserved atm
Res.edx = 0;
}
else if (Leaf == 2) {
Res.eax = SUPPORTS_AVX ? 0x0000'0100 : 0; // YmmSaveStateSize
Res.ebx = SUPPORTS_AVX ? 0x0000'0240 : 0; // YmmSaveStateOffset
// Reserved
Res.ecx = 0;
Res.edx = 0;
}
return Res;
}
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_15h(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
// TSC frequency = ECX * EBX / EAX
uint32_t FrequencyHz = GetCycleCounterFrequency();
@@ -295,7 +474,7 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_15h() {
}
// Highest extended function implemented
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0000h() {
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0000h(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
Res.eax = 0x8000001F;
@@ -314,15 +493,10 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0000h() {
}
// Extended processor and feature bits
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0001h() {
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0001h(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
Res.eax = 0 | // Stepping
(0 << 4) | // Model
(0 << 8) | // Family ID
(0 << 12) | // Processor type
(0 << 16) | // Extended model ID
(0 << 20); // Extended family ID
Res.eax = FAMILY_IDENTIFIER;
Res.ecx =
(1 << 0) | // LAHF/SAHF
@@ -347,13 +521,13 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0001h() {
(0 << 19) | // Reserved
(0 << 20) | // Reserved
(0 << 21) | // Reserved
(1 << 22) | // Topology extensions support
(1 << 23) | // Core performance counter extensions
(1 << 24) | // NB performance counter extensions
(0 << 22) | // Topology extensions support
(0 << 23) | // Core performance counter extensions
(0 << 24) | // NB performance counter extensions
(0 << 25) | // Reserved
(0 << 26) | // Data breakpoints extensions
(1 << 27) | // Performance TSC
(1 << 28) | // L2 perf counter extensions
(0 << 27) | // Performance TSC
(0 << 28) | // L2 perf counter extensions
(0 << 29) | // Reserved
(0 << 30) | // Reserved
(0 << 31); // Reserved
@@ -385,7 +559,7 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0001h() {
(1 << 23) | // MMX
(1 << 24) | // FXSAVE/FXRSTOR
(1 << 25) | // FXSAVE/FXRSTOR Optimizations
(1 << 26) | // 1 gigabit pages
(0 << 26) | // 1 gigabit pages
(0 << 27) | // RDTSCP
(0 << 28) | // Reserved
(1 << 29) | // Long Mode
@@ -400,26 +574,26 @@ constexpr char ProcessorBrand[48] = {
};
//Processor brand string
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0002h() {
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0002h(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
memcpy(&Res, &ProcessorBrand[0], sizeof(FEXCore::CPUID::FunctionResults));
return Res;
}
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0003h() {
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0003h(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
memcpy(&Res, &ProcessorBrand[16], sizeof(FEXCore::CPUID::FunctionResults));
return Res;
}
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0004h() {
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0004h(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
memcpy(&Res, &ProcessorBrand[32], sizeof(FEXCore::CPUID::FunctionResults));
return Res;
}
// L1 Cache and TLB identifiers
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0005h() {
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0005h(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
// L1 TLB Information for 2MB and 4MB pages
@@ -454,7 +628,7 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0005h() {
}
// L2 Cache identifiers
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0006h() {
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0006h(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
// L2 TLB Information for 2MB and 4MB pages
@@ -488,7 +662,7 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0006h() {
}
// Advanced power management
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0007h() {
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0007h(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
Res.eax = (1 << 2); // APIC timer not affected by p-state
Res.edx =
@@ -497,7 +671,7 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0007h() {
}
// Virtual and physical address sizes
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0008h() {
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0008h(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
Res.eax =
(48 << 0) | // PhysAddrSize = 48-bit
@@ -512,14 +686,14 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0008h() {
uint32_t CoreCount = Cores() - 1;
Res.ecx =
(0 << 16) | // PerfTscSize: Performance timestamp count size
(0 << 12) | // ApicIdSize: Number of bits in ApicID
((uint32_t)std::log2(CoreCount + 1) << 12) | // ApicIdSize: Number of bits in ApicID
(CoreCount << 0); // Count count subtract one
return Res;
}
// TLB 1GB page identifiers
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0019h() {
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0019h(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
Res.eax =
(0xF << 28) | // L1 DTLB associativity for 1GB pages
@@ -535,27 +709,128 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0019h() {
return Res;
}
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_Reserved() {
// Deterministic cache parameters for each level
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_001Dh(uint32_t Leaf) {
// This is nearly a copy of CPUID function 4h
// There are some minor changes though
FEXCore::CPUID::FunctionResults Res{};
constexpr uint32_t CacheType_Data = 1;
constexpr uint32_t CacheType_Instruction = 2;
constexpr uint32_t CacheType_Unified = 3;
if (Leaf == 0) {
// Report L1D
Res.eax = CacheType_Data | // Cache type
(0b001 << 5) | // Cache level
(1 << 8) | // Self initializing cache level
(0 << 9) | // Fully associative
(0 << 14); // Maximum number of addressable IDs for logical processors sharing this cache (With SMT this would be 1)
Res.ebx =
(63 << 0) | // Line Size - 1 : Claiming 64 byte
(0 << 12) | // Physical Line partitions
(7 << 22); // Associativity - 1 : Claiming 8 way
// 32KB
Res.ecx = 63; // Number of sets - 1 : Claiming 64 sets
Res.edx =
(0 << 0) | // Write-back invalidate
(0 << 1); // Cache inclusiveness - Includes lower caches
}
else if (Leaf == 1) {
// Report L1I
Res.eax = CacheType_Instruction | // Cache type
(0b001 << 5) | // Cache level
(1 << 8) | // Self initializing cache level
(0 << 9) | // Fully associative
(0 << 14); // Maximum number of addressable IDs for logical processors sharing this cache (With SMT this would be 1)
Res.ebx =
(63 << 0) | // Line Size - 1 : Claiming 64 byte
(0 << 12) | // Physical Line partitions
(7 << 22); // Associativity - 1 : Claiming 8 way
// 32KB
Res.ecx = 63; // Number of sets - 1 : Claiming 64 sets
Res.edx =
(0 << 0) | // Write-back invalidate
(0 << 1); // Cache inclusiveness - Includes lower caches
}
else if (Leaf == 2) {
// Report L2
Res.eax = CacheType_Unified | // Cache type
(0b010 << 5) | // Cache level
(1 << 8) | // Self initializing cache level
(0 << 9) | // Fully associative
(0 << 14); // Maximum number of addressable IDs for logical processors sharing this cache
Res.ebx =
(63 << 0) | // Line Size - 1 : Claiming 64 byte
(0 << 12) | // Physical Line partitions
(7 << 22); // Associativity - 1 : Claiming 8 way
// 512KB
Res.ecx = 0x3FF; // Number of sets - 1 : Claiming 1024 sets
Res.edx =
(0 << 0) | // Write-back invalidate
(0 << 1); // Cache inclusiveness - Includes lower caches
}
else if (Leaf == 3) {
// Report L3
uint32_t CoreCount = Cores() - 1;
Res.eax = CacheType_Unified | // Cache type
(0b011 << 5) | // Cache level
(1 << 8) | // Self initializing cache level
(0 << 9) | // Fully associative
(CoreCount << 14); // Maximum number of addressable IDs for logical processors sharing this cache
Res.ebx =
(63 << 0) | // Line Size - 1 : Claiming 64 byte
(0 << 12) | // Physical Line partitions
(7 << 22); // Associativity - 1 : Claiming 8 way
// 8MB
Res.ecx = 0x4000; // Number of sets - 1 : Claiming 16384 sets
Res.edx =
(0 << 0) | // Write-back invalidate
(0 << 1); // Cache inclusiveness - Includes lower caches
}
return Res;
}
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_Reserved(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
return Res;
}
void CPUIDEmu::Init(FEXCore::Context::Context *ctx) {
CTX = ctx;
RegisterFunction(0, std::bind(&CPUIDEmu::Function_0h, this));
RegisterFunction(1, std::bind(&CPUIDEmu::Function_01h, this));
RegisterFunction(2, std::bind(&CPUIDEmu::Function_02h, this));
using namespace std::placeholders;
RegisterFunction(0, std::bind(&CPUIDEmu::Function_0h, this, _1));
RegisterFunction(1, std::bind(&CPUIDEmu::Function_01h, this, _1));
RegisterFunction(2, std::bind(&CPUIDEmu::Function_02h, this, _1));
// 3: Serial Number(previously), now reserved
// 4: Deterministic cache parameters for each level
#ifndef CPUID_AMD
// Deterministic cache parameters for each level
RegisterFunction(0x4, std::bind(&CPUIDEmu::Function_04h, this, _1));
#endif
// 5: Monitor/mwait
// Thermal and power management
RegisterFunction(6, std::bind(&CPUIDEmu::Function_06h, this));
RegisterFunction(6, std::bind(&CPUIDEmu::Function_06h, this, _1));
// Extended feature flags
RegisterFunction(7, std::bind(&CPUIDEmu::Function_07h, this));
RegisterFunction(7, std::bind(&CPUIDEmu::Function_07h, this, _1));
// 9: Direct Cache Access information
// 0x0A: Architectural performance monitoring
// 0x0B: Extended topology enumeration
// 0x0D: Processor extended state enumeration
RegisterFunction(0x0D, std::bind(&CPUIDEmu::Function_0Dh, this, _1));
// 0x0F: Intel RDT monitoring
// 0x10: Intel RDT allocation enumeration
// 0x12: Intel SGX capability enumeration
@@ -564,43 +839,47 @@ void CPUIDEmu::Init(FEXCore::Context::Context *ctx) {
#ifndef CPUID_AMD
// Timestamp counter information
// Doesn't exist on AMD hardware
RegisterFunction(0x15, std::bind(&CPUIDEmu::Function_15h, this));
RegisterFunction(0x15, std::bind(&CPUIDEmu::Function_15h, this, _1));
#endif
// 0x16: Processor frequency information
// 0x17: SoC vendor attribute enumeration
// Largest extended function number
RegisterFunction(0x8000'0000, std::bind(&CPUIDEmu::Function_8000_0000h, this));
RegisterFunction(0x8000'0000, std::bind(&CPUIDEmu::Function_8000_0000h, this, _1));
// Processor vendor
RegisterFunction(0x8000'0001, std::bind(&CPUIDEmu::Function_8000_0001h, this));
RegisterFunction(0x8000'0001, std::bind(&CPUIDEmu::Function_8000_0001h, this, _1));
// Processor brand string
RegisterFunction(0x8000'0002, std::bind(&CPUIDEmu::Function_8000_0002h, this));
RegisterFunction(0x8000'0002, std::bind(&CPUIDEmu::Function_8000_0002h, this, _1));
// Processor brand string continued
RegisterFunction(0x8000'0003, std::bind(&CPUIDEmu::Function_8000_0003h, this));
RegisterFunction(0x8000'0003, std::bind(&CPUIDEmu::Function_8000_0003h, this, _1));
// Processor brand string continued
RegisterFunction(0x8000'0004, std::bind(&CPUIDEmu::Function_8000_0004h, this));
RegisterFunction(0x8000'0004, std::bind(&CPUIDEmu::Function_8000_0004h, this, _1));
// 0x8000'0005: L1 Cache and TLB identifiers
#ifdef CPUID_AMD
RegisterFunction(0x8000'0005, std::bind(&CPUIDEmu::Function_8000_0005h, this));
RegisterFunction(0x8000'0005, std::bind(&CPUIDEmu::Function_8000_0005h, this, _1));
#else
// This is full reserved on Intel platforms
RegisterFunction(0x8000'0005, std::bind(&CPUIDEmu::Function_Reserved, this));
RegisterFunction(0x8000'0005, std::bind(&CPUIDEmu::Function_Reserved, this, _1));
#endif
// 0x8000'0006: L2 Cache identifiers
RegisterFunction(0x8000'0006, std::bind(&CPUIDEmu::Function_8000_0006h, this));
RegisterFunction(0x8000'0006, std::bind(&CPUIDEmu::Function_8000_0006h, this, _1));
// Advanced power management information
RegisterFunction(0x8000'0007, std::bind(&CPUIDEmu::Function_8000_0007h, this));
RegisterFunction(0x8000'0007, std::bind(&CPUIDEmu::Function_8000_0007h, this, _1));
// Virtual and physical address sizes
RegisterFunction(0x8000'0008, std::bind(&CPUIDEmu::Function_8000_0008h, this));
RegisterFunction(0x8000'0008, std::bind(&CPUIDEmu::Function_8000_0008h, this, _1));
// 0x8000'000A: SVM Revision
// TLB 1GB page identifiers
RegisterFunction(0x8000'0019, std::bind(&CPUIDEmu::Function_8000_0019h, this));
RegisterFunction(0x8000'0019, std::bind(&CPUIDEmu::Function_8000_0019h, this, _1));
// 0x8000'001A: Performance optimization identifiers
// 0x8000'001B: Instruction based sampling identifiers
// 0x8000'001C: Lightweight profiling capabilities
// 0x8000'001D: Cache properties
#ifdef CPUID_AMD
// Deterministic cache parameters for each level
RegisterFunction(0x8000'001D, std::bind(&CPUIDEmu::Function_8000_001Dh, this, _1));
#endif
// 0x8000'001E: Extended APIC ID
// 0x8000'001F: AMD Secure Encryption
}
+25 -25
View File
@@ -24,23 +24,20 @@ private:
public:
void Init(FEXCore::Context::Context *ctx);
FEXCore::CPUID::FunctionResults RunFunction(uint32_t Function, [[maybe_unused]] uint32_t Leaf) {
FEXCore::CPUID::FunctionResults RunFunction(uint32_t Function, uint32_t Leaf) {
auto Handler = FunctionHandlers.find(Function);
if (Handler == FunctionHandlers.end()) {
#ifndef NDEBUG
LogMan::Msg::E("Unhandled CPU ID function, 0x%x", Function);
#endif
return Function_Reserved();
return Function_Reserved(Leaf);
}
return Handler->second();
return Handler->second(Leaf);
}
private:
FEXCore::Context::Context *CTX;
FEX_CONFIG_OPT(Cores, THREADS);
using FunctionHandler = std::function<FEXCore::CPUID::FunctionResults()>;
using FunctionHandler = std::function<FEXCore::CPUID::FunctionResults(uint32_t Leaf)>;
void RegisterFunction(uint32_t Function, FunctionHandler Handler) {
FunctionHandlers[Function] = Handler;
}
@@ -48,23 +45,26 @@ private:
std::unordered_map<uint32_t, FunctionHandler> FunctionHandlers;
// Functions
FEXCore::CPUID::FunctionResults Function_0h();
FEXCore::CPUID::FunctionResults Function_01h();
FEXCore::CPUID::FunctionResults Function_02h();
FEXCore::CPUID::FunctionResults Function_06h();
FEXCore::CPUID::FunctionResults Function_07h();
FEXCore::CPUID::FunctionResults Function_15h();
FEXCore::CPUID::FunctionResults Function_8000_0000h();
FEXCore::CPUID::FunctionResults Function_8000_0001h();
FEXCore::CPUID::FunctionResults Function_8000_0002h();
FEXCore::CPUID::FunctionResults Function_8000_0003h();
FEXCore::CPUID::FunctionResults Function_8000_0004h();
FEXCore::CPUID::FunctionResults Function_8000_0005h();
FEXCore::CPUID::FunctionResults Function_8000_0006h();
FEXCore::CPUID::FunctionResults Function_8000_0007h();
FEXCore::CPUID::FunctionResults Function_8000_0008h();
FEXCore::CPUID::FunctionResults Function_8000_0009h();
FEXCore::CPUID::FunctionResults Function_8000_0019h();
FEXCore::CPUID::FunctionResults Function_Reserved();
FEXCore::CPUID::FunctionResults Function_0h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_01h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_02h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_04h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_06h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_07h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_0Dh(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_15h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_8000_0000h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_8000_0001h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_8000_0002h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_8000_0003h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_8000_0004h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_8000_0005h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_8000_0006h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_8000_0007h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_8000_0008h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_8000_0009h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_8000_0019h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_8000_001Dh(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_Reserved(uint32_t Leaf);
};
}
+58 -71
View File
@@ -33,7 +33,7 @@ $end_info$
#include "Interface/HLE/Thunks/Thunks.h"
#include "FEXCore/Utils/Allocator.h"
#include <xxh3.h>
#include <xxhash.h>
#include <fstream>
#include <unistd.h>
#include <filesystem>
@@ -110,39 +110,7 @@ constexpr std::array<std::string_view const, 16> RegNames = {
std::string_view const& GetGRegName(unsigned Reg) {
return RegNames[Reg];
}
namespace DefaultFallbackCore {
class DefaultFallbackCore final : public FEXCore::CPU::CPUBackend {
public:
explicit DefaultFallbackCore(FEXCore::Core::ThreadState *Thread)
: ThreadState {reinterpret_cast<FEXCore::Core::InternalThreadState*>(Thread)} {
}
~DefaultFallbackCore() override = default;
std::string GetName() override { return "Default Fallback"; }
void *MapRegion(void *HostPtr, uint64_t VirtualGuestPtr, uint64_t Size) override {
return HostPtr;
}
void Initialize() override {}
bool NeedsOpDispatch() override { return false; }
void *CompileCode(uint64_t Entry, FEXCore::IR::IRListView const *IR, FEXCore::Core::DebugData *DebugData, FEXCore::IR::RegisterAllocationData *RAData) override {
LogMan::Msg::E("Fell back to default code handler at RIP: 0x%lx", ThreadState->CurrentFrame->State.rip);
return nullptr;
}
private:
FEXCore::Core::InternalThreadState *ThreadState;
};
FEXCore::CPU::CPUBackend *CPUCreationFactory(FEXCore::Context::Context* CTX, FEXCore::Core::ThreadState *Thread) {
return new DefaultFallbackCore(Thread);
}
}
}
} // namespace FEXCore::Core
namespace FEXCore::Context {
void Context::AOTIRCaptureCacheWriteoutQueue_Flush() {
@@ -248,6 +216,7 @@ namespace FEXCore::Context {
NewThreadState.flags[1] = 1;
NewThreadState.flags[9] = 1;
NewThreadState.FCW = 0x37F;
NewThreadState.FTW = 0xFFFF;
FEXCore::Core::InternalThreadState *Thread = CreateThread(&NewThreadState, 0);
@@ -279,12 +248,12 @@ namespace FEXCore::Context {
Thread->CPUBackend->CallbackPtr(Thread->CurrentFrame, RIP);
}
void Context::RegisterHostSignalHandler(int Signal, HostSignalDelegatorFunction Func) {
SignalDelegation->RegisterHostSignalHandler(Signal, Func);
void Context::RegisterHostSignalHandler(int Signal, HostSignalDelegatorFunction Func, bool Required) {
SignalDelegation->RegisterHostSignalHandler(Signal, Func, Required);
}
void Context::RegisterFrontendHostSignalHandler(int Signal, HostSignalDelegatorFunction Func) {
SignalDelegation->RegisterFrontendHostSignalHandler(Signal, Func);
void Context::RegisterFrontendHostSignalHandler(int Signal, HostSignalDelegatorFunction Func, bool Required) {
SignalDelegation->RegisterFrontendHostSignalHandler(Signal, Func, Required);
}
void Context::WaitForIdle() {
@@ -406,8 +375,6 @@ namespace FEXCore::Context {
}
if (Thread->RunningEvents.Running.load()) {
StopThread(Thread);
} else {
LogMan::Msg::D("Skipping thread %p: Already stopped", Thread);
}
}
}
@@ -617,6 +584,15 @@ namespace FEXCore::Context {
// We now only have one thread
IdleWaitRefCount = 1;
// Clean up dead stacks
FEXCore::Threads::Thread::CleanupAfterFork();
if (LiveThread->CompileService) {
// If this live thread had a compile service then it no longer exists
// Erase the shared_ptr
LiveThread->CompileService.reset();
}
}
void Context::AddBlockMapping(FEXCore::Core::InternalThreadState *Thread, uint64_t Address, void *Ptr, uint64_t Start, uint64_t Length) {
@@ -635,7 +611,7 @@ namespace FEXCore::Context {
}
}
std::tuple<FEXCore::IR::IRListView *, FEXCore::IR::RegisterAllocationData *, uint64_t, uint64_t, uint64_t, uint64_t> Context::GenerateIR(FEXCore::Core::InternalThreadState *Thread, uint64_t GuestRIP) {
Context::GenerateIRResult Context::GenerateIR(FEXCore::Core::InternalThreadState *Thread, uint64_t GuestRIP) {
uint8_t const *GuestCode{};
GuestCode = reinterpret_cast<uint8_t const*>(GuestRIP);
@@ -645,14 +621,14 @@ namespace FEXCore::Context {
uint64_t TotalInstructionsLength {0};
if (!Thread->FrontendDecoder->DecodeInstructionsAtEntry(GuestCode, GuestRIP)) {
return { nullptr, nullptr, 0, 0, 0, 0 };
return {};
}
auto CodeBlocks = Thread->FrontendDecoder->GetDecodedBlocks();
Thread->OpDispatcher->BeginFunction(GuestRIP, CodeBlocks);
uint8_t GPRSize = Config.Is64BitMode ? 8 : 4;
const uint8_t GPRSize = GetGPRSize();
for (size_t j = 0; j < CodeBlocks->size(); ++j) {
FEXCore::Frontend::Decoder::DecodedBlocks const &Block = CodeBlocks->at(j);
@@ -732,7 +708,7 @@ namespace FEXCore::Context {
return { nullptr, nullptr, 0, 0, 0, 0 };
}
else {
uint8_t GPRSize = Config.Is64BitMode ? 8 : 4;
const uint8_t GPRSize = GetGPRSize();
// We had some instructions. Early exit
Thread->OpDispatcher->_ExitFunction(Thread->OpDispatcher->_EntrypointOffset(Block.Entry + BlockInstructionsLength - GuestRIP, GPRSize));
@@ -823,7 +799,14 @@ namespace FEXCore::Context {
Thread->OpDispatcher->ResetWorkingList();
return {IRList, RAData.release(), TotalInstructions, TotalInstructionsLength, Thread->FrontendDecoder->DecodedMinAddress, Thread->FrontendDecoder->DecodedMaxAddress - Thread->FrontendDecoder->DecodedMinAddress };
return {
.IRList = IRList,
.RAData = RAData.release(),
.TotalInstructions = TotalInstructions,
.TotalInstructionsLength = TotalInstructionsLength,
.StartAddr = Thread->FrontendDecoder->DecodedMinAddress,
.Length = Thread->FrontendDecoder->DecodedMaxAddress - Thread->FrontendDecoder->DecodedMinAddress,
};
}
AOTIRInlineEntry *AOTIRInlineIndex::GetInlineEntry(uint64_t DataOffset) {
@@ -866,16 +849,16 @@ namespace FEXCore::Context {
if (Inserted.second) {
//GuestHash
Stream->write((char*)&Hash, sizeof(Hash));
Stream->write((const char*)&Hash, sizeof(Hash));
//GuestLength
Stream->write((char*)&Length, sizeof(Length));
Stream->write((const char*)&Length, sizeof(Length));
// RAData (inline)
// In file, IsShared is always set
auto Shared = RAData->IsShared;
RAData->IsShared = true;
Stream->write((char*)RAData, RAData->Size(RAData->MapCount));
Stream->write((const char*)RAData, RAData->Size(RAData->MapCount));
RAData->IsShared = Shared;
// IRData (inline)
@@ -883,7 +866,7 @@ namespace FEXCore::Context {
}
}
std::tuple<void *, FEXCore::IR::IRListView *, FEXCore::Core::DebugData *, FEXCore::IR::RegisterAllocationData *, bool, uint64_t, uint64_t> Context::CompileCode(FEXCore::Core::InternalThreadState *Thread, uint64_t GuestRIP) {
Context::CompileCodeResult Context::CompileCode(FEXCore::Core::InternalThreadState *Thread, uint64_t GuestRIP) {
FEXCore::IR::IRListView *IRList {};
FEXCore::Core::DebugData *DebugData {};
FEXCore::IR::RegisterAllocationData *RAData {};
@@ -981,10 +964,18 @@ namespace FEXCore::Context {
}
if (IRList == nullptr) {
return { nullptr, nullptr, nullptr, nullptr, false, 0, 0 };
return {};
}
// Attempt to get the CPU backend to compile this code
return { Thread->CPUBackend->CompileCode(GuestRIP, IRList, DebugData, RAData), IRList, DebugData, RAData, GeneratedIR, StartAddr, Length};
return {
.CompiledCode = Thread->CPUBackend->CompileCode(GuestRIP, IRList, DebugData, RAData),
.IRData = IRList,
.DebugData = DebugData,
.RAData = RAData,
.GeneratedIR = GeneratedIR,
.StartAddr = StartAddr,
.Length = Length,
};
}
static bool readAll(int fd, void *data, size_t size) {
@@ -1057,41 +1048,41 @@ namespace FEXCore::Context {
std::unique_lock lk(AOTIRCacheLock);
for (auto &AOTModule: AOTIRCaptureCache) {
if (!AOTModule.second.Stream) {
for (auto& [String, Entry] : AOTIRCaptureCache) {
if (!Entry.Stream) {
continue;
}
auto ModSize = AOTModule.first.size();
auto &stream = AOTModule.second.Stream;
const auto ModSize = String.size();
auto &stream = Entry.Stream;
// pad to 32 bytes
char Zero = 0;
constexpr char Zero = 0;
while(stream->tellp() & 31)
stream->write(&Zero, 1);
// AOTIRInlineIndex
auto FnCount = AOTModule.second.Index.size();
size_t DataBase = -stream->tellp();
const auto FnCount = Entry.Index.size();
const size_t DataBase = -stream->tellp();
stream->write((char*)&FnCount, sizeof(FnCount));
stream->write((char*)&DataBase, sizeof(DataBase));
stream->write((const char*)&FnCount, sizeof(FnCount));
stream->write((const char*)&DataBase, sizeof(DataBase));
for (auto entry: AOTModule.second.Index) {
for (const auto& [GuestStart, DataOffset] : Entry.Index) {
//AOTIRInlineIndexEntry
// GuestStart
stream->write((char*)&entry.first, sizeof(entry.first));
stream->write((const char*)&GuestStart, sizeof(GuestStart));
// DataOffset
stream->write((char*)&entry.second, sizeof(entry.second));
stream->write((const char*)&DataOffset, sizeof(DataOffset));
}
// End of file header
auto IndexSize = FnCount * sizeof(AOTIRInlineIndexEntry) + sizeof(DataBase) + sizeof(FnCount);
stream->write((char*)&IndexSize, sizeof(IndexSize));
stream->write((char*)&AOTModule.first[0], ModSize);
stream->write((char*)&ModSize, sizeof(ModSize));
const auto IndexSize = FnCount * sizeof(AOTIRInlineIndexEntry) + sizeof(DataBase) + sizeof(FnCount);
stream->write((const char*)&IndexSize, sizeof(IndexSize));
stream->write(String.c_str(), ModSize);
stream->write((const char*)&ModSize, sizeof(ModSize));
}
}
@@ -1243,8 +1234,6 @@ namespace FEXCore::Context {
++IdleWaitRefCount;
LogMan::Msg::D("[%d] Waiting to run", Thread->ThreadManager.TID.load());
// Now notify the thread that we are initialized
Thread->ThreadWaiting.NotifyAll();
@@ -1253,8 +1242,6 @@ namespace FEXCore::Context {
Thread->StartRunning.Wait();
}
LogMan::Msg::D("[%d] Running", Thread->ThreadManager.TID.load());
Thread->ExitReason = FEXCore::Context::ExitReason::EXIT_NONE;
Thread->RunningEvents.Running = true;
@@ -1,3 +1,5 @@
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/ArchHelpers/MContext.h"
#include "Interface/Core/Dispatcher/Arm64Dispatcher.h"
@@ -54,6 +54,7 @@ void Dispatcher::StoreThreadState(int Signal, void *ucontext) {
}
void Dispatcher::RestoreThreadState(void *ucontext) {
LOGMAN_THROW_A(!SignalFrames.empty(), "Trying to restore a signal frame when we don't have any");
uint64_t OldSP = SignalFrames.top();
SignalFrames.pop();
uintptr_t NewSP = OldSP;
@@ -105,7 +106,12 @@ bool Dispatcher::HandleGuestSignal(int Signal, void *info, void *ucontext, Guest
// Don't need this offset if we aren't going to be putting siginfo in to it
NewGuestSP -= 128;
if (GuestAction->sa_flags & SA_SIGINFO) {
// siginfo_t
siginfo_t *HostSigInfo = reinterpret_cast<siginfo_t*>(info);
if (GuestAction->sa_flags & SA_SIGINFO &&
!(HostSigInfo->si_code == SI_QUEUE || // If the siginfo comes from sigqueue or user then we don't need to check
HostSigInfo->si_code == SI_USER)) {
if (SRAEnabled) {
if (!IsAddressInJITCode(ArchHelpers::Context::GetPc(ucontext), false)) {
LOGMAN_THROW_A(!IsAddressInJITCode(ArchHelpers::Context::GetPc(ucontext), true), "Signals in dispatcher have unsynchronized context");
@@ -117,7 +123,6 @@ bool Dispatcher::HandleGuestSignal(int Signal, void *info, void *ucontext, Guest
// Setup ucontext a bit
if (CTX->Config.Is64BitMode) {
NewGuestSP -= sizeof(FEXCore::x86_64::ucontext_t);
uint64_t UContextLocation = NewGuestSP;
@@ -133,6 +138,14 @@ bool Dispatcher::HandleGuestSignal(int Signal, void *info, void *ucontext, Guest
// Pointer to where the fpreg memory is
guest_uctx->uc_mcontext.fpregs = &guest_uctx->__fpregs_mem;
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_RIP] = Frame->State.rip;
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_EFL] = 0;
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_CSGSFS] = 0;
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_ERR] = 0;
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_TRAPNO] = Signal;
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_OLDMASK] = 0;
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_CR2] = 0;
#define COPY_REG(x) \
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_##x] = Frame->State.gregs[X86State::REG_##x];
COPY_REG(R8);
@@ -159,6 +172,7 @@ bool Dispatcher::HandleGuestSignal(int Signal, void *info, void *ucontext, Guest
// FCW store default
guest_uctx->__fpregs_mem.fcw = Frame->State.FCW;
guest_uctx->__fpregs_mem.ftw = Frame->State.FTW;
// Reconstruct FSW
guest_uctx->__fpregs_mem.fsw =
@@ -173,26 +187,10 @@ bool Dispatcher::HandleGuestSignal(int Signal, void *info, void *ucontext, Guest
guest_uctx->uc_stack.ss_sp = GuestStack->ss_sp;
guest_uctx->uc_stack.ss_size = GuestStack->ss_size;
// siginfo_t
siginfo_t *HostSigInfo = reinterpret_cast<siginfo_t*>(info);
if (HostSigInfo->si_code == SI_USER) {
// If the signal was a user signal then we need to pass this struct through unaltered
// Guest might be doing something with it
*guest_siginfo = *HostSigInfo;
}
else {
guest_siginfo->si_signo = Signal;
switch (Signal) {
case SIGSEGV:
case SIGBUS:
guest_siginfo->si_code = HostSigInfo->si_code;
guest_siginfo->si_errno = HostSigInfo->si_errno;
// Macro expansion to get the si_addr
guest_siginfo->si_addr = HostSigInfo->si_addr;
break;
default: LogMan::Msg::D("Unhandled siginfo_t signal: %d", Signal); break;
}
}
// aarch64 and x86_64 siginfo_t matches. We can just copy this over
// SI_USER could also potentially have random data in it, needs to be bit perfect
// For guest faults we don't have a real way to reconstruct state to a real guest RIP
*guest_siginfo = *HostSigInfo;
Frame->State.gregs[X86State::REG_RSI] = SigInfoLocation;
Frame->State.gregs[X86State::REG_RDX] = UContextLocation;
@@ -200,19 +198,110 @@ bool Dispatcher::HandleGuestSignal(int Signal, void *info, void *ucontext, Guest
else {
// XXX: 32bit Support
NewGuestSP -= sizeof(FEXCore::x86::ucontext_t);
uint64_t UContextLocation = 0; // NewGuestSP;
uint64_t UContextLocation = NewGuestSP;
NewGuestSP -= sizeof(FEXCore::x86::siginfo_t);
uint64_t SigInfoLocation = 0; // NewGuestSP;
uint64_t SigInfoLocation = NewGuestSP;
FEXCore::x86::ucontext_t *guest_uctx = reinterpret_cast<FEXCore::x86::ucontext_t*>(UContextLocation);
FEXCore::x86::siginfo_t *guest_siginfo = reinterpret_cast<FEXCore::x86::siginfo_t*>(SigInfoLocation);
// We have extended float information
guest_uctx->uc_flags |= FEXCore::x86::UC_FP_XSTATE;
// Pointer to where the fpreg memory is
guest_uctx->uc_mcontext.fpregs = static_cast<uint32_t>(reinterpret_cast<uint64_t>(&guest_uctx->__fpregs_mem));
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_GS] = Frame->State.gs;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_FS] = Frame->State.fs;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_ES] = Frame->State.es;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_DS] = Frame->State.ds;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_TRAPNO] = Signal;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_ERR] = 0;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_EIP] = Frame->State.rip;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_CS] = Frame->State.cs;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_EFL] = 0;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_UESP] = 0;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_SS] = Frame->State.ss;
#define COPY_REG(x) \
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_##x] = Frame->State.gregs[X86State::REG_##x];
COPY_REG(RDI);
COPY_REG(RSI);
COPY_REG(RBP);
COPY_REG(RBX);
COPY_REG(RDX);
COPY_REG(RAX);
COPY_REG(RCX);
COPY_REG(RSP);
#undef COPY_REG
// Copy float registers
memcpy(guest_uctx->__fpregs_mem._st, Frame->State.mm, sizeof(Frame->State.mm));
if (0) {
// XXX: Handle XMM
// memcpy(guest_uctx->__fpregs_mem._xmm, Frame->State.xmm, sizeof(Frame->State.xmm));
guest_uctx->__fpregs_mem.status = FEXCore::x86::fpstate_magic::MAGIC_XFPSTATE;
}
else {
guest_uctx->__fpregs_mem.status = FEXCore::x86::fpstate_magic::MAGIC_FPU;
}
// FCW store default
guest_uctx->__fpregs_mem.fcw = Frame->State.FCW;
guest_uctx->__fpregs_mem.ftw = Frame->State.FTW;
// Reconstruct FSW
guest_uctx->__fpregs_mem.fsw =
(Frame->State.flags[FEXCore::X86State::X87FLAG_TOP_LOC] << 11) |
(Frame->State.flags[FEXCore::X86State::X87FLAG_C0_LOC] << 8) |
(Frame->State.flags[FEXCore::X86State::X87FLAG_C1_LOC] << 9) |
(Frame->State.flags[FEXCore::X86State::X87FLAG_C2_LOC] << 10) |
(Frame->State.flags[FEXCore::X86State::X87FLAG_C3_LOC] << 14);
// Copy over signal stack information
guest_uctx->uc_stack.ss_flags = GuestStack->ss_flags;
guest_uctx->uc_stack.ss_sp = static_cast<uint32_t>(reinterpret_cast<uint64_t>(GuestStack->ss_sp));
guest_uctx->uc_stack.ss_size = GuestStack->ss_size;
// These three elements are in every siginfo
guest_siginfo->si_signo = HostSigInfo->si_signo;
guest_siginfo->si_errno = HostSigInfo->si_errno;
guest_siginfo->si_code = HostSigInfo->si_code;
switch (Signal) {
case SIGSEGV:
case SIGBUS:
// Macro expansion to get the si_addr
// Can't really give a real result here. Pull from the context for now
guest_siginfo->_sifields._sigfault.addr = Frame->State.rip;
break;
case SIGCHLD:
guest_siginfo->_sifields._sigchld.pid = HostSigInfo->si_pid;
guest_siginfo->_sifields._sigchld.uid = HostSigInfo->si_uid;
guest_siginfo->_sifields._sigchld.status = HostSigInfo->si_status;
guest_siginfo->_sifields._sigchld.utime = HostSigInfo->si_utime;
guest_siginfo->_sifields._sigchld.stime = HostSigInfo->si_stime;
break;
default:
// Hope for the best, most things just copy over
memcpy(&guest_siginfo->_sifields, &HostSigInfo->_sifields, sizeof(siginfo_t));
break;
}
NewGuestSP -= 4;
*(uint32_t*)NewGuestSP = UContextLocation;
NewGuestSP -= 4;
*(uint32_t*)NewGuestSP = SigInfoLocation;
NewGuestSP -= 4;
*(uint32_t*)NewGuestSP = Signal;
}
Frame->State.rip = reinterpret_cast<uint64_t>(GuestAction->sigaction_handler.sigaction);
}
else {
if (!CTX->Config.Is64BitMode) {
NewGuestSP -= 4;
*(uint32_t*)NewGuestSP = Signal;
}
Frame->State.rip = reinterpret_cast<uint64_t>(GuestAction->sigaction_handler.handler);
}
@@ -1,3 +1,5 @@
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/Dispatcher/X86Dispatcher.h"
#include "Interface/Core/Interpreter/InterpreterClass.h"
@@ -12,7 +14,9 @@ static constexpr size_t MAX_DISPATCHER_CODE_SIZE = 4096;
X86Dispatcher::X86Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::InternalThreadState *Thread, DispatcherConfig &config)
: Dispatcher(ctx, Thread)
, Xbyak::CodeGenerator(MAX_DISPATCHER_CODE_SIZE, nullptr, this) {
, Xbyak::CodeGenerator(MAX_DISPATCHER_CODE_SIZE,
FEXCore::Allocator::mmap(nullptr, MAX_DISPATCHER_CODE_SIZE, PROT_READ | PROT_WRITE | PROT_EXEC, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0),
nullptr) {
using namespace Xbyak;
using namespace Xbyak::util;
@@ -298,7 +302,7 @@ X86Dispatcher::X86Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::Inte
}
X86Dispatcher::~X86Dispatcher() {
FEXCore::Allocator::munmap(top_, MAX_DISPATCHER_CODE_SIZE);
}
#ifdef _M_X86_64
@@ -9,19 +9,11 @@
namespace FEXCore::CPU {
class X86Dispatcher final : public Dispatcher, public Xbyak::CodeGenerator, public Xbyak::Allocator {
class X86Dispatcher final : public Dispatcher, public Xbyak::CodeGenerator {
public:
X86Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::InternalThreadState *Thread, DispatcherConfig &config);
virtual ~X86Dispatcher() override;
// Xbyak::Allocator
Xbyak::uint8 *alloc(size_t size) override { Size = size; return reinterpret_cast<uint8_t*>(FEXCore::Allocator::mmap(nullptr, size, PROT_READ | PROT_WRITE | PROT_EXEC, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0)); }
void free(Xbyak::uint8 *p) override { FEXCore::Allocator::munmap(p, Size); }
bool useProtect() const override { return false; }
private:
size_t Size{};
};
}
+58 -24
View File
@@ -16,12 +16,17 @@ $end_info$
#include <FEXCore/Debug/X86Tables.h>
#include <FEXCore/Utils/LogManager.h>
#include <set>
#include <sys/mman.h>
namespace FEXCore::Frontend {
#include "Interface/Core/VSyscall/VSyscall.inc"
using namespace FEXCore::X86Tables;
static uint32_t MapModRMToReg(uint8_t REX, uint8_t bits, bool HighBits, bool HasREX, bool HasXMM, bool HasMM, uint8_t InvalidOffset = 16) {
constexpr std::array<uint64_t, 16> GPRIndexes = {
using GPRArray = std::array<uint32_t, 16>;
static constexpr GPRArray GPRIndexes = {
// Classical ordering?
FEXCore::X86State::REG_RAX,
FEXCore::X86State::REG_RCX,
@@ -41,7 +46,7 @@ static uint32_t MapModRMToReg(uint8_t REX, uint8_t bits, bool HighBits, bool Has
FEXCore::X86State::REG_R15,
};
constexpr std::array<uint64_t, 16> GPR8BitHighIndexes = {
static constexpr GPRArray GPR8BitHighIndexes = {
// Classical ordering?
FEXCore::X86State::REG_RAX,
FEXCore::X86State::REG_RCX,
@@ -61,7 +66,7 @@ static uint32_t MapModRMToReg(uint8_t REX, uint8_t bits, bool HighBits, bool Has
FEXCore::X86State::REG_R15,
};
constexpr std::array<uint64_t, 16> XMMIndexes = {
static constexpr GPRArray XMMIndexes = {
FEXCore::X86State::REG_XMM_0,
FEXCore::X86State::REG_XMM_1,
FEXCore::X86State::REG_XMM_2,
@@ -80,7 +85,7 @@ static uint32_t MapModRMToReg(uint8_t REX, uint8_t bits, bool HighBits, bool Has
FEXCore::X86State::REG_XMM_15,
};
constexpr std::array<uint64_t, 16> MMIndexes = {
static constexpr GPRArray MMIndexes = {
FEXCore::X86State::REG_MM_0,
FEXCore::X86State::REG_MM_1,
FEXCore::X86State::REG_MM_2,
@@ -99,7 +104,7 @@ static uint32_t MapModRMToReg(uint8_t REX, uint8_t bits, bool HighBits, bool Has
FEXCore::X86State::REG_INVALID
};
const std::array<uint64_t, 16> *GPRs = &GPRIndexes;
const GPRArray *GPRs = &GPRIndexes;
if (HasXMM) {
GPRs = &XMMIndexes;
}
@@ -119,8 +124,18 @@ static uint32_t MapModRMToReg(uint8_t REX, uint8_t bits, bool HighBits, bool Has
}
Decoder::Decoder(FEXCore::Context::Context *ctx)
: CTX {ctx} {
DecodedBuffer.resize(DefaultDecodedBufferSize);
: CTX {ctx}
, OSABI { ctx->SyscallHandler ? ctx->SyscallHandler->GetOSABI() : FEXCore::HLE::SyscallOSABI::OS_UNKNOWN } {
// Using mmap is a start-up time optimization
// Take advantage of page faulting to reduce startup time for minimal runtime cost
DecodedBuffer =
reinterpret_cast<FEXCore::X86Tables::DecodedInst *>(
FEXCore::Allocator::mmap(0, sizeof(FEXCore::X86Tables::DecodedInst) * DefaultDecodedBufferSize,
PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0));
}
Decoder::~Decoder() {
FEXCore::Allocator::munmap(DecodedBuffer, sizeof(FEXCore::X86Tables::DecodedInst) * DefaultDecodedBufferSize);
}
uint8_t Decoder::ReadByte() {
@@ -212,28 +227,28 @@ void Decoder::DecodeModRM_16(X86Tables::DecodedOperand *Operand, X86Tables::ModR
{FEXCore::X86State::REG_RBX, FEXCore::X86State::REG_RDI},
{FEXCore::X86State::REG_RBP, FEXCore::X86State::REG_RSI},
{FEXCore::X86State::REG_RBP, FEXCore::X86State::REG_RDI},
{FEXCore::X86State::REG_RSI, 255},
{FEXCore::X86State::REG_RDI, 255},
{255, 255},
{FEXCore::X86State::REG_RBX, 255},
{FEXCore::X86State::REG_RSI, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_RDI, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_INVALID, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_RBX, FEXCore::X86State::REG_INVALID},
// Mod = 0b01
{FEXCore::X86State::REG_RBX, FEXCore::X86State::REG_RSI},
{FEXCore::X86State::REG_RBX, FEXCore::X86State::REG_RDI},
{FEXCore::X86State::REG_RBP, FEXCore::X86State::REG_RSI},
{FEXCore::X86State::REG_RBP, FEXCore::X86State::REG_RDI},
{FEXCore::X86State::REG_RSI, 255},
{FEXCore::X86State::REG_RDI, 255},
{FEXCore::X86State::REG_RBP, 255},
{FEXCore::X86State::REG_RBX, 255},
{FEXCore::X86State::REG_RSI, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_RDI, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_RBP, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_RBX, FEXCore::X86State::REG_INVALID},
// Mod = 0b10
{FEXCore::X86State::REG_RBX, FEXCore::X86State::REG_RSI},
{FEXCore::X86State::REG_RBX, FEXCore::X86State::REG_RDI},
{FEXCore::X86State::REG_RBP, FEXCore::X86State::REG_RSI},
{FEXCore::X86State::REG_RBP, FEXCore::X86State::REG_RDI},
{FEXCore::X86State::REG_RSI, 255},
{FEXCore::X86State::REG_RDI, 255},
{FEXCore::X86State::REG_RBP, 255},
{FEXCore::X86State::REG_RBX, 255},
{FEXCore::X86State::REG_RSI, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_RDI, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_RBP, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_RBX, FEXCore::X86State::REG_INVALID},
}};
uint8_t LookupIndex = ModRM.mod << 3 | ModRM.rm;
@@ -927,7 +942,7 @@ bool Decoder::DecodeInstruction(uint64_t PC) {
}
if (DecodeInst->Dest.IsGPR()) {
assert(DecodeInst->Dest.Data.GPR.GPR != 255);
assert(DecodeInst->Dest.Data.GPR.GPR != FEXCore::X86State::REG_INVALID);
}
return true;
@@ -939,7 +954,7 @@ void Decoder::BranchTargetInMultiblockRange() {
// If the RIP setting is conditional AND within our symbol range then it can be considered for multiblock
uint64_t TargetRIP = 0;
uint8_t GPRSize = CTX->Config.Is64BitMode ? 8 : 4;
const uint8_t GPRSize = CTX->GetGPRSize();
bool Conditional = true;
switch (DecodeInst->OP) {
@@ -1002,6 +1017,25 @@ void Decoder::BranchTargetInMultiblockRange() {
}
}
const uint8_t *Decoder::AdjustAddrForSpecialRegion(uint8_t const* _InstStream, uint64_t EntryPoint, uint64_t RIP) {
constexpr uint64_t VSyscall_Base = 0xFFFF'FFFF'FF60'0000ULL;
constexpr uint64_t VSyscall_End = VSyscall_Base + 0x1000;
if (OSABI == FEXCore::HLE::SyscallOSABI::OS_LINUX64 &&
RIP >= VSyscall_Base &&
RIP < VSyscall_End) {
// VSyscall
// This doesn't exist on AArch64 and on x86_64 hosts this is emulated with faults to a region mapped with --xp permissions
// Offset 0: vgettimeofday
// Offset 0x400: vtime
// Offset 0x800: vgetcpu
uint64_t Offset = RIP - VSyscall_Base;
return VSyscallData + Offset;
}
return _InstStream - EntryPoint + RIP;
}
bool Decoder::DecodeInstructionsAtEntry(uint8_t const* _InstStream, uint64_t PC) {
Blocks.clear();
BlocksToDecode.clear();
@@ -1045,7 +1079,7 @@ bool Decoder::DecodeInstructionsAtEntry(uint8_t const* _InstStream, uint64_t PC)
uint64_t BlockStartOffset = DecodedSize;
// Do a bit of pointer math to figure out where we are in code
InstStream = _InstStream - EntryPoint + RIPToDecode;
InstStream = AdjustAddrForSpecialRegion(_InstStream, EntryPoint, RIPToDecode);
while (1) {
ErrorDuringDecoding = !DecodeInstruction(RIPToDecode + PCOffset);
@@ -1089,7 +1123,7 @@ bool Decoder::DecodeInstructionsAtEntry(uint8_t const* _InstStream, uint64_t PC)
}
if (DecodedSize >= CTX->Config.MaxInstPerBlock ||
DecodedSize >= DecodedBuffer.size()) {
DecodedSize >= DefaultDecodedBufferSize) {
break;
}
@@ -1106,7 +1140,7 @@ bool Decoder::DecodeInstructionsAtEntry(uint8_t const* _InstStream, uint64_t PC)
// Copy over only the number of instructions we decoded
CurrentBlockDecoding.NumInstructions = BlockNumberOfInstructions;
CurrentBlockDecoding.DecodedInstructions = &DecodedBuffer.at(BlockStartOffset);
CurrentBlockDecoding.DecodedInstructions = &DecodedBuffer[BlockStartOffset];
}
+7 -1
View File
@@ -1,6 +1,8 @@
#pragma once
#include <FEXCore/Debug/X86Tables.h>
#include <FEXCore/HLE/SyscallHandler.h>
#include <array>
#include <cstdint>
#include <utility>
@@ -24,6 +26,7 @@ public:
};
Decoder(FEXCore::Context::Context *ctx);
~Decoder();
bool DecodeInstructionsAtEntry(uint8_t const* InstStream, uint64_t PC);
std::vector<DecodedBlocks> const *GetDecodedBlocks() const {
@@ -37,6 +40,7 @@ public:
void SetExternalBranches(std::set<uint64_t> *v) { ExternalBranches = v; }
private:
FEXCore::Context::Context *CTX;
const FEXCore::HLE::SyscallOSABI OSABI{};
bool DecodeInstruction(uint64_t PC);
@@ -50,7 +54,7 @@ private:
bool NormalOpHeader(FEXCore::X86Tables::X86InstInfo const *Info, uint16_t Op);
static constexpr size_t DefaultDecodedBufferSize = 0x10000;
std::vector<FEXCore::X86Tables::DecodedInst> DecodedBuffer;
FEXCore::X86Tables::DecodedInst *DecodedBuffer{};
size_t DecodedSize {};
uint8_t const *InstStream;
@@ -83,5 +87,7 @@ private:
&FEXCore::Frontend::Decoder::DecodeModRM_64,
&FEXCore::Frontend::Decoder::DecodeModRM_16,
};
const uint8_t *AdjustAddrForSpecialRegion(uint8_t const* _InstStream, uint64_t EntryPoint, uint64_t RIP);
};
}
+4 -4
View File
@@ -47,11 +47,11 @@ void GdbServer::Break(int signal) {
}
GdbServer::GdbServer(FEXCore::Context::Context *ctx) : CTX(ctx) {
ctx->CustomExitHandler = [this](uint64_t ThreadId, FEXCore::Context::ExitReason ExitReason) {
Context::SetExitHandler(ctx, [this](uint64_t ThreadId, FEXCore::Context::ExitReason ExitReason) {
if (ExitReason == FEXCore::Context::ExitReason::EXIT_DEBUG) {
this->Break(SIGTRAP);
}
};
});
// This is a total hack as there is currently no way to resume once hitting a segfault
// But it's semi-useful for debugging.
@@ -64,7 +64,7 @@ GdbServer::GdbServer(FEXCore::Context::Context *ctx) : CTX(ctx) {
usleep(100000);
return true;
});
}, true);
StartThread();
}
@@ -939,7 +939,7 @@ void GdbServer::GdbServerLoop() {
{
std::lock_guard lk(sendMutex);
CommsStream.release();
CommsStream.reset();
}
}
}
@@ -1,6 +1,5 @@
#pragma once
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/InternalThreadState.h"
#include "Interface/Core/Dispatcher/Dispatcher.h"
@@ -78,6 +78,18 @@ bool InterpreterCore::HandleSIGBUS(int Signal, void *info, void *ucontext) {
return false;
}
}
else if ((Instr & FEXCore::ArchHelpers::Arm64::LDAXR_MASK) == FEXCore::ArchHelpers::Arm64::LDAXR_INST) { // LDAXR*
uint64_t BytesToSkip = FEXCore::ArchHelpers::Arm64::HandleAtomicLoadstoreExclusive(ucontext, info);
if (BytesToSkip) {
// Skip this instruction now
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) + BytesToSkip);
return true;
}
else {
LogMan::Msg::E("Unhandled JIT SIGBUS LDAXR: PC: %p Instruction: 0x%08x\n", PC, PC[0]);
return false;
}
}
}
return false;
}
@@ -93,12 +105,12 @@ InterpreterCore::InterpreterCore(FEXCore::Context::Context *ctx, FEXCore::Core::
CTX->SignalDelegation->RegisterHostSignalHandler(SignalDelegator::SIGNAL_FOR_PAUSE, [](FEXCore::Core::InternalThreadState *Thread, int Signal, void *info, void *ucontext) -> bool {
InterpreterCore *Core = reinterpret_cast<InterpreterCore*>(Thread->CPUBackend.get());
return Core->Dispatcher->HandleSignalPause(Signal, info, ucontext);
});
}, true);
CTX->SignalDelegation->RegisterHostSignalHandler(SIGBUS, [](FEXCore::Core::InternalThreadState *Thread, int Signal, void *info, void *ucontext) -> bool {
InterpreterCore *Core = reinterpret_cast<InterpreterCore*>(Thread->CPUBackend.get());
return Core->HandleSIGBUS(Signal, info, ucontext);
});
}, true);
auto GuestSignalHandler = [](FEXCore::Core::InternalThreadState *Thread, int Signal, void *info, void *ucontext, GuestSigAction *GuestAction, stack_t *GuestStack) -> bool {
InterpreterCore *Core = reinterpret_cast<InterpreterCore*>(Thread->CPUBackend.get());
File diff suppressed because it is too large. Load diff
+36 -37
View File
@@ -46,7 +46,7 @@ DEF_OP(TruncElementPair) {
mov(Dst.second, Src.second);
break;
}
default: LOGMAN_MSG_A("Unhandled Truncation size: %d", Op->Size); break;
default: LOGMAN_MSG_A_FMT("Unhandled Truncation size: {}", Op->Size); break;
}
}
@@ -95,7 +95,7 @@ DEF_OP(Add) {
case 8:
add(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()), Const);
break;
default: LOGMAN_MSG_A("Unsupported Add size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported Add size: {}", OpSize);
}
} else {
switch (OpSize) {
@@ -105,7 +105,7 @@ DEF_OP(Add) {
case 8:
add(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()), GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unsupported Add size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported Add size: {}", OpSize);
}
}
}
@@ -121,7 +121,7 @@ DEF_OP(Sub) {
case 8:
sub(GRS(Node), GRS(Op->Header.Args[0].ID()), Const);
break;
default: LOGMAN_MSG_A("Unsupported Sub size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported Sub size: {}", OpSize);
}
} else {
switch (OpSize) {
@@ -131,7 +131,7 @@ DEF_OP(Sub) {
case 8:
sub(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()), GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unsupported Sub size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported Sub size: {}", OpSize);
}
}
@@ -147,7 +147,7 @@ DEF_OP(Neg) {
case 8:
neg(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unsupported Not size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported Neg size: {}", OpSize);
}
}
@@ -163,7 +163,7 @@ DEF_OP(Mul) {
case 8:
mul(Dst, GetReg<RA_64>(Op->Header.Args[0].ID()), GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unknown Sext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown Mul size: %d", OpSize);
}
}
@@ -179,7 +179,7 @@ DEF_OP(UMul) {
case 8:
mul(Dst, GetReg<RA_64>(Op->Header.Args[0].ID()), GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unknown Sext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown UMul size: {}", OpSize);
}
}
@@ -216,7 +216,7 @@ DEF_OP(Div) {
sdiv(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()), GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown DIV Size: %d", Size); break;
default: LOGMAN_MSG_A_FMT("Unknown DIV Size: {}", Size); break;
}
}
@@ -243,7 +243,7 @@ DEF_OP(UDiv) {
udiv(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()), GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown UDIV Size: %d", Size); break;
default: LOGMAN_MSG_A_FMT("Unknown UDIV Size: {}", Size); break;
}
}
@@ -290,7 +290,7 @@ DEF_OP(Rem) {
msub(GetReg<RA_64>(Node), TMP1, Divisor, Dividend);
break;
}
default: LOGMAN_MSG_A("Unknown REM Size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown REM Size: {}", OpSize); break;
}
}
@@ -332,7 +332,7 @@ DEF_OP(URem) {
msub(GetReg<RA_64>(Node), TMP1, Divisor, Dividend);
break;
}
default: LOGMAN_MSG_A("Unknown UREM Size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown UREM Size: {}", OpSize); break;
}
}
@@ -349,7 +349,7 @@ DEF_OP(MulH) {
case 8:
smulh(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()), GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unknown Sext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown Sext size: {}", OpSize);
}
}
@@ -366,7 +366,7 @@ DEF_OP(UMulH) {
case 8:
umulh(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()), GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unknown Sext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown Sext size: {}", OpSize);
}
}
@@ -462,7 +462,7 @@ DEF_OP(Ror) {
break;
}
default: LOGMAN_MSG_A("Unhandled ROR size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled ROR size: {}", OpSize);
}
} else {
switch (OpSize) {
@@ -475,7 +475,7 @@ DEF_OP(Ror) {
break;
}
default: LOGMAN_MSG_A("Unhandled ROR size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled ROR size: {}", OpSize);
}
}
}
@@ -494,7 +494,7 @@ DEF_OP(Extr) {
break;
}
default: LOGMAN_MSG_A("Unhandled EXTR size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled EXTR size: {}", OpSize);
}
}
@@ -539,7 +539,7 @@ DEF_OP(LDiv) {
mov(GetReg<RA_64>(Node), x0);
break;
}
default: LOGMAN_MSG_A("Unknown LDIV Size: %d", Size); break;
default: LOGMAN_MSG_A_FMT("Unknown LDIV Size: {}", Size); break;
}
}
@@ -582,7 +582,7 @@ DEF_OP(LUDiv) {
mov(GetReg<RA_64>(Node), x0);
break;
}
default: LOGMAN_MSG_A("Unknown LUDIV Size: %d", Size); break;
default: LOGMAN_MSG_A_FMT("Unknown LUDIV Size: {}", Size); break;
}
}
@@ -635,7 +635,7 @@ DEF_OP(LRem) {
mov(GetReg<RA_64>(Node), x0);
break;
}
default: LOGMAN_MSG_A("Unknown LREM Size: %d", Size); break;
default: LOGMAN_MSG_A_FMT("Unknown LREM Size: {}", Size); break;
}
}
@@ -685,7 +685,7 @@ DEF_OP(LURem) {
mov(GetReg<RA_64>(Node), x0);
break;
}
default: LOGMAN_MSG_A("Unknown LUREM Size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown LUREM Size: {}", OpSize); break;
}
}
@@ -699,7 +699,7 @@ DEF_OP(Not) {
case 8:
mvn(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unsupported Not size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported Not size: {}", OpSize);
}
}
@@ -730,7 +730,7 @@ DEF_OP(Popcount) {
// fmov has zero extended, unused bytes are zero
addv(VTMP1.B(), VTMP1.V8B());
break;
default: LOGMAN_MSG_A("Unsupported Popcount size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported Popcount size: {}", OpSize);
}
auto Dst = GetReg<RA_32>(Node);
@@ -779,7 +779,7 @@ DEF_OP(FindMSB) {
clz(Dst, GetReg<RA_64>(Op->Header.Args[0].ID()));
sub(Dst, TMP1, Dst);
break;
default: LOGMAN_MSG_A("Unknown REV size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown FindMSB size: {}", OpSize); break;
}
}
@@ -800,7 +800,7 @@ DEF_OP(FindTrailingZeros) {
rbit(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()));
clz(GetReg<RA_64>(Node), GetReg<RA_64>(Node));
break;
default: LOGMAN_MSG_A("Unknown size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown FindTrailingZeros size: {}", OpSize); break;
}
}
@@ -819,7 +819,7 @@ DEF_OP(CountLeadingZeroes) {
case 8:
clz(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unknown size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown CountLeadingZeroes size: {}", OpSize); break;
}
}
@@ -837,7 +837,7 @@ DEF_OP(Rev) {
case 8:
rev(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unknown REV size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown REV size: {}", OpSize); break;
}
}
@@ -859,15 +859,14 @@ DEF_OP(Bfi) {
bfi(TMP1, GetReg<RA_64>(Op->Header.Args[1].ID()), Op->lsb, Op->Width);
mov(GetReg<RA_64>(Node), TMP1);
break;
default: LOGMAN_MSG_A("Unknown BFI size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown BFI size: {}", OpSize); break;
}
}
DEF_OP(Bfe) {
auto Op = IROp->C<IR::IROp_Bfe>();
uint8_t OpSize = IROp->Size;
LOGMAN_THROW_A(OpSize <= 8, "OpSize is too large for BFE: %d", OpSize);
LOGMAN_THROW_A(Op->Width != 0, "Invalid BFE width of 0");
LOGMAN_THROW_A_FMT(IROp->Size <= 8, "OpSize is too large for BFE: {}", IROp->Size);
LOGMAN_THROW_A_FMT(Op->Width != 0, "Invalid BFE width of 0");
auto Dst = GetReg<RA_64>(Node);
ubfx(Dst, GetReg<RA_64>(Op->Header.Args[0].ID()), Op->lsb, Op->Width);
@@ -881,7 +880,7 @@ DEF_OP(Sbfe) {
if (OpSize == 8) {
sbfx(Dst, GetReg<RA_64>(Op->Header.Args[0].ID()), Op->lsb, Op->Width);
} else {
LogMan::Msg::D("Unimplemented Sbfe size");
LogMan::Msg::DFmt("Unimplemented Sbfe size");
}
}
@@ -912,7 +911,7 @@ Condition MapSelectCC(IR::CondClassType Cond) {
case FEXCore::IR::COND_MI:
case FEXCore::IR::COND_PL:
default:
LOGMAN_MSG_A("Unsupported compare type");
LOGMAN_MSG_A_FMT("Unsupported compare type");
return Condition::nv;
}
}
@@ -930,7 +929,7 @@ DEF_OP(Select) {
} else if (IsFPR(Op->Cmp1.ID())) {
fcmp(GRFCMP(Op->Cmp1.ID()), GRFCMP(Op->Cmp2.ID()));
} else {
LOGMAN_MSG_A("Select: Expected GPR or FPR");
LOGMAN_MSG_A_FMT("Select: Expected GPR or FPR");
}
auto cc = MapSelectCC(Op->Cond);
@@ -941,7 +940,7 @@ DEF_OP(Select) {
if (is_const_true || is_const_false) {
if (is_const_false != true || is_const_true != true || const_true != 1 || const_false != 0) {
LOGMAN_MSG_A("Select: Unsupported compare inline parameters");
LOGMAN_MSG_A_FMT("Select: Unsupported compare inline parameters");
}
cset(GRS(Node), cc);
} else {
@@ -965,7 +964,7 @@ DEF_OP(VExtractToGPR) {
case 8:
umov(GetReg<RA_64>(Node), GetSrc(Op->Header.Args[0].ID()).V2D(), Op->Idx);
break;
default: LOGMAN_MSG_A("Unhandled ExtractElementSize: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled ExtractElementSize: {}", OpSize);
}
}
@@ -1028,7 +1027,7 @@ DEF_OP(FCmp) {
bool set = false;
if (Op->Flags & (1 << IR::FCMP_FLAG_EQ)) {
LOGMAN_THROW_A(IR::FCMP_FLAG_EQ == 0, "IR::FCMP_FLAG_EQ must equal 0");
LOGMAN_THROW_A_FMT(IR::FCMP_FLAG_EQ == 0, "IR::FCMP_FLAG_EQ must equal 0");
// EQ or unordered
cset(Dst, Condition::eq); // Z = 1
csinc(Dst, Dst, xzr, Condition::vc); // IF !V ? Z : 1
@@ -34,7 +34,7 @@ DEF_OP(CASPair) {
mov(Dst.first, TMP3);
mov(Dst.second, TMP4);
break;
default: LOGMAN_MSG_A("Unsupported: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported: {}", OpSize);
}
}
else {
@@ -44,11 +44,15 @@ DEF_OP(CASPair) {
aarch64::Label LoopNotExpected;
aarch64::Label LoopExpected;
bind(&LoopTop);
nop();
ldaxp(TMP2.W(), TMP3.W(), MemOperand(MemSrc));
nop();
cmp(TMP2.W(), Expected.first.W());
ccmp(TMP3.W(), Expected.second.W(), NoFlag, Condition::eq);
b(&LoopNotExpected, Condition::ne);
nop();
stlxp(TMP2.W(), Desired.first.W(), Desired.second.W(), MemOperand(MemSrc));
nop();
cbnz(TMP2.W(), &LoopTop);
mov(Dst.first.W(), Expected.first.W());
mov(Dst.second.W(), Expected.second.W());
@@ -69,11 +73,15 @@ DEF_OP(CASPair) {
aarch64::Label LoopNotExpected;
aarch64::Label LoopExpected;
bind(&LoopTop);
nop();
ldaxp(TMP2.X(), TMP3.X(), MemOperand(MemSrc));
nop();
cmp(TMP2.X(), Expected.first.X());
ccmp(TMP3.X(), Expected.second.X(), NoFlag, Condition::eq);
b(&LoopNotExpected, Condition::ne);
nop();
stlxp(TMP2.X(), Desired.first.X(), Desired.second.X(), MemOperand(MemSrc));
nop();
cbnz(TMP2.X(), &LoopTop);
mov(Dst.first.X(), Expected.first.X());
mov(Dst.second.X(), Expected.second.X());
@@ -89,7 +97,7 @@ DEF_OP(CASPair) {
bind(&LoopExpected);
break;
}
default: LOGMAN_MSG_A("Unsupported: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported: {}", OpSize);
}
}
}
@@ -115,7 +123,7 @@ DEF_OP(CAS) {
case 2: casalh(TMP2.W(), Desired.W(), MemOperand(MemSrc)); break;
case 4: casal(TMP2.W(), Desired.W(), MemOperand(MemSrc)); break;
case 8: casal(TMP2.X(), Desired.X(), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unsupported: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported: {}", OpSize);
}
mov(GetReg<RA_64>(Node), TMP2);
}
@@ -206,7 +214,7 @@ DEF_OP(CAS) {
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", OpSize);
}
}
}
@@ -222,7 +230,7 @@ DEF_OP(AtomicAdd) {
case 2: staddlh(GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
case 4: staddl(GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
case 8: staddl(GetReg<RA_64>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
else {
@@ -264,7 +272,7 @@ DEF_OP(AtomicAdd) {
cbnz(TMP2, &LoopTop);
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
}
@@ -281,7 +289,7 @@ DEF_OP(AtomicSub) {
case 2: staddlh(TMP2.W(), MemOperand(MemSrc)); break;
case 4: staddl(TMP2.W(), MemOperand(MemSrc)); break;
case 8: staddl(TMP2.X(), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
else {
@@ -323,7 +331,7 @@ DEF_OP(AtomicSub) {
cbnz(TMP2, &LoopTop);
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
}
@@ -340,7 +348,7 @@ DEF_OP(AtomicAnd) {
case 2: stclrlh(TMP2.W(), MemOperand(MemSrc)); break;
case 4: stclrl(TMP2.W(), MemOperand(MemSrc)); break;
case 8: stclrl(TMP2.X(), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
else {
@@ -382,7 +390,7 @@ DEF_OP(AtomicAnd) {
cbnz(TMP2, &LoopTop);
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
}
@@ -398,7 +406,7 @@ DEF_OP(AtomicOr) {
case 2: stsetlh(GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
case 4: stsetl(GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
case 8: stsetl(GetReg<RA_64>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
else {
@@ -440,7 +448,7 @@ DEF_OP(AtomicOr) {
cbnz(TMP2, &LoopTop);
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
}
@@ -456,7 +464,7 @@ DEF_OP(AtomicXor) {
case 2: steorlh(GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
case 4: steorl(GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
case 8: steorl(GetReg<RA_64>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
else {
@@ -498,7 +506,7 @@ DEF_OP(AtomicXor) {
cbnz(TMP2, &LoopTop);
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
}
@@ -515,36 +523,35 @@ DEF_OP(AtomicSwap) {
case 2: swplh(TMP2.W(), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 4: swpl(TMP2.W(), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 8: swpl(TMP2.X(), GetReg<RA_64>(Node), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
else {
// TMP2-TMP3
mov(TMP3, GetReg<RA_64>(Op->Header.Args[1].ID()));
switch (Op->Size) {
case 1: {
aarch64::Label LoopTop;
bind(&LoopTop);
ldaxrb(TMP2.W(), MemOperand(MemSrc));
stlxrb(TMP4.W(), TMP3.W(), MemOperand(MemSrc));
stlxrb(TMP4.W(), GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc));
cbnz(TMP4.W(), &LoopTop);
uxtb(GetReg<RA_64>(Node), TMP2.W());
uxtb(GetReg<RA_32>(Node), TMP2.W());
break;
}
case 2: {
aarch64::Label LoopTop;
bind(&LoopTop);
ldaxrh(TMP2.W(), MemOperand(MemSrc));
stlxrh(TMP4.W(), TMP3.W(), MemOperand(MemSrc));
stlxrh(TMP4.W(), GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc));
cbnz(TMP4.W(), &LoopTop);
uxtw(GetReg<RA_64>(Node), TMP2.W());
uxtw(GetReg<RA_32>(Node), TMP2.W());
break;
}
case 4: {
aarch64::Label LoopTop;
bind(&LoopTop);
ldaxr(TMP2.W(), MemOperand(MemSrc));
stlxr(TMP4.W(), TMP3.W(), MemOperand(MemSrc));
stlxr(TMP4.W(), GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc));
cbnz(TMP4.W(), &LoopTop);
mov(GetReg<RA_32>(Node), TMP2.W());
break;
@@ -553,12 +560,12 @@ DEF_OP(AtomicSwap) {
aarch64::Label LoopTop;
bind(&LoopTop);
ldaxr(TMP2, MemOperand(MemSrc));
stlxr(TMP4, TMP3.X(), MemOperand(MemSrc));
stlxr(TMP4, GetReg<RA_64>(Op->Header.Args[1].ID()), MemOperand(MemSrc));
cbnz(TMP4, &LoopTop);
mov(GetReg<RA_64>(Node), TMP2.X());
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
}
@@ -573,7 +580,7 @@ DEF_OP(AtomicFetchAdd) {
case 2: ldaddalh(GetReg<RA_32>(Op->Header.Args[1].ID()), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 4: ldaddal(GetReg<RA_32>(Op->Header.Args[1].ID()), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 8: ldaddal(GetReg<RA_64>(Op->Header.Args[1].ID()), GetReg<RA_64>(Node), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
else {
@@ -619,7 +626,7 @@ DEF_OP(AtomicFetchAdd) {
mov(GetReg<RA_64>(Node), TMP2);
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
}
@@ -635,7 +642,7 @@ DEF_OP(AtomicFetchSub) {
case 2: ldaddalh(TMP2.W(), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 4: ldaddal(TMP2.W(), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 8: ldaddal(TMP2.X(), GetReg<RA_64>(Node), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
else {
@@ -681,7 +688,7 @@ DEF_OP(AtomicFetchSub) {
mov(GetReg<RA_64>(Node), TMP2);
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
}
@@ -697,7 +704,7 @@ DEF_OP(AtomicFetchAnd) {
case 2: ldclralh(TMP2.W(), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 4: ldclral(TMP2.W(), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 8: ldclral(TMP2.X(), GetReg<RA_64>(Node), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
else {
@@ -743,7 +750,7 @@ DEF_OP(AtomicFetchAnd) {
mov(GetReg<RA_64>(Node), TMP2);
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
}
@@ -758,7 +765,7 @@ DEF_OP(AtomicFetchOr) {
case 2: ldsetalh(GetReg<RA_32>(Op->Header.Args[1].ID()), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 4: ldsetal(GetReg<RA_32>(Op->Header.Args[1].ID()), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 8: ldsetal(GetReg<RA_64>(Op->Header.Args[1].ID()), GetReg<RA_64>(Node), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
else {
@@ -804,7 +811,7 @@ DEF_OP(AtomicFetchOr) {
mov(GetReg<RA_64>(Node), TMP2);
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
}
@@ -819,7 +826,7 @@ DEF_OP(AtomicFetchXor) {
case 2: ldeoralh(GetReg<RA_32>(Op->Header.Args[1].ID()), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 4: ldeoral(GetReg<RA_32>(Op->Header.Args[1].ID()), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 8: ldeoral(GetReg<RA_64>(Op->Header.Args[1].ID()), GetReg<RA_64>(Node), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
else {
@@ -865,11 +872,61 @@ DEF_OP(AtomicFetchXor) {
mov(GetReg<RA_64>(Node), TMP2);
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
}
DEF_OP(AtomicFetchNeg) {
auto Op = IROp->C<IR::IROp_AtomicFetchNeg>();
auto MemSrc = GetReg<RA_64>(Op->Header.Args[0].ID());
// TMP2-TMP3
switch (Op->Size) {
case 1: {
aarch64::Label LoopTop;
bind(&LoopTop);
ldaxrb(TMP2.W(), MemOperand(MemSrc));
neg(TMP3.W(), TMP2.W());
stlxrb(TMP4.W(), TMP3.W(), MemOperand(MemSrc));
cbnz(TMP4.W(), &LoopTop);
mov(GetReg<RA_32>(Node), TMP2.W());
break;
}
case 2: {
aarch64::Label LoopTop;
bind(&LoopTop);
ldaxrh(TMP2.W(), MemOperand(MemSrc));
neg(TMP3.W(), TMP2.W());
stlxrh(TMP4.W(), TMP3.W(), MemOperand(MemSrc));
cbnz(TMP4.W(), &LoopTop);
mov(GetReg<RA_32>(Node), TMP2.W());
break;
}
case 4: {
aarch64::Label LoopTop;
bind(&LoopTop);
ldaxr(TMP2.W(), MemOperand(MemSrc));
neg(TMP3.W(), TMP2.W());
stlxr(TMP4.W(), TMP3.W(), MemOperand(MemSrc));
cbnz(TMP4.W(), &LoopTop);
mov(GetReg<RA_32>(Node), TMP2.W());
break;
}
case 8: {
aarch64::Label LoopTop;
bind(&LoopTop);
ldaxr(TMP2, MemOperand(MemSrc));
neg(TMP3, TMP2);
stlxr(TMP4, TMP3, MemOperand(MemSrc));
cbnz(TMP4, &LoopTop);
mov(GetReg<RA_64>(Node), TMP2);
break;
}
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
#undef DEF_OP
void Arm64JITCore::RegisterAtomicHandlers() {
#define REGISTER_OP(op, x) OpHandlers[FEXCore::IR::IROps::OP_##op] = &Arm64JITCore::Op_##x
@@ -886,6 +943,7 @@ void Arm64JITCore::RegisterAtomicHandlers() {
REGISTER_OP(ATOMICFETCHAND, AtomicFetchAnd);
REGISTER_OP(ATOMICFETCHOR, AtomicFetchOr);
REGISTER_OP(ATOMICFETCHXOR, AtomicFetchXor);
REGISTER_OP(ATOMICFETCHNEG, AtomicFetchNeg);
#undef REGISTER_OP
}
}
@@ -4,6 +4,8 @@ tags: backend|arm64
$end_info$
*/
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/JIT/Arm64/JITClass.h"
#include "Interface/Core/InternalThreadState.h"
@@ -16,15 +18,15 @@ using namespace vixl;
using namespace vixl::aarch64;
#define DEF_OP(x) void Arm64JITCore::Op_##x(FEXCore::IR::IROp_Header *IROp, uint32_t Node)
DEF_OP(GuestCallDirect) {
LogMan::Msg::D("Unimplemented");
LogMan::Msg::DFmt("Unimplemented");
}
DEF_OP(GuestCallIndirect) {
LogMan::Msg::D("Unimplemented");
LogMan::Msg::DFmt("Unimplemented");
}
DEF_OP(GuestReturn) {
LogMan::Msg::D("Unimplemented");
LogMan::Msg::DFmt("Unimplemented");
}
DEF_OP(SignalReturn) {
@@ -142,7 +144,7 @@ Condition MapBranchCC(IR::CondClassType Cond) {
case FEXCore::IR::COND_MI:
case FEXCore::IR::COND_PL:
default:
LOGMAN_MSG_A("Unsupported compare type");
LOGMAN_MSG_A_FMT("Unsupported compare type");
return Condition::nv;
}
}
@@ -169,10 +171,10 @@ DEF_OP(CondJump) {
bool isConst = IsInlineConstant(Op->Cmp2, &Const);
if (isConst && Const == 0 && Op->Cond.Val == FEXCore::IR::COND_EQ) {
LOGMAN_THROW_A(IsGPR(Op->Cmp1.ID()), "CondJump: Expected GPR");
LOGMAN_THROW_A_FMT(IsGPR(Op->Cmp1.ID()), "CondJump: Expected GPR");
cbz(GRCMP(Op->Cmp1.ID()), TrueTargetLabel);
} else if (isConst && Const == 0 && Op->Cond.Val == FEXCore::IR::COND_NEQ) {
LOGMAN_THROW_A(IsGPR(Op->Cmp1.ID()), "CondJump: Expected GPR");
LOGMAN_THROW_A_FMT(IsGPR(Op->Cmp1.ID()), "CondJump: Expected GPR");
cbnz(GRCMP(Op->Cmp1.ID()), TrueTargetLabel);
} else {
if (IsGPR(Op->Cmp1.ID())) {
@@ -183,7 +185,7 @@ DEF_OP(CondJump) {
} else if (IsFPR(Op->Cmp1.ID())) {
fcmp(GRFCMP(Op->Cmp1.ID()), GRFCMP(Op->Cmp2.ID()));
} else {
LOGMAN_MSG_A("CondJump: Expected GPR or FPR");
LOGMAN_MSG_A_FMT("CondJump: Expected GPR or FPR");
}
b(TrueTargetLabel, MapBranchCC(Op->Cond));
@@ -257,7 +259,7 @@ DEF_OP(Thunk) {
DEF_OP(ValidateCode) {
auto Op = IROp->C<IR::IROp_ValidateCode>();
uint8_t *OldCode = (uint8_t *)&Op->CodeOriginalLow;
const auto *OldCode = (const uint8_t *)&Op->CodeOriginalLow;
int len = Op->CodeLength;
int idx = 0;
@@ -268,7 +270,7 @@ DEF_OP(ValidateCode) {
while (len >= 8)
{
ldr(x2, MemOperand(x0, idx));
LoadConstant(x3, *(uint32_t *)(OldCode + idx));
LoadConstant(x3, *(const uint32_t *)(OldCode + idx));
cmp(x2, x3);
csel(GetReg<RA_64>(Node), GetReg<RA_64>(Node), x1, Condition::eq);
len -= 8;
@@ -277,7 +279,7 @@ DEF_OP(ValidateCode) {
while (len >= 4)
{
ldr(w2, MemOperand(x0, idx));
LoadConstant(w3, *(uint32_t *)(OldCode + idx));
LoadConstant(w3, *(const uint32_t *)(OldCode + idx));
cmp(w2, w3);
csel(GetReg<RA_64>(Node), GetReg<RA_64>(Node), x1, Condition::eq);
len -= 4;
@@ -286,7 +288,7 @@ DEF_OP(ValidateCode) {
while (len >= 2)
{
ldrh(w2, MemOperand(x0, idx));
LoadConstant(w3, *(uint16_t *)(OldCode + idx));
LoadConstant(w3, *(const uint16_t *)(OldCode + idx));
cmp(w2, w3);
csel(GetReg<RA_64>(Node), GetReg<RA_64>(Node), x1, Condition::eq);
len -= 2;
@@ -295,7 +297,7 @@ DEF_OP(ValidateCode) {
while (len >= 1)
{
ldrb(w2, MemOperand(x0, idx));
LoadConstant(w3, *(uint8_t *)(OldCode + idx));
LoadConstant(w3, *(const uint8_t *)(OldCode + idx));
cmp(w2, w3);
csel(GetReg<RA_64>(Node), GetReg<RA_64>(Node), x1, Condition::eq);
len -= 1;
@@ -31,7 +31,7 @@ DEF_OP(VInsGPR) {
ins(GetDst(Node).V2D(), Op->Index, GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -52,7 +52,7 @@ DEF_OP(VCastFromGPR) {
case 8:
fmov(GetDst(Node).D(), GetReg<RA_64>(Op->Header.Args[0].ID()).X());
break;
default: LOGMAN_MSG_A("Unknown castGPR element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown castGPR element size: {}", Op->Header.ElementSize);
}
}
@@ -91,7 +91,7 @@ DEF_OP(Float_FToF) {
fcvt(GetDst(Node).S(), GetSrc(Op->Header.Args[0].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown FCVT sizes: 0x%x", Conv);
default: LOGMAN_MSG_A_FMT("Unknown FCVT sizes: 0x{:x}", Conv);
}
}
@@ -104,7 +104,7 @@ DEF_OP(Vector_SToF) {
case 8:
scvtf(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
default: LOGMAN_MSG_A("Unknown castGPR element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Vector_SToF element size: {}", Op->Header.ElementSize);
}
}
@@ -117,7 +117,7 @@ DEF_OP(Vector_FToZS) {
case 8:
fcvtzs(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
default: LOGMAN_MSG_A("Unknown castGPR element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Vector_FToZS element size: {}", Op->Header.ElementSize);
}
}
@@ -132,7 +132,7 @@ DEF_OP(Vector_FToS) {
frinti(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
fcvtzs(GetDst(Node).V2D(), GetDst(Node).V2D());
break;
default: LOGMAN_MSG_A("Unknown castGPR element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Vector_FToS element size: {}", Op->Header.ElementSize);
}
}
@@ -149,7 +149,7 @@ DEF_OP(Vector_FToF) {
fcvtn(GetDst(Node).V2S(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Conversion Type : 0%04x", Conv); break;
default: LOGMAN_MSG_A_FMT("Unknown Vector_FToF Type : 0x{:04x}", Conv); break;
}
}
+81 -39
View File
@@ -11,6 +11,7 @@ $end_info$
*/
#include "Interface/Context/Context.h"
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/ArchHelpers/Arm64.h"
#include "Interface/Core/ArchHelpers/MContext.h"
@@ -44,8 +45,9 @@ using namespace vixl::aarch64;
void Arm64JITCore::Op_Unhandled(FEXCore::IR::IROp_Header *IROp, uint32_t Node) {
FallbackInfo Info;
if (!InterpreterOps::GetFallbackHandler(IROp, &Info)) {
auto Name = FEXCore::IR::GetName(IROp->Op);
LOGMAN_MSG_A("Unhandled IR Op: %s", std::string(Name).c_str());
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
LOGMAN_MSG_A_FMT("Unhandled IR Op: {}", FEXCore::IR::GetName(IROp->Op));
#endif
} else {
switch(Info.ABI) {
case FABI_VOID_U16:{
@@ -292,8 +294,10 @@ void Arm64JITCore::Op_Unhandled(FEXCore::IR::IROp_Header *IROp, uint32_t Node) {
case FABI_UNKNOWN:
default:
auto Name = FEXCore::IR::GetName(IROp->Op);
LOGMAN_MSG_A("Unhandled IR Fallback abi: %s %d", std::string(Name).c_str(), Info.ABI);
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
LOGMAN_MSG_A_FMT("Unhandled IR Fallback ABI: {} {}", FEXCore::IR::GetName(IROp->Op), Info.ABI);
#endif
break;
}
}
}
@@ -310,7 +314,7 @@ Arm64JITCore::CodeBuffer Arm64JITCore::AllocateNewCodeBuffer(size_t Size) {
PROT_READ | PROT_WRITE | PROT_EXEC,
MAP_PRIVATE | MAP_ANONYMOUS,
-1, 0));
LOGMAN_THROW_A(!!Buffer.Ptr, "Couldn't allocate code buffer");
LOGMAN_THROW_A_FMT(!!Buffer.Ptr, "Couldn't allocate code buffer");
Dispatcher->RegisterCodeBuffer(Buffer.Ptr, Buffer.Size);
return Buffer;
}
@@ -340,26 +344,52 @@ bool Arm64JITCore::HandleSIGBUS(int Signal, void *info, void *ucontext) {
0b1011'0000'0000; // Inner shareable all
if ((Instr & 0x3F'FF'FC'00) == 0x08'DF'FC'00 || // LDAR*
(Instr & 0x3F'FF'FC'00) == 0x38'BF'C0'00) { // LDAPR*
uint32_t LDR = 0b0011'1000'0111'1111'0110'1000'0000'0000;
LDR |= Size << 30;
LDR |= AddrReg << 5;
LDR |= DataReg;
PC[-1] = DMB;
PC[0] = LDR;
PC[1] = DMB;
// Back up one instruction and have another go
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) - 4);
if (ParanoidTSO()) {
if (FEXCore::ArchHelpers::Arm64::HandleAtomicLoad(ucontext, info, Instr)) {
// Skip this instruction now
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) + 4);
return true;
}
else {
LogMan::Msg::EFmt("Unhandled JIT SIGBUS LDAR*: PC: {} Instruction: 0x{:08x}\n", fmt::ptr(PC), PC[0]);
return false;
}
}
else {
uint32_t LDR = 0b0011'1000'0111'1111'0110'1000'0000'0000;
LDR |= Size << 30;
LDR |= AddrReg << 5;
LDR |= DataReg;
PC[-1] = DMB;
PC[0] = LDR;
PC[1] = DMB;
// Back up one instruction and have another go
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) - 4);
}
}
else if ( (Instr & 0x3F'FF'FC'00) == 0x08'9F'FC'00) { // STLR*
uint32_t STR = 0b0011'1000'0011'1111'0110'1000'0000'0000;
STR |= Size << 30;
STR |= AddrReg << 5;
STR |= DataReg;
PC[-1] = DMB;
PC[0] = STR;
PC[1] = DMB;
// Back up one instruction and have another go
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) - 4);
if (ParanoidTSO()) {
if (FEXCore::ArchHelpers::Arm64::HandleAtomicStore(ucontext, info, Instr)) {
// Skip this instruction now
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) + 4);
return true;
}
else {
LogMan::Msg::EFmt("Unhandled JIT SIGBUS STLR*: PC: {} Instruction: 0x{:08x}\n", fmt::ptr(PC), PC[0]);
return false;
}
}
else {
uint32_t STR = 0b0011'1000'0011'1111'0110'1000'0000'0000;
STR |= Size << 30;
STR |= AddrReg << 5;
STR |= DataReg;
PC[-1] = DMB;
PC[0] = STR;
PC[1] = DMB;
// Back up one instruction and have another go
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) - 4);
}
}
else if ((Instr & FEXCore::ArchHelpers::Arm64::LDAXP_MASK) == FEXCore::ArchHelpers::Arm64::LDAXP_INST) { // LDAXP
uint32_t DataReg2 = (Instr >> 10) & 0x1F;
@@ -396,7 +426,7 @@ bool Arm64JITCore::HandleSIGBUS(int Signal, void *info, void *ucontext) {
return true;
}
else {
LogMan::Msg::E("Unhandled JIT SIGBUS CASPAL: PC: %p Instruction: 0x%08x\n", PC, PC[0]);
LogMan::Msg::EFmt("Unhandled JIT SIGBUS CASPAL: PC: {} Instruction: 0x{:08x}\n", fmt::ptr(PC), PC[0]);
return false;
}
}
@@ -407,7 +437,7 @@ bool Arm64JITCore::HandleSIGBUS(int Signal, void *info, void *ucontext) {
return true;
}
else {
LogMan::Msg::E("Unhandled JIT SIGBUS CASAL: PC: %p Instruction: 0x%08x\n", PC, PC[0]);
LogMan::Msg::EFmt("Unhandled JIT SIGBUS CASAL: PC: {} Instruction: 0x{:08x}\n", fmt::ptr(PC), PC[0]);
return false;
}
}
@@ -419,12 +449,24 @@ bool Arm64JITCore::HandleSIGBUS(int Signal, void *info, void *ucontext) {
}
else {
uint8_t Op = (PC[0] >> 12) & 0xF;
LogMan::Msg::E("Unhandled JIT SIGBUS Atomic mem op 0x%02x: PC: %p Instruction: 0x%08x\n", Op, PC, PC[0]);
LogMan::Msg::EFmt("Unhandled JIT SIGBUS Atomic mem op 0x{:02x}: PC: {} Instruction: 0x{:08x}\n", Op, fmt::ptr(PC), PC[0]);
return false;
}
}
else if ((Instr & FEXCore::ArchHelpers::Arm64::LDAXR_MASK) == FEXCore::ArchHelpers::Arm64::LDAXR_INST) { // LDAXR*
uint64_t BytesToSkip = FEXCore::ArchHelpers::Arm64::HandleAtomicLoadstoreExclusive(ucontext, info);
if (BytesToSkip) {
// Skip this instruction now
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) + BytesToSkip);
return true;
}
else {
LogMan::Msg::EFmt("Unhandled JIT SIGBUS LDAXR: PC: {} Instruction: 0x{:08x}\n", fmt::ptr(PC), PC[0]);
return false;
}
}
else {
LogMan::Msg::E("Unhandled JIT SIGBUS: PC: %p Instruction: 0x%08x\n", PC, PC[0]);
LogMan::Msg::EFmt("Unhandled JIT SIGBUS: PC: {} Instruction: 0x{:08x}\n", fmt::ptr(PC), PC[0]);
return false;
}
@@ -502,17 +544,17 @@ Arm64JITCore::Arm64JITCore(FEXCore::Context::Context *ctx, FEXCore::Core::Intern
CTX->SignalDelegation->RegisterHostSignalHandler(SIGILL, [](FEXCore::Core::InternalThreadState *Thread, int Signal, void *info, void *ucontext) -> bool {
Arm64JITCore *Core = reinterpret_cast<Arm64JITCore*>(Thread->CPUBackend.get());
return Core->Dispatcher->HandleSIGILL(Signal, info, ucontext);
});
}, true);
CTX->SignalDelegation->RegisterHostSignalHandler(SIGBUS, [](FEXCore::Core::InternalThreadState *Thread, int Signal, void *info, void *ucontext) -> bool {
Arm64JITCore *Core = reinterpret_cast<Arm64JITCore*>(Thread->CPUBackend.get());
return Core->HandleSIGBUS(Signal, info, ucontext);
});
}, true);
CTX->SignalDelegation->RegisterHostSignalHandler(SignalDelegator::SIGNAL_FOR_PAUSE, [](FEXCore::Core::InternalThreadState *Thread, int Signal, void *info, void *ucontext) -> bool {
Arm64JITCore *Core = reinterpret_cast<Arm64JITCore*>(Thread->CPUBackend.get());
return Core->Dispatcher->HandleSignalPause(Signal, info, ucontext);
});
}, true);
auto GuestSignalHandler = [](FEXCore::Core::InternalThreadState *Thread, int Signal, void *info, void *ucontext, GuestSigAction *GuestAction, stack_t *GuestStack) -> bool {
Arm64JITCore *Core = reinterpret_cast<Arm64JITCore*>(Thread->CPUBackend.get());
@@ -579,7 +621,7 @@ Arm64JITCore::~Arm64JITCore() {
IR::PhysicalRegister Arm64JITCore::GetPhys(uint32_t Node) const {
auto PhyReg = RAData->GetNodeRegister(Node);
LOGMAN_THROW_A(!PhyReg.IsInvalid(), "Couldn't Allocate register for node: ssa%d. Class: %d", Node, PhyReg.Class);
LOGMAN_THROW_A_FMT(!PhyReg.IsInvalid(), "Couldn't Allocate register for node: ssa{}. Class: {}", Node, PhyReg.Class);
return PhyReg;
}
@@ -593,7 +635,7 @@ aarch64::Register Arm64JITCore::GetReg<Arm64JITCore::RA_32>(uint32_t Node) const
} else if (Reg.Class == IR::GPRClass.Val) {
return RA64[Reg.Reg].W();
} else {
LOGMAN_THROW_A(false, "Unexpected Class: %d", Reg.Class);
LOGMAN_THROW_A_FMT(false, "Unexpected Class: {}", Reg.Class);
}
FEX_UNREACHABLE;
@@ -608,7 +650,7 @@ aarch64::Register Arm64JITCore::GetReg<Arm64JITCore::RA_64>(uint32_t Node) const
} else if (Reg.Class == IR::GPRClass.Val) {
return RA64[Reg.Reg];
} else {
LOGMAN_THROW_A(false, "Unexpected Class: %d", Reg.Class);
LOGMAN_THROW_A_FMT(false, "Unexpected Class: {}", Reg.Class);
}
FEX_UNREACHABLE;
@@ -634,7 +676,7 @@ aarch64::VRegister Arm64JITCore::GetSrc(uint32_t Node) const {
} else if (Reg.Class == IR::FPRClass.Val) {
return RAFPR[Reg.Reg];
} else {
LOGMAN_THROW_A(false, "Unexpected Class: %d", Reg.Class);
LOGMAN_THROW_A_FMT(false, "Unexpected Class: {}", Reg.Class);
}
FEX_UNREACHABLE;
@@ -648,7 +690,7 @@ aarch64::VRegister Arm64JITCore::GetDst(uint32_t Node) const {
} else if (Reg.Class == IR::FPRClass.Val) {
return RAFPR[Reg.Reg];
} else {
LOGMAN_THROW_A(false, "Unexpected Class: %d", Reg.Class);
LOGMAN_THROW_A_FMT(false, "Unexpected Class: {}", Reg.Class);
}
FEX_UNREACHABLE;
@@ -707,8 +749,6 @@ void *Arm64JITCore::CompileCode(uint64_t Entry, [[maybe_unused]] FEXCore::IR::IR
this->Entry = Entry;
this->RAData = RAData;
auto HeaderOp = IR->GetHeader();
#ifndef NDEBUG
LoadConstant(x0, Entry);
#endif
@@ -786,8 +826,10 @@ void *Arm64JITCore::CompileCode(uint64_t Entry, [[maybe_unused]] FEXCore::IR::IR
for (auto [BlockNode, BlockHeader] : IR->GetBlocks()) {
using namespace FEXCore::IR;
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
auto BlockIROp = BlockHeader->CW<FEXCore::IR::IROp_CodeBlock>();
LOGMAN_THROW_A(BlockIROp->Header.Op == IR::OP_CODEBLOCK, "IR type failed to be a code block");
LOGMAN_THROW_A_FMT(BlockIROp->Header.Op == IR::OP_CODEBLOCK, "IR type failed to be a code block");
#endif
{
uint32_t Node = IR->GetID(BlockNode);
@@ -851,7 +893,7 @@ uint64_t Arm64JITCore::ExitFunctionLink(Arm64JITCore *core, FEXCore::Core::CpuSt
auto HostCode = Thread->LookupCache->FindBlock(GuestRip);
if (!HostCode) {
//printf("ExitFunctionLink: Aborting, %lX not in cache\n", GuestRip);
//fmt::print("ExitFunctionLink: Aborting, {:X} not in cache\n", GuestRip);
Frame->State.rip = GuestRip;
return core->ThreadSharedData.Dispatcher->AbsoluteLoopTopAddress;
}
@@ -6,7 +6,6 @@ $end_info$
#pragma once
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/ArchHelpers/Arm64Emitter.h"
#include "Interface/Core/Dispatcher/Dispatcher.h"
@@ -257,6 +256,7 @@ private:
DEF_OP(AtomicFetchAnd);
DEF_OP(AtomicFetchOr);
DEF_OP(AtomicFetchXor);
DEF_OP(AtomicFetchNeg);
///< Branch ops
DEF_OP(GuestCallDirect);
@@ -30,7 +30,7 @@ DEF_OP(LoadContext) {
case 8:
ldr(GetReg<RA_64>(Node), MemOperand(STATE, Op->Offset));
break;
default: LOGMAN_MSG_A("Unhandled LoadContext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled LoadContext size: {}", OpSize);
}
}
else {
@@ -51,7 +51,7 @@ DEF_OP(LoadContext) {
case 16:
ldr(Dst, MemOperand(STATE, Op->Offset));
break;
default: LOGMAN_MSG_A("Unhandled LoadContext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled LoadContext size: {}", OpSize);
}
}
}
@@ -73,7 +73,7 @@ DEF_OP(StoreContext) {
case 8:
str(GetReg<RA_64>(Op->Header.Args[0].ID()), MemOperand(STATE, Op->Offset));
break;
default: LOGMAN_MSG_A("Unhandled StoreContext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled StoreContext size: {}", OpSize);
}
}
else {
@@ -94,7 +94,7 @@ DEF_OP(StoreContext) {
case 16:
str(Src, MemOperand(STATE, Op->Offset));
break;
default: LOGMAN_MSG_A("Unhandled LoadContext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled StoreContext size: {}", OpSize);
}
}
}
@@ -107,29 +107,29 @@ DEF_OP(LoadRegister) {
auto regId = (Op->Offset - offsetof(FEXCore::Core::CpuStateFrame, State.gregs[0])) / 8;
auto regOffs = Op->Offset & 7;
LOGMAN_THROW_A(regId < SRA64.size(), "out of range regId");
LOGMAN_THROW_A_FMT(regId < SRA64.size(), "out of range regId");
auto reg = SRA64[regId];
switch(Op->Header.Size) {
case 1:
LOGMAN_THROW_A(regOffs == 0 || regOffs == 1, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0 || regOffs == 1, "unexpected regOffs");
ubfx(GetReg<RA_64>(Node), reg, regOffs * 8, 8);
break;
case 2:
LOGMAN_THROW_A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
ubfx(GetReg<RA_64>(Node), reg, 0, 16);
break;
case 4:
LOGMAN_THROW_A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
if (GetReg<RA_64>(Node).GetCode() != reg.GetCode())
mov(GetReg<RA_32>(Node), reg.W());
break;
case 8:
LOGMAN_THROW_A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
if (GetReg<RA_64>(Node).GetCode() != reg.GetCode())
mov(GetReg<RA_64>(Node), reg);
break;
@@ -138,24 +138,24 @@ DEF_OP(LoadRegister) {
auto regId = (Op->Offset - offsetof(FEXCore::Core::CpuStateFrame, State.xmm[0][0])) / 16;
auto regOffs = Op->Offset & 15;
LOGMAN_THROW_A(regId < SRAFPR.size(), "out of range regId");
LOGMAN_THROW_A_FMT(regId < SRAFPR.size(), "out of range regId");
auto guest = SRAFPR[regId];
auto host = GetSrc(Node);
switch(Op->Header.Size) {
case 1:
LOGMAN_THROW_A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
mov(host.B(), guest.B());
break;
case 2:
LOGMAN_THROW_A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
fmov(host.H(), guest.H());
break;
case 4:
LOGMAN_THROW_A((regOffs & 3) == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT((regOffs & 3) == 0, "unexpected regOffs");
if (regOffs == 0) {
if (host.GetCode() != guest.GetCode())
fmov(host.S(), guest.S());
@@ -165,7 +165,7 @@ DEF_OP(LoadRegister) {
break;
case 8:
LOGMAN_THROW_A((regOffs & 7) == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT((regOffs & 7) == 0, "unexpected regOffs");
if (regOffs == 0) {
if (host.GetCode() != guest.GetCode())
mov(host.D(), guest.D());
@@ -175,13 +175,13 @@ DEF_OP(LoadRegister) {
break;
case 16:
LOGMAN_THROW_A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
if (host.GetCode() != guest.GetCode())
mov(host.Q(), guest.Q());
break;
}
} else {
LOGMAN_THROW_A(false, "Unhandled Op->Class %d", Op->Class);
LOGMAN_THROW_A_FMT(false, "Unhandled Op->Class {}", Op->Class);
}
}
@@ -192,28 +192,28 @@ DEF_OP(StoreRegister) {
auto regId = Op->Offset / 8 - 1;
auto regOffs = Op->Offset & 7;
LOGMAN_THROW_A(regId < SRA64.size(), "out of range regId");
LOGMAN_THROW_A_FMT(regId < SRA64.size(), "out of range regId");
auto reg = SRA64[regId];
switch(Op->Header.Size) {
case 1:
LOGMAN_THROW_A(regOffs == 0 || regOffs == 1, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0 || regOffs == 1, "unexpected regOffs");
bfi(reg, GetReg<RA_64>(Op->Value.ID()), regOffs * 8, 8);
break;
case 2:
LOGMAN_THROW_A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
bfi(reg, GetReg<RA_64>(Op->Value.ID()), 0, 16);
break;
case 4:
LOGMAN_THROW_A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
bfi(reg, GetReg<RA_64>(Op->Value.ID()), 0, 32);
break;
case 8:
LOGMAN_THROW_A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
if (GetReg<RA_64>(Op->Value.ID()).GetCode() != reg.GetCode())
mov(reg, GetReg<RA_64>(Op->Value.ID()));
break;
@@ -222,7 +222,7 @@ DEF_OP(StoreRegister) {
auto regId = (Op->Offset - offsetof(FEXCore::Core::CpuStateFrame, State.xmm[0][0])) / 16;
auto regOffs = Op->Offset & 15;
LOGMAN_THROW_A(regId < SRAFPR.size(), "regId out of range");
LOGMAN_THROW_A_FMT(regId < SRAFPR.size(), "regId out of range");
auto guest = SRAFPR[regId];
auto host = GetSrc(Op->Value.ID());
@@ -233,28 +233,28 @@ DEF_OP(StoreRegister) {
break;
case 2:
LOGMAN_THROW_A((regOffs & 1) == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT((regOffs & 1) == 0, "unexpected regOffs");
ins(guest.V8H(), regOffs/2, host.V8H(), 0);
break;
case 4:
LOGMAN_THROW_A((regOffs & 3) == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT((regOffs & 3) == 0, "unexpected regOffs");
ins(guest.V4S(), regOffs/4, host.V4S(), 0);
break;
case 8:
LOGMAN_THROW_A((regOffs & 7) == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT((regOffs & 7) == 0, "unexpected regOffs");
ins(guest.V2D(), regOffs / 8, host.V2D(), 0);
break;
case 16:
LOGMAN_THROW_A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
if (guest.GetCode() != host.GetCode())
mov(guest.Q(), host.Q());
break;
}
} else {
LOGMAN_THROW_A(false, "Unhandled Op->Class %d", Op->Class);
LOGMAN_THROW_A_FMT(false, "Unhandled Op->Class {}", Op->Class);
}
}
@@ -288,15 +288,17 @@ DEF_OP(LoadContextIndexed) {
ldr(GetReg<RA_64>(Node), MemOperand(TMP1, Op->BaseOffset));
break;
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed size: %d", Op->Size);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed size: {}", Op->Size);
break;
}
break;
}
case 16:
LOGMAN_MSG_A("Invalid Class load of size 16");
LOGMAN_MSG_A_FMT("Invalid Class load of size 16");
break;
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed stride: %d", Op->Stride);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed stride: {}", Op->Stride);
break;
}
}
else {
@@ -333,12 +335,14 @@ DEF_OP(LoadContextIndexed) {
}
break;
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed size: %d", Op->Size);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed size: {}", Op->Size);
break;
}
break;
}
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed stride: %d", Op->Stride);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed stride: {}", Op->Stride);
break;
}
}
}
@@ -374,15 +378,17 @@ DEF_OP(StoreContextIndexed) {
str(value, MemOperand(TMP1, Op->BaseOffset));
break;
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed size: %d", Op->Size);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed size: {}", Op->Size);
break;
}
break;
}
case 16:
LOGMAN_MSG_A("Invalid Class load of size 16");
LOGMAN_MSG_A_FMT("Invalid Class store of size 16");
break;
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed stride: %d", Op->Stride);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed stride: {}", Op->Stride);
break;
}
}
else {
@@ -421,12 +427,14 @@ DEF_OP(StoreContextIndexed) {
}
break;
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed size: %d", Op->Size);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed size: {}", Op->Size);
break;
}
break;
}
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed stride: %d", Op->Stride);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed stride: {}", Op->Stride);
break;
}
}
}
@@ -434,7 +442,7 @@ DEF_OP(StoreContextIndexed) {
DEF_OP(SpillRegister) {
auto Op = IROp->C<IR::IROp_SpillRegister>();
uint8_t OpSize = IROp->Size;
uint32_t SlotOffset = Op->Slot * 16 + 16;
uint32_t SlotOffset = Op->Slot * 16;
if (Op->Class == FEXCore::IR::GPRClass) {
switch (OpSize) {
@@ -454,7 +462,7 @@ DEF_OP(SpillRegister) {
str(GetReg<RA_64>(Op->Header.Args[0].ID()), MemOperand(sp, SlotOffset));
break;
}
default: LOGMAN_MSG_A("Unhandled SpillRegister size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled SpillRegister size: {}", OpSize);
}
} else if (Op->Class == FEXCore::IR::FPRClass) {
switch (OpSize) {
@@ -470,17 +478,17 @@ DEF_OP(SpillRegister) {
str(GetSrc(Op->Header.Args[0].ID()), MemOperand(sp, SlotOffset));
break;
}
default: LOGMAN_MSG_A("Unhandled SpillRegister size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled SpillRegister size: {}", OpSize);
}
} else {
LOGMAN_MSG_A("Unhandled SpillRegister class: %d", Op->Class.Val);
LOGMAN_MSG_A_FMT("Unhandled SpillRegister class: {}", Op->Class.Val);
}
}
DEF_OP(FillRegister) {
auto Op = IROp->C<IR::IROp_FillRegister>();
uint8_t OpSize = IROp->Size;
uint32_t SlotOffset = Op->Slot * 16 + 16;
uint32_t SlotOffset = Op->Slot * 16;
if (Op->Class == FEXCore::IR::GPRClass) {
switch (OpSize) {
@@ -500,7 +508,7 @@ DEF_OP(FillRegister) {
ldr(GetReg<RA_64>(Node), MemOperand(sp, SlotOffset));
break;
}
default: LOGMAN_MSG_A("Unhandled SpillRegister size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled FillRegister size: {}", OpSize);
}
} else if (Op->Class == FEXCore::IR::FPRClass) {
switch (OpSize) {
@@ -516,10 +524,10 @@ DEF_OP(FillRegister) {
ldr(GetDst(Node), MemOperand(sp, SlotOffset));
break;
}
default: LOGMAN_MSG_A("Unhandled SpillRegister size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled FillRegister size: {}", OpSize);
}
} else {
LOGMAN_MSG_A("Unhandled FillRegister class: %d", Op->Class.Val);
LOGMAN_MSG_A_FMT("Unhandled FillRegister class: {}", Op->Class.Val);
}
}
@@ -539,7 +547,7 @@ MemOperand Arm64JITCore::GenerateMemOperand(uint8_t AccessSize, aarch64::Registe
return MemOperand(Base);
} else {
if (OffsetScale != 1 && OffsetScale != AccessSize) {
LOGMAN_MSG_A("Unhandled GenerateMemOperand OffsetScale: %d", OffsetScale);
LOGMAN_MSG_A_FMT("Unhandled GenerateMemOperand OffsetScale: {}", OffsetScale);
}
uint64_t Const;
if (IsInlineConstant(Offset, &Const)) {
@@ -551,7 +559,7 @@ MemOperand Arm64JITCore::GenerateMemOperand(uint8_t AccessSize, aarch64::Registe
case IR::MEM_OFFSET_UXTW.Val: return MemOperand(Base, RegOffset.W(), Extend::UXTW, (int)std::log2(OffsetScale) );
case IR::MEM_OFFSET_SXTW.Val: return MemOperand(Base, RegOffset.W(), Extend::SXTW, (int)std::log2(OffsetScale) );
default: LOGMAN_MSG_A("Unhandled GenerateMemOperand OffsetType: %d", OffsetType.Val); break;
default: LOGMAN_MSG_A_FMT("Unhandled GenerateMemOperand OffsetType: {}", OffsetType.Val); break;
}
}
}
@@ -580,7 +588,7 @@ DEF_OP(LoadMem) {
case 8:
ldr(Dst, MemSrc);
break;
default: LOGMAN_MSG_A("Unhandled LoadMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled LoadMem size: {}", Op->Size);
}
}
else {
@@ -601,7 +609,7 @@ DEF_OP(LoadMem) {
case 16:
ldr(Dst, MemSrc);
break;
default: LOGMAN_MSG_A("Unhandled LoadMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled LoadMem size: {}", Op->Size);
}
}
}
@@ -612,7 +620,7 @@ DEF_OP(LoadMemTSO) {
auto MemSrc = MemOperand(GetReg<RA_64>(Op->Header.Args[0].ID()));
if (!Op->Offset.IsInvalid()) {
LOGMAN_MSG_A("LoadMemTSO: No offset allowed");
LOGMAN_MSG_A_FMT("LoadMemTSO: No offset allowed");
}
if (SupportsRCPC && Op->Class == FEXCore::IR::GPRClass) {
@@ -635,7 +643,7 @@ DEF_OP(LoadMemTSO) {
case 8:
ldapr(Dst, MemSrc);
break;
default: LOGMAN_MSG_A("Unhandled LoadMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled LoadMemTSO size: {}", Op->Size);
}
nop();
}
@@ -660,7 +668,7 @@ DEF_OP(LoadMemTSO) {
case 8:
ldar(Dst, MemSrc);
break;
default: LOGMAN_MSG_A("Unhandled LoadMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled LoadMemTSO size: {}", Op->Size);
}
nop();
}
@@ -681,7 +689,7 @@ DEF_OP(LoadMemTSO) {
case 16:
ldr(Dst, MemSrc);
break;
default: LOGMAN_MSG_A("Unhandled LoadMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled LoadMemTSO size: {}", Op->Size);
}
dmb(InnerShareable, BarrierAll);
}
@@ -708,7 +716,7 @@ DEF_OP(StoreMem) {
case 8:
str(GetReg<RA_64>(Op->Header.Args[1].ID()), MemSrc);
break;
default: LOGMAN_MSG_A("Unhandled StoreMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled StoreMem size: {}", Op->Size);
}
}
else {
@@ -729,7 +737,7 @@ DEF_OP(StoreMem) {
case 16:
str(Src, MemSrc);
break;
default: LOGMAN_MSG_A("Unhandled StoreMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled StoreMem size: {}", Op->Size);
}
}
}
@@ -739,7 +747,7 @@ DEF_OP(StoreMemTSO) {
auto MemSrc = MemOperand(GetReg<RA_64>(Op->Header.Args[0].ID()));
if (!Op->Offset.IsInvalid()) {
LOGMAN_MSG_A("StoreMemTSO: No offset allowed");
LOGMAN_MSG_A_FMT("StoreMemTSO: No offset allowed");
}
if (Op->Class == FEXCore::IR::GPRClass) {
@@ -759,7 +767,7 @@ DEF_OP(StoreMemTSO) {
case 8:
stlr(GetReg<RA_64>(Op->Header.Args[1].ID()), MemSrc);
break;
default: LOGMAN_MSG_A("Unhandled StoreMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled StoreMemTSO size: {}", Op->Size);
}
nop();
}
@@ -783,7 +791,7 @@ DEF_OP(StoreMemTSO) {
case 16:
str(Src, MemSrc);
break;
default: LOGMAN_MSG_A("Unhandled StoreMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled StoreMemTSO size: {}", Op->Size);
}
dmb(InnerShareable, BarrierAll);
}
@@ -795,7 +803,7 @@ DEF_OP(ParanoidLoadMemTSO) {
auto MemSrc = MemOperand(GetReg<RA_64>(Op->Header.Args[0].ID()));
if (!Op->Offset.IsInvalid()) {
LOGMAN_MSG_A("LoadMemTSO: No offset allowed");
LOGMAN_MSG_A_FMT("ParanoidLoadMemTSO: No offset allowed");
}
if (Op->Class == FEXCore::IR::GPRClass) {
@@ -806,7 +814,6 @@ DEF_OP(ParanoidLoadMemTSO) {
}
else {
auto Dst = GetReg<RA_64>(Node);
nop();
switch (Op->Size) {
case 2:
ldarh(Dst, MemSrc);
@@ -817,31 +824,24 @@ DEF_OP(ParanoidLoadMemTSO) {
case 8:
ldar(Dst, MemSrc);
break;
default: LOGMAN_MSG_A("Unhandled LoadMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled ParanoidLoadMemTSO size: {}", Op->Size);
}
nop();
}
}
else {
auto Dst = GetDst(Node);
switch (Op->Size) {
case 2:
nop();
ldarh(TMP1, MemSrc);
nop();
fmov(Dst, TMP1);
ldarh(TMP1.W(), MemSrc);
fmov(Dst.H(), TMP1.W());
break;
case 4:
nop();
ldar(TMP1.W(), MemSrc);
nop();
fmov(Dst, TMP1);
fmov(Dst.S(), TMP1.W());
break;
case 8:
nop();
ldar(TMP1, MemSrc);
nop();
fmov(Dst, TMP1);
fmov(Dst.D(), TMP1);
break;
case 16:
nop();
@@ -850,7 +850,7 @@ DEF_OP(ParanoidLoadMemTSO) {
mov(Dst.V2D(), 0, TMP1);
mov(Dst.V2D(), 1, TMP2);
break;
default: LOGMAN_MSG_A("Unhandled LoadMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled ParanoidLoadMemTSO size: {}", Op->Size);
}
}
}
@@ -860,7 +860,7 @@ DEF_OP(ParanoidStoreMemTSO) {
auto MemSrc = MemOperand(GetReg<RA_64>(Op->Header.Args[0].ID()));
if (!Op->Offset.IsInvalid()) {
LOGMAN_MSG_A("StoreMemTSO: No offset allowed");
LOGMAN_MSG_A_FMT("ParanoidStoreMemTSO: No offset allowed");
}
if (Op->Class == FEXCore::IR::GPRClass) {
@@ -869,7 +869,6 @@ DEF_OP(ParanoidStoreMemTSO) {
stlrb(GetReg<RA_64>(Op->Header.Args[1].ID()), MemSrc);
}
else {
nop();
switch (Op->Size) {
case 2:
stlrh(GetReg<RA_64>(Op->Header.Args[1].ID()), MemSrc);
@@ -880,37 +879,30 @@ DEF_OP(ParanoidStoreMemTSO) {
case 8:
stlr(GetReg<RA_64>(Op->Header.Args[1].ID()), MemSrc);
break;
default: LOGMAN_MSG_A("Unhandled StoreMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled ParanoidStoreMemTSO size: {}", Op->Size);
}
nop();
}
}
else {
auto Src = GetSrc(Op->Header.Args[1].ID());
if (Op->Size == 1) {
// 8bit load is always aligned to natural alignment
mov(TMP1, Src.V4S(), 0);
mov(TMP1.W(), Src.V16B(), 0);
stlrb(TMP1, MemSrc);
}
else {
switch (Op->Size) {
case 2:
mov(TMP1, Src.V4S(), 0);
nop();
mov(TMP1.W(), Src.V8H(), 0);
stlrh(TMP1, MemSrc);
nop();
break;
case 4:
mov(TMP1, Src.V4S(), 0);
nop();
mov(TMP1.W(), Src.V4S(), 0);
stlr(TMP1.W(), MemSrc);
nop();
break;
case 8:
mov(TMP1, Src.V2D(), 0);
nop();
stlr(TMP1, MemSrc);
nop();
break;
case 16: {
// Move vector to GPRs
@@ -927,18 +919,18 @@ DEF_OP(ParanoidStoreMemTSO) {
cbnz(TMP3, &B); // < Overwritten with DMB
break;
}
default: LOGMAN_MSG_A("Unhandled StoreMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled ParanoidStoreMemTSO size: {}", Op->Size);
}
}
}
}
DEF_OP(VLoadMemElement) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(VStoreMemElement) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(CacheLineClear) {
@@ -8,11 +8,11 @@ $end_info$
namespace FEXCore::CPU {
static void PrintValue(uint64_t Value) {
LogMan::Msg::D("Value: 0x%lx", Value);
LogMan::Msg::DFmt("Value: 0x{:x}", Value);
}
static void PrintVectorValue(uint64_t Value, uint64_t ValueUpper) {
LogMan::Msg::D("Value: 0x%016lx'%016lx", ValueUpper, Value);
LogMan::Msg::DFmt("Value: 0x{:016x}'{:016x}", ValueUpper, Value);
}
using namespace vixl;
@@ -31,7 +31,7 @@ DEF_OP(Fence) {
case IR::Fence_Store.Val:
dmb(FullSystem, BarrierWrites);
break;
default: LOGMAN_MSG_A("Unknown Fence: %d", Op->Fence); break;
default: LOGMAN_MSG_A_FMT("Unknown Fence: {}", Op->Fence); break;
}
}
@@ -69,7 +69,7 @@ DEF_OP(Break) {
br(TMP1);
break;
}
default: LOGMAN_MSG_A("Unknown Break reason: %d", Op->Reason);
default: LOGMAN_MSG_A_FMT("Unknown Break reason: {}", Op->Reason);
}
}
@@ -26,7 +26,7 @@ DEF_OP(ExtractElementPair) {
mov (GetReg<RA_64>(Node), Regs[Op->Element]);
break;
}
default: LOGMAN_MSG_A("Unknown Size"); break;
default: LOGMAN_MSG_A_FMT("Unknown Size"); break;
}
}
@@ -52,7 +52,7 @@ DEF_OP(CreateElementPair) {
RegTmp = x0;
break;
}
default: LOGMAN_MSG_A("Unknown Size"); break;
default: LOGMAN_MSG_A_FMT("Unknown Size"); break;
}
if (Dst.first.GetCode() != RegSecond.GetCode()) {
+132 -134
View File
@@ -22,7 +22,7 @@ DEF_OP(VectorZero) {
eor(GetDst(Node).V16B(), GetDst(Node).V16B(), GetDst(Node).V16B());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", OpSize); break;
}
}
@@ -43,47 +43,47 @@ DEF_OP(VectorImm) {
}
DEF_OP(CreateVector2) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(CreateVector4) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(SplatVector2) {
auto Op = IROp->C<IR::IROp_SplatVector2>();
auto Op = IROp->C<IR::IROp_SplatVector2>();
uint8_t OpSize = IROp->Size;
LOGMAN_THROW_A(OpSize <= 16, "Can't handle a vector of size: %d", OpSize);
LOGMAN_THROW_A_FMT(OpSize <= 16, "Can't handle a vector of size: {}", OpSize);
uint8_t ElementSize = OpSize / 2;
uint8_t ElementSize = OpSize / 2;
switch (ElementSize) {
case 4:
dup(GetDst(Node).V4S(), GetSrc(Op->Header.Args[0].ID()).V4S(), 0);
break;
case 8:
dup(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), 0);
break;
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.Size); break;
}
switch (ElementSize) {
case 4:
dup(GetDst(Node).V4S(), GetSrc(Op->Header.Args[0].ID()).V4S(), 0);
break;
case 8:
dup(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), 0);
break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.Size); break;
}
}
DEF_OP(SplatVector4) {
auto Op = IROp->C<IR::IROp_SplatVector4>();
auto Op = IROp->C<IR::IROp_SplatVector4>();
uint8_t OpSize = IROp->Size;
LOGMAN_THROW_A(OpSize <= 16, "Can't handle a vector of size: %d", OpSize);
LOGMAN_THROW_A_FMT(OpSize <= 16, "Can't handle a vector of size: {}", OpSize);
uint8_t ElementSize = OpSize / 4;
uint8_t ElementSize = OpSize / 4;
switch (ElementSize) {
case 4:
dup(GetDst(Node).V4S(), GetSrc(Op->Header.Args[0].ID()).V4S(), 0);
break;
case 8:
dup(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), 0);
break;
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.Size); break;
}
switch (ElementSize) {
case 4:
dup(GetDst(Node).V4S(), GetSrc(Op->Header.Args[0].ID()).V4S(), 0);
break;
case 8:
dup(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), 0);
break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.Size); break;
}
}
DEF_OP(VMov) {
@@ -118,7 +118,7 @@ DEF_OP(VMov) {
mov(GetDst(Node).V16B(), GetSrc(Op->Header.Args[0].ID()).V16B());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", OpSize); break;
}
}
@@ -161,7 +161,7 @@ DEF_OP(VAdd) {
add(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -184,7 +184,7 @@ DEF_OP(VSub) {
sub(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -207,7 +207,7 @@ DEF_OP(VUQAdd) {
uqadd(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -230,7 +230,7 @@ DEF_OP(VUQSub) {
uqsub(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -253,7 +253,7 @@ DEF_OP(VSQAdd) {
sqadd(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -276,7 +276,7 @@ DEF_OP(VSQSub) {
sqsub(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -298,7 +298,7 @@ DEF_OP(VAddP) {
addp(GetDst(Node).V2S(), GetSrc(Op->Header.Args[0].ID()).V2S(), GetSrc(Op->Header.Args[1].ID()).V2S());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -319,7 +319,7 @@ DEF_OP(VAddP) {
addp(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -338,7 +338,7 @@ DEF_OP(VAddV) {
case 8:
addp(GetDst(Node).VCast(OpSize * 8, 1), GetSrc(Op->Header.Args[0].ID()).VCast(OpSize * 8, Elements));
break;
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -353,7 +353,7 @@ DEF_OP(VUMinV) {
case 4:
uminv(GetDst(Node).VCast(Op->Header.ElementSize * 8, 1), GetSrc(Op->Header.Args[0].ID()).VCast(OpSize * 8, Elements));
break;
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -368,7 +368,7 @@ DEF_OP(VURAvg) {
urhadd(GetDst(Node).V8H(), GetSrc(Op->Header.Args[0].ID()).V8H(), GetSrc(Op->Header.Args[1].ID()).V8H());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -383,7 +383,7 @@ DEF_OP(VAbs) {
abs(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -395,7 +395,7 @@ DEF_OP(VAbs) {
case 8:
abs(GetDst(Node).VCast(OpSize * 8, Elements), GetSrc(Op->Header.Args[0].ID()).VCast(OpSize * 8, Elements));
break;
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -410,7 +410,7 @@ DEF_OP(VPopcount) {
cnt(GetDst(Node).V8B(), GetSrc(Op->Header.Args[0].ID()).V8B());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -419,7 +419,7 @@ DEF_OP(VPopcount) {
case 1:
cnt(GetDst(Node).V16B(), GetSrc(Op->Header.Args[0].ID()).V16B());
break;
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -438,7 +438,7 @@ DEF_OP(VFAdd) {
fadd(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), GetSrc(Op->Header.Args[1].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -452,7 +452,7 @@ DEF_OP(VFAdd) {
fadd(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -468,7 +468,7 @@ DEF_OP(VFAddP) {
faddp(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -486,7 +486,7 @@ DEF_OP(VFSub) {
fsub(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), GetSrc(Op->Header.Args[1].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -500,7 +500,7 @@ DEF_OP(VFSub) {
fsub(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -519,7 +519,7 @@ DEF_OP(VFMul) {
fmul(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), GetSrc(Op->Header.Args[1].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -533,7 +533,7 @@ DEF_OP(VFMul) {
fmul(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -552,7 +552,7 @@ DEF_OP(VFDiv) {
fdiv(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), GetSrc(Op->Header.Args[1].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -566,7 +566,7 @@ DEF_OP(VFDiv) {
fdiv(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -587,7 +587,7 @@ DEF_OP(VFMin) {
fcsel(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), GetSrc(Op->Header.Args[1].ID()).D(), Condition::mi);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -607,7 +607,7 @@ DEF_OP(VFMin) {
mov(GetDst(Node).V2D(), VTMP2.V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -628,7 +628,7 @@ DEF_OP(VFMax) {
fcsel(GetDst(Node).D(), GetSrc(Op->Header.Args[1].ID()).D(), GetSrc(Op->Header.Args[0].ID()).D(), Condition::mi);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -648,7 +648,7 @@ DEF_OP(VFMax) {
mov(GetDst(Node).V2D(), VTMP2.V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -669,7 +669,7 @@ DEF_OP(VFRecp) {
fdiv(GetDst(Node).D(), VTMP1.D(), GetSrc(Op->Header.Args[0].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -685,7 +685,7 @@ DEF_OP(VFRecp) {
fdiv(GetDst(Node).V2D(), VTMP1.V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -704,7 +704,7 @@ DEF_OP(VFSqrt) {
fsqrt(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -718,7 +718,7 @@ DEF_OP(VFSqrt) {
fsqrt(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -741,7 +741,7 @@ DEF_OP(VFRSqrt) {
fdiv(GetDst(Node).D(), VTMP1.D(), VTMP2.D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -759,14 +759,13 @@ DEF_OP(VFRSqrt) {
fdiv(GetDst(Node).V2D(), VTMP1.V2D(), VTMP2.V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
DEF_OP(VNeg) {
auto Op = IROp->C<IR::IROp_VNeg>();
uint8_t OpSize = IROp->Size;
switch (Op->Header.ElementSize) {
case 1:
neg(GetDst(Node).V16B(), GetSrc(Op->Header.Args[0].ID()).V16B());
@@ -780,13 +779,12 @@ DEF_OP(VNeg) {
case 8:
neg(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
default: LOGMAN_MSG_A("Unsupported Not size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported VNeg size: {}", IROp->Size);
}
}
DEF_OP(VFNeg) {
auto Op = IROp->C<IR::IROp_VFNeg>();
uint8_t OpSize = IROp->Size;
switch (Op->Header.ElementSize) {
case 4:
fneg(GetDst(Node).V4S(), GetSrc(Op->Header.Args[0].ID()).V4S());
@@ -794,7 +792,7 @@ DEF_OP(VFNeg) {
case 8:
fneg(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
default: LOGMAN_MSG_A("Unsupported Not size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported VFNeg size: {}", IROp->Size);
}
}
@@ -825,7 +823,7 @@ DEF_OP(VUMin) {
mov(GetDst(Node).V2D(), VTMP2.V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -851,7 +849,7 @@ DEF_OP(VSMin) {
mov(GetDst(Node).V2D(), VTMP2.V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -877,7 +875,7 @@ DEF_OP(VUMax) {
mov(GetDst(Node).V2D(), VTMP2.V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -903,7 +901,7 @@ DEF_OP(VSMax) {
mov(GetDst(Node).V2D(), VTMP2.V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -924,7 +922,7 @@ DEF_OP(VZip) {
zip1(GetDst(Node).V2S(), GetSrc(Op->Header.Args[0].ID()).V2S(), GetSrc(Op->Header.Args[1].ID()).V2S());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -945,7 +943,7 @@ DEF_OP(VZip) {
zip1(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -967,7 +965,7 @@ DEF_OP(VZip2) {
zip2(GetDst(Node).V2S(), GetSrc(Op->Header.Args[0].ID()).V2S(), GetSrc(Op->Header.Args[1].ID()).V2S());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -988,7 +986,7 @@ DEF_OP(VZip2) {
zip2(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1010,7 +1008,7 @@ DEF_OP(VUnZip) {
uzp1(GetDst(Node).V2S(), GetSrc(Op->Header.Args[0].ID()).V2S(), GetSrc(Op->Header.Args[1].ID()).V2S());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1031,7 +1029,7 @@ DEF_OP(VUnZip) {
uzp1(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1053,7 +1051,7 @@ DEF_OP(VUnZip2) {
uzp2(GetDst(Node).V2S(), GetSrc(Op->Header.Args[0].ID()).V2S(), GetSrc(Op->Header.Args[1].ID()).V2S());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1074,7 +1072,7 @@ DEF_OP(VUnZip2) {
uzp2(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1107,7 +1105,7 @@ DEF_OP(VCMPEQ) {
cmeq(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), GetSrc(Op->Header.Args[1].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1129,7 +1127,7 @@ DEF_OP(VCMPEQ) {
cmeq(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1148,7 +1146,7 @@ DEF_OP(VCMPEQZ) {
cmeq(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), 0);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1170,7 +1168,7 @@ DEF_OP(VCMPEQZ) {
cmeq(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), 0);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1189,7 +1187,7 @@ DEF_OP(VCMPGT) {
cmgt(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), GetSrc(Op->Header.Args[1].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1211,7 +1209,7 @@ DEF_OP(VCMPGT) {
cmgt(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1230,7 +1228,7 @@ DEF_OP(VCMPGTZ) {
cmgt(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), 0);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1252,7 +1250,7 @@ DEF_OP(VCMPGTZ) {
cmgt(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), 0);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1271,7 +1269,7 @@ DEF_OP(VCMPLTZ) {
cmlt(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), 0);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1293,7 +1291,7 @@ DEF_OP(VCMPLTZ) {
cmlt(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), 0);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1312,7 +1310,7 @@ DEF_OP(VFCMPEQ) {
fcmeq(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), GetSrc(Op->Header.Args[1].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1330,7 +1328,7 @@ DEF_OP(VFCMPEQ) {
fcmeq(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1349,7 +1347,7 @@ DEF_OP(VFCMPNEQ) {
fcmeq(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), GetSrc(Op->Header.Args[1].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
mvn(GetDst(Node).V8B(), GetDst(Node).V8B());
}
@@ -1368,7 +1366,7 @@ DEF_OP(VFCMPNEQ) {
fcmeq(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
mvn(GetDst(Node).V16B(), GetDst(Node).V16B());
}
@@ -1388,7 +1386,7 @@ DEF_OP(VFCMPLT) {
fcmgt(GetDst(Node).D(), GetSrc(Op->Header.Args[1].ID()).D(), GetSrc(Op->Header.Args[0].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1406,7 +1404,7 @@ DEF_OP(VFCMPLT) {
fcmgt(GetDst(Node).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1425,7 +1423,7 @@ DEF_OP(VFCMPGT) {
fcmgt(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), GetSrc(Op->Header.Args[1].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1443,7 +1441,7 @@ DEF_OP(VFCMPGT) {
fcmgt(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1462,7 +1460,7 @@ DEF_OP(VFCMPLE) {
fcmge(GetDst(Node).D(), GetSrc(Op->Header.Args[1].ID()).D(), GetSrc(Op->Header.Args[0].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1480,7 +1478,7 @@ DEF_OP(VFCMPLE) {
fcmge(GetDst(Node).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1503,7 +1501,7 @@ DEF_OP(VFCMPORD) {
orr(GetDst(Node).V8B(), VTMP1.V8B(), VTMP2.V8B());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1527,7 +1525,7 @@ DEF_OP(VFCMPORD) {
orr(GetDst(Node).V16B(), VTMP1.V16B(), VTMP2.V16B());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1552,7 +1550,7 @@ DEF_OP(VFCMPUNO) {
mvn(GetDst(Node).V8B(), GetDst(Node).V8B());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1579,21 +1577,21 @@ DEF_OP(VFCMPUNO) {
mvn(GetDst(Node).V16B(), GetDst(Node).V16B());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
DEF_OP(VUShl) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(VUShr) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(VSShr) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(VUShlS) {
@@ -1620,7 +1618,7 @@ DEF_OP(VUShlS) {
ushl(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), VTMP1.V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1652,7 +1650,7 @@ DEF_OP(VUShrS) {
ushl(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), VTMP1.V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1684,7 +1682,7 @@ DEF_OP(VSShrS) {
sshl(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), VTMP1.V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1715,7 +1713,7 @@ DEF_OP(VInsElement) {
mov(reg.V2D(), Op->DestIdx, GetSrc(Op->Header.Args[1].ID()).V2D(), Op->SrcIdx);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
if (GetDst(Node).GetCode() != reg.GetCode()) {
@@ -1750,7 +1748,7 @@ DEF_OP(VInsScalarElement) {
mov(reg.V2D(), Op->DestIdx, GetSrc(Op->Header.Args[1].ID()).V2D(), 0);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
if (GetDst(Node).GetCode() != reg.GetCode()) {
@@ -1774,7 +1772,7 @@ DEF_OP(VExtractElement) {
case 8:
mov(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).V2D(), Op->Index);
break;
default: LOGMAN_MSG_A("Unhandled ExtractElementSize: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled VExtractElement element size: {}", OpSize);
}
}
@@ -1793,7 +1791,7 @@ DEF_OP(VDupElement) {
case 8:
dup(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), Op->Index);
break;
default: LOGMAN_MSG_A("Unhandled DupElementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unhandled VDupElement element size: {}", Op->Header.ElementSize);
}
}
@@ -1908,7 +1906,7 @@ DEF_OP(VUShrI) {
ushr(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), Op->BitShift);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1933,7 +1931,7 @@ DEF_OP(VSShrI) {
sshr(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), std::min((uint8_t)(Op->Header.ElementSize * 8 - 1), Op->BitShift));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1961,7 +1959,7 @@ DEF_OP(VShlI) {
shl(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), Op->BitShift);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1982,7 +1980,7 @@ DEF_OP(VUShrNI) {
shrn(GetDst(Node).V2S(), GetSrc(Op->Header.Args[0].ID()).V2D(), Op->BitShift);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -2002,7 +2000,7 @@ DEF_OP(VUShrNI2) {
shrn2(VTMP1.V4S(), GetSrc(Op->Header.Args[1].ID()).V2D(), Op->BitShift);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
mov(GetDst(Node), VTMP1);
@@ -2025,7 +2023,7 @@ DEF_OP(VSXTL) {
case 8:
sxtl(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2S());
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
@@ -2041,7 +2039,7 @@ DEF_OP(VSXTL2) {
case 8:
sxtl2(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V4S());
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
@@ -2057,7 +2055,7 @@ DEF_OP(VUXTL) {
case 8:
uxtl(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2S());
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
@@ -2073,7 +2071,7 @@ DEF_OP(VUXTL2) {
case 8:
uxtl2(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V4S());
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
@@ -2089,7 +2087,7 @@ DEF_OP(VSQXTN) {
case 4:
sqxtn(GetDst(Node).V2S(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
@@ -2111,7 +2109,7 @@ DEF_OP(VSQXTN2) {
sqxtn(VTMP2.V2S(), GetSrc(Op->Header.Args[1].ID()).V2D());
ins(VTMP1.V4S(), 1, VTMP2.V4S(), 0);
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
else {
@@ -2125,7 +2123,7 @@ DEF_OP(VSQXTN2) {
case 4:
sqxtn2(VTMP1.V4S(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
mov(GetDst(Node), VTMP1);
@@ -2143,7 +2141,7 @@ DEF_OP(VSQXTUN) {
case 4:
sqxtun(GetDst(Node).V2S(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
@@ -2165,7 +2163,7 @@ DEF_OP(VSQXTUN2) {
sqxtun(VTMP2.V2S(), GetSrc(Op->Header.Args[1].ID()).V2D());
ins(VTMP1.V4S(), 1, VTMP2.V4S(), 0);
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
else {
@@ -2179,7 +2177,7 @@ DEF_OP(VSQXTUN2) {
case 4:
sqxtun2(VTMP1.V4S(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
mov(GetDst(Node), VTMP1);
@@ -2204,7 +2202,7 @@ DEF_OP(VMul) {
mul(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -2223,7 +2221,7 @@ DEF_OP(VUMull) {
umull(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2S(), GetSrc(Op->Header.Args[1].ID()).V2S());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize >> 1); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize >> 1); break;
}
}
@@ -2242,7 +2240,7 @@ DEF_OP(VSMull) {
smull(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2S(), GetSrc(Op->Header.Args[1].ID()).V2S());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize >> 1); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize >> 1); break;
}
}
@@ -2261,7 +2259,7 @@ DEF_OP(VUMull2) {
umull2(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V4S(), GetSrc(Op->Header.Args[1].ID()).V4S());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize >> 1); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize >> 1); break;
}
}
@@ -2280,7 +2278,7 @@ DEF_OP(VSMull2) {
smull2(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V4S(), GetSrc(Op->Header.Args[1].ID()).V4S());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize >> 1); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize >> 1); break;
}
}
@@ -2299,7 +2297,7 @@ DEF_OP(VUABDL) {
uabdl(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2S(), GetSrc(Op->Header.Args[1].ID()).V2S());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize >> 1); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize >> 1); break;
}
}
@@ -2316,7 +2314,7 @@ DEF_OP(VTBL1) {
tbl(GetDst(Node).V16B(), GetSrc(Op->Header.Args[0].ID()).V16B(), GetSrc(Op->Header.Args[1].ID()).V16B());
break;
}
default: LOGMAN_MSG_A("Unknown OpSize: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown OpSize: {}", OpSize); break;
}
}
+34 -36
View File
@@ -20,7 +20,7 @@ DEF_OP(TruncElementPair) {
mov(Dst.second, Src.second);
break;
}
default: LOGMAN_MSG_A("Unhandled Truncation size: %d", Op->Size); break;
default: LOGMAN_MSG_A_FMT("Unhandled Truncation size: {}", Op->Size); break;
}
}
@@ -70,7 +70,7 @@ DEF_OP(Add) {
case 8:
add(rax, Const);
break;
default: LOGMAN_MSG_A("Unhandled Add size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled Add size: {}", OpSize);
break;
}
} else {
@@ -81,7 +81,7 @@ DEF_OP(Add) {
case 8:
add(rax, GetSrc<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unhandled Add size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled Add size: {}", OpSize);
break;
}
}
@@ -103,7 +103,7 @@ DEF_OP(Sub) {
case 8:
sub(rax, Const);
break;
default: LOGMAN_MSG_A("Unhandled Sub size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled Sub size: {}", OpSize);
break;
}
} else {
@@ -114,7 +114,7 @@ DEF_OP(Sub) {
case 8:
sub(rax, GetSrc<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unhandled Sub size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled Sub size: {}", OpSize);
break;
}
}
@@ -136,7 +136,7 @@ DEF_OP(Neg) {
Src = GetSrc<RA_64>(Op->Header.Args[0].ID());
Dst = GetDst<RA_64>(Node);
break;
default: LOGMAN_MSG_A("Unhandled Neg size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled Neg size: {}", OpSize);
break;
}
mov(Dst, Src);
@@ -160,7 +160,7 @@ DEF_OP(Mul) {
imul(rax, GetSrc<RA_64>(Op->Header.Args[1].ID()));
mov(Dst, rax);
break;
default: LOGMAN_MSG_A("Unknown Sext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown Mul size: {}", OpSize);
}
}
@@ -179,7 +179,7 @@ DEF_OP(UMul) {
mul(GetSrc<RA_64>(Op->Header.Args[1].ID()));
mov(GetDst<RA_64>(Node), rax);
break;
default: LOGMAN_MSG_A("Unknown Sext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown UMul size: {}", OpSize);
}
}
@@ -218,7 +218,7 @@ DEF_OP(Div) {
mov(GetDst<RA_64>(Node), rax);
break;
}
default: LOGMAN_MSG_A("Unknown UDIV Size: %d", Size); break;
default: LOGMAN_MSG_A_FMT("Unknown DIV Size: {}", Size); break;
}
}
@@ -261,7 +261,7 @@ DEF_OP(UDiv) {
mov(GetDst<RA_64>(Node), rax);
break;
}
default: LOGMAN_MSG_A("Unknown UDIV OpSize: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown UDIV OpSize: {}", OpSize); break;
}
}
@@ -298,7 +298,7 @@ DEF_OP(Rem) {
mov(GetDst<RA_64>(Node), rdx);
break;
}
default: LOGMAN_MSG_A("Unknown UDIV Size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Rem Size: {}", OpSize); break;
}
}
@@ -341,7 +341,7 @@ DEF_OP(URem) {
mov(GetDst<RA_64>(Node), rdx);
break;
}
default: LOGMAN_MSG_A("Unknown UDIV OpSize: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown URem OpSize: {}", OpSize); break;
}
}
@@ -360,7 +360,7 @@ DEF_OP(MulH) {
imul(GetSrc<RA_64>(Op->Header.Args[1].ID()));
mov(GetDst<RA_64>(Node), rdx);
break;
default: LOGMAN_MSG_A("Unknown Sext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown MulH size: {}", OpSize);
}
}
@@ -379,7 +379,7 @@ DEF_OP(UMulH) {
mul(GetSrc<RA_64>(Op->Header.Args[1].ID()));
mov(GetDst<RA_64>(Node), rdx);
break;
default: LOGMAN_MSG_A("Unknown Sext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown UMulH size: {}", OpSize);
}
}
@@ -441,7 +441,7 @@ DEF_OP(Lshl) {
mov(GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
shl(GetDst<RA_64>(Node), Const);
break;
default: LOGMAN_MSG_A("Unknown LSHL Size: %d\n", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown LSHL Size: {}\n", OpSize); break;
};
} else {
mov(rcx, GetSrc<RA_64>(Op->Header.Args[1].ID()));
@@ -456,7 +456,7 @@ DEF_OP(Lshl) {
mov(GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
shl(GetDst<RA_64>(Node), cl);
break;
default: LOGMAN_MSG_A("Unknown LSHL Size: %d\n", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown LSHL Size: {}\n", OpSize); break;
};
}
}
@@ -488,7 +488,7 @@ DEF_OP(Lshr) {
mov(GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
shr(GetDst<RA_64>(Node), Const);
break;
default: LOGMAN_MSG_A("Unknown Size: %d\n", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Size: {}\n", OpSize); break;
};
} else {
@@ -512,7 +512,7 @@ DEF_OP(Lshr) {
mov(GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
shr(GetDst<RA_64>(Node), cl);
break;
default: LOGMAN_MSG_A("Unknown Size: %d\n", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Size: {}\n", OpSize); break;
};
}
}
@@ -546,7 +546,7 @@ DEF_OP(Ashr) {
mov(GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
sar(GetDst<RA_64>(Node), Const);
break;
default: LOGMAN_MSG_A("Unknown ASHR Size: %d\n", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown ASHR Size: {}\n", OpSize); break;
};
} else {
@@ -571,7 +571,7 @@ DEF_OP(Ashr) {
mov(GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
sar(GetDst<RA_64>(Node), cl);
break;
default: LOGMAN_MSG_A("Unknown ASHR Size: %d\n", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown ASHR Size: {}\n", OpSize); break;
};
}
}
@@ -596,7 +596,7 @@ DEF_OP(Ror) {
ror(rax, Const);
break;
}
default: LOGMAN_MSG_A("Unknown ROR Size: %d\n", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown ROR Size: {}\n", OpSize); break;
}
} else {
mov (rcx, GetSrc<RA_64>(Op->Header.Args[1].ID()));
@@ -612,7 +612,7 @@ DEF_OP(Ror) {
ror(rax, cl);
break;
}
default: LOGMAN_MSG_A("Unknown ROR Size: %d\n", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown ROR Size: {}\n", OpSize); break;
}
}
mov(GetDst<RA_64>(Node), rax);
@@ -668,7 +668,7 @@ DEF_OP(LDiv) {
mov(GetDst<RA_64>(Node), rax);
break;
}
default: LOGMAN_MSG_A("Unknown LDIV OpSize: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown LDIV OpSize: {}", OpSize); break;
}
}
@@ -700,7 +700,7 @@ DEF_OP(LUDiv) {
mov(GetDst<RA_64>(Node), rax);
break;
}
default: LOGMAN_MSG_A("Unknown LUDIV OpSize: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown LUDIV OpSize: {}", OpSize); break;
}
}
@@ -732,7 +732,7 @@ DEF_OP(LRem) {
mov(GetDst<RA_64>(Node), rdx);
break;
}
default: LOGMAN_MSG_A("Unknown LREM OpSize: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown LREM OpSize: {}", OpSize); break;
}
}
@@ -764,7 +764,7 @@ DEF_OP(LURem) {
mov(GetDst<RA_64>(Node), rdx);
break;
}
default: LOGMAN_MSG_A("Unknown LUDIV OpSize: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown LUREM OpSize: {}", OpSize); break;
}
}
@@ -829,7 +829,7 @@ DEF_OP(FindMSB) {
case 8:
bsr(GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unknown OpSize: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown FindMSB OpSize: {}", OpSize);
}
}
@@ -853,7 +853,7 @@ DEF_OP(FindTrailingZeros) {
mov(rax, 0x40);
cmovz(GetDst<RA_64>(Node), rax);
break;
default: LOGMAN_MSG_A("Unknown size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown FindTrailingZeros size: {}", OpSize); break;
}
}
@@ -876,7 +876,7 @@ DEF_OP(CountLeadingZeroes) {
lzcnt(GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown CountLeadingZeros size: {}", OpSize); break;
}
}
else {
@@ -915,7 +915,7 @@ DEF_OP(CountLeadingZeroes) {
mov(GetDst<RA_64>(Node), rax);
break;
}
default: LOGMAN_MSG_A("Unknown size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown CountLeadingZeros size: {}", OpSize); break;
}
}
}
@@ -937,7 +937,7 @@ DEF_OP(Rev) {
mov (GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
bswap(GetDst<RA_64>(Node).cvt64());
break;
default: LOGMAN_MSG_A("Unknown REV size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown REV size: {}", OpSize); break;
}
}
@@ -970,9 +970,7 @@ DEF_OP(Bfi) {
DEF_OP(Bfe) {
auto Op = IROp->C<IR::IROp_Bfe>();
uint8_t OpSize = IROp->Size;
LOGMAN_THROW_A(OpSize <= 8, "OpSize is too large for BFE: %d", OpSize);
LOGMAN_THROW_A_FMT(IROp->Size <= 8, "OpSize is too large for BFE: {}", IROp->Size);
auto Dst = GetDst<RA_64>(Node);
@@ -1073,7 +1071,7 @@ DEF_OP(Select) {
if (is_const_true || is_const_false) {
if (is_const_false != true || is_const_true != true || const_true != 1 || const_false != 0) {
LOGMAN_MSG_A("Select: Unsupported compare inline parameters");
LOGMAN_MSG_A_FMT("Select: Unsupported compare inline parameters");
}
(this->*SetCC)(al);
movzx(Dst, al);
@@ -1104,7 +1102,7 @@ DEF_OP(VExtractToGPR) {
pextrq(GetDst<RA_64>(Node), GetSrc(Op->Header.Args[0].ID()), Op->Idx);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -55,7 +55,7 @@ DEF_OP(CASPair) {
mov(Dst.second, rdx);
break;
}
default: LOGMAN_MSG_A("Unsupported: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported: {}", OpSize);
}
}
@@ -104,7 +104,7 @@ DEF_OP(CAS) {
mov (GetDst<RA_64>(Node), rax);
break;
}
default: LOGMAN_MSG_A("Unsupported: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported: {}", OpSize);
}
}
@@ -127,7 +127,7 @@ DEF_OP(AtomicAdd) {
case 8:
add(qword [MemReg], GetSrc<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unhandled AtomicAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicAdd size: {}", Op->Size);
}
}
@@ -149,7 +149,7 @@ DEF_OP(AtomicSub) {
case 8:
sub(qword [MemReg], GetSrc<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unhandled AtomicAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicAdd size: {}", Op->Size);
}
}
@@ -171,7 +171,7 @@ DEF_OP(AtomicAnd) {
case 8:
and_(qword [MemReg], GetSrc<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unhandled AtomicAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicAdd size: {}", Op->Size);
}
}
@@ -193,7 +193,7 @@ DEF_OP(AtomicOr) {
case 8:
or_(qword [MemReg], GetSrc<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unhandled AtomicAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicAdd size: {}", Op->Size);
}
}
@@ -215,7 +215,7 @@ DEF_OP(AtomicXor) {
case 8:
xor_(qword [MemReg], GetSrc<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unhandled AtomicAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicAdd size: {}", Op->Size);
}
}
@@ -246,7 +246,7 @@ DEF_OP(AtomicSwap) {
lock();
xchg(qword [MemReg], GetDst<RA_64>(Node));
break;
default: LOGMAN_MSG_A("Unhandled AtomicAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicSwap size: {}", Op->Size);
}
}
@@ -279,7 +279,7 @@ DEF_OP(AtomicFetchAdd) {
xadd(qword [MemReg], rcx);
mov(GetDst<RA_64>(Node), rcx);
break;
default: LOGMAN_MSG_A("Unhandled AtomicFetchAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicFetchAdd size: {}", Op->Size);
}
}
@@ -316,7 +316,7 @@ DEF_OP(AtomicFetchSub) {
xadd(qword [MemReg], rcx);
mov(GetDst<RA_64>(Node), rcx);
break;
default: LOGMAN_MSG_A("Unhandled AtomicFetchAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicFetchSub size: {}", Op->Size);
}
}
@@ -394,7 +394,7 @@ DEF_OP(AtomicFetchAnd) {
mov(GetDst<RA_64>(Node), TMP3.cvt64());
break;
}
default: LOGMAN_MSG_A("Unhandled AtomicFetchAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicFetchAnd size: {}", Op->Size);
}
}
@@ -471,7 +471,7 @@ DEF_OP(AtomicFetchOr) {
mov(GetDst<RA_64>(Node), TMP3.cvt64());
break;
}
default: LOGMAN_MSG_A("Unhandled AtomicFetchAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicFetchOr size: {}", Op->Size);
}
}
@@ -548,7 +548,83 @@ DEF_OP(AtomicFetchXor) {
mov(GetDst<RA_64>(Node), TMP3.cvt64());
break;
}
default: LOGMAN_MSG_A("Unhandled AtomicFetchAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicFetchXor size: {}", Op->Size);
}
}
DEF_OP(AtomicFetchNeg) {
auto Op = IROp->C<IR::IROp_AtomicFetchNeg>();
Xbyak::Reg MemReg = GetSrc<RA_64>(Op->Header.Args[0].ID());
switch (Op->Size) {
case 1: {
mov(TMP1.cvt8(), byte [MemReg]);
Label Loop;
L(Loop);
mov(TMP2.cvt8(), TMP1.cvt8());
mov(TMP3.cvt8(), TMP1.cvt8());
neg(TMP2.cvt8());
// Updates RAX with the value from memory
lock(); cmpxchg(byte [MemReg], TMP2.cvt8());
jne(Loop);
// Result is the previous value from memory, which is currently in TMP3
movzx(GetDst<RA_64>(Node), TMP3.cvt8());
break;
}
case 2: {
mov(TMP1.cvt16(), word [MemReg]);
Label Loop;
L(Loop);
mov(TMP2.cvt16(), TMP1.cvt16());
mov(TMP3.cvt16(), TMP1.cvt16());
neg(TMP2.cvt16());
// Updates RAX with the value from memory
lock(); cmpxchg(word [MemReg], TMP2.cvt16());
jne(Loop);
// Result is the previous value from memory, which is currently in TMP3
movzx(GetDst<RA_64>(Node), TMP3.cvt16());
break;
}
case 4: {
mov(TMP1.cvt32(), dword [MemReg]);
Label Loop;
L(Loop);
mov(TMP2.cvt32(), TMP1.cvt32());
mov(TMP3.cvt32(), TMP1.cvt32());
neg(TMP2.cvt32());
// Updates RAX with the value from memory
lock(); cmpxchg(dword [MemReg], TMP2.cvt32());
jne(Loop);
// Result is the previous value from memory, which is currently in TMP3
mov(GetDst<RA_32>(Node), TMP3.cvt32());
break;
}
case 8: {
mov(TMP1.cvt64(), qword [MemReg]);
Label Loop;
L(Loop);
mov(TMP2.cvt64(), TMP1.cvt64());
mov(TMP3.cvt64(), TMP1.cvt64());
neg(TMP2.cvt64());
// Updates RAX with the value from memory
lock(); cmpxchg(qword [MemReg], TMP2.cvt64());
jne(Loop);
// Result is the previous value from memory, which is currently in TMP3
mov(GetDst<RA_64>(Node), TMP3.cvt64());
break;
}
default: LOGMAN_MSG_A_FMT("Unhandled AtomicFetchNeg size: {}", Op->Size);
}
}
@@ -568,6 +644,7 @@ void X86JITCore::RegisterAtomicHandlers() {
REGISTER_OP(ATOMICFETCHAND, AtomicFetchAnd);
REGISTER_OP(ATOMICFETCHOR, AtomicFetchOr);
REGISTER_OP(ATOMICFETCHXOR, AtomicFetchXor);
REGISTER_OP(ATOMICFETCHNEG, AtomicFetchNeg);
#undef REGISTER_OP
}
}
@@ -4,6 +4,8 @@ tags: backend|x86-64
$end_info$
*/
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/JIT/x86_64/JITClass.h"
#include "Interface/IR/Passes/RegisterAllocationPass.h"
@@ -14,15 +16,15 @@ $end_info$
namespace FEXCore::CPU {
#define DEF_OP(x) void X86JITCore::Op_##x(FEXCore::IR::IROp_Header *IROp, uint32_t Node)
DEF_OP(GuestCallDirect) {
LogMan::Msg::D("Unimplemented");
LogMan::Msg::DFmt("Unimplemented");
}
DEF_OP(GuestCallIndirect) {
LogMan::Msg::D("Unimplemented");
LogMan::Msg::DFmt("Unimplemented");
}
DEF_OP(GuestReturn) {
LogMan::Msg::D("Unimplemented");
LogMan::Msg::DFmt("Unimplemented");
}
DEF_OP(SignalReturn) {
@@ -248,7 +250,7 @@ DEF_OP(Thunk) {
DEF_OP(ValidateCode) {
auto Op = IROp->C<IR::IROp_ValidateCode>();
uint8_t* OldCode = (uint8_t*)&Op->CodeOriginalLow;
const auto* OldCode = (const uint8_t*)&Op->CodeOriginalLow;
int len = Op->CodeLength;
int idx = 0;
@@ -256,20 +258,20 @@ DEF_OP(ValidateCode) {
mov(rax, Entry + Op->Offset);
mov(rbx, 1);
while (len >= 4) {
cmp(dword[rax + idx], *(uint32_t*)(OldCode + idx));
cmp(dword[rax + idx], *(const uint32_t*)(OldCode + idx));
cmovne(GetDst<RA_64>(Node), rbx);
len-=4;
idx+=4;
}
while (len >= 2) {
mov(rcx, *(uint16_t*)(OldCode + idx));
mov(rcx, *(const uint16_t*)(OldCode + idx));
cmp(word[rax + idx], cx);
cmovne(GetDst<RA_64>(Node), rbx);
len-=2;
idx+=2;
}
while (len >= 1) {
cmp(byte[rax + idx], *(uint8_t*)(OldCode + idx));
cmp(byte[rax + idx], *(const uint8_t*)(OldCode + idx));
cmovne(GetDst<RA_64>(Node), rbx);
len-=1;
idx+=1;
@@ -319,8 +321,9 @@ DEF_OP(CPUID) {
//
// Result: RAX, RDX. 4xi32
mov (rsi, GetSrc<RA_64>(Op->Header.Args[0].ID()));
mov (rdx, GetSrc<RA_64>(Op->Header.Args[1].ID()));
// rsi can be in the source registers, so copy argument to edx first
mov (edx, GetSrc<RA_32>(Op->Header.Args[1].ID()));
mov (esi, GetSrc<RA_32>(Op->Header.Args[0].ID()));
mov (rdi, reinterpret_cast<uint64_t>(&CTX->CPUID));
auto NumPush = RA64.size();
@@ -31,7 +31,7 @@ DEF_OP(VInsGPR) {
pinsrq(GetDst(Node), GetSrc<RA_64>(Op->Header.Args[1].ID()), Op->Index);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -52,7 +52,7 @@ DEF_OP(VCastFromGPR) {
case 8:
vmovq(GetDst(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()).cvt64());
break;
default: LOGMAN_MSG_A("Unknown castGPR element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown VCastFromGPR element size: {}", Op->Header.ElementSize);
}
}
@@ -91,7 +91,7 @@ DEF_OP(Float_FToF) {
cvtsd2ss(GetDst(Node), GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown FCVT sizes: 0x%x", Conv);
default: LOGMAN_MSG_A_FMT("Unknown Float_FToF sizes: 0x{:x}", Conv);
}
}
@@ -113,7 +113,7 @@ DEF_OP(Vector_SToF) {
cvtsi2sd(xmm15, rax);
movlhps(GetDst(Node), xmm15);
break;
default: LOGMAN_MSG_A("Unknown castGPR element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Vector_SToF element size: {}", Op->Header.ElementSize);
}
}
@@ -126,7 +126,7 @@ DEF_OP(Vector_FToZS) {
case 8:
cvttpd2dq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unknown castGPR element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Vector_FToZS element size: {}", Op->Header.ElementSize);
}
}
@@ -139,7 +139,7 @@ DEF_OP(Vector_FToS) {
case 8:
cvtpd2dq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unknown castGPR element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Vector_FToS element size: {}", Op->Header.ElementSize);
}
}
@@ -156,7 +156,7 @@ DEF_OP(Vector_FToF) {
cvtpd2ps(GetDst(Node), GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Conversion Type : 0%04x", Conv); break;
default: LOGMAN_MSG_A_FMT("Unknown Vector_FToF conversion type : 0x{:04x}", Conv); break;
}
}
+25 -18
View File
@@ -6,6 +6,7 @@ $end_info$
*/
#include "Interface/Context/Context.h"
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/Dispatcher/X86Dispatcher.h"
#include "Interface/Core/JIT/x86_64/JITClass.h"
@@ -36,7 +37,7 @@ CodeBuffer AllocateNewCodeBuffer(size_t Size) {
PROT_READ | PROT_WRITE | PROT_EXEC,
MAP_PRIVATE | MAP_ANONYMOUS,
-1, 0));
LOGMAN_THROW_A(Buffer.Ptr != reinterpret_cast<uint8_t*>(~0ULL), "Couldn't allocate code buffer");
LOGMAN_THROW_A_FMT(Buffer.Ptr != reinterpret_cast<uint8_t*>(~0ULL), "Couldn't allocate code buffer");
return Buffer;
}
@@ -54,9 +55,9 @@ void X86JITCore::CopyNecessaryDataForCompileThread(CPUBackend *Original) {
}
void X86JITCore::PushRegs() {
for (auto &Xmm : RAXMM_x) {
sub(rsp, 16);
movaps(ptr[rsp], Xmm);
sub(rsp, 16 * RAXMM_x.size());
for (size_t i = 0; i < RAXMM_x.size(); ++i) {
movaps(ptr[rsp + i * 16], RAXMM_x[i]);
}
for (auto &Reg : RA64)
@@ -75,17 +76,19 @@ void X86JITCore::PopRegs() {
for (uint32_t i = RA64.size(); i > 0; --i)
pop(RA64[i - 1]);
for (uint32_t i = RAXMM_x.size(); i > 0; --i) {
movaps(RAXMM_x[i - 1], ptr[rsp]);
add(rsp, 16);
for (size_t i = 0; i < RAXMM_x.size(); ++i) {
movaps(RAXMM_x[i], ptr[rsp + i * 16]);
}
add(rsp, 16 * RAXMM_x.size());
}
void X86JITCore::Op_Unhandled(FEXCore::IR::IROp_Header *IROp, uint32_t Node) {
FallbackInfo Info;
if (!InterpreterOps::GetFallbackHandler(IROp, &Info)) {
auto Name = FEXCore::IR::GetName(IROp->Op);
LOGMAN_MSG_A("Unhandled IR Op: %s", std::string(Name).c_str());
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
LOGMAN_MSG_A_FMT("Unhandled IR Op: {}", FEXCore::IR::GetName(IROp->Op));
#endif
} else {
switch(Info.ABI) {
case FABI_VOID_U16: {
@@ -282,8 +285,10 @@ void X86JITCore::Op_Unhandled(FEXCore::IR::IROp_Header *IROp, uint32_t Node) {
case FABI_UNKNOWN:
default:
auto Name = FEXCore::IR::GetName(IROp->Op);
LOGMAN_MSG_A("Unhandled IR Fallback abi: %s %d", std::string(Name).c_str(), Info.ABI);
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
LOGMAN_MSG_A_FMT("Unhandled IR Fallback ABI: {} {}", FEXCore::IR::GetName(IROp->Op), Info.ABI);
#endif
break;
}
}
}
@@ -343,12 +348,12 @@ X86JITCore::X86JITCore(FEXCore::Context::Context *ctx, FEXCore::Core::InternalTh
CTX->SignalDelegation->RegisterHostSignalHandler(SIGILL, [](FEXCore::Core::InternalThreadState *Thread, int Signal, void *info, void *ucontext) -> bool {
X86JITCore *Core = reinterpret_cast<X86JITCore*>(Thread->CPUBackend.get());
return Core->Dispatcher->HandleSIGILL(Signal, info, ucontext);
});
}, true);
CTX->SignalDelegation->RegisterHostSignalHandler(SignalDelegator::SIGNAL_FOR_PAUSE, [](FEXCore::Core::InternalThreadState *Thread, int Signal, void *info, void *ucontext) -> bool {
X86JITCore *Core = reinterpret_cast<X86JITCore*>(Thread->CPUBackend.get());
return Core->Dispatcher->HandleSignalPause(Signal, info, ucontext);
});
}, true);
auto GuestSignalHandler = [](FEXCore::Core::InternalThreadState *Thread, int Signal, void *info, void *ucontext, GuestSigAction *GuestAction, stack_t *GuestStack) -> bool {
X86JITCore *Core = reinterpret_cast<X86JITCore*>(Thread->CPUBackend.get());
@@ -414,7 +419,7 @@ void X86JITCore::ClearCache() {
IR::PhysicalRegister X86JITCore::GetPhys(uint32_t Node) const {
auto PhyReg = RAData->GetNodeRegister(Node);
LOGMAN_THROW_A(PhyReg.Raw != 255, "Couldn't Allocate register for node: ssa%d. Class: %d", Node, PhyReg.Class);
LOGMAN_THROW_A_FMT(PhyReg.Raw != 255, "Couldn't Allocate register for node: ssa{}. Class: {}", Node, PhyReg.Class);
return PhyReg;
}
@@ -563,7 +568,7 @@ std::tuple<X86JITCore::SetCC, X86JITCore::CMovCC, X86JITCore::JCC> X86JITCore::G
case FEXCore::IR::COND_VS:
case FEXCore::IR::COND_VC:
default:
LOGMAN_MSG_A("Unsupported compare type");
LOGMAN_MSG_A_FMT("Unsupported compare type");
break;
}
@@ -606,7 +611,7 @@ void *X86JITCore::CompileCode(uint64_t Entry, [[maybe_unused]] FEXCore::IR::IRLi
L(RunBlock);
}
LOGMAN_THROW_A(RAData != nullptr, "Needs RA");
LOGMAN_THROW_A_FMT(RAData != nullptr, "Needs RA");
SpillSlots = RAData->SpillSlots();
@@ -662,8 +667,10 @@ void *X86JITCore::CompileCode(uint64_t Entry, [[maybe_unused]] FEXCore::IR::IRLi
for (auto [BlockNode, BlockHeader] : IR->GetBlocks()) {
using namespace FEXCore::IR;
{
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
auto BlockIROp = BlockHeader->CW<IROp_CodeBlock>();
LOGMAN_THROW_A(BlockIROp->Header.Op == IR::OP_CODEBLOCK, "IR type failed to be a code block");
LOGMAN_THROW_A_FMT(BlockIROp->Header.Op == IR::OP_CODEBLOCK, "IR type failed to be a code block");
#endif
uint32_t Node = IR->GetID(BlockNode);
auto IsTarget = JumpTargets.find(Node);
@@ -714,7 +721,7 @@ void *X86JITCore::CompileCode(uint64_t Entry, [[maybe_unused]] FEXCore::IR::IRLi
Inst << "Reg" << GetPhys(ArgNode) << (i + 1 == NumArgs ? "" : ", ");
}
LogMan::Msg::D("%s", Inst.str().c_str());
LogMan::Msg::DFmt("{}", Inst.str());
}
#endif
uint32_t ID = IR->GetID(CodeNode);
@@ -6,7 +6,6 @@ $end_info$
#pragma once
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/BlockSamplingData.h"
#include "Interface/Core/Dispatcher/Dispatcher.h"
@@ -260,6 +259,7 @@ private:
DEF_OP(AtomicFetchAnd);
DEF_OP(AtomicFetchOr);
DEF_OP(AtomicFetchXor);
DEF_OP(AtomicFetchNeg);
///< Branch ops
DEF_OP(GuestCallDirect);
@@ -36,10 +36,10 @@ DEF_OP(LoadContext) {
}
break;
case 16: {
LOGMAN_MSG_A("Invalid GPR load of size 16");
LOGMAN_MSG_A_FMT("Invalid GPR load of size 16");
}
break;
default: LOGMAN_MSG_A("Unhandled LoadContext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled LoadContext size: {}", OpSize);
}
}
else {
@@ -69,7 +69,7 @@ DEF_OP(LoadContext) {
movups(GetDst(Node), xword [STATE + Op->Offset]);
}
break;
default: LOGMAN_MSG_A("Unhandled LoadContext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled LoadContext size: {}", OpSize);
}
}
}
@@ -98,9 +98,9 @@ DEF_OP(StoreContext) {
}
break;
case 16:
LogMan::Msg::D("Invalid store size of 16");
LogMan::Msg::DFmt("Invalid store size of 16");
break;
default: LOGMAN_MSG_A("Unhandled StoreContext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled StoreContext size: {}", OpSize);
}
}
else {
@@ -129,7 +129,7 @@ DEF_OP(StoreContext) {
movups(xword [STATE + Op->Offset], GetSrc(Op->Header.Args[0].ID()));
}
break;
default: LOGMAN_MSG_A("Unhandled StoreContext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled StoreContext size: {}", OpSize);
}
}
}
@@ -160,17 +160,18 @@ DEF_OP(LoadContextIndexed) {
mov(GetDst<RA_64>(Node), qword [rax + index * Op->Stride]);
break;
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed size: %d", Op->Size);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed size: {}", Op->Size);
break;
}
break;
}
case 16:
LOGMAN_MSG_A("Invalid Class load of size 16");
LOGMAN_MSG_A_FMT("Invalid Class load of size 16");
break;
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed stride: %d", Op->Stride);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed stride: {}", Op->Stride);
break;
}
}
else {
switch (Op->Stride) {
@@ -195,7 +196,8 @@ DEF_OP(LoadContextIndexed) {
vmovq(GetDst(Node), qword [rax + index * Op->Stride]);
break;
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed size: %d", Op->Size);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed size: {}", Op->Size);
break;
}
break;
}
@@ -223,12 +225,14 @@ DEF_OP(LoadContextIndexed) {
movups(GetDst(Node), xword [STATE + rax]);
break;
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed size: %d", Op->Size);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed size: {}", Op->Size);
break;
}
break;
}
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed stride: %d", Op->Stride);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed stride: {}", Op->Stride);
break;
}
}
}
@@ -248,13 +252,14 @@ DEF_OP(StoreContextIndexed) {
case 4:
case 8: {
if (!(size == 1 || size == 2 || size == 4 || size == 8)) {
LOGMAN_MSG_A("Unhandled StoreContextIndexed size: %d", Op->Size);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed size: {}", Op->Size);
}
mov(AddressFrame(Op->Size * 8) [rax + index * Op->Stride], value);
break;
}
default:
LOGMAN_MSG_A("Unhandled StoreContextIndexed stride: %d", Op->Stride);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed stride: {}", Op->Stride);
break;
}
}
else {
@@ -279,7 +284,8 @@ DEF_OP(StoreContextIndexed) {
vmovq(AddressFrame(Op->Size * 8) [rax + index * Op->Stride], value);
break;
default:
LOGMAN_MSG_A("Unhandled StoreContextIndexed size: %d", size);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed size: {}", size);
break;
}
break;
}
@@ -307,12 +313,14 @@ DEF_OP(StoreContextIndexed) {
movups(xword [STATE + rax], value);
break;
default:
LOGMAN_MSG_A("Unhandled StoreContextIndexed size: %d", size);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed size: {}", size);
break;
}
break;
}
default:
LOGMAN_MSG_A("Unhandled StoreContextIndexed stride: %d", Op->Stride);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed stride: {}", Op->Stride);
break;
}
}
}
@@ -340,7 +348,7 @@ DEF_OP(SpillRegister) {
mov(qword [rsp + SlotOffset], GetSrc<RA_64>(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unhandled SpillRegister size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled SpillRegister size: {}", OpSize);
}
} else if (Op->Class == FEXCore::IR::FPRClass) {
switch (OpSize) {
@@ -356,10 +364,10 @@ DEF_OP(SpillRegister) {
movaps(xword [rsp + SlotOffset], GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unhandled SpillRegister size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled SpillRegister size: {}", OpSize);
}
} else {
LOGMAN_MSG_A("Unhandled SpillRegister class: %d", Op->Class.Val);
LOGMAN_MSG_A_FMT("Unhandled SpillRegister class: {}", Op->Class.Val);
}
@@ -388,7 +396,7 @@ DEF_OP(FillRegister) {
mov(GetDst<RA_64>(Node), qword [rsp + SlotOffset]);
break;
}
default: LOGMAN_MSG_A("Unhandled FillRegister size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled FillRegister size: {}", OpSize);
}
} else if (Op->Class == FEXCore::IR::FPRClass) {
switch (OpSize) {
@@ -404,10 +412,10 @@ DEF_OP(FillRegister) {
movaps(GetDst(Node), xword [rsp + SlotOffset]);
break;
}
default: LOGMAN_MSG_A("Unhandled FillRegister size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled FillRegister size: {}", OpSize);
}
} else {
LOGMAN_MSG_A("Unhandled FillRegister class: %d", Op->Class.Val);
LOGMAN_MSG_A_FMT("Unhandled FillRegister class: {}", Op->Class.Val);
}
}
@@ -430,11 +438,11 @@ Xbyak::RegExp X86JITCore::GenerateModRM(Xbyak::Reg Base, IR::OrderedNodeWrapper
return Base;
} else {
if (OffsetScale != 1 && OffsetScale != 2 && OffsetScale != 4 && OffsetScale != 8) {
LOGMAN_MSG_A("Unhandled GenerateModRM OffsetScale: %d", OffsetScale);
LOGMAN_MSG_A_FMT("Unhandled GenerateModRM OffsetScale: {}", OffsetScale);
}
if (OffsetType != IR::MEM_OFFSET_SXTX) {
LOGMAN_MSG_A("Unhandled GenerateModRM OffsetType: %d", OffsetType.Val);
LOGMAN_MSG_A_FMT("Unhandled GenerateModRM OffsetType: {}", OffsetType.Val);
}
uint64_t Const;
@@ -475,7 +483,7 @@ DEF_OP(LoadMem) {
mov(Dst, qword [MemPtr]);
}
break;
default: LOGMAN_MSG_A("Unhandled LoadMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled LoadMem size: {}", Op->Size);
}
}
else
@@ -511,7 +519,7 @@ DEF_OP(LoadMem) {
}
}
break;
default: LOGMAN_MSG_A("Unhandled LoadMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled LoadMem size: {}", Op->Size);
}
}
}
@@ -537,7 +545,7 @@ DEF_OP(StoreMem) {
case 8:
mov(qword [MemPtr], GetSrc<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unhandled StoreMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled StoreMem size: {}", Op->Size);
}
}
else {
@@ -560,17 +568,17 @@ DEF_OP(StoreMem) {
else
movups(xword [MemPtr], GetSrc(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unhandled StoreMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled StoreMem size: {}", Op->Size);
}
}
}
DEF_OP(VLoadMemElement) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(VStoreMemElement) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(CacheLineClear) {
@@ -9,11 +9,11 @@ $end_info$
namespace FEXCore::CPU {
static void PrintValue(uint64_t Value) {
LogMan::Msg::D("Value: 0x%lx", Value);
LogMan::Msg::DFmt("Value: 0x{:x}", Value);
}
static void PrintVectorValue(uint64_t Value, uint64_t ValueUpper) {
LogMan::Msg::D("Value: 0x%016lx'%016lx", ValueUpper, Value);
LogMan::Msg::DFmt("Value: 0x{:016x}'{:016x}", ValueUpper, Value);
}
#define DEF_OP(x) void X86JITCore::Op_##x(FEXCore::IR::IROp_Header *IROp, uint32_t Node)
@@ -30,7 +30,7 @@ DEF_OP(Fence) {
case IR::Fence_Store.Val:
sfence();
break;
default: LOGMAN_MSG_A("Unknown Fence: %d", Op->Fence); break;
default: LOGMAN_MSG_A_FMT("Unknown Fence: {}", Op->Fence); break;
}
}
@@ -83,7 +83,7 @@ DEF_OP(Break) {
}
break;
}
default: LOGMAN_MSG_A("Unknown Break reason: %d", Op->Reason);
default: LOGMAN_MSG_A_FMT("Unknown Break reason: {}", Op->Reason);
}
}
@@ -25,7 +25,7 @@ DEF_OP(ExtractElementPair) {
mov (GetDst<RA_64>(Node), Regs[Op->Element]);
break;
}
default: LOGMAN_MSG_A("Unknown Size"); break;
default: LOGMAN_MSG_A_FMT("Unknown Size"); break;
}
}
@@ -51,7 +51,7 @@ DEF_OP(CreateElementPair) {
RegTmp = rax;
break;
}
default: LOGMAN_MSG_A("Unknown Size"); break;
default: LOGMAN_MSG_A_FMT("Unknown Size"); break;
}
if (Dst.first != RegSecond) {
+101 -101
View File
@@ -62,24 +62,24 @@ DEF_OP(VectorImm) {
}
DEF_OP(CreateVector2) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(CreateVector4) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(SplatVector) {
auto Op = IROp->C<IR::IROp_SplatVector2>();
uint8_t OpSize = IROp->Size;
LOGMAN_THROW_A(OpSize <= 16, "Can't handle a vector of size: %d", OpSize);
LOGMAN_THROW_A_FMT(OpSize <= 16, "Can't handle a vector of size: {}", OpSize);
uint8_t Elements = 0;
switch (Op->Header.Op) {
case IR::OP_SPLATVECTOR4: Elements = 4; break;
case IR::OP_SPLATVECTOR2: Elements = 2; break;
default: LOGMAN_MSG_A("Uknown Splat size"); break;
default: LOGMAN_MSG_A_FMT("Unknown Splat size"); break;
}
uint8_t ElementSize = OpSize / Elements;
@@ -92,7 +92,7 @@ DEF_OP(SplatVector) {
case 8:
movddup(GetDst(Node), GetSrc(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.Size); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.Size); break;
}
}
@@ -130,7 +130,7 @@ DEF_OP(VMov) {
movaps(GetDst(Node), GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", OpSize); break;
}
}
@@ -176,7 +176,7 @@ DEF_OP(VAdd) {
vpaddq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -199,7 +199,7 @@ DEF_OP(VSub) {
vpsubq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -214,7 +214,7 @@ DEF_OP(VUQAdd) {
vpaddusw(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -229,7 +229,7 @@ DEF_OP(VUQSub) {
vpsubusw(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -244,7 +244,7 @@ DEF_OP(VSQAdd) {
vpaddsw(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -259,7 +259,7 @@ DEF_OP(VSQSub) {
vpsubsw(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -296,7 +296,7 @@ DEF_OP(VAddP) {
case 4:
vphaddd(GetDst(Node), xmm15, xmm14);
break;
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -325,7 +325,7 @@ DEF_OP(VAddP) {
case 4:
vphaddd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -357,7 +357,7 @@ DEF_OP(VAddV) {
pinsrd(xmm15, eax, 0);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
movaps(Dest, xmm15);
@@ -376,7 +376,7 @@ DEF_OP(VUMinV) {
pinsrw(Dest, eax, 1);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -391,7 +391,7 @@ DEF_OP(VURAvg) {
vpavgw(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -414,7 +414,7 @@ DEF_OP(VAbs) {
vpabsq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -438,7 +438,7 @@ DEF_OP(VPopcount) {
}
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
movaps(Dest, xmm15);
@@ -459,7 +459,7 @@ DEF_OP(VFAdd) {
vaddsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -473,7 +473,7 @@ DEF_OP(VFAdd) {
vaddpd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -487,7 +487,7 @@ DEF_OP(VFAddP) {
case 8:
vhaddpd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -506,7 +506,7 @@ DEF_OP(VFSub) {
vsubsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -520,7 +520,7 @@ DEF_OP(VFSub) {
vsubpd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -540,7 +540,7 @@ DEF_OP(VFMul) {
vmulsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -554,7 +554,7 @@ DEF_OP(VFMul) {
vmulpd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -574,7 +574,7 @@ DEF_OP(VFDiv) {
vdivsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -588,7 +588,7 @@ DEF_OP(VFDiv) {
vdivpd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -608,7 +608,7 @@ DEF_OP(VFMin) {
vminsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -622,7 +622,7 @@ DEF_OP(VFMin) {
vminpd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -642,7 +642,7 @@ DEF_OP(VFMax) {
vmaxsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -656,7 +656,7 @@ DEF_OP(VFMax) {
vmaxpd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -674,7 +674,7 @@ DEF_OP(VFRecp) {
vdivss(GetDst(Node), xmm15, GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -687,7 +687,7 @@ DEF_OP(VFRecp) {
vdivps(GetDst(Node), xmm15, GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -707,7 +707,7 @@ DEF_OP(VFSqrt) {
vsqrtsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -721,7 +721,7 @@ DEF_OP(VFSqrt) {
vsqrtpd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -747,7 +747,7 @@ DEF_OP(VFRSqrt) {
divsd(GetDst(Node), xmm15);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -761,7 +761,7 @@ DEF_OP(VFRSqrt) {
divps(GetDst(Node), xmm15);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -786,7 +786,7 @@ DEF_OP(VNeg) {
vpsubq(GetDst(Node), xmm15, GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -807,7 +807,7 @@ DEF_OP(VFNeg) {
vxorpd(GetDst(Node), xmm15, GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -830,7 +830,7 @@ DEF_OP(VUMin) {
pinsrq(GetDst(Node), TMP2, 0);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -847,7 +847,7 @@ DEF_OP(VUMin) {
vpminud(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -867,7 +867,7 @@ DEF_OP(VSMin) {
vpminsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -886,7 +886,7 @@ DEF_OP(VUMax) {
vpmaxud(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -905,7 +905,7 @@ DEF_OP(VSMax) {
vpmaxsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -930,7 +930,7 @@ DEF_OP(VZip) {
punpcklqdq(xmm15, GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
movapd(GetDst(Node), xmm15);
}
@@ -957,7 +957,7 @@ DEF_OP(VZip2) {
vpunpckhdq(GetDst(Node), xmm15, xmm14);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -978,7 +978,7 @@ DEF_OP(VZip2) {
punpckhqdq(xmm15, GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
movapd(GetDst(Node), xmm15);
}
@@ -989,7 +989,7 @@ DEF_OP(VUnZip) {
uint8_t OpSize = IROp->Size;
if (OpSize == 8) {
LOGMAN_MSG_A("Unsupported registersize on VunZip");
LOGMAN_MSG_A_FMT("Unsupported register size on VUnZip");
}
else {
switch (Op->Header.ElementSize) {
@@ -1031,7 +1031,7 @@ DEF_OP(VUnZip) {
0b0'0);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1042,7 +1042,7 @@ DEF_OP(VUnZip2) {
if (OpSize == 8) {
LOGMAN_MSG_A("Unsupported registersize on VunZip");
LOGMAN_MSG_A_FMT("Unsupported register size on VUnZip2");
}
else {
switch (Op->Header.ElementSize) {
@@ -1084,7 +1084,7 @@ DEF_OP(VUnZip2) {
0b1'1);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1113,7 +1113,7 @@ DEF_OP(VCMPEQ) {
case 8:
vpcmpeqq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
@@ -1134,7 +1134,7 @@ DEF_OP(VCMPEQZ) {
case 8:
vpcmpeqq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), xmm15);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
@@ -1154,7 +1154,7 @@ DEF_OP(VCMPGT) {
case 8:
vpcmpgtq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
@@ -1175,7 +1175,7 @@ DEF_OP(VCMPGTZ) {
case 8:
vpcmpgtq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), xmm15);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
@@ -1196,7 +1196,7 @@ DEF_OP(VCMPLTZ) {
case 8:
vpcmpgtq(GetDst(Node), xmm15, GetSrc(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
@@ -1212,7 +1212,7 @@ DEF_OP(VFCMPEQ) {
case 8:
vcmpsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 0);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
else {
@@ -1223,7 +1223,7 @@ DEF_OP(VFCMPEQ) {
case 8:
vcmppd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 0);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
}
@@ -1240,7 +1240,7 @@ DEF_OP(VFCMPNEQ) {
case 8:
vcmpsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 4);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
@@ -1252,7 +1252,7 @@ DEF_OP(VFCMPNEQ) {
case 8:
vcmppd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 4);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
}
@@ -1269,7 +1269,7 @@ DEF_OP(VFCMPLT) {
case 8:
vcmpsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 1);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
else {
@@ -1280,7 +1280,7 @@ DEF_OP(VFCMPLT) {
case 8:
vcmppd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 1);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
}
@@ -1297,7 +1297,7 @@ DEF_OP(VFCMPGT) {
case 8:
vcmpsd(GetDst(Node), GetSrc(Op->Header.Args[1].ID()), GetSrc(Op->Header.Args[0].ID()), 1);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
else {
@@ -1308,7 +1308,7 @@ DEF_OP(VFCMPGT) {
case 8:
vcmppd(GetDst(Node), GetSrc(Op->Header.Args[1].ID()), GetSrc(Op->Header.Args[0].ID()), 1);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
}
@@ -1325,7 +1325,7 @@ DEF_OP(VFCMPLE) {
case 8:
vcmpsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 2);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
else {
@@ -1336,7 +1336,7 @@ DEF_OP(VFCMPLE) {
case 8:
vcmppd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 2);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
}
@@ -1353,7 +1353,7 @@ DEF_OP(VFCMPORD) {
case 8:
vcmpsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 7);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
else {
@@ -1364,7 +1364,7 @@ DEF_OP(VFCMPORD) {
case 8:
vcmppd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 7);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
}
@@ -1381,7 +1381,7 @@ DEF_OP(VFCMPUNO) {
case 8:
vcmpsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 3);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
else {
@@ -1392,21 +1392,21 @@ DEF_OP(VFCMPUNO) {
case 8:
vcmppd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 3);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
}
DEF_OP(VUShl) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(VUShr) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(VSShr) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(VUShlS) {
@@ -1425,7 +1425,7 @@ DEF_OP(VUShlS) {
vpsllq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1445,7 +1445,7 @@ DEF_OP(VUShrS) {
vpsrlq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1462,7 +1462,7 @@ DEF_OP(VSShrS) {
break;
}
case 8: // Doesn't exist on x86
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1495,7 +1495,7 @@ DEF_OP(VInsElement) {
pinsrq(xmm15, rax, Op->DestIdx);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
movapd(GetDst(Node), xmm15);
@@ -1530,7 +1530,7 @@ DEF_OP(VInsScalarElement) {
pinsrq(xmm15, rax, Op->DestIdx);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
movapd(GetDst(Node), xmm15);
@@ -1560,7 +1560,7 @@ DEF_OP(VExtractElement) {
pinsrq(GetDst(Node), rax, 0);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1613,7 +1613,7 @@ DEF_OP(VDupElement) {
(Op->Index << 1));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1666,7 +1666,7 @@ DEF_OP(VUShrI) {
psrlq(GetDst(Node), Op->BitShift);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1704,7 +1704,7 @@ DEF_OP(VSShrI) {
}
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1724,7 +1724,7 @@ DEF_OP(VShlI) {
psllq(GetDst(Node), Op->BitShift);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1754,7 +1754,7 @@ DEF_OP(VUShrNI) {
mov(rcx, 0x80'80'80'80'80'80'80'80); // Upper
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
vmovq(xmm15, rax);
@@ -1790,7 +1790,7 @@ DEF_OP(VUShrNI2) {
mov(rcx, 0x0B'0A'09'08'03'02'01'00); // Upper
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
vmovq(xmm15, rax);
@@ -1817,7 +1817,7 @@ DEF_OP(VSXTL) {
case 8:
pmovsxdq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
@@ -1836,7 +1836,7 @@ DEF_OP(VSXTL2) {
case 8:
pmovsxdq(GetDst(Node), GetDst(Node));
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
@@ -1852,7 +1852,7 @@ DEF_OP(VUXTL) {
case 8:
pmovzxdq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
@@ -1871,7 +1871,7 @@ DEF_OP(VUXTL2) {
case 8:
pmovzxdq(GetDst(Node), GetDst(Node));
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
@@ -1884,7 +1884,7 @@ DEF_OP(VSQXTN) {
case 2:
packssdw(xmm15, GetSrc(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
psrldq(xmm15, 8);
movaps(GetDst(Node), xmm15);
@@ -1903,7 +1903,7 @@ DEF_OP(VSQXTN2) {
case 2:
packssdw(xmm15, GetSrc(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
if (OpSize == 8) {
@@ -1921,7 +1921,7 @@ DEF_OP(VSQXTUN) {
case 2:
packusdw(xmm15, GetSrc(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
psrldq(xmm15, 8);
movaps(GetDst(Node), xmm15);
@@ -1940,7 +1940,7 @@ DEF_OP(VSQXTUN2) {
case 2:
packusdw(xmm15, GetSrc(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
if (OpSize == 8) {
psrldq(xmm15, OpSize / 2);
@@ -1960,7 +1960,7 @@ DEF_OP(VMul) {
vpmulld(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1991,7 +1991,7 @@ DEF_OP(VUMull) {
vpmuludq(GetDst(Node), xmm14, xmm15);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -2026,7 +2026,7 @@ DEF_OP(VSMull) {
vpmuldq(GetDst(Node), xmm14, xmm15);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -2062,7 +2062,7 @@ DEF_OP(VUMull2) {
vpmuludq(GetDst(Node), xmm14, xmm15);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -2102,7 +2102,7 @@ DEF_OP(VSMull2) {
vpmuldq(GetDst(Node), xmm14, xmm15);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -2123,7 +2123,7 @@ DEF_OP(VUABDL) {
vpabsd(GetDst(Node), GetDst(Node));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -2141,7 +2141,7 @@ DEF_OP(VTBL1) {
vpshufb(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown OpSize: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown OpSize: {}", OpSize); break;
}
}
+4 -1
View File
@@ -38,7 +38,10 @@ public:
std::map<uint64_t, std::vector<uint64_t>> CodePages;
void AddBlockMapping(uint64_t Address, void *HostCode, uint64_t Start, uint64_t Length) {
auto InsertPoint = BlockList.emplace(Address, (uintptr_t)HostCode);
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
auto InsertPoint =
#endif
BlockList.emplace(Address, (uintptr_t)HostCode);
LOGMAN_THROW_A(InsertPoint.second == true, "Dupplicate block mapping added");
for (auto CurrentPage = Start >> 12, EndPage = (Start + Length) >> 12; CurrentPage <= EndPage; CurrentPage++) {
File diff suppressed because it is too large. Load diff
+29 -9
View File
@@ -85,7 +85,7 @@ public:
auto it = JumpTargets.find(NextRIP);
if (it == JumpTargets.end()) {
uint8_t GPRSize = CTX->Config.Is64BitMode ? 8 : 4;
const uint8_t GPRSize = CTX->GetGPRSize();
// If we don't have a jump target to a new block then we have to leave
// Set the RIP to the next instruction and leave
auto RelocatedNextRIP = _EntrypointOffset(NextRIP - Entry, GPRSize);
@@ -270,15 +270,15 @@ public:
template<size_t ElementSize, bool HalfSize, bool Low>
void PSHUFDOp(OpcodeArgs);
void MOVDOp(OpcodeArgs);
template<size_t ElementSize, bool Scalar, uint32_t SrcIndex>
template<size_t ElementSize>
void PSRLDOp(OpcodeArgs);
template<size_t ElementSize>
void PSRLI(OpcodeArgs);
template<size_t ElementSize>
void PSLLI(OpcodeArgs);
template<size_t ElementSize, bool Scalar, uint32_t SrcIndex>
template<size_t ElementSize>
void PSLL(OpcodeArgs);
template<size_t ElementSize, bool Scalar, uint32_t SrcIndex>
template<size_t ElementSize>
void PSRAOp(OpcodeArgs);
void PSRLDQ(OpcodeArgs);
void PSLLDQ(OpcodeArgs);
@@ -299,9 +299,9 @@ public:
void Vector_CVT_Float_To_Float(OpcodeArgs);
template<size_t SrcElementSize, bool Narrow, bool HostRoundingMode>
void Vector_CVT_Float_To_Int(OpcodeArgs);
template<size_t SrcElementSize, bool Signed, bool Widen>
template<size_t SrcElementSize, bool Widen>
void MMX_To_XMM_Vector_CVT_Int_To_Float(OpcodeArgs);
template<size_t SrcElementSize, bool Narrow, bool HostRoundingMode>
template<size_t SrcElementSize, bool HostRoundingMode>
void XMM_To_MMX_Vector_CVT_Float_To_Int(OpcodeArgs);
void MASKMOVOp(OpcodeArgs);
void MOVBetweenGPR_FPR(OpcodeArgs);
@@ -380,6 +380,8 @@ public:
void X87FRSTOR(OpcodeArgs);
void X87FXAM(OpcodeArgs);
void X87FCMOV(OpcodeArgs);
void X87EMMS(OpcodeArgs);
void X87FFREE(OpcodeArgs);
void FXCH(OpcodeArgs);
@@ -501,9 +503,19 @@ private:
uint8_t GetSrcSize(FEXCore::X86Tables::DecodedOp Op) const;
template<unsigned BitOffset>
void SetRFLAG(OrderedNode *Value);
void SetRFLAG(OrderedNode *Value, unsigned BitOffset);
OrderedNode *GetRFLAG(unsigned BitOffset);
void SetRFLAG(OrderedNode *Value) {
flagsOp = FLAGS_OP_NONE;
_StoreFlag(_Bfe(1, 0, Value), BitOffset);
}
void SetRFLAG(OrderedNode *Value, unsigned BitOffset) {
flagsOp = FLAGS_OP_NONE;
_StoreFlag(_Bfe(1, 0, Value), BitOffset);
}
OrderedNode *GetRFLAG(unsigned BitOffset) {
return _LoadFlag(BitOffset);
}
OrderedNode *SelectCC(uint8_t OP, OrderedNode *TrueValue, OrderedNode *FalseValue);
@@ -526,6 +538,14 @@ private:
void GenerateFlags_RotateLeftImmediate(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, uint64_t Shift);
OrderedNode * GetX87Top();
enum X87Tag {
TAG_VALID = 0b00,
TAG_ZERO = 0b01,
TAG_SPECIAL = 0b10,
TAG_EMPTY = 0b11
};
void SetX87TopTag(OrderedNode *Value, uint32_t Tag);
OrderedNode *GetX87FTW(OrderedNode *Value);
void SetX87Top(OrderedNode *Value);
bool DestIsLockedMem(FEXCore::X86Tables::DecodedOp Op) const {
@@ -0,0 +1,58 @@
/*
$info$
tags: frontend|x86-to-ir, opcodes|dispatcher-implementations
desc: Handles x86/64 Crypto instructions to IR
$end_info$
*/
#include "Interface/Core/OpcodeDispatcher.h"
#include <FEXCore/Core/X86Enums.h>
namespace FEXCore::IR {
#define OpcodeArgs [[maybe_unused]] FEXCore::X86Tables::DecodedOp Op
void OpDispatchBuilder::AESImcOp(OpcodeArgs) {
OrderedNode *Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
auto Res = _VAESImc(Src);
StoreResult(FPRClass, Op, Res, -1);
}
void OpDispatchBuilder::AESEncOp(OpcodeArgs) {
OrderedNode *Dest = LoadSource(FPRClass, Op, Op->Dest, Op->Flags, -1);
OrderedNode *Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
auto Res = _VAESEnc(Dest, Src);
StoreResult(FPRClass, Op, Res, -1);
}
void OpDispatchBuilder::AESEncLastOp(OpcodeArgs) {
OrderedNode *Dest = LoadSource(FPRClass, Op, Op->Dest, Op->Flags, -1);
OrderedNode *Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
auto Res = _VAESEncLast(Dest, Src);
StoreResult(FPRClass, Op, Res, -1);
}
void OpDispatchBuilder::AESDecOp(OpcodeArgs) {
OrderedNode *Dest = LoadSource(FPRClass, Op, Op->Dest, Op->Flags, -1);
OrderedNode *Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
auto Res = _VAESDec(Dest, Src);
StoreResult(FPRClass, Op, Res, -1);
}
void OpDispatchBuilder::AESDecLastOp(OpcodeArgs) {
OrderedNode *Dest = LoadSource(FPRClass, Op, Op->Dest, Op->Flags, -1);
OrderedNode *Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
auto Res = _VAESDecLast(Dest, Src);
StoreResult(FPRClass, Op, Res, -1);
}
void OpDispatchBuilder::AESKeyGenAssist(OpcodeArgs) {
OrderedNode *Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
LOGMAN_THROW_A(Op->Src[1].IsLiteral(), "Src1 needs to be literal here");
uint64_t RCON = Op->Src[1].Data.Literal.Value;
auto Res = _VAESKeyGenAssist(Src, RCON);
StoreResult(FPRClass, Op, Res, -1);
}
}
@@ -0,0 +1,810 @@
/*
$info$
tags: frontend|x86-to-ir, opcodes|dispatcher-implementations
desc: Handles x86/64 flag generation
$end_info$
*/
#include "Interface/Core/OpcodeDispatcher.h"
#include <FEXCore/Core/X86Enums.h>
namespace FEXCore::IR {
constexpr std::array<uint32_t, 17> FlagOffsets = {
FEXCore::X86State::RFLAG_CF_LOC,
FEXCore::X86State::RFLAG_PF_LOC,
FEXCore::X86State::RFLAG_AF_LOC,
FEXCore::X86State::RFLAG_ZF_LOC,
FEXCore::X86State::RFLAG_SF_LOC,
FEXCore::X86State::RFLAG_TF_LOC,
FEXCore::X86State::RFLAG_IF_LOC,
FEXCore::X86State::RFLAG_DF_LOC,
FEXCore::X86State::RFLAG_OF_LOC,
FEXCore::X86State::RFLAG_IOPL_LOC,
FEXCore::X86State::RFLAG_NT_LOC,
FEXCore::X86State::RFLAG_RF_LOC,
FEXCore::X86State::RFLAG_VM_LOC,
FEXCore::X86State::RFLAG_AC_LOC,
FEXCore::X86State::RFLAG_VIF_LOC,
FEXCore::X86State::RFLAG_VIP_LOC,
FEXCore::X86State::RFLAG_ID_LOC,
};
void OpDispatchBuilder::SetPackedRFLAG(bool Lower8, OrderedNode *Src) {
uint8_t NumFlags = FlagOffsets.size();
if (Lower8) {
NumFlags = 5;
}
auto OneConst = _Constant(1);
for (int i = 0; i < NumFlags; ++i) {
auto Tmp = _And(_Lshr(Src, _Constant(FlagOffsets[i])), OneConst);
SetRFLAG(Tmp, FlagOffsets[i]);
}
}
OrderedNode *OpDispatchBuilder::GetPackedRFLAG(bool Lower8) {
OrderedNode *Original = _Constant(2);
uint8_t NumFlags = FlagOffsets.size();
if (Lower8) {
NumFlags = 5;
}
for (int i = 0; i < NumFlags; ++i) {
OrderedNode *Flag = _LoadFlag(FlagOffsets[i]);
Flag = _Bfe(4, 32, 0, Flag);
Flag = _Lshl(Flag, _Constant(FlagOffsets[i]));
Original = _Or(Original, Flag);
}
return Original;
}
void OpDispatchBuilder::GenerateFlags_ADC(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2, OrderedNode *CF) {
auto Size = GetSrcSize(Op) * 8;
// AF
{
OrderedNode *AFRes = _Xor(_Xor(Src1, Src2), Res);
AFRes = _Bfe(1, 4, AFRes);
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(AFRes);
}
// SF
{
auto SignBitConst = _Constant(GetSrcSize(Op) * 8 - 1);
auto LshrOp = _Lshr(Res, SignBitConst);
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(LshrOp);
}
// PF
if (!CTX->Config.ABINoPF) {
auto PopCountOp = _Popcount(_And(Res, _Constant(0xFF)));
auto XorOp = _Xor(PopCountOp, _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(SelectOp);
}
// CF
// Unsigned
{
auto SelectOpLT = _Select(FEXCore::IR::COND_ULT, Res, Src2, _Constant(1), _Constant(0));
auto SelectOpLE = _Select(FEXCore::IR::COND_ULE, Res, Src2, _Constant(1), _Constant(0));
auto SelectCF = _Select(FEXCore::IR::COND_EQ, CF, _Constant(1), SelectOpLE, SelectOpLT);
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(SelectCF);
}
// OF
// Signed
{
auto NegOne = _Constant(~0ULL);
auto XorOp1 = _Xor(_Xor(Src1, Src2), NegOne);
auto XorOp2 = _Xor(Res, Src1);
OrderedNode *AndOp1 = _And(XorOp1, XorOp2);
switch (Size) {
case 8:
AndOp1 = _Bfe(1, 7, AndOp1);
break;
case 16:
AndOp1 = _Bfe(1, 15, AndOp1);
break;
case 32:
AndOp1 = _Bfe(1, 31, AndOp1);
break;
case 64:
AndOp1 = _Bfe(1, 63, AndOp1);
break;
default: LOGMAN_MSG_A("Unknown BFESize: %d", Size); break;
}
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(AndOp1);
}
}
void OpDispatchBuilder::GenerateFlags_SBB(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2, OrderedNode *CF) {
// AF
{
OrderedNode *AFRes = _Xor(_Xor(Src1, Src2), Res);
AFRes = _Bfe(1, 4, AFRes);
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(AFRes);
}
// SF
{
auto SignBitConst = _Constant(GetSrcSize(Op) * 8 - 1);
auto LshrOp = _Lshr(Res, SignBitConst);
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(LshrOp);
}
// PF
if (!CTX->Config.ABINoPF) {
auto PopCountOp = _Popcount(_And(Res, _Constant(0xFF)));
auto XorOp = _Xor(PopCountOp, _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(SelectOp);
}
// CF
// Unsigned
{
auto SelectOpLT = _Select(FEXCore::IR::COND_UGT, Res, Src1, _Constant(1), _Constant(0));
auto SelectOpLE = _Select(FEXCore::IR::COND_UGE, Res, Src1, _Constant(1), _Constant(0));
auto SelectCF = _Select(FEXCore::IR::COND_EQ, CF, _Constant(1), SelectOpLE, SelectOpLT);
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(SelectCF);
}
// OF
// Signed
{
auto XorOp1 = _Xor(Src1, Src2);
auto XorOp2 = _Xor(Res, Src1);
OrderedNode *AndOp1 = _And(XorOp1, XorOp2);
switch (GetSrcSize(Op)) {
case 1:
AndOp1 = _Bfe(1, 7, AndOp1);
break;
case 2:
AndOp1 = _Bfe(1, 15, AndOp1);
break;
case 4:
AndOp1 = _Bfe(1, 31, AndOp1);
break;
case 8:
AndOp1 = _Bfe(1, 63, AndOp1);
break;
default: LOGMAN_MSG_A("Unknown BFESize: %d", GetSrcSize(Op)); break;
}
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(AndOp1);
}
}
void OpDispatchBuilder::GenerateFlags_SUB(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2, bool UpdateCF) {
// AF
{
OrderedNode *AFRes = _Xor(_Xor(Src1, Src2), Res);
AFRes = _Bfe(1, 4, AFRes);
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(AFRes);
}
// SF
{
auto SignBitConst = _Constant(GetSrcSize(Op) * 8 - 1);
auto LshrOp = _Lshr(Res, SignBitConst);
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(LshrOp);
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// ZF
{
auto ZeroConst = _Constant(0);
auto OneConst = _Constant(1);
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, ZeroConst, OneConst, ZeroConst);
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(SelectOp);
}
// CF
if (UpdateCF) {
auto ZeroConst = _Constant(0);
auto OneConst = _Constant(1);
auto SelectOp = _Select(FEXCore::IR::COND_ULT,
Src1, Src2, OneConst, ZeroConst);
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(SelectOp);
}
// OF
{
auto XorOp1 = _Xor(Src1, Src2);
auto XorOp2 = _Xor(Res, Src1);
OrderedNode *FinalAnd = _And(XorOp1, XorOp2);
FinalAnd = _Bfe(1, GetSrcSize(Op) * 8 - 1, FinalAnd);
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(FinalAnd);
}
}
void OpDispatchBuilder::GenerateFlags_ADD(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2, bool UpdateCF) {
// AF
{
OrderedNode *AFRes = _Xor(_Xor(Src1, Src2), Res);
AFRes = _Bfe(1, 4, AFRes);
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(AFRes);
}
// SF
{
auto SignBitConst = _Constant(GetSrcSize(Op) * 8 - 1);
auto LshrOp = _Lshr(Res, SignBitConst);
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(LshrOp);
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(SelectOp);
}
// CF
if (UpdateCF) {
auto SelectOp = _Select(FEXCore::IR::COND_ULT, Res, Src2, _Constant(1), _Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(SelectOp);
}
// OF
{
auto NegOne = _Constant(~0ULL);
auto XorOp1 = _Xor(_Xor(Src1, Src2), NegOne);
auto XorOp2 = _Xor(Res, Src1);
OrderedNode *AndOp1 = _And(XorOp1, XorOp2);
switch (GetSrcSize(Op)) {
case 1:
AndOp1 = _Bfe(1, 7, AndOp1);
break;
case 2:
AndOp1 = _Bfe(1, 15, AndOp1);
break;
case 4:
AndOp1 = _Bfe(1, 31, AndOp1);
break;
case 8:
AndOp1 = _Bfe(1, 63, AndOp1);
break;
default: LOGMAN_MSG_A("Unknown BFESize: %d", GetSrcSize(Op)); break;
}
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(AndOp1);
}
}
void OpDispatchBuilder::GenerateFlags_MUL(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *High) {
// PF/AF/ZF/SF
// Undefined
{
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(_Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(_Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(_Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(_Constant(0));
}
// CF/OF
{
// CF and OF are set if the result of the operation can't be fit in to the destination register
// If the value can fit then the top bits will be zero
auto SignBit = _Sbfe(1, GetSrcSize(Op) * 8 - 1, Res);
auto SelectOp = _Select(FEXCore::IR::COND_EQ, High, SignBit, _Constant(0), _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(SelectOp);
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(SelectOp);
}
}
void OpDispatchBuilder::GenerateFlags_UMUL(FEXCore::X86Tables::DecodedOp Op, OrderedNode *High) {
// AF/SF/PF/ZF
// Undefined
{
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(_Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(_Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(_Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(_Constant(0));
}
// CF/OF
{
// CF and OF are set if the result of the operation can't be fit in to the destination register
// The result register will be all zero if it can't fit due to how multiplication behaves
auto SelectOp = _Select(FEXCore::IR::COND_EQ, High, _Constant(0), _Constant(0), _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(SelectOp);
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(SelectOp);
}
}
void OpDispatchBuilder::GenerateFlags_Logical(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2) {
// AF
{
// Undefined
// Set to zero anyway
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(_Constant(0));
}
// SF
{
auto SignBitConst = _Constant(GetSrcSize(Op) * 8 - 1);
auto LshrOp = _Lshr(Res, SignBitConst);
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(LshrOp);
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(SelectOp);
}
// CF/OF
{
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(_Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(_Constant(0));
}
}
#define COND_FLAG_SET(cond, flag, newflag) \
auto oldflag = GetRFLAG(FEXCore::X86State::flag);\
auto newval = _Select(FEXCore::IR::COND_EQ, cond, _Constant(0), oldflag, newflag);\
SetRFLAG<FEXCore::X86State::flag>(newval);
void OpDispatchBuilder::GenerateFlags_ShiftLeft(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2) {
// CF
{
// Extract the last bit shifted in to CF
auto Size = _Constant(GetSrcSize(Op) * 8);
auto ShiftAmt = _Sub(Size, Src2);
auto LastBit = _And(_Lshr(Src1, ShiftAmt), _Constant(1));
COND_FLAG_SET(Src2, RFLAG_CF_LOC, LastBit);
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
COND_FLAG_SET(Src2, RFLAG_PF_LOC, XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// AF
{
// Undefined
// Set to zero anyway
COND_FLAG_SET(Src2, RFLAG_AF_LOC, _Constant(0));
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
COND_FLAG_SET(Src2, RFLAG_ZF_LOC, SelectOp);
}
// SF
{
auto val = _Bfe(1, GetSrcSize(Op) * 8 - 1, Res);
COND_FLAG_SET(Src2, RFLAG_SF_LOC, val);
}
// OF
{
// In the case of left shift. OF is only set from the result of <Top Source Bit> XOR <Top Result Bit>
// When Shift > 1 then OF is undefined
auto val = _Bfe(1, GetSrcSize(Op) * 8 - 1, _Xor(Src1, Res));
COND_FLAG_SET(Src2, RFLAG_OF_LOC, val);
}
}
void OpDispatchBuilder::GenerateFlags_ShiftRight(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2) {
// CF
{
// Extract the last bit shifted in to CF
auto ShiftAmt = _Sub(Src2, _Constant(1));
auto LastBit = _And(_Lshr(Src1, ShiftAmt), _Constant(1));
COND_FLAG_SET(Src2, RFLAG_CF_LOC, LastBit);
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
COND_FLAG_SET(Src2, RFLAG_PF_LOC, XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// AF
{
// Undefined
// Set to zero anyway
COND_FLAG_SET(Src2, RFLAG_AF_LOC, _Constant(0));
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
COND_FLAG_SET(Src2, RFLAG_ZF_LOC, SelectOp);
}
// SF
{
auto val =_Bfe(1, GetSrcSize(Op) * 8 - 1, Res);
COND_FLAG_SET(Src2, RFLAG_SF_LOC, val);
}
// OF
{
// Only defined when Shift is 1 else undefined
// OF flag is set if a sign change occurred
auto val = _Bfe(1, GetSrcSize(Op) * 8 - 1, _Xor(Src1, Res));
COND_FLAG_SET(Src2, RFLAG_OF_LOC, val);
}
}
void OpDispatchBuilder::GenerateFlags_SignShiftRight(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2) {
// CF
{
// Extract the last bit shifted in to CF
auto ShiftAmt = _Sub(Src2, _Constant(1));
auto LastBit = _And(_Lshr(Src1, ShiftAmt), _Constant(1));
COND_FLAG_SET(Src2, RFLAG_CF_LOC, LastBit);
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
COND_FLAG_SET(Src2, RFLAG_PF_LOC, XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// AF
{
// Undefined
// Set to zero anyway
COND_FLAG_SET(Src2, RFLAG_AF_LOC, _Constant(0));
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
COND_FLAG_SET(Src2, RFLAG_ZF_LOC, SelectOp);
}
// SF
{
auto SignBitConst = _Constant(GetSrcSize(Op) * 8 - 1);
auto LshrOp = _Lshr(Res, SignBitConst);
COND_FLAG_SET(Src2, RFLAG_SF_LOC, LshrOp);
}
// OF
{
COND_FLAG_SET(Src2, RFLAG_OF_LOC, _Constant(0));
}
}
void OpDispatchBuilder::GenerateFlags_ShiftLeftImmediate(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, uint64_t Shift) {
// No flags changed if shift is zero
if (Shift == 0) return;
// CF
{
// Extract the last bit shifted in to CF
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(_Bfe(1, GetSrcSize(Op) * 8 - Shift, Src1));
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// AF
{
// Undefined
// Set to zero anyway
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(_Constant(0));
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(SelectOp);
}
// SF
{
auto LshrOp = _Bfe(1, GetSrcSize(Op) * 8 - 1, Res);
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(LshrOp);
// OF
// In the case of left shift. OF is only set from the result of <Top Source Bit> XOR <Top Result Bit>
if (Shift == 1) {
auto SourceBit = _Bfe(1, GetSrcSize(Op) * 8 - 1, Src1);
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(_Xor(SourceBit, LshrOp));
}
}
}
void OpDispatchBuilder::GenerateFlags_SignShiftRightImmediate(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, uint64_t Shift) {
// No flags changed if shift is zero
if (Shift == 0) return;
// CF
{
// Extract the last bit shifted in to CF
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(_Bfe(1, Shift-1, Src1));
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// AF
{
// Undefined
// Set to zero anyway
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(_Constant(0));
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(SelectOp);
}
// SF
{
auto SignBitConst = _Constant(GetSrcSize(Op) * 8 - 1);
auto LshrOp = _Lshr(Res, SignBitConst);
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(LshrOp);
// OF
// Only defined when Shift is 1 else undefined
// Only is set if the top bit was set to 1 when shifted
// So it is set to same value as SF
if (Shift == 1) {
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(_Constant(0));
}
}
}
void OpDispatchBuilder::GenerateFlags_ShiftRightImmediate(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, uint64_t Shift) {
// No flags changed if shift is zero
if (Shift == 0) return;
// CF
{
// Extract the last bit shifted in to CF
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(_Bfe(1, Shift-1, Src1));
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// AF
{
// Undefined
// Set to zero anyway
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(_Constant(0));
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(SelectOp);
}
// SF
{
auto SignBitConst = _Constant(GetSrcSize(Op) * 8 - 1);
auto LshrOp = _Lshr(Res, SignBitConst);
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(LshrOp);
}
// OF
{
// Only defined when Shift is 1 else undefined
// Is set to the MSB of the original value
if (Shift == 1) {
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(_Bfe(1, GetSrcSize(Op) * 8 - 1, Src1));
}
}
}
void OpDispatchBuilder::GenerateFlags_RotateRight(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2) {
auto OpSize = GetSrcSize(Op) * 8;
// Extract the last bit shifted in to CF
auto NewCF = _Bfe(1, OpSize - 1, Res);
// CF
{
auto OldCF = GetRFLAG(FEXCore::X86State::RFLAG_CF_LOC);
auto CF = _Select(FEXCore::IR::COND_EQ, Src2, _Constant(0), OldCF, NewCF);
// Extract the last bit shifted in to CF
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(CF);
}
// OF
{
auto OldOF = GetRFLAG(FEXCore::X86State::RFLAG_OF_LOC);
// OF is set to the XOR of the new CF bit and the most significant bit of the result
auto NewOF = _Xor(_Bfe(1, OpSize - 2, Res), NewCF);
// If shift == 0, don't update flags
auto OF = _Select(FEXCore::IR::COND_EQ, Src2, _Constant(0), OldOF, NewOF);
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(OF);
}
}
void OpDispatchBuilder::GenerateFlags_RotateLeft(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2) {
auto OpSize = GetSrcSize(Op) * 8;
// Extract the last bit shifted in to CF
//auto Size = _Constant(GetSrcSize(Res) * 8);
//auto ShiftAmt = _Sub(Size, Src2);
auto NewCF = _Bfe(1, 0, Res);
// CF
{
auto OldCF = GetRFLAG(FEXCore::X86State::RFLAG_CF_LOC);
auto CF = _Select(FEXCore::IR::COND_EQ, Src2, _Constant(0), OldCF, NewCF);
// Extract the last bit shifted in to CF
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(CF);
}
// OF
{
auto OldOF = GetRFLAG(FEXCore::X86State::RFLAG_OF_LOC);
// OF is set to the XOR of the new CF bit and the most significant bit of the result
auto NewOF = _Xor(_Bfe(1, OpSize - 1, Res), NewCF);
auto OF = _Select(FEXCore::IR::COND_EQ, Src2, _Constant(0), OldOF, NewOF);
// If shift == 0, don't update flags
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(OF);
}
}
void OpDispatchBuilder::GenerateFlags_RotateRightImmediate(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, uint64_t Shift) {
if (Shift == 0) return;
auto OpSize = GetSrcSize(Op) * 8;
auto NewCF = _Bfe(1, OpSize - Shift, Src1);
// CF
{
// Extract the last bit shifted in to CF
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(NewCF);
}
// OF
{
if (Shift == 1) {
// OF is set to the XOR of the new CF bit and the most significant bit of the result
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(_Xor(_Bfe(1, OpSize - 1, Res), NewCF));
}
}
}
void OpDispatchBuilder::GenerateFlags_RotateLeftImmediate(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, uint64_t Shift) {
if (Shift == 0) return;
auto OpSize = GetSrcSize(Op) * 8;
// CF
{
// Extract the last bit shifted in to CF
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(_Bfe(1, Shift, Src1));
}
// OF
{
if (Shift == 1) {
// OF is the top two MSBs XOR'd together
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(_Xor(_Bfe(1, OpSize - 1, Src1), _Bfe(1, OpSize - 2, Src1)));
}
}
}
}
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,285 @@
// This is the vsyscall page for x86_64 guest code
// This was compiled with nasm with the following source then exported to binary
//BITS 64;
//
//align 4096, db 0xcc
// ; __NR_gettimeofday
// mov rax, 96
// syscall
// ret
//
//align 1024, db 0xcc
// ; __NR_time
// mov rax, 201
// syscall
// ret
//
//align 1024, db 0xcc
// ; __NR_getcpu
// mov rax, 309
// syscall
// ret
//
//align 4096, db 0xcc
// We only want one of these pages per FEX process
// One page
const static uint8_t VSyscallData[0x1000] = {
0xB8, 0x60, 0x00, 0x00, 0x00, 0x0F, 0x05, 0xC3, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xB8, 0xC9, 0x00, 0x00, 0x00, 0x0F, 0x05, 0xC3, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xB8, 0x35, 0x01, 0x00, 0x00, 0x0F, 0x05, 0xC3, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC
};
@@ -114,8 +114,6 @@ void InstallDebugInfo() {
GenerateDebugTable(PrimaryInstGroupOps, PrimaryGroupOpTable);
GenerateDebugTable(SecondInstGroupOps, SecondaryExtensionOpTable);
LogMan::Msg::D("Installing debug info");
}
}
#endif
+2 -2
View File
@@ -50,14 +50,14 @@ void* X86GeneratedCode::AllocateGuestCodeSpace(size_t Size) {
// We need to have the sigret handler in the lower 32bits of memory space
// Scan top down and try to allocate a location
for (size_t Location = 0xFFFF'E000; Location != 0x0; Location -= 0x1000) {
void *Ptr = FEXCore::Allocator::mmap(reinterpret_cast<void*>(Location), Size, PROT_READ | PROT_WRITE, MAP_FIXED_NOREPLACE | MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
void *Ptr = ::mmap(reinterpret_cast<void*>(Location), Size, PROT_READ | PROT_WRITE, MAP_FIXED_NOREPLACE | MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
if (Ptr != MAP_FAILED &&
reinterpret_cast<uintptr_t>(Ptr) >= LOCATION_MAX) {
// Failed to map in the lower 32bits
// Try again
// Can happen in the case that host kernel ignores MAP_FIXED_NOREPLACE
FEXCore::Allocator::munmap(Ptr, Size);
::munmap(Ptr, Size);
continue;
}
-1
View File
@@ -108,7 +108,6 @@ void InitializeInfoTables(Context::OperatingMode Mode) {
#ifndef NDEBUG
X86InstDebugInfo::InstallDebugInfo();
LogMan::Msg::D("X86Tables had %ld total insts, and %ld labeled as understood", Total, NumInsts);
#endif
}
@@ -4,6 +4,8 @@ tags: frontend|x86-tables
$end_info$
*/
#include <FEXCore/Core/Context.h>
#include "Interface/Core/X86Tables/X86Tables.h"
namespace FEXCore::X86Tables {
@@ -27,7 +27,7 @@ void InitializeSecondaryTables(Context::OperatingMode Mode) {
{0x0B, 1, X86InstInfo{"UD2", TYPE_INST, FLAGS_DEBUG | FLAGS_BLOCK_END | FLAGS_NO_OVERLAY, 0, nullptr}},
{0x0C, 1, X86InstInfo{"", TYPE_INVALID, FLAGS_NO_OVERLAY, 0, nullptr}},
{0x0D, 1, X86InstInfo{"", TYPE_GROUP_P, FLAGS_MODRM | FLAGS_NO_OVERLAY, 0, nullptr}},
{0x0E, 1, X86InstInfo{"FEMMS", TYPE_INST, FLAGS_BLOCK_END | FLAGS_NO_OVERLAY, 0, nullptr}},
{0x0E, 1, X86InstInfo{"FEMMS", TYPE_INST, FLAGS_NO_OVERLAY, 0, nullptr}},
{0x0F, 1, X86InstInfo{"", TYPE_3DNOW_TABLE, FLAGS_NO_OVERLAY, 0, nullptr}},
{0x10, 1, X86InstInfo{"MOVUPS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
@@ -6,6 +6,7 @@ $end_info$
#pragma once
#include <FEXCore/Debug/X86Tables.h>
#include <FEXCore/Core/Context.h>
#include <FEXCore/Utils/LogManager.h>
@@ -113,5 +114,7 @@ static inline void GenerateX87Table(X86InstInfo *FinalTable, U16U8InfoStruct con
}
};
void InitializeInfoTables(Context::OperatingMode Mode);
}
+18
View File
@@ -1462,6 +1462,24 @@
]
},
"AtomicFetchNeg": {
"HasSideEffects": true,
"Desc": ["Atomic integer fetch and two's complement negate",
"Dest is the value prior to operating on the value in memory"
],
"OpClass": "Atomic",
"HasDest": true,
"DestClass": "GPR",
"DestSize": "Size",
"SSAArgs": "1",
"SSANames": [
"Addr"
],
"Args": [
"uint8_t", "Size"
]
},
"VExtractToGPR": {
"Desc": ["Extracts an element from a vector and places it in a GPR",
"The element that is extracted from the vector is zero extended to the GPR size"
+2 -2
View File
@@ -82,7 +82,7 @@ IREmitter::IRPair<IROp_CodeBlock> IREmitter::CreateNewCodeBlockAfter(OrderedNode
if (insertAfter) {
LinkCodeBlocks(insertAfter, CodeNode);
} else {
LOGMAN_THROW_A(CurrentCodeBlock != nullptr, "CurrentCodeBlock must not be null here");
LOGMAN_THROW_A_FMT(CurrentCodeBlock != nullptr, "CurrentCodeBlock must not be null here");
// Find last block
auto LastBlock = CurrentCodeBlock;
@@ -101,7 +101,7 @@ IREmitter::IRPair<IROp_CodeBlock> IREmitter::CreateNewCodeBlockAfter(OrderedNode
void IREmitter::SetCurrentCodeBlock(OrderedNode *Node) {
CurrentCodeBlock = Node;
LOGMAN_THROW_A(Node->Op(DualListData.DataBegin())->Op == OP_CODEBLOCK, "Node wasn't codeblock. It was '%s'", std::string(IR::GetName(Node->Op(DualListData.DataBegin())->Op)).c_str());
LOGMAN_THROW_A_FMT(Node->Op(DualListData.DataBegin())->Op == OP_CODEBLOCK, "Node wasn't codeblock. It was '{}'", IR::GetName(Node->Op(DualListData.DataBegin())->Op));
SetWriteCursor(Node->Op(DualListData.DataBegin())->CW<IROp_CodeBlock>()->Begin.GetNode(DualListData.ListBegin()));
}
+2 -4
View File
@@ -45,10 +45,9 @@ void PassManager::AddDefaultPasses(bool InlineConstants, bool StaticRegisterAllo
InsertPass(CreateStaticRegisterAllocationPass());
}
CompactionPass = CreateIRCompaction();
// If the IR is compacted post-RA then the node indexing gets messed up and the backend isn't able to find the register assigned to a node
// Compact before IR, don't worry about RA generating spills/fills
InsertPass(CompactionPass);
CompactionPass = InsertPass(CreateIRCompaction());
}
void PassManager::AddDefaultValidationPasses() {
@@ -60,8 +59,7 @@ void PassManager::AddDefaultValidationPasses() {
}
void PassManager::InsertRegisterAllocationPass(bool OptimizeSRA) {
RAPass = IR::CreateRegisterAllocationPass(CompactionPass, OptimizeSRA);
InsertPass(RAPass);
RAPass = InsertPass(IR::CreateRegisterAllocationPass(CompactionPass, OptimizeSRA));
}
bool PassManager::Run(IREmitter *IREmit) {
+6 -6
View File
@@ -42,9 +42,9 @@ class PassManager final {
public:
void AddDefaultPasses(bool InlineConstants, bool StaticRegisterAllocation);
void AddDefaultValidationPasses();
void InsertPass(Pass *Pass) {
Pass* InsertPass(std::unique_ptr<Pass> Pass) {
Pass->RegisterPassManager(this);
Passes.emplace_back(Pass);
return Passes.emplace_back(std::move(Pass)).get();
}
void InsertRegisterAllocationPass(bool OptimizeSRA);
@@ -52,7 +52,7 @@ public:
bool Run(IREmitter *IREmit);
void RegisterExitHandler(ShouldExitHandler Handler) {
ExitHandler = Handler;
ExitHandler = std::move(Handler);
}
bool HasRAPass() const {
@@ -73,15 +73,15 @@ protected:
private:
Pass *RAPass{};
FEXCore::IR::Pass *CompactionPass{};
Pass *CompactionPass{};
std::vector<std::unique_ptr<Pass>> Passes;
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
std::vector<std::unique_ptr<Pass>> ValidationPasses;
void InsertValidationPass(Pass *Pass) {
void InsertValidationPass(std::unique_ptr<Pass> Pass) {
Pass->RegisterPassManager(this);
ValidationPasses.emplace_back(Pass);
ValidationPasses.emplace_back(std::move(Pass));
}
#endif
+15 -13
View File
@@ -1,25 +1,27 @@
#pragma once
#include <memory>
namespace FEXCore::IR {
class Pass;
class RegisterAllocationPass;
class RegisterAllocationData;
FEXCore::IR::Pass* CreateConstProp(bool InlineConstants);
FEXCore::IR::Pass* CreateContextLoadStoreElimination();
FEXCore::IR::Pass* CreateSyscallOptimization();
FEXCore::IR::Pass* CreateDeadFlagCalculationEliminination();
FEXCore::IR::Pass* CreateDeadStoreElimination();
FEXCore::IR::Pass* CreatePassDeadCodeElimination();
FEXCore::IR::Pass* CreateIRCompaction();
FEXCore::IR::RegisterAllocationPass* CreateRegisterAllocationPass(FEXCore::IR::Pass* CompactionPass, bool OptimizeSRA);
FEXCore::IR::Pass* CreateStaticRegisterAllocationPass();
FEXCore::IR::Pass* CreateLongDivideEliminationPass();
std::unique_ptr<FEXCore::IR::Pass> CreateConstProp(bool InlineConstants);
std::unique_ptr<FEXCore::IR::Pass> CreateContextLoadStoreElimination();
std::unique_ptr<FEXCore::IR::Pass> CreateSyscallOptimization();
std::unique_ptr<FEXCore::IR::Pass> CreateDeadFlagCalculationEliminination();
std::unique_ptr<FEXCore::IR::Pass> CreateDeadStoreElimination();
std::unique_ptr<FEXCore::IR::Pass> CreatePassDeadCodeElimination();
std::unique_ptr<FEXCore::IR::Pass> CreateIRCompaction();
std::unique_ptr<FEXCore::IR::RegisterAllocationPass> CreateRegisterAllocationPass(FEXCore::IR::Pass* CompactionPass, bool OptimizeSRA);
std::unique_ptr<FEXCore::IR::Pass> CreateStaticRegisterAllocationPass();
std::unique_ptr<FEXCore::IR::Pass> CreateLongDivideEliminationPass();
namespace Validation {
FEXCore::IR::Pass* CreateIRValidation();
FEXCore::IR::Pass* CreatePhiValidation();
FEXCore::IR::Pass* CreateValueDominanceValidation();
std::unique_ptr<FEXCore::IR::Pass> CreateIRValidation();
std::unique_ptr<FEXCore::IR::Pass> CreatePhiValidation();
std::unique_ptr<FEXCore::IR::Pass> CreateValueDominanceValidation();
}
}
+7 -5
View File
@@ -448,8 +448,10 @@ bool ConstProp::ZextAndMaskingElimination(IREmitter *IREmit, const IRListView& C
IREmit->ReplaceAllUsesWith(CodeNode, CurrentIR.GetNode(source));
} else if (IROp->Size == sourceHeader->Size) {
// VMOV of same size
// XXX: This is unsafe of an optimization since in some cases we can't see through garbage data in the upper bits of a vector
// RCLSE generates VMOV instructions which are being used as a zero extension
//printf("printf vmov of same size?!\n");
IREmit->ReplaceAllUsesWith(CodeNode, CurrentIR.GetNode(source));
//IREmit->ReplaceAllUsesWith(CodeNode, CurrentIR.GetNode(source));
}
break;
}
@@ -489,7 +491,7 @@ bool ConstProp::ConstantPropagation(IREmitter *IREmit, const IRListView& Current
if (IREmit->IsValueConstant(IROp->Args[0], &Constant1) &&
IREmit->IsValueConstant(IROp->Args[1], &Constant2)) {
LOGMAN_MSG_A("Could const prop op: %s", std::string(IR::GetName(IROp->Op)).c_str());
LOGMAN_MSG_A_FMT("Could const prop op: {}", IR::GetName(IROp->Op));
}
break;
}
@@ -505,7 +507,7 @@ bool ConstProp::ConstantPropagation(IREmitter *IREmit, const IRListView& Current
uint64_t Constant1;
if (IREmit->IsValueConstant(IROp->Args[0], &Constant1)) {
LOGMAN_MSG_A("Could const prop op: %s", std::string(IR::GetName(IROp->Op)).c_str());
LOGMAN_MSG_A_FMT("Could const prop op: {}", IR::GetName(IROp->Op));
}
break;
}
@@ -992,8 +994,8 @@ bool ConstProp::Run(IREmitter *IREmit) {
return Changed;
}
FEXCore::IR::Pass* CreateConstProp(bool InlineConstants) {
return new ConstProp(InlineConstants);
std::unique_ptr<FEXCore::IR::Pass> CreateConstProp(bool InlineConstants) {
return std::make_unique<ConstProp>(InlineConstants);
}
}
@@ -59,10 +59,8 @@ void DeadCodeElimination::markUsed(OrderedNodeWrapper *CodeOp, IROp_Header *IROp
}
FEXCore::IR::Pass* CreatePassDeadCodeElimination() {
return new DeadCodeElimination{};
std::unique_ptr<FEXCore::IR::Pass> CreatePassDeadCodeElimination() {
return std::make_unique<DeadCodeElimination>();
}
}
@@ -17,14 +17,36 @@ namespace {
};
enum LastAccessType {
ACCESS_NONE, ///< Was never previously accessed
ACCESS_WRITE, ///< Was fully overwritten
ACCESS_READ, ///< Was fully read
ACCESS_PARTIAL_WRITE, ///< Was partially written
ACCESS_PARTIAL_READ, ///< Was partially read
ACCESS_INVALID, ///< Accessing this is invalid
ACCESS_NONE = (0b000 << 0), ///< Was never previously accessed
ACCESS_WRITE = (0b001 << 0), ///< Was fully overwritten
ACCESS_READ = (0b010 << 0), ///< Was fully read
ACCESS_INVALID = (0b011 << 0), ///< Accessing this is invalid
ACCESS_TYPE_MASK = (0b011 << 0),
ACCESS_PARTIAL = (0b100 << 0),
ACCESS_PARTIAL_WRITE = (ACCESS_PARTIAL | ACCESS_WRITE), ///< Was partially written
ACCESS_PARTIAL_READ = (ACCESS_PARTIAL | ACCESS_READ), ///< Was partially read
};
static bool IsWriteAccess(LastAccessType Type) {
return (Type & ACCESS_TYPE_MASK) == ACCESS_WRITE;
}
static bool IsReadAccess(LastAccessType Type) {
return (Type & ACCESS_TYPE_MASK) == ACCESS_READ;
}
static bool IsInvalidAccess(LastAccessType Type) {
return (Type & ACCESS_TYPE_MASK) == ACCESS_INVALID;
}
static bool IsPartialAccess(LastAccessType Type) {
return (Type & ACCESS_PARTIAL) == ACCESS_PARTIAL;
}
static bool IsFullAccess(LastAccessType Type) {
return (Type & ACCESS_PARTIAL) == 0;
}
struct ContextMemberInfo {
ContextMemberClassification Class;
LastAccessType Accessed;
@@ -40,7 +62,7 @@ namespace {
std::vector<ContextMemberInfo> ClassificationInfo;
};
constexpr static std::array<LastAccessType, 15> DefaultAccess = {
constexpr static std::array<LastAccessType, 16> DefaultAccess = {
ACCESS_NONE,
ACCESS_NONE,
ACCESS_INVALID, // PAD
@@ -56,6 +78,7 @@ namespace {
ACCESS_NONE,
ACCESS_NONE,
ACCESS_NONE,
ACCESS_NONE,
};
static void ClassifyContextStruct(ContextInfo *ContextClassificationInfo) {
@@ -145,7 +168,7 @@ namespace {
DefaultAccess[4],
FEXCore::IR::InvalidClass,
});
ContextClassification->emplace_back(ContextMemberInfo{
ContextMemberClassification {
offsetof(FEXCore::Core::CPUState, fs),
@@ -208,6 +231,17 @@ namespace {
FEXCore::IR::InvalidClass,
});
// FTW
ContextClassification->emplace_back(ContextMemberInfo {
ContextMemberClassification {
offsetof(FEXCore::Core::CPUState, FTW),
sizeof(FEXCore::Core::CPUState::FTW),
},
DefaultAccess[15],
FEXCore::IR::InvalidClass,
});
size_t ClassifiedStructSize{};
ContextClassificationInfo->Lookup.reserve(sizeof(FEXCore::Core::CPUState));
for (auto &it : *ContextClassification) {
@@ -270,6 +304,7 @@ namespace {
}
SetAccess(Offset++, DefaultAccess[14]);
SetAccess(Offset++, DefaultAccess[15]);
}
struct BlockInfo {
@@ -283,7 +318,7 @@ class RCLSE final : public FEXCore::IR::Pass {
public:
RCLSE() {
ClassifyContextStruct(&ClassifiedStruct);
DCE.reset(FEXCore::IR::CreatePassDeadCodeElimination());
DCE = FEXCore::IR::CreatePassDeadCodeElimination();
}
bool Run(FEXCore::IR::IREmitter *IREmit) override;
private:
@@ -431,6 +466,8 @@ bool RCLSE::RedundantStoreLoadElimination(FEXCore::IR::IREmitter *IREmit) {
ContextInfo &LocalInfo = ClassifiedStruct;
for (auto [BlockNode, BlockHeader] : CurrentIR.GetBlocks()) {
auto BlockOp = BlockHeader->CW<FEXCore::IR::IROp_CodeBlock>();
auto BlockEnd = IREmit->GetIterator(BlockOp->Last);
ResetClassificationAccesses(&LocalInfo);
@@ -443,9 +480,9 @@ bool RCLSE::RedundantStoreLoadElimination(FEXCore::IR::IREmitter *IREmit) {
uint8_t LastSize = Info->AccessSize;
LastAccessType LastAccess = Info->Accessed;
OrderedNode *LastStoreNode = Info->StoreNode;
RecordAccess(Info, Op->Class, Op->Offset, IROp->Size, ACCESS_WRITE, CurrentIR.GetNode(Op->Header.Args[0]), CodeNode);
RecordAccess(Info, Op->Class, Op->Offset, IROp->Size, ACCESS_WRITE, CurrentIR.GetNode(Op->Value), CodeNode);
if ((LastAccess == ACCESS_WRITE || LastAccess == ACCESS_PARTIAL_WRITE) &&
if (IsWriteAccess(LastAccess) &&
LastClass == Op->Class &&
LastOffset == Op->Offset &&
LastSize <= IROp->Size) {
@@ -470,7 +507,7 @@ bool RCLSE::RedundantStoreLoadElimination(FEXCore::IR::IREmitter *IREmit) {
OrderedNode *LastStoreNode = Info->StoreNode;
RecordAccess(Info, Op->Class, Op->Offset, IROp->Size, ACCESS_READ, CodeNode);
if ((LastAccess == ACCESS_WRITE || LastAccess == ACCESS_PARTIAL_WRITE) &&
if (IsWriteAccess(LastAccess) &&
LastClass == Op->Class &&
LastOffset == Op->Offset &&
IROp->Size <= LastSize) {
@@ -491,46 +528,52 @@ bool RCLSE::RedundantStoreLoadElimination(FEXCore::IR::IREmitter *IREmit) {
if (TruncateSize != IREmit->GetOpSize(LastNode)) {
// We need to insert an explict truncation
if (LastClass == FPRClass) {
LastNode = IREmit->_VMov(LastNode, TruncateSize); // Vmov truncates and zexts when register width is smaller than source
}
else if (LastClass == GPRPairClass) {
LastNode = IREmit->_TruncElementPair(LastNode, TruncateSize);
}
else if (LastClass == GPRClass) {
LastNode = IREmit->_Bfe(Info->AccessSize, TruncateSize * 8, 0, LastNode);
} else {
LOGMAN_MSG_A("Unhandled Register class");
}
LastNode = IREmit->_Bfe(Info->AccessSize, TruncateSize * 8, 0, LastNode);
}
IREmit->ReplaceAllUsesWith(CodeNode, LastNode);
IREmit->ReplaceAllUsesWithRange(CodeNode, LastNode, IREmit->GetIterator(IREmit->WrapNode(CodeNode)), BlockEnd);
RecordAccess(Info, Op->Class, Op->Offset, IROp->Size, ACCESS_READ, LastNode);
Changed = true;
} else {
if (LastClass == FPRClass && LastSize == IROp->Size && LastSize == IREmit->GetOpSize(LastNode)) {
// LoadCtx matches StoreCtx and Node Size
IREmit->ReplaceAllUsesWith(CodeNode, LastNode);
RecordAccess(Info, Op->Class, Op->Offset, IROp->Size, ACCESS_READ, LastNode);
Changed = true;
} else if (LastClass == FPRClass && LastSize >= IROp->Size && IROp->Size == IREmit->GetOpSize(LastNode)) {
} else if (LastClass == FPRClass) {
if (LastSize == IROp->Size && LastSize == IREmit->GetOpSize(LastNode)) {
if (IsFullAccess(Info->Accessed)) {
// LoadCtx matches StoreCtx and Node Size
IREmit->ReplaceAllUsesWithRange(CodeNode, LastNode, IREmit->GetIterator(IREmit->WrapNode(CodeNode)), BlockEnd);
RecordAccess(Info, Op->Class, Op->Offset, IROp->Size, ACCESS_READ, LastNode);
Changed = true;
}
else {
// If this load size is a partial load then it may be expecting a zext of
// the vector element
IREmit->SetWriteCursor(CodeNode);
// zext to size
LastNode = IREmit->_VMov(LastNode, IROp->Size);
IREmit->ReplaceAllUsesWithRange(CodeNode, LastNode, IREmit->GetIterator(IREmit->WrapNode(CodeNode)), BlockEnd);
RecordAccess(Info, Op->Class, Op->Offset, IROp->Size, ACCESS_READ, LastNode);
Changed = true;
}
} else if (LastSize >= IROp->Size &&
IROp->Size == IREmit->GetOpSize(LastNode)) {
// LoadCtx is <= StoreCtx and Node is LoadCtx
IREmit->ReplaceAllUsesWith(CodeNode, LastNode);
IREmit->ReplaceAllUsesWithRange(CodeNode, LastNode, IREmit->GetIterator(IREmit->WrapNode(CodeNode)), BlockEnd);
RecordAccess(Info, Op->Class, Op->Offset, IROp->Size, ACCESS_READ, LastNode);
Changed = true;
} else if (LastClass == FPRClass && LastSize >= IROp->Size && IROp->Size < IREmit->GetOpSize(LastNode)) {
} else if (LastSize >= IROp->Size &&
IROp->Size < IREmit->GetOpSize(LastNode)) {
IREmit->SetWriteCursor(CodeNode);
// trucate to size
LastNode = IREmit->_VMov(LastNode, IROp->Size);
IREmit->ReplaceAllUsesWith(CodeNode, LastNode);
IREmit->ReplaceAllUsesWithRange(CodeNode, LastNode, IREmit->GetIterator(IREmit->WrapNode(CodeNode)), BlockEnd);
RecordAccess(Info, Op->Class, Op->Offset, IROp->Size, ACCESS_READ, LastNode);
Changed = true;
} else if (LastClass == FPRClass && LastSize >= IROp->Size && IROp->Size > IREmit->GetOpSize(LastNode)) {
} else if (LastSize >= IROp->Size &&
IROp->Size > IREmit->GetOpSize(LastNode)) {
IREmit->SetWriteCursor(CodeNode);
// zext to size
LastNode = IREmit->_VMov(LastNode, IROp->Size);
IREmit->ReplaceAllUsesWith(CodeNode, LastNode);
IREmit->ReplaceAllUsesWithRange(CodeNode, LastNode, IREmit->GetIterator(IREmit->WrapNode(CodeNode)), BlockEnd);
RecordAccess(Info, Op->Class, Op->Offset, IROp->Size, ACCESS_READ, LastNode);
Changed = true;
} else {
@@ -538,13 +581,13 @@ bool RCLSE::RedundantStoreLoadElimination(FEXCore::IR::IREmitter *IREmit) {
}
}
}
else if ((LastAccess == ACCESS_READ || LastAccess == ACCESS_PARTIAL_READ) &&
else if (IsReadAccess(LastAccess) &&
IsReadAccess(Info->Accessed) &&
LastClass == Op->Class &&
LastOffset == Op->Offset &&
LastSize == IROp->Size &&
(Info->Accessed == ACCESS_READ || Info->Accessed == ACCESS_PARTIAL_READ)) {
LastSize == IROp->Size) {
// Did we read and then read again?
IREmit->ReplaceAllUsesWith(CodeNode, LastNode);
IREmit->ReplaceAllUsesWithRange(CodeNode, LastNode, IREmit->GetIterator(IREmit->WrapNode(CodeNode)), BlockEnd);
RecordAccess(Info, Op->Class, Op->Offset, IROp->Size, ACCESS_READ, LastNode);
Changed = true;
}
@@ -591,21 +634,21 @@ bool RCLSE::RedundantStoreLoadElimination(FEXCore::IR::IREmitter *IREmit) {
LastAccessType LastAccess = Info->Accessed;
OrderedNode *LastNode = Info->Node;
if (LastAccess == ACCESS_WRITE) { // 1 byte so always a full write
if (IsWriteAccess(LastAccess)) { // 1 byte so always a full write
// If the last store matches this load value then we can replace the loaded value with the previous valid one
IREmit->SetWriteCursor(CodeNode);
IREmit->ReplaceAllUsesWith(CodeNode, LastNode);
RecordAccess(Info, FEXCore::IR::GPRClass, offsetof(FEXCore::Core::CPUState, flags[0]) + Op->Flag, 1, ACCESS_READ, LastNode);
Changed = true;
}
else if (LastAccess == ACCESS_READ) {
else if (IsReadAccess(LastAccess)) {
IREmit->ReplaceAllUsesWith(CodeNode, LastNode);
RecordAccess(Info, FEXCore::IR::GPRClass, offsetof(FEXCore::Core::CPUState, flags[0]) + Op->Flag, 1, ACCESS_READ, LastNode);
Changed = true;
}
}
else if (IROp->Op == OP_STORECONTEXTINDEXED ||
IROp->Op == OP_LOADCONTEXTINDEXED ||
IROp->Op == OP_LOADCONTEXTINDEXED ||
IROp->Op == OP_SYSCALL) {
// We can't track through these
ResetClassificationAccesses(&LocalInfo);
@@ -622,7 +665,7 @@ bool RCLSE::Run(FEXCore::IR::IREmitter *IREmit) {
// XXX: We don't do cross-block optimizations yet
//CalculateControlFlowInfo(IREmit);
bool Changed = false;
// Run up to 5 times
for( int i = 0; i < 5 && RedundantStoreLoadElimination(IREmit); i++) {
Changed = true;
@@ -636,8 +679,8 @@ bool RCLSE::Run(FEXCore::IR::IREmitter *IREmit) {
namespace FEXCore::IR {
FEXCore::IR::Pass* CreateContextLoadStoreElimination() {
return new RCLSE{};
std::unique_ptr<FEXCore::IR::Pass> CreateContextLoadStoreElimination() {
return std::make_unique<RCLSE>();
}
}
@@ -331,8 +331,8 @@ bool DeadStoreElimination::Run(IREmitter *IREmit) {
return Changed;
}
FEXCore::IR::Pass* CreateDeadStoreElimination() {
return new DeadStoreElimination{};
std::unique_ptr<FEXCore::IR::Pass> CreateDeadStoreElimination() {
return std::make_unique<DeadStoreElimination>();
}
}
@@ -153,8 +153,10 @@ bool IRCompaction::Run(IREmitter *IREmit) {
{
// Fixup the arguments of all the IROps
for (auto &Block : GeneratedCodeBlocks) {
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
auto BlockIROp = LocalIR.GetOp<FEXCore::IR::IROp_CodeBlock>(Block.NewNode);
LOGMAN_THROW_A(BlockIROp->Header.Op == OP_CODEBLOCK, "IR type failed to be a code block");
#endif
for (auto [LocalNode, LocalIROp] : LocalIR.GetCode(Block.NewNode)) {
@@ -199,8 +201,8 @@ bool IRCompaction::Run(IREmitter *IREmit) {
return true;
}
FEXCore::IR::Pass* CreateIRCompaction() {
return new IRCompaction{};
std::unique_ptr<FEXCore::IR::Pass> CreateIRCompaction() {
return std::make_unique<IRCompaction>();
}
}
@@ -11,7 +11,7 @@ $end_info$
#include "Interface/Core/OpcodeDispatcher.h"
#include "Common/BitSet.h"
#include <iostream>
#include <sstream>
namespace {
struct BlockInfo {
@@ -54,8 +54,10 @@ bool IRValidation::Run(IREmitter *IREmit) {
std::vector<uint32_t> Uses(CurrentIR.GetSSACount(), 0);
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
auto HeaderOp = CurrentIR.GetHeader();
LOGMAN_THROW_A(HeaderOp->Header.Op == OP_IRHEADER, "First op wasn't IRHeader");
LOGMAN_THROW_A_FMT(HeaderOp->Header.Op == OP_IRHEADER, "First op wasn't IRHeader");
#endif
IR::RegisterAllocationData * RAData{};
if (Manager->HasRAPass()) {
@@ -66,7 +68,7 @@ bool IRValidation::Run(IREmitter *IREmit) {
for (auto [BlockNode, BlockHeader] : CurrentIR.GetBlocks()) {
auto BlockIROp = BlockHeader->CW<FEXCore::IR::IROp_CodeBlock>();
LOGMAN_THROW_A(BlockIROp->Header.Op == OP_CODEBLOCK, "IR type failed to be a code block");
LOGMAN_THROW_A_FMT(BlockIROp->Header.Op == OP_CODEBLOCK, "IR type failed to be a code block");
uint32_t BlockID = CurrentIR.GetID(BlockNode);
@@ -209,7 +211,7 @@ bool IRValidation::Run(IREmitter *IREmit) {
break;
}
default:
// LOGMAN_MSG_A("Unknown IR Op: %d(%s)", IROp->Op, FEXCore::IR::GetName(IROp->Op).data());
// LOGMAN_MSG_A_FMT("Unknown IR Op: {}({})", IROp->Op, FEXCore::IR::GetName(IROp->Op));
break;
}
}
@@ -265,10 +267,9 @@ bool IRValidation::Run(IREmitter *IREmit) {
}
}
std::stringstream Out;
HadWarning = false;
if (HadError || HadWarning) {
std::stringstream Out;
FEXCore::IR::Dump(&Out, &CurrentIR, RAData);
if (HadError) {
@@ -279,13 +280,13 @@ bool IRValidation::Run(IREmitter *IREmit) {
Out << "Warnings:" << std::endl << Warnings.str() << std::endl;
}
LogMan::Msg::E("%s", Out.str().c_str());
LogMan::Msg::EFmt("{}", Out.str());
}
return false;
}
FEXCore::IR::Pass* CreateIRValidation() {
return new IRValidation{};
std::unique_ptr<FEXCore::IR::Pass> CreateIRValidation() {
return std::make_unique<IRValidation>();
}
}
@@ -106,7 +106,7 @@ bool LongDivideEliminationPass::Run(IREmitter *IREmit) {
return Changed;
}
FEXCore::IR::Pass* CreateLongDivideEliminationPass() {
return new LongDivideEliminationPass{};
std::unique_ptr<FEXCore::IR::Pass> CreateLongDivideEliminationPass() {
return std::make_unique<LongDivideEliminationPass>();
}
}
@@ -8,7 +8,7 @@ $end_info$
#include "Interface/IR/PassManager.h"
#include "Interface/Core/OpcodeDispatcher.h"
#include <iostream>
#include <sstream>
namespace FEXCore::IR::Validation {
@@ -52,22 +52,18 @@ bool PhiValidation::Run(IREmitter *IREmit) {
}
}
std::stringstream Out;
if (HadError) {
std::stringstream Out;
FEXCore::IR::Dump(&Out, &CurrentIR, nullptr);
Out << "Errors:" << std::endl << Errors.str() << std::endl;
LogMan::Msg::E(Out.str().c_str());
LogMan::Msg::EFmt("{}", Out.str());
}
return false;
}
FEXCore::IR::Pass* CreatePhiValidation() {
return new PhiValidation{};
std::unique_ptr<FEXCore::IR::Pass> CreatePhiValidation() {
return std::make_unique<PhiValidation>();
}
}
@@ -59,8 +59,8 @@ bool DeadFlagCalculationEliminination::Run(IREmitter *IREmit) {
return Changed;
}
FEXCore::IR::Pass* CreateDeadFlagCalculationEliminination() {
return new DeadFlagCalculationEliminination{};
std::unique_ptr<FEXCore::IR::Pass> CreateDeadFlagCalculationEliminination() {
return std::make_unique<DeadFlagCalculationEliminination>();
}
}
@@ -12,6 +12,7 @@ $end_info$
#include <iterator>
#include <unordered_set>
#include <sys/mman.h>
#define SRA_DEBUG(...) // printf(__VA_ARGS__)
@@ -46,7 +47,7 @@ namespace {
for (int i = 1; i < Size; i++)
Items[i] = 0xDEADBEEF;
#endif
Next.release();
Next.reset();
}
BucketList() {
@@ -144,7 +145,7 @@ namespace {
}
else if (++i == Size) {
if (that->Next->Items[0] == 0) {
that->Next.release();
that->Next.reset();
foundThat->Items[foundI] = that->Items[Size-1];
that->Items[Size-1] = 0;
break;
@@ -177,6 +178,7 @@ namespace {
};
static_assert(sizeof(RegisterNode) == 128 * 4);
constexpr size_t REGISTER_NODES_PER_PAGE = FEXCore::Core::PAGE_SIZE / sizeof(RegisterNode);
struct RegisterSet {
std::vector<RegisterClass> Classes;
@@ -204,8 +206,8 @@ namespace {
struct RegisterGraph {
std::unique_ptr<FEXCore::IR::RegisterAllocationData, FEXCore::IR::RegisterAllocationDataDeleter> AllocData;
RegisterSet Set;
std::vector<RegisterNode> Nodes;
uint32_t NodeCount;
std::vector<RegisterNode> Nodes{};
uint32_t NodeCount{};
std::vector<SpillStackUnit> SpillStack;
std::unordered_map<uint32_t, std::unordered_set<uint32_t>> BlockPredecessors;
std::unordered_map<uint32_t, std::unordered_set<uint32_t>> VisitedNodePredecessors;
@@ -258,11 +260,14 @@ namespace {
}
void ResetRegisterGraph(RegisterGraph *Graph, uint64_t NodeCount) {
NodeCount = AlignUp(NodeCount, sizeof(uint64_t));
NodeCount = AlignUp(NodeCount, REGISTER_NODES_PER_PAGE);
// Clear to free the Bucketlists which have unique_ptrs
// Resize to our correct size
Graph->Nodes.clear();
Graph->Nodes.resize(NodeCount);
Graph->VisitedNodePredecessors.clear();
Graph->AllocData.reset();
Graph->AllocData.reset((FEXCore::IR::RegisterAllocationData*)FEXCore::Allocator::malloc(FEXCore::IR::RegisterAllocationData::Size(NodeCount)));
memset(&Graph->AllocData->Map[0], INVALID_REGCLASS.Raw, NodeCount);
Graph->AllocData->MapCount = NodeCount;
@@ -1399,11 +1404,13 @@ namespace FEXCore::IR {
if (InterferenceNode != ~0U) {
FEXCore::IR::RegisterClassType InterferenceRegClass = FEXCore::IR::RegisterClassType{Graph->AllocData->Map[InterferenceNode].Class};
uint32_t SpillSlot = FindSpillSlot(InterferenceNode, InterferenceRegClass);
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
RegisterNode *InterferenceRegisterNode = &Graph->Nodes[InterferenceNode];
LOGMAN_THROW_A(SpillSlot != ~0U, "Interference Node doesn't have a spill slot!");
//LOGMAN_THROW_A(InterferenceRegisterNode->Head.RegAndClass.Reg != INVALID_REG, "Interference node never assigned a register?");
LOGMAN_THROW_A(InterferenceRegClass != ~0U, "Interference node never assigned a register class?");
LOGMAN_THROW_A(InterferenceRegisterNode->Head.PhiPartner == nullptr, "We don't support spilling PHI nodes currently");
#endif
// This is the op that we need to dump
auto [InterferenceOrderedNode, InterferenceIROp] = IR.at(InterferenceNode)();
@@ -1538,7 +1545,7 @@ namespace FEXCore::IR {
return Changed;
}
FEXCore::IR::RegisterAllocationPass* CreateRegisterAllocationPass(FEXCore::IR::Pass* CompactionPass, bool OptimizeSRA) {
return new ConstrainedRAPass{CompactionPass, OptimizeSRA};
std::unique_ptr<FEXCore::IR::RegisterAllocationPass> CreateRegisterAllocationPass(FEXCore::IR::Pass* CompactionPass, bool OptimizeSRA) {
return std::make_unique<ConstrainedRAPass>(CompactionPass, OptimizeSRA);
}
}
@@ -94,8 +94,8 @@ bool StaticRegisterAllocationPass::Run(IREmitter *IREmit) {
return true;
}
FEXCore::IR::Pass* CreateStaticRegisterAllocationPass() {
return new StaticRegisterAllocationPass{};
std::unique_ptr<FEXCore::IR::Pass> CreateStaticRegisterAllocationPass() {
return std::make_unique<StaticRegisterAllocationPass>();
}
}
@@ -44,12 +44,11 @@ bool SyscallOptimization::Run(IREmitter *IREmit) {
}
}
return Changed;
}
FEXCore::IR::Pass* CreateSyscallOptimization() {
return new SyscallOptimization{};
std::unique_ptr<FEXCore::IR::Pass> CreateSyscallOptimization() {
return std::make_unique<SyscallOptimization>();
}
}
@@ -8,9 +8,9 @@ $end_info$
#include "Interface/IR/PassManager.h"
#include "Interface/Core/OpcodeDispatcher.h"
#include <iostream>
#include <map>
#include <list>
#include <sstream>
#include <unordered_map>
namespace {
@@ -28,12 +28,9 @@ public:
bool ValueDominanceValidation::Run(IREmitter *IREmit) {
bool HadError = false;
bool HadWarning = false;
auto CurrentIR = IREmit->ViewIR();
std::ostringstream Errors;
std::ostringstream Warnings;
std::unordered_map<IR::OrderedNodeWrapper::NodeOffsetType, BlockInfo> OffsetToBlockMap;
for (auto [BlockNode, BlockHeader] : CurrentIR.GetBlocks()) {
@@ -193,27 +190,18 @@ bool ValueDominanceValidation::Run(IREmitter *IREmit) {
}
}
std::stringstream Out;
if (HadError || HadWarning) {
if (HadError) {
std::stringstream Out;
FEXCore::IR::Dump(&Out, &CurrentIR, nullptr);
if (HadError) {
Out << "Errors:" << std::endl << Errors.str() << std::endl;
}
if (HadWarning) {
Out << "Warnings:" << std::endl << Warnings.str() << std::endl;
}
LogMan::Msg::E(Out.str().c_str());
Out << "Errors:" << std::endl << Errors.str() << std::endl;
LogMan::Msg::EFmt("{}", Out.str());
}
return false;
}
FEXCore::IR::Pass* CreateValueDominanceValidation() {
return new ValueDominanceValidation{};
std::unique_ptr<FEXCore::IR::Pass> CreateValueDominanceValidation() {
return std::make_unique<ValueDominanceValidation>();
}
}
+18 -55
View File
@@ -1,39 +1,36 @@
#include "Utils/Allocator/HostAllocator.h"
#include <FEXCore/Utils/Allocator.h>
#include <sys/mman.h>
#ifdef ENABLE_JEMALLOC
#include <jemalloc/jemalloc.h>
#endif
#include <memory>
#include <malloc.h>
extern "C" {
extern void *__libc_malloc(size_t size);
extern void *__libc_realloc(void *ptr, size_t size);
extern void __libc_free(void *ptr);
typedef void* (*mmap_hook_type)(
void *addr, size_t length, int prot, int flags,
int fd, off_t offset);
typedef int (*munmap_hook_type)(void *addr, size_t length);
#ifdef ENABLE_JEMALLOC
extern mmap_hook_type __mmap_hook;
extern munmap_hook_type __munmap_hook;
static FEXCore::Allocator::MALLOC_Hook global_malloc {::__libc_malloc};
static FEXCore::Allocator::REALLOC_Hook global_realloc {::__libc_realloc};
static FEXCore::Allocator::FREE_Hook global_free {::__libc_free};
// Override the global functions
FEX_DEFAULT_VISIBILITY void *malloc(size_t size) { return global_malloc(size); }
FEX_DEFAULT_VISIBILITY void *realloc(void *ptr, size_t size) { return global_realloc(ptr, size); }
FEX_DEFAULT_VISIBILITY void free(void *ptr) { return global_free(ptr); }
#endif
}
namespace FEXCore::Allocator {
MMAP_Hook mmap {::mmap};
MUNMAP_Hook munmap {::munmap};
MALLOC_Hook malloc {::__libc_malloc};
REALLOC_Hook realloc {::__libc_realloc};
FREE_Hook free {::__libc_free};
#ifdef ENABLE_JEMALLOC
MALLOC_Hook malloc {::je_malloc};
REALLOC_Hook realloc {::je_realloc};
FREE_Hook free {::je_free};
#else
MALLOC_Hook malloc {::malloc};
REALLOC_Hook realloc {::realloc};
FREE_Hook free {::free};
#endif
using GLIBC_MALLOC_Hook = void*(*)(size_t, const void *caller);
using GLIBC_REALLOC_Hook = void*(*)(void*, size_t, const void *caller);
@@ -59,60 +56,26 @@ namespace FEXCore::Allocator {
return Result;
}
void *FEX_malloc_hook(size_t size, const void *caller) {
return ::je_malloc(size);
}
void *FEX_realloc_hook(void *ptr, size_t size, const void *caller) {
return ::je_realloc(ptr, size);
}
void FEX_free_hook(void *ptr, const void *caller) {
return ::je_free(ptr);
}
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wdeprecated-declarations"
void SetupHooks() {
Alloc64.reset(Alloc::OSAllocator::Create64BitAllocator());
Alloc64 = Alloc::OSAllocator::Create64BitAllocator();
#ifdef ENABLE_JEMALLOC
__mmap_hook = FEX_mmap;
__munmap_hook = FEX_munmap;
#endif
FEXCore::Allocator::mmap = FEX_mmap;
FEXCore::Allocator::munmap = FEX_munmap;
FEXCore::Allocator::malloc = ::je_malloc;
FEXCore::Allocator::realloc = ::je_realloc;
FEXCore::Allocator::free = ::je_free;
global_malloc = ::je_malloc;
global_realloc = ::je_realloc;
global_free = ::je_free;
__malloc_hook = FEXCore::Allocator::FEX_malloc_hook;
__realloc_hook = FEXCore::Allocator::FEX_realloc_hook;
__free_hook = FEXCore::Allocator::FEX_free_hook;
}
void ClearHooks() {
#ifdef ENABLE_JEMALLOC
__mmap_hook = ::mmap;
__munmap_hook = ::munmap;
#endif
FEXCore::Allocator::mmap = ::mmap;
FEXCore::Allocator::munmap = ::munmap;
FEXCore::Allocator::malloc = ::__libc_malloc;
FEXCore::Allocator::realloc = ::__libc_realloc;
FEXCore::Allocator::free = ::__libc_free;
global_malloc = ::__libc_malloc;
global_realloc = ::__libc_realloc;
global_free = ::__libc_free;
// Reset's glibc hooks
__malloc_hook = 0;
__realloc_hook = 0;
__free_hook = 0;
}
#pragma GCC diagnostic pop
}
extern "C" {
}
+71 -25
View File
@@ -122,7 +122,9 @@ namespace Alloc::OSAllocator {
size_t SizeOfLiveRegion = AlignUp(LiveVMARegion::GetSizeWithFlexSet(ReservedRegion->RegionSize), PAGE_SIZE);
size_t SizePlusManagedData = UsedSize + SizeOfLiveRegion;
mprotect(reinterpret_cast<void*>(ReservedRegion->Base), SizePlusManagedData, PROT_READ | PROT_WRITE);
[[maybe_unused]] auto Res = mprotect(reinterpret_cast<void*>(ReservedRegion->Base), SizePlusManagedData, PROT_READ | PROT_WRITE);
LOGMAN_THROW_A(Res == 0, "Couldn't mprotect region: %d '%s' Likely occurs when running out of memory or Maximum VMAs", errno, strerror(errno));
LiveVMARegion *LiveRange = new (reinterpret_cast<void*>(ReservedRegion->Base)) LiveVMARegion();
@@ -147,7 +149,7 @@ namespace Alloc::OSAllocator {
};
void OSAllocator_64Bit::DetermineVASize() {
const std::vector<uintptr_t> TLBSizes = {{
static constexpr std::array<uintptr_t, 7> TLBSizes = {
1ULL << 57,
1ULL << 52,
1ULL << 48,
@@ -155,7 +157,7 @@ void OSAllocator_64Bit::DetermineVASize() {
1ULL << 42,
1ULL << 39,
1ULL << 36,
}};
};
for (auto Size : TLBSizes) {
// Just try allocating
@@ -264,15 +266,17 @@ void *OSAllocator_64Bit::Mmap(void *addr, size_t length, int prot, int flags, in
(StartingPosition - Region->SlabInfo->Base) >> PAGE_SHIFT
: Region->LastPageAllocation;
size_t RegionNumberOfPages = Region->SlabInfo->RegionSize >> PAGE_SHIFT;
try_again:
for (size_t CurrentPage = LastAllocation;
CurrentPage < (RegionNumberOfPages - NumberOfPages);) {
// If we have enough free space, check if we have enough free pages that are contiguous
size_t Remaining = NumberOfPages;
assert((CurrentPage + Remaining - 1) < RegionNumberOfPages);
// Backward scan
// We need to do a backward scan first to fill any holes
// Otherwise we will very quickly run out of VMA regions (65k maximum)
for (size_t CurrentPage = LastAllocation;
CurrentPage >= NumberOfPages;) {
size_t Remaining = NumberOfPages;
assert(Remaining <= CurrentPage);
while (Remaining) {
if (Region->UsedPages[CurrentPage + Remaining - 1]) {
if (Region->UsedPages[CurrentPage - Remaining]) {
// Has an intersecting range
break;
}
@@ -281,20 +285,51 @@ void *OSAllocator_64Bit::Mmap(void *addr, size_t length, int prot, int flags, in
if (Remaining) {
// Didn't find a slab range
CurrentPage += Remaining;
CurrentPage -= Remaining;
}
else {
// We have a slab range
CurrentPage -= NumberOfPages;
// Keep scanning backwards to not introduce ANOTHER gap
while (CurrentPage >= 1) {
if (Region->UsedPages[CurrentPage - 1]) {
// Found a used page, we can leave now
break;
}
--CurrentPage;
}
AllocatedPage = CurrentPage;
break;
}
}
if (!AllocatedPage && LastAllocation != 0) {
// Try again but starting from the beginning
LastAllocation = 0;
// Using goto so we don't have recursive mutex shenanigans
goto try_again;
// Foward Scan
if (AllocatedPage == 0) {
for (size_t CurrentPage = LastAllocation;
CurrentPage < (RegionNumberOfPages - NumberOfPages);) {
// If we have enough free space, check if we have enough free pages that are contiguous
size_t Remaining = NumberOfPages;
assert((CurrentPage + Remaining - 1) < RegionNumberOfPages);
while (Remaining) {
if (Region->UsedPages[CurrentPage + Remaining - 1]) {
// Has an intersecting range
break;
}
--Remaining;
}
if (Remaining) {
// Didn't find a slab range
CurrentPage += Remaining;
}
else {
// We have a slab range
AllocatedPage = CurrentPage;
break;
}
}
}
if (AllocatedPage) {
@@ -457,7 +492,7 @@ int OSAllocator_64Bit::Munmap(void *addr, size_t length) {
// Live region fully encompasses slab range
uint64_t FreedPages{};
uint64_t SlabPageBegin = (PtrBegin - RegionBegin) >> PAGE_SHIFT;
uint32_t SlabPageBegin = (PtrBegin - RegionBegin) >> PAGE_SHIFT;
uint64_t PagesToFree = length >> PAGE_SHIFT;
for (size_t i = 0; i < PagesToFree; ++i) {
@@ -476,6 +511,10 @@ int OSAllocator_64Bit::Munmap(void *addr, size_t length) {
(*it)->FreeSpace += FreedPages * 4096;
// Set the last allocated page to the minimum of last page allocation or this slab
// This will let us more quickly fill holes
(*it)->LastPageAllocation = std::min((*it)->LastPageAllocation, SlabPageBegin);
// XXX: Move region back to reserved list
return 0;
}
@@ -542,7 +581,7 @@ OSAllocator_64Bit::PtrCache *OSAllocator_64Bit::Steal32BitIfOldKernel() {
// If we managed to allocate and not get the address we want then unmap it
// This happens with kernels older than 4.17
if (reinterpret_cast<uintptr_t>(Ptr) + AllocationSize > UPPER_BOUND_32) {
munmap(Ptr, AllocationSize);
::munmap(Ptr, AllocationSize);
Ptr = reinterpret_cast<void*>(~0ULL);
}
@@ -594,7 +633,7 @@ void OSAllocator_64Bit::Clear32BitOnOldKernel(OSAllocator_64Bit::PtrCache *Base)
for (size_t i = 0;; ++i) {
void *Ptr = reinterpret_cast<void*>(Base[i].Ptr);
size_t Size = Base[i].Size;
munmap(Ptr, Size);
::munmap(Ptr, Size);
if (Ptr == Base) {
break;
}
@@ -602,7 +641,6 @@ void OSAllocator_64Bit::Clear32BitOnOldKernel(OSAllocator_64Bit::PtrCache *Base)
}
OSAllocator_64Bit::OSAllocator_64Bit() {
malloc_trim(0);
DetermineVASize();
auto ArrayPtr = Steal32BitIfOldKernel();
@@ -641,7 +679,7 @@ OSAllocator_64Bit::OSAllocator_64Bit() {
// This happens with kernels older than 4.17
if (reinterpret_cast<uintptr_t>(Ptr) != MemoryOffset &&
reinterpret_cast<uintptr_t>(Ptr) < LOWER_BOUND) {
munmap(Ptr, AllocationSize);
::munmap(Ptr, AllocationSize);
Ptr = reinterpret_cast<void*>(~0ULL);
}
@@ -659,13 +697,21 @@ OSAllocator_64Bit::OSAllocator_64Bit() {
if (!ObjectAlloc) {
// Steal the first allocation for an intrusive allocator
// Will be mprotected correctly already
int Result = mprotect(Ptr, AllocationSize, PROT_READ | PROT_WRITE);
LogMan::Throw::A(Result == 0, "mprotect(%p, 0x%lx) -> %d (%s)", Ptr, AllocationSize, Result, strerror(errno));
mprotect(Ptr, AllocationSize, PROT_READ | PROT_WRITE);
ObjectAlloc = new (Ptr) Alloc::ForwardOnlyIntrusiveArenaAllocator(Ptr, AllocationSize);
ReservedRegions = ObjectAlloc->new_construct(ReservedRegions, ObjectAlloc);
LiveRegions = ObjectAlloc->new_construct(LiveRegions, ObjectAlloc);
}
else {
// If the allocation size is large than a page, then try allowing it to be a huge page
// This enables the kernel to use transparent large pages in the allocator which can reduce memory pressure
// Considering we are allocating the entire VA space, this is a good thing
// If MADV_HUGEPAGE isn't support then this will fail harmlessly
if (AllocationSize > 4096) {
::madvise(Ptr, AllocationSize, MADV_HUGEPAGE);
}
bool Merged = false;
if (PrevReserved) {
Merged = MergeReservedRegionIfPossible(PrevReserved, reinterpret_cast<uint64_t>(Ptr), AllocationSize);
@@ -709,7 +755,7 @@ OSAllocator_64Bit::~OSAllocator_64Bit() {
}
}
Alloc::HostAllocator *Create64BitAllocator() {
return new OSAllocator_64Bit{};
std::unique_ptr<Alloc::HostAllocator> Create64BitAllocator() {
return std::make_unique<OSAllocator_64Bit>();
}
}
+7 -7
View File
@@ -14,18 +14,18 @@ struct FlexBitSet final {
T Memory[];
bool Get(T Element) {
bool Get(size_t Element) const {
return (Memory[Element / MinimumSizeBits] & (1ULL << (Element % MinimumSizeBits))) != 0;
}
bool TestAndClear(T Element) {
bool TestAndClear(size_t Element) {
bool Value = Get(Element);
Memory[Element / MinimumSizeBits] &= ~(1ULL << (Element % MinimumSizeBits));
return Value;
}
void Set(T Element) {
void Set(size_t Element) {
Memory[Element / MinimumSizeBits] |= (1ULL << (Element % MinimumSizeBits));
}
void Clear(T Element) {
void Clear(size_t Element) {
Memory[Element / MinimumSizeBits] &= ~(1ULL << (Element % MinimumSizeBits));
}
void MemClear(size_t Elements) {
@@ -37,14 +37,14 @@ struct FlexBitSet final {
// This very explicitly doesn't let you take an address
// Is only a getter
bool operator[](T Element) {
bool operator[](size_t Element) const {
return Get(Element);
}
static size_t Size(T Elements) {
static size_t Size(uint64_t Elements) {
return Alloc::AlignUp(Elements / MinimumSizeBits, MinimumSizeBits);
}
};
static_assert(sizeof(FlexBitSet<uint64_t>) == 0, "This needs to be a flex member");
static_assert(std::is_trivially_copyable<FlexBitSet<uint64_t>>::value, "Needsto be trivially copyable");
static_assert(std::is_trivially_copyable_v<FlexBitSet<uint64_t>>, "Needs to be trivially copyable");
+4 -3
View File
@@ -1,6 +1,8 @@
#pragma once
#include <cstddef>
#include <cstdint>
#include <memory>
#include <sys/types.h>
constexpr static uint64_t PAGE_SIZE = 4096;
@@ -29,16 +31,15 @@ static inline uint64_t AlignUp(uint64_t value, uint64_t size) {
GlobalAllocator(HostAllocator *_Alloc)
: Alloc {_Alloc} {}
virtual ~GlobalAllocator() = default;
virtual void *malloc(size_t Size) = 0;
virtual void *calloc(size_t num, size_t size) = 0;
virtual void *realloc(void *ptr, size_t size) = 0;
virtual void *memalign(size_t alignment, size_t size) = 0;
virtual void free(void *ptr) = 0;
};
GlobalAllocator *CreateBasicAllocator(HostAllocator *Alloc);
}
namespace Alloc::OSAllocator {
Alloc::HostAllocator *Create64BitAllocator();
std::unique_ptr<Alloc::HostAllocator> Create64BitAllocator();
}
+65 -11
View File
@@ -14,30 +14,48 @@ namespace FEXCore::Threads {
void *Ptr;
size_t Size;
};
std::mutex StackPoolMutex{};
std::deque<StackPoolItem> StackPool;
std::mutex DeadStackPoolMutex{};
std::mutex LiveStackPoolMutex{};
std::deque<StackPoolItem> DeadStackPool;
std::deque<StackPoolItem> LiveStackPool;
void *AllocateStackObject(size_t Size) {
std::unique_lock<std::mutex> lk{StackPoolMutex};
if (StackPool.size() == 0) {
std::lock_guard lk{DeadStackPoolMutex};
if (DeadStackPool.size() == 0) {
// Nothing in the pool, just allocate
return FEXCore::Allocator::mmap(nullptr, Size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS | MAP_GROWSDOWN, -1, 0);
}
// Keep the first item in the stack pool
auto Result = StackPool.front().Ptr;
StackPool.pop_front();
auto Result = DeadStackPool.front().Ptr;
DeadStackPool.pop_front();
// Erase the rest as a garbage collection step
for (auto &Item : StackPool) {
for (auto &Item : DeadStackPool) {
FEXCore::Allocator::munmap(Item.Ptr, Item.Size);
}
return Result;
}
void AddStackToPool(void *Ptr, size_t Size) {
std::unique_lock<std::mutex> lk{StackPoolMutex};
StackPool.emplace_back(StackPoolItem{Ptr, Size});
void AddStackToDeadPool(void *Ptr, size_t Size) {
std::lock_guard lk{DeadStackPoolMutex};
DeadStackPool.emplace_back(StackPoolItem{Ptr, Size});
}
void AddStackToLivePool(void *Ptr, size_t Size) {
std::lock_guard lk{LiveStackPoolMutex};
LiveStackPool.emplace_back(StackPoolItem{Ptr, Size});
}
void RemoveStackFromLivePool(void *Ptr) {
std::lock_guard lk{LiveStackPoolMutex};
for (auto it = LiveStackPool.begin(); it != LiveStackPool.end(); ++it) {
if (it->Ptr == Ptr) {
LiveStackPool.erase(it);
return;
}
}
}
void *InitializeThread(void *Ptr);
@@ -49,6 +67,7 @@ namespace FEXCore::Threads {
, UserArg {Arg} {
pthread_attr_t Attr{};
Stack = AllocateStackObject(STACK_SIZE);
AddStackToLivePool(Stack, STACK_SIZE);
pthread_attr_init(&Attr);
pthread_attr_setstack(&Attr, Stack, STACK_SIZE);
pthread_create(&Thread, &Attr, Func, Arg);
@@ -87,7 +106,8 @@ namespace FEXCore::Threads {
}
void FreeStack() {
AddStackToPool(Stack, STACK_SIZE);
RemoveStackFromLivePool(Stack);
AddStackToDeadPool(Stack, STACK_SIZE);
}
private:
@@ -115,8 +135,38 @@ namespace FEXCore::Threads {
return std::make_unique<PThread>(Func, Arg);
}
void CleanupAfterFork_PThread() {
// We don't need to pull the mutex here
// After a fork we are the only thread running
// Just need to make sure not to delete our own stack
uintptr_t StackLocation = reinterpret_cast<uintptr_t>(alloca(0));
auto ClearStackPool = [&](auto &StackPool) {
for (auto it = StackPool.begin(); it != StackPool.end(); ) {
StackPoolItem &Item = *it;
uintptr_t ItemStack = reinterpret_cast<uintptr_t>(Item.Ptr);
if (ItemStack <= StackLocation && (ItemStack + Item.Size) > StackLocation) {
// This is our stack item, skip it
++it;
}
else {
// Untracked stack. Clean it up
FEXCore::Allocator::munmap(Item.Ptr, Item.Size);
it = StackPool.erase(it);
}
}
};
// Clear both dead stacks and live stacks
ClearStackPool(DeadStackPool);
ClearStackPool(LiveStackPool);
LogMan::Throw::A((DeadStackPool.size() + LiveStackPool.size()) <= 1, "After fork we should only have zero or one tracked stacks!");
}
static FEXCore::Threads::Pointers Ptrs = {
.CreateThread = CreateThread_PThread,
.CleanupAfterFork = CleanupAfterFork_PThread,
};
std::unique_ptr<FEXCore::Threads::Thread> FEXCore::Threads::Thread::Create(
@@ -125,6 +175,10 @@ namespace FEXCore::Threads {
return Ptrs.CreateThread(Func, Arg);
}
void FEXCore::Threads::Thread::CleanupAfterFork() {
return Ptrs.CleanupAfterFork();
}
void FEXCore::Threads::Thread::SetInternalPointers(Pointers const &_Ptrs) {
memcpy(&Ptrs, &_Ptrs, sizeof(FEXCore::Threads::Pointers));
}
+7 -4
View File
@@ -46,8 +46,11 @@ namespace FEXCore::Context {
MODE_32BIT,
MODE_64BIT,
};
using CustomCPUFactoryType = std::function<std::unique_ptr<FEXCore::CPU::CPUBackend> (FEXCore::Context::Context*, FEXCore::Core::InternalThreadState *Thread)>;
using ExitHandler = std::function<void(uint64_t ThreadId, FEXCore::Context::ExitReason)>;
/**
* @brief This initializes internal FEXCore state that is shared between contexts and requires overhead to setup
*/
@@ -90,8 +93,8 @@ namespace FEXCore::Context {
*/
FEX_DEFAULT_VISIBILITY bool InitCore(FEXCore::Context::Context *CTX, FEXCore::CodeLoader *Loader);
FEX_DEFAULT_VISIBILITY void SetExitHandler(FEXCore::Context::Context *CTX, std::function<void(uint64_t ThreadId, FEXCore::Context::ExitReason)> handler);
FEX_DEFAULT_VISIBILITY std::function<void(uint64_t ThreadId, FEXCore::Context::ExitReason)> GetExitHandler(FEXCore::Context::Context *CTX);
FEX_DEFAULT_VISIBILITY void SetExitHandler(FEXCore::Context::Context *CTX, ExitHandler handler);
FEX_DEFAULT_VISIBILITY ExitHandler GetExitHandler(FEXCore::Context::Context *CTX);
/**
* @brief Pauses execution on the CPU core
@@ -216,8 +219,8 @@ namespace FEXCore::Context {
FEX_DEFAULT_VISIBILITY void HandleCallback(FEXCore::Context::Context *CTX, uint64_t RIP);
FEX_DEFAULT_VISIBILITY void RegisterHostSignalHandler(FEXCore::Context::Context *CTX, int Signal, HostSignalDelegatorFunction Func);
FEX_DEFAULT_VISIBILITY void RegisterFrontendHostSignalHandler(FEXCore::Context::Context *CTX, int Signal, HostSignalDelegatorFunction Func);
FEX_DEFAULT_VISIBILITY void RegisterHostSignalHandler(FEXCore::Context::Context *CTX, int Signal, HostSignalDelegatorFunction Func, bool Required);
FEX_DEFAULT_VISIBILITY void RegisterFrontendHostSignalHandler(FEXCore::Context::Context *CTX, int Signal, HostSignalDelegatorFunction Func, bool Required);
FEX_DEFAULT_VISIBILITY FEXCore::Core::InternalThreadState* CreateThread(FEXCore::Context::Context *CTX, FEXCore::Core::CPUState *NewThreadState, uint64_t ParentTID);
FEX_DEFAULT_VISIBILITY void InitializeThread(FEXCore::Context::Context *CTX, FEXCore::Core::InternalThreadState *Thread);
+1
View File
@@ -26,6 +26,7 @@ namespace FEXCore::Core {
uint32_t base;
} gdt[32];
uint16_t FCW;
uint16_t FTW;
};
static_assert(offsetof(CPUState, xmm) % 16 == 0, "xmm needs to be 128bit aligned!");
+2 -2
View File
@@ -54,8 +54,8 @@ namespace Core {
*
* It's a process level signal handler so one must be careful
*/
virtual void RegisterHostSignalHandler(int Signal, HostSignalDelegatorFunction Func) = 0;
virtual void RegisterFrontendHostSignalHandler(int Signal, HostSignalDelegatorFunction Func) = 0;
virtual void RegisterHostSignalHandler(int Signal, HostSignalDelegatorFunction Func, bool Required) = 0;
virtual void RegisterFrontendHostSignalHandler(int Signal, HostSignalDelegatorFunction Func, bool Required) = 0;
/**
* @brief Registers a signal handler for the host to handle a signal specifically for guest handling
+94 -2
View File
@@ -95,13 +95,105 @@ namespace FEXCore {
}
namespace x86 {
// uc_flags flags
///< Has extended FP state
constexpr uint64_t UC_FP_XSTATE = (1ULL << 0);
///< The order of these must match the GNU ordering
enum ContextRegs {
FEX_REG_GS = 0,
FEX_REG_FS,
FEX_REG_ES,
FEX_REG_DS,
FEX_REG_RDI,
FEX_REG_RSI,
FEX_REG_RBP,
FEX_REG_RSP,
FEX_REG_RBX,
FEX_REG_RDX,
FEX_REG_RCX,
FEX_REG_RAX,
FEX_REG_TRAPNO,
FEX_REG_ERR,
FEX_REG_EIP,
FEX_REG_CS,
FEX_REG_EFL,
FEX_REG_UESP,
FEX_REG_SS
};
static_assert(FEX_REG_SS == 18, "Oops");
struct FEX_PACKED siginfo_t {
uint32_t pad[32];
int si_signo;
int si_errno;
int si_code;
union {
uint32_t pad[29];
/* SIGILL, SIGFPE, SIGSEGV, SIBUS */
struct {
uint32_t addr;
} _sigfault;
/* SIGCHLD */
struct {
int32_t pid;
int32_t uid;
int32_t status;
int32_t utime;
int32_t stime;
} _sigchld;
} _sifields;
};
static_assert(sizeof(FEXCore::x86::siginfo_t) == 128, "This needs to be the right size");
struct FEX_PACKED stack_t {
uint32_t ss_sp; // XXX: should be compat_ptr<void>
int ss_flags;
uint32_t ss_size;
};
static_assert(sizeof(FEXCore::x86::stack_t) == 12, "This needs to be the right size");
struct FEX_PACKED mcontext_t {
uint32_t gregs[19];
uint32_t fpregs; // XXX: should be compat_ptr<FEXCore::x86::_libc_fpstate>
uint32_t oldmask;
uint32_t cr2;
};
static_assert(sizeof(FEXCore::x86::mcontext_t) == 88, "This needs to be the right size");
struct _libc_fpreg {
uint16_t significand[4];
uint16_t exponent;
};
static_assert(sizeof(FEXCore::x86::_libc_fpreg) == 10, "This needs to be the right size");
enum fpstate_magic {
// Legacy fpstate
MAGIC_FPU = 0xFFFF'0000,
// Contains extended state information
MAGIC_XFPSTATE = 0x0,
};
struct FEX_PACKED _libc_fpstate {
uint32_t fcw;
uint32_t fsw;
uint32_t ftw;
uint32_t fop;
uint32_t cssel;
uint32_t dataoff;
uint32_t datasel;
FEXCore::x86::_libc_fpreg _st[8];
uint32_t status;
};
static_assert(sizeof(FEXCore::x86::_libc_fpstate) == 112, "This needs to be the right size");
struct FEX_PACKED ucontext_t {
uint32_t pad[91];
uint32_t uc_flags;
uint32_t uc_link; // XXX: should be a compat_ptr<FEXCore::x86::ucontext_t>
FEXCore::x86::stack_t uc_stack;
FEXCore::x86::mcontext_t uc_mcontext;
FEXCore::x86_64::sigset_t uc_sigmask; // This matches across architectures
FEXCore::x86::_libc_fpstate __fpregs_mem;
uint32_t __ssp[4];
};
static_assert(sizeof(FEXCore::x86::ucontext_t) == 364, "This needs to be the right size");
-3
View File
@@ -1,6 +1,5 @@
#pragma once
#include <FEXCore/Core/Context.h>
#include <FEXCore/Utils/CompilerDefs.h>
#include <cstdint>
@@ -500,6 +499,4 @@ extern FEX_DEFAULT_VISIBILITY X86InstInfo XOPTableGroupOps[MAX_XOP_GROUP_TABLE_S
// EVEX
extern FEX_DEFAULT_VISIBILITY X86InstInfo EVEXTableOps[MAX_EVEX_TABLE_SIZE];
FEX_DEFAULT_VISIBILITY void InitializeInfoTables(Context::OperatingMode Mode);
}
+1
View File
@@ -27,6 +27,7 @@ namespace FEXCore::HLE {
};
enum class SyscallOSABI {
OS_UNKNOWN,
OS_LINUX64,
OS_LINUX32,
OS_WIN64,
+27 -23
View File
@@ -398,50 +398,50 @@ friend class FEXCore::IR::PassManager;
}
void SetJumpTarget(IR::IROp_Jump *Op, OrderedNode *Target) {
LOGMAN_THROW_A(Target->Op(DualListData.DataBegin())->Op == OP_CODEBLOCK,
"Tried setting Jump target to %%ssa%d %s",
LOGMAN_THROW_A_FMT(Target->Op(DualListData.DataBegin())->Op == OP_CODEBLOCK,
"Tried setting Jump target to %ssa{} {}",
Target->Wrapped(DualListData.ListBegin()).ID(),
std::string(IR::GetName(Target->Op(DualListData.DataBegin())->Op)).c_str());
IR::GetName(Target->Op(DualListData.DataBegin())->Op));
Op->Header.Args[0].NodeOffset = Target->Wrapped(DualListData.ListBegin()).NodeOffset;
}
void SetTrueJumpTarget(IR::IROp_CondJump *Op, OrderedNode *Target) {
LOGMAN_THROW_A(Target->Op(DualListData.DataBegin())->Op == OP_CODEBLOCK,
"Tried setting CondJump target to %%ssa%d %s",
LOGMAN_THROW_A_FMT(Target->Op(DualListData.DataBegin())->Op == OP_CODEBLOCK,
"Tried setting CondJump target to %ssa{} {}",
Target->Wrapped(DualListData.ListBegin()).ID(),
std::string(IR::GetName(Target->Op(DualListData.DataBegin())->Op)).c_str());
IR::GetName(Target->Op(DualListData.DataBegin())->Op));
Op->TrueBlock.NodeOffset = Target->Wrapped(DualListData.ListBegin()).NodeOffset;
}
void SetFalseJumpTarget(IR::IROp_CondJump *Op, OrderedNode *Target) {
LOGMAN_THROW_A(Target->Op(DualListData.DataBegin())->Op == OP_CODEBLOCK,
"Tried setting CondJump target to %%ssa%d %s",
LOGMAN_THROW_A_FMT(Target->Op(DualListData.DataBegin())->Op == OP_CODEBLOCK,
"Tried setting CondJump target to %ssa{} {}",
Target->Wrapped(DualListData.ListBegin()).ID(),
std::string(IR::GetName(Target->Op(DualListData.DataBegin())->Op)).c_str());
IR::GetName(Target->Op(DualListData.DataBegin())->Op));
Op->FalseBlock.NodeOffset = Target->Wrapped(DualListData.ListBegin()).NodeOffset;
}
void SetJumpTarget(IRPair<IROp_Jump> Op, OrderedNode *Target) {
LOGMAN_THROW_A(Target->Op(DualListData.DataBegin())->Op == OP_CODEBLOCK,
"Tried setting Jump target to %%ssa%d %s",
LOGMAN_THROW_A_FMT(Target->Op(DualListData.DataBegin())->Op == OP_CODEBLOCK,
"Tried setting Jump target to %ssa{} {}",
Target->Wrapped(DualListData.ListBegin()).ID(),
std::string(IR::GetName(Target->Op(DualListData.DataBegin())->Op)).c_str());
IR::GetName(Target->Op(DualListData.DataBegin())->Op));
Op.first->Header.Args[0].NodeOffset = Target->Wrapped(DualListData.ListBegin()).NodeOffset;
}
void SetTrueJumpTarget(IRPair<IROp_CondJump> Op, OrderedNode *Target) {
LOGMAN_THROW_A(Target->Op(DualListData.DataBegin())->Op == OP_CODEBLOCK,
"Tried setting CondJump target to %%ssa%d %s",
LOGMAN_THROW_A_FMT(Target->Op(DualListData.DataBegin())->Op == OP_CODEBLOCK,
"Tried setting CondJump target to %ssa{} {}",
Target->Wrapped(DualListData.ListBegin()).ID(),
std::string(IR::GetName(Target->Op(DualListData.DataBegin())->Op)).c_str());
IR::GetName(Target->Op(DualListData.DataBegin())->Op));
Op.first->TrueBlock.NodeOffset = Target->Wrapped(DualListData.ListBegin()).NodeOffset;
}
void SetFalseJumpTarget(IRPair<IROp_CondJump> Op, OrderedNode *Target) {
LOGMAN_THROW_A(Target->Op(DualListData.DataBegin())->Op == OP_CODEBLOCK,
"Tried setting CondJump target to %%ssa%d %s",
LOGMAN_THROW_A_FMT(Target->Op(DualListData.DataBegin())->Op == OP_CODEBLOCK,
"Tried setting CondJump target to %ssa{} {}",
Target->Wrapped(DualListData.ListBegin()).ID(),
std::string(IR::GetName(Target->Op(DualListData.DataBegin())->Op)).c_str());
IR::GetName(Target->Op(DualListData.DataBegin())->Op));
Op.first->FalseBlock.NodeOffset = Target->Wrapped(DualListData.ListBegin()).NodeOffset;
}
@@ -507,7 +507,7 @@ friend class FEXCore::IR::PassManager;
ReplaceUsesWithAfter(Node, NewNode, Start);
LOGMAN_THROW_A(Node->NumUses == 0, "Node still used");
LOGMAN_THROW_A_FMT(Node->NumUses == 0, "Node still used");
// Since we have deleted ALL uses, we can safely delete the node.
Remove(Node);
@@ -521,8 +521,8 @@ friend class FEXCore::IR::PassManager;
OrderedNode *GetPackedRFLAG(bool Lower8);
void CopyData(IREmitter const &rhs) {
LOGMAN_THROW_A(rhs.DualListData.DataBackingSize() <= DualListData.DataBackingSize(), "Trying to take ownership of data that is too large");
LOGMAN_THROW_A(rhs.DualListData.ListBackingSize() <= DualListData.ListBackingSize(), "Trying to take ownership of data that is too large");
LOGMAN_THROW_A_FMT(rhs.DualListData.DataBackingSize() <= DualListData.DataBackingSize(), "Trying to take ownership of data that is too large");
LOGMAN_THROW_A_FMT(rhs.DualListData.ListBackingSize() <= DualListData.ListBackingSize(), "Trying to take ownership of data that is too large");
DualListData.CopyData(rhs.DualListData);
InvalidNode = rhs.InvalidNode->Wrapped(rhs.DualListData.ListBegin()).GetNode(DualListData.ListBegin());
CurrentWriteCursor = rhs.CurrentWriteCursor;
@@ -583,8 +583,12 @@ friend class FEXCore::IR::PassManager;
* @{ */
/** @} */
void LinkCodeBlocks(OrderedNode *CodeNode, OrderedNode *Next) {
FEXCore::IR::IROp_CodeBlock *CurrentIROp = CodeNode->Op(DualListData.DataBegin())->CW<FEXCore::IR::IROp_CodeBlock>();
LOGMAN_THROW_A(CurrentIROp->Header.Op == IROps::OP_CODEBLOCK, "Invalid");
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
FEXCore::IR::IROp_CodeBlock *CurrentIROp =
#endif
CodeNode->Op(DualListData.DataBegin())->CW<FEXCore::IR::IROp_CodeBlock>();
LOGMAN_THROW_A_FMT(CurrentIROp->Header.Op == IROps::OP_CODEBLOCK, "Invalid");
CodeNode->append(DualListData.ListBegin(), Next);
}
+6 -6
View File
@@ -69,8 +69,8 @@ class DualIntrusiveAllocator final {
size_t ListSize() const { return ListCurrentOffset; }
size_t ListBackingSize() const { return MemorySize; }
uintptr_t const DataBegin() const { return Data; }
uintptr_t const ListBegin() const { return List; }
uintptr_t DataBegin() const { return Data; }
uintptr_t ListBegin() const { return List; }
void Reset() { DataCurrentOffset = 0; ListCurrentOffset = 0; }
@@ -159,7 +159,7 @@ public:
stream.write((char*)GetListData(), ListSize);
}
size_t GetInlineSize() {
size_t GetInlineSize() const {
static_assert(sizeof(*this) == 40);
return sizeof(*this) + DataSize + ListSize;
}
@@ -197,7 +197,7 @@ public:
// If we are casting to something narrower than just the header, check the opcode.
if constexpr (!std::is_same<T, IROp_Header>::value) {
LOGMAN_THROW_A(Op->OPCODE == Op->Header.Op, "Expected Node to be '%s'. Found '%s' instead", GetName(Op->OPCODE), GetName(Op->Header.Op));
LOGMAN_THROW_A_FMT(Op->OPCODE == Op->Header.Op, "Expected Node to be '{}'. Found '{}' instead", GetName(Op->OPCODE), GetName(Op->Header.Op));
}
return Op;
@@ -317,11 +317,11 @@ public:
return iterator(reinterpret_cast<uintptr_t>(GetListData()), reinterpret_cast<uintptr_t>(GetData()), Wrapped);
}
uintptr_t const GetData() const {
uintptr_t GetData() const {
return reinterpret_cast<uintptr_t>(IRDataInternal ? IRDataInternal : InlineData);
}
uintptr_t const GetListData() const {
uintptr_t GetListData() const {
return reinterpret_cast<uintptr_t>(ListDataInternal ? ListDataInternal : &InlineData[DataSize]);
}
+17
View File
@@ -6,6 +6,7 @@
#include <climits>
#include <cstddef>
#include <cstdint>
#include <cstring>
#include <type_traits>
namespace FEXCore {
@@ -61,4 +62,20 @@ template <typename T>
return trailing_zeroes + 1;
}
// Stand-in for std::bit_cast until libc++ implements it.
template <typename To, typename From>
[[nodiscard]] inline To BitCast(const From& source) noexcept
{
static_assert(sizeof(From) == sizeof(To),
"BitCast source and destination types must be equal in size.");
static_assert(std::is_trivially_copyable_v<From>,
"BitCast source type must be trivially copyable.");
static_assert(std::is_trivially_copyable_v<To>,
"BitCast destination type must be trivially copyable.");
std::aligned_storage_t<sizeof(To), alignof(To)> storage;
std::memcpy(&storage, &source, sizeof(storage));
return reinterpret_cast<To&>(storage);
}
} // namespace FEXCore
+12
View File
@@ -7,8 +7,11 @@ namespace FEXCore::Threads {
class Thread;
using CreateThreadFunc = std::function<std::unique_ptr<Thread>(ThreadFunc Func, void* Arg)>;
using CleanupAfterForkFunc = std::function<void()>;
struct Pointers {
CreateThreadFunc CreateThread;
CleanupAfterForkFunc CleanupAfterFork;
};
// API
@@ -19,10 +22,19 @@ namespace FEXCore::Threads {
virtual bool join(void **ret) = 0;
virtual bool detach() = 0;
virtual bool IsSelf() = 0;
/**
* @name Calls provided API functions
* @{ */
static std::unique_ptr<Thread> Create(
ThreadFunc Func,
void* Arg);
static void CleanupAfterFork();
/** @} */
// Set API functions
static void SetInternalPointers(Pointers const &_Ptrs);
};
}
Loaded 100 of 189 files, more files were not shown because too many files have changed in this diff. Show more