Compare commits

...
679 Commits
Author SHA1 Message Date
Ryan Houdek 1f65bf9e47 Docs: Update for release FEX-2108 2021-08-02 23:21:04 -07:00
Ryan Houdek 4c431518b0 Merge pull request #1192 from Sonicadvance1/workaround_static_pie
AArch64: Workaround static-pie crashing
2021-08-02 23:19:45 -07:00
Ryan Houdek b586592643 Make sure our std::filesystem users use the std::error_code versions
Otherwise these crash out.
2021-08-02 23:10:23 -07:00
Ryan Houdek 83bdbc8392 AArch64: Check if we launched with FD argument
This will give us another way to check if the interpreter is installed rather than file checks.
The binfmt_misc files won't exist inside of a rootfs
2021-08-02 22:47:49 -07:00
Ryan Houdek c49fa5a64f Safely check if we can create paths rather than crashing on failure. 2021-08-02 22:47:42 -07:00
Ryan Houdek d052e87d47 AArch64: Workaround static-pie crashing
Static-pie can only work with GNU ld instead of lld on AArch64 hosts.
This is a known problem on the llvm side: https://bugs.llvm.org/show_bug.cgi?id=49672
2021-08-02 22:02:12 -07:00
Ryan Houdek ff98a43f70 Merge pull request #1191 from Sonicadvance1/improve_fexmountdaemon_stability
FEXMountDaemon: Make squashfs mounting more robust
2021-07-30 16:49:20 -07:00
Ryan Houdek a6c4ad889b FEXMountDaemon: Make squashfs mounting more robust
Instead of watching to ensure our parent process is still alive. Mount the
squashfs once and use a combination of file leases and inotify to
ref count how many processes are using the rootfs.

This makes it so in the common case, the FEXMountDaemon only ever executes once
and runs until all FEX processes stop running.
In the rare edge case there is a race condition where multiple FEXMountDaemon
applications will start, but only one will end up mounting the squashfs.
In this case, one application wins and the one that failed to grab the lease
will wait until the other one completes.

With this change, squashfs should be reasonable to use now.
2021-07-29 21:45:30 -07:00
Ryan Houdek 7e8577a50a Merge pull request #1180 from Sonicadvance1/support_setxid
Linux: Implements support for signals 32 and 33
2021-07-28 00:22:30 -07:00
Ryan Houdek a21cead9dc Fixes memcpy of siginfo_t data
_sifields are offset by a uint32_t between a 32-bit and 64-bit arch.

Fixes thread cancelling on 32-bit processes
2021-07-28 00:10:15 -07:00
Ryan Houdek 555cdb558b Linux: Implements support for signals 32 and 33
When a guest tries to use the setxid syscalls, the guest glibc has
a mechanism in place to ensure that the process wide setxid is handled.
The mechanism is fairly complex but it uses signal 33 and sends the signal to all
active threads to ensure every thread sets the correct state here.

We need to intercept this and pass the context information correctly to the guest instead.
Otherwise FEX just crashes when the glibc HOST handler tries handling the guest applications signal.

Fixes the game SOMA https://store.steampowered.com/app/282140/SOMA/
2021-07-28 00:10:08 -07:00
Ryan Houdek ceaaac3b7a Linux: Pass setxid syscalls directly to the kernel
We don't want these to be captured by the host glibc handlers around these
2021-07-27 18:55:30 -07:00
Ryan Houdek de0337db47 Merge pull request #1188 from Sonicadvance1/wine_fixes
Wine fixes
2021-07-27 18:07:53 -07:00
Ryan Houdek a48e41e4a9 Merge pull request #1187 from Sonicadvance1/cleanup_compile_service_on_fork
Cleanup compile service on fork
2021-07-27 18:07:42 -07:00
Ryan Houdek 896aa52851 Merge pull request #1186 from Sonicadvance1/fix_partial_rclse
RCLSE: Fixes an assumption in RCLSE
2021-07-27 18:07:32 -07:00
Ryan Houdek 0315964848 Merge pull request #1185 from Sonicadvance1/allow_stall_config
Adds an option to stall processes on launch
2021-07-27 18:07:19 -07:00
Ryan Houdek dec822192a Merge pull request #1182 from Sonicadvance1/disable_jemalloc_option
Adds an option to disable jemalloc from cmake
2021-07-27 18:07:09 -07:00
Ryan Houdek a5b142cf02 Merge pull request #1178 from Sonicadvance1/implement_vsyscall
Linux: Implements support for vsyscall
2021-07-27 18:06:54 -07:00
Ryan Houdek 007b1cbe9d Merge pull request #1174 from Sonicadvance1/binfmt_support_preserve_and_fd
binfmt_misc: Support loading ELFs from FD and support preserve
2021-07-27 18:06:39 -07:00
Ryan Houdek b0a28e595f Merge pull request #1181 from Sonicadvance1/fix_rapass
Fixes bug in RAPass
2021-07-27 18:06:27 -07:00
Ryan Houdek b8c5cf2736 Adds FXAM unit tests for ABI conformance
Wine was doing a fun x87 stack clean by checking tag bits for empty.
We hadn't previously supported anything but valid. Which cause wine to overflow the stack
trying to save all of the x87 values.

We now support valid and empty well enough that this doesn't break and we can unit test it
2021-07-27 08:25:40 -07:00
Ryan Houdek 2803113e35 Improves 32-bit signal handling
A decent amount of this was broken.
To the point that the ABI was incorrect even, causing 32-bit signals to completely crash in most instances.
This is enough to get my 32-bit signal handler test application to work and basic SIGUSR behaviour
2021-07-27 08:23:25 -07:00
Ryan Houdek 7a84ab8cfc Implements partial support for x87 FTW
This only implements the Empty and Valid tags.
Anything that is not empty is considered valid currently.
2021-07-27 08:22:11 -07:00
Ryan Houdek 77a7ecbae0 Add x87 FTW to CoreState
This will be needed for x87 FTW emulation
2021-07-27 07:57:41 -07:00
Ryan Houdek 2310ea83bb Cleanup compile service on fork
On fork this thread will be gone. Make sure to delete the shared_ptr.
2021-07-27 07:53:02 -07:00
Ryan Houdek 109df27bba Amd64: Very minor optimization in Push/Pop Regs
Just generates a bit nicer code to look at. Doesn't really change much.
2021-07-24 08:45:38 -07:00
Ryan Houdek 2afa14d1f3 ConstProp: Removes VMOV assumption
ConstProp has an assumption that a VMOV of the same size arguments is a no-op.
This isn't quite 100% true as this could be behaving as a zext and the pass can't see through the op correctly.

Something to be said that the vector ops should be looked over to see which op is leaving dirty bits in the upper bits
of the register. But that's a battle for a different time
2021-07-24 08:43:35 -07:00
Ryan Houdek 744b778013 RCLSE: Fixes an assumption in RCLSE
There is an assumption in the RCLSE pass that if a StoreContext + LoadContext pair of the same size
with a vector register that it is safe to remove the LoadContext.
This isn't quite safe since it can't entirely see through the IR to determine if the store op
had already zext the upper bits of the register.

This was causing 8byte loadcontext operations to "load" with garbage data in the upper
bits when it was expecting a zext.

Which granted it is likely a bug that not all vector ops zero their upper bits but that's a battle for a different day.

Additionally:
- Removes a couple dead optimizations in the GPR path that was never hit.
- Cleans up how access types are checked
- Uses ReplaceAllUsesWithRange to reinforce that this is purely a per block optimization still
2021-07-24 08:37:03 -07:00
Ryan Houdek b3965ab9b4 Adds an option to stall processes on launch
For misbehaving applications before we can fully diagnose, allow
a config that hangs a process on load.
2021-07-23 15:18:52 -07:00
Ryan Houdek ec6cc5cb4f Merge pull request #1184 from lioncash/vec-alloc
64BitAllocator: Convert std::vector table into std::array
2021-07-23 14:29:03 -07:00
Ryan Houdek 7931dc2c09 Merge pull request #1183 from lioncash/size
FlexBitSet: Use non-template type parameter for index parameters
2021-07-23 14:17:13 -07:00
Lioncash 22590806a0 64BitAllocator: Convert std::vector table into std::array
This is small enough that it can be converted over to a std::array so
the heap use isn't necessary
2021-07-23 17:12:37 -04:00
Lioncash f9313f4000 FlexBitSet: Use non-template type parameter for index parameters
These parameters are used as indexes into the tracked memory, so sizing
the index variable relative to the type being stored is kind of sketchy.

e.g. If the tracked types were uint8_t for example, we'd still want the
indexing parameters to be regularly sized so that we aren't implicitly
truncating values all the time when passing values to a uint8_t
parameter (and while all usages are currently using uint64_t as type
T, we may as well address this).

While we're at it, we can make both Get() and operator[] const member
functions, since they don't directly modify any underlying data, they
only read it.
2021-07-23 17:07:41 -04:00
Ryan Houdek ddcc18277b Check for valgrind for memory region check
If running under valgrind then ignore memory region check.
Otherwise you'd need to modify source to get past this check
2021-07-22 18:17:16 -07:00
Ryan Houdek 19de33f9ca Adds an option to disable jemalloc from cmake
While not recommended. It is necessary to allow disabling jemalloc if you want to run asan or tsan
2021-07-22 18:16:24 -07:00
Ryan Houdek a9ee4123ef Fixes bug in RAPass
RegisterNode isn't a POD so we can't memset it.
This was overwriting some unique_ptrs which were memory leaking and causing crashes in other PRs
2021-07-22 18:13:44 -07:00
Ryan Houdek 138e88bc80 Linux: Implements support for vsyscall
vsyscall is a legacy interface to expose three syscalls in a faster fashion.
Most applications should have moved to vDSO but this isn't entirely the case.

golang specifically still used vsyscall in the case that vdso isn't available.
Which that isn't completely safe since vsyscall can be completely disabled.
This is fixed in a newer version of their runtime but we must support legacy behaviour.

This makes it so FEX checks if the incoming RIP is a "special" region and adjusts where it
loads instructions from accordingly. Lying to the guest application about the instruction stream
while maintaining the illusion that its RIP is in the correct location.
This was fairly straightforward since we already decoupled the instruction stream from the RIP location.

This fixes golang execution as long as you also disable their preempt signaling.
environment variable to do so is `GODEBUG=asyncpreemptoff=1`
2021-07-22 16:58:17 -07:00
Ryan Houdek 62775161fe Merge pull request #1176 from Sonicadvance1/fix_static_pie_crashing
Works around static-pie crashing
2021-07-20 19:37:56 -07:00
Ryan Houdek 310cfcbcc4 Works around static-pie crashing
static-pie is crashing early due to an issue with pthread symbols being exposed as weak.
For some reason the weak symbols never get resolved and jump to zero or ELF base.

When linking static-pie, force search for the pthread symbols using an undefined glob.

With this, static-pie works as long as you link with a new enough version of lld (version 13.0)
2021-07-20 05:24:57 -07:00
Ryan Houdek 2fd41de5e0 Merge pull request #1153 from Sonicadvance1/split_opdispatcher
OpcodeDispatcher: Split the opcode handling to multiple files
2021-07-20 04:43:53 -07:00
Ryan Houdek c2a97d4fc1 Minor header shuffling to improve compilation time
Shaves a few seconds off compile time
2021-07-20 04:27:28 -07:00
Ryan Houdek ebb21e86b0 OpcodeDispatcher: Split the opcode handling to multiple files
These are some fairly large separate files still.
Don't want these to be too terribly small but helps significantly with compile time when working
in the OpcodeDispatcher
2021-07-20 04:27:28 -07:00
Ryan Houdek d8da4ce2a5 binfmt_misc: Support loading ELFs from FD and support preserve
This lets us support the final two flags in binfmt_misc that we needed.
1) Support open_binary
2) Support preserve

1)
We already supported the credentials flag, which implied open_binary, but we weren't handling half of it.
With the open_binary flag, the Linux kernel passes us the executable in an FD instead of as a pathname.
This can be found inside of auxv on startup, inside of AT_EXECFD.
If AT_EXECFD is available then we prioritize using that instead of the pathname passed in.
This fixes a potential permissions issue where an executable is executed without read permissions.

2)
The preserve flag has the Linux kernel preserve the original argv[0] that was passed to the application.
Prior to supporting this flag, the kernel would provide us with a resolved program path.
This can happen in the instance where something like `blah` resolves to `/usr/loca/bin/blah` which
isn't what the user originally typed.
This works around this problem by handing the interpreter both the resolved path and the original typed path.

Alongside open_binary, we can just use the FD passed in instead of the resolved path, this means
we can just drop the argv[0] for the guest (which is the kernel resolved path) and pass through arguments
unmangled.
We do have to make a minor assumption here that if we are using EXECFD that we assume preserve.
It isn't until kernel v5.12 that we can actually check AT_FLAGS to see if that was true.
2021-07-20 01:35:23 -07:00
Ryan Houdek aa767b3922 Merge pull request #1166 from Sonicadvance1/static_pie_support
Adds cmake option ENABLE_STATIC_PIE
2021-07-19 23:33:46 -07:00
Ryan Houdek 14480f3ee0 Merge pull request #1167 from Sonicadvance1/tsan_problems
Fixes some issues I found when running under tsan and asan
2021-07-19 23:30:25 -07:00
Ryan Houdek 6ac23003e1 Merge pull request #1168 from Sonicadvance1/fix_spill_slot
Arm64: Fixes fill and spill slot offset calculation
2021-07-19 23:29:01 -07:00
Ryan Houdek 04aacea58b Merge pull request #1173 from Sonicadvance1/fix_binfmt_misc_not_usr
Fixes binfmt_misc install when not installed to /usr
2021-07-19 23:26:33 -07:00
Ryan Houdek 3d46c40aaf Fixes binfmt_misc install when not installed to /usr
We had hardcoded paths in our binfmt_misc files. Change it to a generated
file instead.

Additionally for the binfmt_misc install targets, since we aren't installing to the
global binfmt_misc folder, we need to pass in the import directory

Fixes #1169
2021-07-18 13:05:12 -07:00
Ryan Houdek 7052d022c1 Merge pull request #1172 from lioncash/jits
Jits: Migrate logs over to fmt where applicable
2021-07-16 11:10:27 -07:00
Lioncash 9c7ebca235 JIT/x86_64: Migrate logs over to fmt 2021-07-16 13:57:15 -04:00
Lioncash 22656aedb2 JIT/Arm64: Migrate logs over to fmt 2021-07-16 13:57:12 -04:00
Ryan Houdek 80caf9038f Merge pull request #1171 from lioncash/string
General: Remove redundant string constructions in log calls related to IR op name retrieval
2021-07-16 10:09:43 -07:00
Lioncash 7fddac7b22 General: Remove redundant string constructions in log calls
Prior to the introduction of fmt, there wasn't a nice and easy way to
format std::string_view, but now that we have it in place, we can modify
relevant log calls to make use of it and get rid of the need to
construct a string around the view.

While we're in the same area, we can make the rest of the modified file
use fmt where appropriate.
2021-07-16 09:53:13 -04:00
Ryan Houdek 7f5b352453 Merge pull request #1170 from lioncash/dead
Validation Passes: Remove unused variables
2021-07-16 06:13:35 -07:00
Lioncash f8968b8296 IRValidation: Move stringstream int HadError conditonal body
Same behavior, but now we only construct it when we need it.
2021-07-16 08:54:09 -04:00
Lioncash 914861ba42 PhiValidation: Move stringstream int HadError conditonal body
Same behavior, but now we don't construct it unless we need to.
2021-07-16 08:52:51 -04:00
Lioncash 5d4efa8caa ValueDominanceValidation: Remove unused variables
No warnings are ever appended to the warning string stream, and
HadWarning is never set to anything other than false, so we can remove
it to bring it in line with the PhiValidation pass.
2021-07-16 08:50:33 -04:00
Ryan Houdek 0075e4ed19 Arm64: Fixes fill and spill slot offset calculation
+16 doesn't matter anymore. This can cause stack corruption
2021-07-15 06:47:28 -07:00
Scott Mansell 3a0da68b94 Merge pull request #1165 from Sonicadvance1/fix_lock_flags_ops
Fixes flag setting for 8bit and 16bit LOCK ALU ops
2021-07-15 14:00:33 +12:00
Ryan Houdek d0edd04e58 Amd64: Fixes X86Dispatcher allocator
The entire object didn't need to be an Allocator.
Additionally there was an overwrite happening on the Size variable.
This was causing the shutdown to free a size of zero.
2021-07-14 13:25:29 -07:00
Ryan Houdek 4332c22797 Linux: Fixes BRK deallocation
The SyscallHandler takes over ownership of BRK from the ELFLoader.
We were failing to deallocate the full range.
Additionally if the guest changed BRK size to something smaller than initial load then the size would underflow.
2021-07-14 13:23:58 -07:00
Ryan Houdek 5e12c38298 SignalDelegator: Switch Required flag to atomic bool
This is shared between multiple threads but it isn't never changed beyond first set.
Change it to atomic bool with relaxed loads to quiet tsan
2021-07-14 13:22:38 -07:00
Ryan Houdek bf8e28f935 Fixes TLS for softfloat
softfloat uses TLS for setting the rounding mode per thread.
We had never set the `THREAD_LOCAL` environment variable, so this
rounding mode was being set globally rather than per thread.
We are expecting per thread state here.

Noticed this while running through tsan
2021-07-14 13:21:02 -07:00
Ryan Houdek 6b90663be3 Adds cmake option ENABLE_STATIC_PIE
This option does cmake checks to determine if your system can handle static-pie.
With upstream projects static-pie only works if you use the binutils linker.
Using lld doesn't currently work because it defines __rela_iplt_{start,end} symbols.

Our cmake file will now compile a test application and check for these symbols.
Either the symbols will not exist at all or they will exist but be a null address

Once your system passes the checks then it will allow you to enable static-pie
2021-07-14 09:56:47 -07:00
Ryan Houdek c362bd883d Update jemalloc for more symbols 2021-07-14 08:21:29 -07:00
Ryan Houdek 0ade57614a Fixes flag setting for 8bit and 16bit LOCK ALU ops
MUL is handled in a different function, so this isn't needed there anymore.
This was causing a problem where if you force enabled LOCK on all capable ops it was causing a crash.

Issue #1164 is set to track getting unit tests for these.
2021-07-13 01:37:17 -07:00
Ryan Houdek f4e044a53c Merge pull request #1163 from Sonicadvance1/add_named_thunk_configs
Adds a convenience of named thunk configs
2021-07-13 00:26:36 -07:00
Ryan Houdek 108081e2cd Merge pull request #1162 from Sonicadvance1/update_man_page_with_json_key
Update man page to show JSON key for config options
2021-07-13 00:26:31 -07:00
Ryan Houdek 6ff9b9664d Merge pull request #1161 from Sonicadvance1/fix_thunks_finding_gl
Fixes Host thunks lib not being able to GL
2021-07-13 00:26:26 -07:00
Ryan Houdek 0a0d8f8885 Adds a convenience of named thunk configs
If the thunk config is not a path then search for the filename in $XDG_DATA_DIR/.fex-emu/ThunkConfigs/
for the file.

Just makes it easier to use rather than having full paths
2021-07-12 19:30:03 -07:00
Ryan Houdek bc09d73b9d Update man page to show JSON key for config options
Allows users to reference the man page for the json key instead of code diving
2021-07-12 19:22:57 -07:00
Ryan Houdek ed891f09f1 Fixes Host thunks lib not being able to GL 2021-07-12 19:21:58 -07:00
Ryan Houdek a24d63b5f9 Merge pull request #1160 from Sonicadvance1/allocator_fixes
Allocator fixes
2021-07-12 16:38:22 -07:00
Ryan Houdek 9434a59902 Remove some logs that aren't worth seeing anymore 2021-07-12 00:25:09 -07:00
Ryan Houdek 786a341652 BRKHandler: Don't rely on 64Bit Allocator 32-bit hack.
Use the actual 32-bit allocator if we are running a 32-bit process.
This hack in the 64bit allocator is likely to go away in the future
2021-07-12 00:22:59 -07:00
Ryan Houdek 74bff6c3a5 64BitAllocator: Add a backward scan path
First thing, do a backwards scan to try and fill any holes in the allocator
This reduces the number of holes we have in our VA space
2021-07-12 00:22:44 -07:00
Ryan Houdek 8cfda9c86d RAPass: Mostly revert the PR switching this over to mmap.
1) We needed to switch it to the FEXCore allocator so it didn't live in 32-bit space
2) This generates holes in the VA space due to how the allocations line up.
This makes us almost immediate hit the 65k VMA region.
Switching over to jemalloc allocations means we don't hit that.

Not worth creating an mmap based allocator that can work around that.
2021-07-12 00:16:20 -07:00
Ryan Houdek f79c832f88 X86HelperGen: Use System mmap for 32-bit fixed location
This needs to end up 32-bit space which means we shouldn't use the FEXCore Allocator.
FEXCore allocator has a hack in it for fixed offset in 32-bit space, but we shouldn't rely on it
2021-07-12 00:13:32 -07:00
Ryan Houdek 1fa51f7f74 Frontend: Use mmap helper for DecodeBuffer
This ensures that on 32-bit applications we don't consume 416KB per thread inside of 32-bit space
2021-07-12 00:12:30 -07:00
Ryan Houdek 53417e5322 Merge pull request #1158 from Sonicadvance1/atomic_improvements
Atomic improvements
2021-07-11 18:17:16 -07:00
Ryan Houdek 94e38bb556 Arm64: Fixes duplicated symbols problem in the interpreter
Uses local symbol numbers so the compiler doesn't get upset if this is inlined.
2021-07-10 23:51:02 -07:00
Ryan Houdek 59a4481e5b unittests: Adds explicit 16bit, 32bit, 64bit unaligned unit tests 2021-07-10 23:40:18 -07:00
Ryan Houdek 31a672b1d5 unittests: Adds explicit 128bit vector unaligned load test 2021-07-10 23:39:04 -07:00
Ryan Houdek e03ece0ee6 unittests: Adds LOCK NEG unit tests 2021-07-10 23:38:19 -07:00
Ryan Houdek b1151428e8 Arm64: Stop backpatching loadstores in paranoid TSO mode
When paroid tso is enabled, stop backpatching almost entirely.
Only code still backpatching is 128bit loadstores
2021-07-10 23:37:10 -07:00
Ryan Houdek 818808c3e1 Arm64: Implements unaligned atomic loads store handler
In the case that paranoid TSO is enabled, we can force all unaligned loadstores
down this path.

Instead of backpatching, just do the unaligned loadstore in the handler
2021-07-10 23:32:18 -07:00
Ryan Houdek ee40300eb5 OpcodeDispatcher: Implements support for LOCK NEG
Previously this was unimplemented since it isn't really used on x86.
Fixes #844
2021-07-10 23:30:42 -07:00
Ryan Houdek 484d756a26 Arm64: Implements AtomicFetchNeg IR op 2021-07-10 23:29:49 -07:00
Ryan Houdek d0bcdeda6b amd64: Implements AtomicFetchNeg IR op 2021-07-10 23:29:04 -07:00
Ryan Houdek ea218f8758 Interpreter: Implements AtomicFetchNeg
Does a two's complement negation of what is in memory atomically

Requires a loadstore exclusive loop to implement correctly
2021-07-10 23:27:45 -07:00
Ryan Houdek 821634e300 Arm64: Make sure ARMv8.0 Atomic Swap follows our pattern
The most minor of optimizations as well
2021-07-10 23:26:26 -07:00
Ryan Houdek 5df8354339 Arm64: Allow capturing loadstore exclusive pairing for atomic memory ops
This allows us to capture LDAXR* instructions which appear in ARMv8.0 memory ops.
These scan the memory space to see what the loop is doing and then make an atomic decision to handle
the loadstore exclusive loop with our signal handler based CAS atomics.

This will be necessary to support `LOCK NEG` on ARMv8.1+
2021-07-10 23:23:16 -07:00
Ryan Houdek fe10db9396 Adds an assert if we tried restoring a signal without known frames
Noticed this happen while tinkering. Best to get a debug only message in that case.
2021-07-10 23:19:46 -07:00
Ryan Houdek 3b4119dd31 Arm64: Add some helpers to unaligned atomic load
If we want to do atomic memory loads that are unaligned then we need some helpers to do these.

LoadAcquire128 needs to specifically avoid the std::atomic helper because it can try storing to read only memory
It does this to clear the exclusive lock from its ldaxp but we can just use clrex.
2021-07-10 23:18:46 -07:00
Ryan Houdek 60da70b07e ARMv8.0: Adds some NOPs around CASPair loadstore exclusive pair
The unaligned signal handler will backpatch these with DMBs.
Previously generating corrupted code. This will fix that corruption but not the atomicity problem
2021-07-10 23:15:49 -07:00
Ryan Houdek 35a4f52d7e Merge pull request #1156 from Sonicadvance1/actually_fix_jemalloc
Actually fix jemalloc for FEXConfig
2021-07-09 04:58:51 -07:00
Ryan Houdek 78a9dcf505 Actually fix jemalloc for FEXConfig 2021-07-09 03:48:29 -07:00
Ryan Houdek 1eee2dd004 Merge pull request #1155 from Sonicadvance1/improve_startup_time
Switch some memory allocations over to mmap
2021-07-09 03:47:05 -07:00
Ryan Houdek 25866a00ed Disable rtsignal gvisor test
This test is getting frustrating and is becoming MORE flakey as threads become faster
2021-07-09 01:57:48 -07:00
Ryan Houdek a27c05f577 Switch some memory allocations over to mmap
This cuts start up time from 8.962ms to 3.949ms and lets us use less physical memory.
We pay a small amortized cost from the faulting to populate pages which will be fairly low
2021-07-09 01:57:47 -07:00
Ryan Houdek 0d9ab9ca42 Merge pull request #1154 from Sonicadvance1/fix_jemalloc_override
Update jemalloc to fix glibc override.
2021-07-08 14:48:43 -07:00
Ryan Houdek 9c4c9ea566 Update jemalloc to fix glibc override.
For some reason jemalloc was missing an override
2021-07-08 14:33:35 -07:00
Stefanos Kornilios Mitsis Poiitidis a9bb33b8c2 Merge pull request #1152 from Sonicadvance1/fix_jemalloc_malloc_replace
Fix jemalloc malloc replace
2021-07-08 18:13:20 +03:00
Ryan Houdek faf09fec5f Removes glibc symbol replacement inside FEX
FEX_jemalloc now handles this entirely by itself.
It now replaces glibc during startup rather than when we call for it.
2021-07-06 21:52:57 -07:00
Ryan Houdek 4cd2b1a959 Update Externals jemalloc 2021-07-06 21:20:17 -07:00
Stefanos Kornilios Mitsis Poiitidis 8f09bd9215 Merge pull request #1150 from Sonicadvance1/fix_old_kernel_defines
Fixes old kernel defines for sockios
2021-07-06 15:39:54 +03:00
Stefanos Kornilios Mitsis Poiitidis 84dee03a16 Merge pull request #1151 from Sonicadvance1/error_on_immediate_failure
FEXLoader: stderr on immediate failure
2021-07-06 14:36:11 +03:00
Stefanos Kornilios Mitsis Poiitidis 0a8ee43081 Merge pull request #1149 from Sonicadvance1/enable_local_xxhash
Use externals xxhash if not found installed
2021-07-06 14:35:18 +03:00
Ryan Houdek 9fbeba3b79 FEXLoader: stderr on immediate failure
In the case that there is an immediate configuration failure. Use stderr specifically for outputting.
These errors won't be output typically because silent logging is enabled by default.
In the case of executable missing or rootfs configuration failure, print directly to stderr.
Previously it looked like FEX just exited for no reason.
We had multiple users encounter this and be confused
2021-07-05 18:52:06 -07:00
Ryan Houdek b3bab814de Fixes old kernel defines for sockios
These defines were changed at in kernel v5.2 but the debian defines didn't seem to change?
Redefine them if missing
2021-07-05 18:26:44 -07:00
Ryan Houdek 728e9fc187 Use externals xxhash if not found installed
Also in the case that you don't have v0.8.0 minimum installed
2021-07-05 17:42:35 -07:00
Ryan Houdek e01e6d9293 Adds xxhash external submodule 2021-07-05 17:39:19 -07:00
Stefanos Kornilios Mitsis Poiitidis 1a64c908e5 Merge pull request #1146 from Sonicadvance1/implement_sigqueueinfo
Linux: Implements rt_{tg,}sigqueueinfo
2021-07-05 16:55:34 +03:00
Stefanos Kornilios Mitsis Poiitidis 7dd700394e Merge pull request #1147 from Sonicadvance1/fix_imul_flags
OpDispatcher: Fixes imul flags calculations
2021-07-05 16:55:16 +03:00
Ryan Houdek b7b9d4089f unittests: Adds more imul unit tests for flags 2021-07-03 21:48:18 -07:00
Ryan Houdek 5a7dd844b0 OpDispatcher: Fixes imul flags calculations
We were calculating the high bits incorrectly in a couple variants
2021-07-03 21:45:15 -07:00
Ryan Houdek 80da08b15b Fixes Signal handler check for queue signals
Any signals sent with sigqueue or kill/sigsend are safe on the siginfo path.
2021-07-03 17:12:42 -07:00
Ryan Houdek 20a519a250 unittests: Update posix tests that have been fixed 2021-07-02 23:00:48 -07:00
Ryan Houdek b9efbd6159 Linux: Implements rt_{tg,}sigqueueinfo
We already support receiving user signals in our signal handlers for this.
Just need to push through syscalls.

This fixes Dead Island Definitive Edition so it runs.
2021-07-02 22:53:27 -07:00
Ryan Houdek 749f1eba51 Merge pull request #1145 from lioncash/stub
GdbStub: Fix memory leak in GdbServerLoop()
2021-07-02 03:21:00 -07:00
Lioncash bc52a16fde GdbStub: Fix memory leak in GdbServerLoop()
Previously the opened socket stream would be leaked.
2021-07-02 05:53:59 -04:00
Stefanos Kornilios Mitsis Poiitidis 2db44cf2ed Merge pull request #1142 from Sonicadvance1/implement_base_signalfd
Linux: Implements a base implementation of signalfd{4,}
2021-07-02 10:04:26 +03:00
Ryan Houdek 336eb9a8b4 Merge pull request #1143 from Sonicadvance1/remove_fork_log
Linux: Remove logs about parent or child stack usage
2021-07-01 12:46:18 -07:00
Ryan Houdek b923a82c39 unittests: fallocate gvisor test no longer fails 2021-07-01 07:11:00 -07:00
Ryan Houdek 1304279eb5 Linux: Remove logs about parent or child stack usage
We don't need to care about what the guest stacks are doing now
2021-07-01 07:06:14 -07:00
Ryan Houdek 1f6926a245 Linux: Implements a base implementation of signalfd{4,}
This is a base implementation of signalfd.
Signalfd allows the application to receive siginfo_t information through an FD.
The FD is either provided by the application or created by the kernel depending.
This specifically doesn't pick up *true* synchronous signals. tgkill of the number
should theoretically go through this interface.

This very specifically skips our internal required signals for now.
This means it won't pick up SIGILL, SIGBUS, or SIG63.

This is enough to capture an application that just wants to poll for SIGCHLD.
Anything more complex has the same problems of the guest handling a siginfo_t.
2021-07-01 06:51:39 -07:00
Stefanos Kornilios Mitsis Poiitidis 2d11df03cb Merge pull request #1141 from Sonicadvance1/modify_cpack_name
CPack: Update package name to remove conflict
2021-07-01 16:44:44 +03:00
Stefanos Kornilios Mitsis Poiitidis d142d7c363 Merge pull request #1140 from Sonicadvance1/cpuid_tm
CPUID: Update TM and TM2 CPUID bits
2021-07-01 16:44:09 +03:00
Ryan Houdek b546869d9d CPack: Update package name to remove conflict
'fex' is already taken by another projects. Use 'fex-emu' instead; Similar to other emulator package names.
2021-07-01 05:08:57 -07:00
Ryan Houdek 2a49ad941f CPUID: Update TM and TM2 CPUID bits
These are in place to let applications know that we do our own thermal management

TM was introduced with Pentium M, it would dispatch idle uops to counter thermals.
TM2 was later and is what allows the CPU to dynamically change its multiplier and frequency.
All in an effort to not cook themselves.

Our ARM CPUs also do all this. Claim it as well.
2021-07-01 04:26:39 -07:00
Ryan Houdek 8170307aef Docs: Update for release FEX-2107 2021-07-01 04:20:44 -07:00
Ryan Houdek 12cc979ec6 Merge pull request #1139 from Sonicadvance1/more_unaligned_atomics_fixes
ARM64: Fixes bugs in unaligned atomic signal handlers
2021-06-30 23:21:55 -07:00
Ryan Houdek d6ecd6364c Disables the new tests on ARMv8.0
Still aren't supported correctly there.
2021-06-30 21:48:36 -07:00
Ryan Houdek 7d6dafe25d Adds cmpxchg unit tests for new bugs encountered
Ensures we are testing when Desired != Memory && Memory == Expected to ensure it fails on all alignments
2021-06-30 21:43:51 -07:00
Ryan Houdek db6f4786d2 ARM64: Fixes bugs in unaligned atomic signal handlers
There were two bugs in here.

The first bug here is with with the CMPXCHG emulation.
1) If the *Expected* value did *NOT* match what was in memory
2) *AND* The *Desired* value matched the memory value
3) The CAS would incorrectly return success for this CMPXCHG
4) Thus setting ZF incorrectly

The second bug comes from atomic memory operations (Add, CLR, EOR, SET, SWAP).
This operation is a Load + <Op> + CAS
1) If the memory backing between the Load and CAS changes
2) The CAS would then fail
3) On Atomic memory operations this should then retry to ensure it completes successfully
4) We were not retrying on failure in this case
5) Thus something like `LOCK INC` would have never atomically incremented correctly
6) We can't use our ASM unit tests to test this, since it needs thread contention.
2021-06-30 21:43:44 -07:00
Ryan Houdek 49fa69d4e2 Merge pull request #1138 from lioncash/casfn
Arm64: Use regular function pointers with CAS handling functions
2021-06-30 19:46:38 -07:00
Lioncash 448cbc8e3d Arm64: Use regular function pointers with CAS handling functions
Given these are called in a loop repeatedly, where we know we'll always
have a set function to call, std::function adds a little bit of
overhead.
2021-06-30 22:03:08 -04:00
Ryan Houdek 81ce7b3f16 Merge pull request #1137 from lioncash/cexpr
OpcodeDispatcher: Make use of if constexpr
2021-06-30 18:21:20 -07:00
Lioncash c9615032a9 OpcodeDispatcher: Make use of if constexpr
Given these arguments are template arguments we can use if constexpr
with these branches to guarantee their elision.
2021-06-30 21:11:28 -04:00
Ryan Houdek 52e2c4abc5 Merge pull request #1136 from lioncash/handler
Context: Add alias for exit handler
2021-06-30 17:12:24 -07:00
Lioncash c4216755ab Context: Add alias for exit handler
Places the definition in one place so it doesn't need to be written
several times.
2021-06-30 19:50:57 -04:00
Ryan Houdek 09071e4279 Merge pull request #1135 from lioncash/ctx
Context: Place return value from GenerateIR into a struct
2021-06-30 16:39:24 -07:00
Lioncash 46a6e79233 Context: Place return value from GenerateIR into a struct
This makes it a little more straightforward to see what these return
values mean without needing to look at the implementation.

These tuples were also getting a little bit large.
2021-06-30 19:30:03 -04:00
Ryan Houdek 09363f1b1b Merge pull request #1134 from lioncash/lut
OpcodeDispatcher: Mark lookup tables as static in Get{Src,Dst}Size
2021-06-30 15:51:34 -07:00
Lioncash 9d5f9d6783 OpcodeDispatcher: Mark lookup tables as static in Get{Src,Dst}Size
Same behavior, but allows clang to elide pushing all of these values on
and off the stack, particularly given these are called quite frequently
throughout the opcode dispatcher.
2021-06-30 18:22:14 -04:00
Stefanos Kornilios Mitsis Poiitidis 7d0535e0d9 Merge pull request #1133 from Sonicadvance1/fix_zero_size_pt_load
Fixes PT_LOAD with zero file size
2021-06-29 09:39:22 +03:00
Stefanos Kornilios Mitsis Poiitidis 176b3292ed Merge pull request #1132 from Sonicadvance1/move_elfutils_frontend
Moves ELF handlers from FEXCore to frontend
2021-06-29 09:38:30 +03:00
Stefanos Kornilios Mitsis Poiitidis 3d25f594a9 Merge pull request #1131 from Sonicadvance1/disallow_disabling_cpuid
syscalls: Disallow disabling CPUID from arch_prctl
2021-06-29 09:37:02 +03:00
Stefanos Kornilios Mitsis Poiitidis 861d89e8a8 Merge pull request #1130 from Sonicadvance1/fix_fcntl
x32: Fixes fcntl OP_GETLK64_32
2021-06-29 09:36:19 +03:00
Stefanos Kornilios Mitsis Poiitidis 59c36ebc36 Merge pull request #1129 from Sonicadvance1/fix_msg_alignment
x32: Fixes sendmmsg cmsg alignment
2021-06-29 09:35:00 +03:00
Stefanos Kornilios Mitsis Poiitidis 2d8304547e Merge pull request #1128 from Sonicadvance1/remove_logs
Remove logs that are just noise at this point
2021-06-29 09:34:12 +03:00
Stefanos Kornilios Mitsis Poiitidis 4880097f89 Merge pull request #1122 from Sonicadvance1/CreateAppConfig
FEXConfig: Have it create AppConfig folder on save
2021-06-29 09:29:59 +03:00
Stefanos Kornilios Mitsis Poiitidis 3a11b88d30 Merge pull request #1120 from Sonicadvance1/fix_a_few_32bit_syscalls
x32: Fixes shmdt, sendmsg, recvmsg syscalls
2021-06-29 09:28:58 +03:00
Stefanos Kornilios Mitsis Poiitidis 76a054e50a Merge pull request #1113 from Sonicadvance1/fix_argument_epoll_ctl
Wrap 32-bit epoll_ctl argument in compat_ptr
2021-06-29 09:26:23 +03:00
Ryan Houdek 279d9219a6 Fixes PT_LOAD with zero file size
gzip ships a PT_LOAD program section without a file size and only a memory size.
In the case of a zero file size PT_LOAD then return success immediately in this section loader.
This fixes Steam using gzip to package up crash logs
2021-06-27 02:10:21 -07:00
Ryan Houdek ed4c5d9704 Remove logs that are just noise at this point
Don't print how many instructions are installed in the tables.
  This isn't useful anymore

Not installing signal 32 and 33 are something we don't support right now. Stop complaining in that case.

Stop printing when a thread is starting up and shutting down. If you want to see this then gdb shows it well.

Don't print clone flags unless we are hitting a case where we are printing another log message.
2021-06-27 02:09:27 -07:00
Ryan Houdek 069e279a18 Moves ELF handlers from FEXCore to frontend
Only the frontends need to deal with ELF files specifically.
The backend doesn't need to be aware of them at all.
Since the ELF handling is the frontend's responsibility, move all the code to the frontend.
2021-06-27 02:07:56 -07:00
Ryan Houdek 9ba1948375 syscalls: Disallow disabling CPUID from arch_prctl
In a newer version of the kernel there was a feature to disallow cpuid.
We can emulate this by saying it is always enabled and disallow the ability to disable it.
2021-06-27 02:03:21 -07:00
Ryan Houdek 835ca9cdf9 x32: Fixes fcntl OP_GETLK64_32
This was overwriting the cmd argument and then being checked in the switch statement
after the call.
Since it was overwritten, it wasn't falling down the correct path, returning a flock_32 instead of a flock64_32
2021-06-27 02:01:37 -07:00
Ryan Houdek 59a429ec19 x32: Fixes sendmmsg cmsg alignment
This wasn't aligning like the other recvmsg and sendmsg variants
2021-06-27 02:00:39 -07:00
Ryan Houdek 36b5863c7c Merge pull request #1125 from lioncash/ptr
FEXLoader: Make use of unique_ptr for syscall handlers
2021-06-25 18:36:32 -07:00
Ryan Houdek bdfbebe24d Merge pull request #1126 from lioncash/socket
x32/Socket: Amend std::vector initialization bug in sendmmsg
2021-06-25 17:24:13 -07:00
Ryan Houdek 336ecb8cc7 Merge pull request #1112 from Sonicadvance1/deferred_signal_installation
Implements support for deferred signal handler installation
2021-06-25 16:44:42 -07:00
Lioncash ea9f08e481 x32/Socket: Amend std::vector initialization in sendmmsg
Since the vector was being initialized with braces, this caused the
arrays to only ever have one element within it instead of the size of
the array.

While we're at it, we can also construct a vector in place with the
necessary size.
2021-06-25 10:00:37 -04:00
Lioncash da8c5dc460 FEXLoader: Make use of unique_ptr for syscall handlers
Makes the ownership requirements explicit in the interface.

Also makes it harder to unintentionally/accidentally leak memory.
2021-06-25 09:19:48 -04:00
Ryan Houdek fd8dcda67e Merge pull request #1124 from lioncash/timeofday
x32/Time: Implement settimeofday
2021-06-25 05:59:00 -07:00
Lioncash b2fb48d711 x32/Time: Implement settimeofday 2021-06-25 08:48:05 -04:00
Ryan Houdek 8e6328e946 Merge pull request #1123 from lioncash/time
x32/Time: Fix a few edge cases in time related syscalls
2021-06-25 05:45:23 -07:00
Lioncash 9262264525 x32/Time: Fix potential null pointer dereference in nanosleep
Prevents potential null dereferences that could occur from user code.
2021-06-25 08:30:45 -04:00
Lioncash 4553564c9f x32/Time: Fix potential null pointer dereference in clock_nanosleep
Prevents a potential null dereference in our code that could result from
user code.
2021-06-25 08:30:36 -04:00
Lioncash 2a86421172 x32/Time: Fix potential null pointer dereference in clock_settime
The kernel handles the case where a null address is passed in for tp.

Instead we can handle this case so we don't have a dereference inside
our code.
2021-06-25 08:19:04 -04:00
Ryan Houdek bb85f90f78 FEXConfig: Have it create AppConfig folder on save
Otherwise it will claim that the AppConfig was saved and won't actually be
2021-06-25 04:03:16 -07:00
Ryan Houdek 4a1da85214 Merge pull request #1121 from lioncash/utimesat
x32/Time: Implement futimesat
2021-06-25 03:36:32 -07:00
Lioncash ff9268d6c7 x32/Time: Implement futimesat 2021-06-25 06:20:51 -04:00
Ryan Houdek c4f8e6c934 x32: Fixes non-ipc sendmsg/recvmsg syscalls
We were sending these non-ipc variants of these syscalls through
the generic path. This is broken but luckily glibc from Ubuntu isn't shipping
a version using these yet.
2021-06-25 03:02:31 -07:00
Ryan Houdek 52c50735d5 x32: Fixes shmdt not going through allocator
This was missed. Which means we were leaking SHM allocations if the 32-bit allocator was used
2021-06-25 03:01:38 -07:00
Ryan Houdek aca8903b56 GVisor: Update gvisor tests that have now changed behaviour 2021-06-24 21:08:59 -07:00
Ryan Houdek 2e9b22e042 POSIX: Updates known failures
Now that signal behaviours have changed a little bit, update the tests that have changed
2021-06-24 20:42:39 -07:00
Ryan Houdek d4559ea0c6 Update host signal mask on guest update
Now that are aren't consuming all signals we need to more aggressively handle the host signal mask.
Now more signals are getting masked and blocked how they should be.
2021-06-24 20:42:39 -07:00
Ryan Houdek d4dd4d0972 Copy over siginfo_t to the guest more correctly
si_addr will still be incorrect. What matters more here is that SIGCHLD gets correct information.
The guest needs SIGCHLD ifnromation to be filled out correctly, otherwise TTY handoff hangs
with the child process stopped.
2021-06-24 20:42:39 -07:00
Ryan Houdek 79db23b7d3 Define some of 32-bit x86 siginfo_t 2021-06-24 20:42:39 -07:00
Ryan Houdek fbfc774446 Implements support for deferred signal handler installation
I saw a red herring that I thought the high cpu usage in steamwebhelper could come from signal handlers.
This turned out to not be the case, but now I've got this implemented.

Installs the few signal handlers that we need upfront but for everything that isn't a mandatory signal
we instead now wait until the guest also installs that signal handler.
This fixes #1107
2021-06-24 20:42:39 -07:00
Ryan Houdek 35157c1251 Merge pull request #1118 from lioncash/utimes
x32/Time: Implement utimes
2021-06-24 17:02:02 -07:00
Lioncash 17d836a178 x32/Time: Fix edge case in utimensat
utimensat allows passing null to signify that timestamps should be set
to the current time.

Check for this to avoid dereferencing null.
2021-06-24 18:49:55 -04:00
Lioncash 28d754b466 x32/Time: Implement utimes 2021-06-24 18:45:03 -04:00
Ryan Houdek 01667bdfb2 Merge pull request #1117 from lioncash/fsuid
x32/Thread: Implement setfsuid32/setfsgid32
2021-06-24 09:36:00 -07:00
Lioncash 2ddcaef525 x32/Thread: Implement setfsuid32/setfsgid32
Fairly straightforward to implement.
2021-06-24 12:22:48 -04:00
Ryan Houdek c7cd2241c8 Merge pull request #1116 from lioncash/groups
x32/Thread: Implement getgroups32/setgroups32
2021-06-24 09:11:39 -07:00
Lioncash f67f3fe4de x32/Thread: Implement getgroups32/setgroups32
Implements two other straightforward syscalls for the 32-bit side of
things.
2021-06-24 08:15:11 -04:00
Ryan Houdek 5d7822c987 Merge pull request #1115 from lioncash/chown
x32: Implement chown32/fchown32/lchown32
2021-06-24 04:47:25 -07:00
Lioncash 738354a52d x32/FD: Implement chown32/fchown32/lchown32
Implements three basic hooks for the chown32 family of syscalls.
2021-06-24 07:08:46 -04:00
Stefanos Kornilios Mitsis Poiitidis 26531e9b96 Merge pull request #1111 from Sonicadvance1/32bit_sigpending
Implements 32-bit sigpending
2021-06-24 13:24:38 +03:00
Stefanos Kornilios Mitsis Poiitidis 282402a80c Merge pull request #1110 from Sonicadvance1/cpack_setup
Implements support for cpack debian package building
2021-06-24 13:24:23 +03:00
Stefanos Kornilios Mitsis Poiitidis 28254a3163 Merge pull request #1105 from Sonicadvance1/fix_32bit_syscall_checks
Fixes a couple of 32-bit checks
2021-06-24 13:24:00 +03:00
Ryan Houdek 0dd568f7fb Wrap 32-bit epoll_ctl argument in compat_ptr
Doesn't change behaviour, just makes sure we know it is a compat_ptr
2021-06-24 00:43:14 -07:00
Ryan Houdek 12adc5bc6e Implements 32-bit sigpending
This is a early version of the syscall that only returns the lower 32 signals.
Easy enough to support
2021-06-24 00:35:23 -07:00
Ryan Houdek 6ac942266f Implements support for cpack debian package building
This doesn't currently install thunks which can come a bit later.
We require a postinst and prerm step for importing and unimporting the binfmt_misc files.
Easy enough
2021-06-23 18:47:04 -07:00
Ryan Houdek 24a2a0c4eb Switches binfmt_misc install step to use registration files
This is easier to represent than the raw files. Once we do a debian file install then
this becomes more important
2021-06-22 20:22:09 -07:00
Stefanos Kornilios Mitsis Poiitidis 910c624a8b Merge pull request #1104 from Sonicadvance1/implement_32bit_iret
Implements 32-bit iret instruction
2021-06-22 09:22:00 +03:00
Stefanos Kornilios Mitsis Poiitidis 767ea0fc9e Merge pull request #1103 from Sonicadvance1/deprioritize_aot
Lower priority of AOT compilation threads.
2021-06-22 09:20:23 +03:00
Stefanos Kornilios Mitsis Poiitidis 087e5a9576 Merge pull request #1102 from Sonicadvance1/implement_truncate64
Implements 32-bit truncate64 syscall
2021-06-22 09:20:04 +03:00
Stefanos Kornilios Mitsis Poiitidis 67e6ffbc93 Merge pull request #1101 from Sonicadvance1/fixed_stack_fix
Fixes fixed stack offset for 32-bit applications
2021-06-22 09:19:36 +03:00
Stefanos Kornilios Mitsis Poiitidis 6734d745c6 Merge pull request #1099 from Sonicadvance1/fix_strace_32
Fixes debug strace output
2021-06-22 09:17:14 +03:00
Stefanos Kornilios Mitsis Poiitidis 332123a38c Merge pull request #1098 from Sonicadvance1/fix_more_gdt
Fixes a 32-bit processes wanting another TLS space
2021-06-22 09:16:40 +03:00
Stefanos Kornilios Mitsis Poiitidis 24fdbe4e6d Merge pull request #1096 from Sonicadvance1/fix_kotor2
Fix Star Wars Knights of the Old Republic 2
2021-06-22 09:14:02 +03:00
Ryan Houdek ea48f36511 Fixes a couple of 32-bit checks
The allocator functions don't return -1 on error, instead return the actual error.
This was setup to match the syscall behaviour rather than the glibc/errno behaviour
2021-06-21 00:25:01 -07:00
Ryan Houdek d587485383 Implements 32-bit iret instruction
This is necessary for wine and some wine-like emulation layers.

With this implemented then some Saint's Row games start running.

Fixes #1100
2021-06-21 00:20:44 -07:00
Ryan Houdek c9211ae77a Lower priority of AOT compilation threads.
Set these threads to minimum priority to not complete starve the system if there are other tasks running.
Fixes #1075
2021-06-20 17:24:36 -07:00
Ryan Houdek 7cbe9579ba Fixes 32-bit allocator returning 64-bit pointers on old kernels
In the case of a kernel older than 4.17 then MAP_FIXED_NOREPLACE doesn't exist.
This adds a check in to see if the pointer returned was valid but not what we asked for
2021-06-20 17:22:26 -07:00
Ryan Houdek ca482e3e96 Implements 32-bit truncate64 syscall
A 32-bit game was using this syscall and now it gets farther in game
2021-06-20 01:10:19 -07:00
Ryan Houdek eaa3df9cba Fixes fixed stack offset for 32-bit applications
Instead of forcing a fixed offset for the stack. Allow it to get placed automatically.
64-bit was already doing this; Now we can also do it in 32-bit.
This is possible because the mapper that is mapping the code will always map in the 32-bit space
for a 32-bit guest.
2021-06-20 01:07:58 -07:00
Ryan Houdek c252dc99fc Fixes debug strace output
This custom definition was missing
2021-06-19 22:01:47 -07:00
Ryan Houdek 70dc5217ec Fixes a 32-bit processes wanting another TLS space
Currently our 32-bit code only gave one TLS slot and crashed
if you needed more.

First switch over to the same initial slot as the Linux kernel.
Then allow searching the slots for a free spot.
This allows us to more closely match the behaviour of the Linux kernel just in case
anything has hardcoded the TLS slots

This makes it so Saints Row: The Third stops crashing at boot, plays a few intro videos, then hangs instead.
2021-06-19 21:58:49 -07:00
Ryan Houdek 1745bcceb6 Fixes 32bit statfs and fstatfs
These 32bit syscalls use a compat statfs which is only 64bytes in size.
This was overwriting data on the guest stack and causing crashes.

Describe a 32-bit statfs struct and ensure with struct verifier that it matches.

Fixes a crash in KOTOR2 that would happen just before the main menu.
2021-06-19 07:26:41 -07:00
Ryan Houdek cc8fa9d934 Fixes 32bit sysinfo syscall
This was altered in kernel 2.3.23 to include a mem_unit variable.
mem_unit states what each of the memory units is scaled by.

The kernel will always have this be 1 or page size on x86, so follow that behaviour.
Instead of maxing out the values to what uint32_t can handle, scale all of these by page size instead.

This fixes a crash in KOTOR2's boot sequence where it would divide by zero if everything was maxed out.
2021-06-19 07:23:25 -07:00
Ryan Houdek 424b93b5f8 Merge pull request #1095 from Sonicadvance1/disable_flake2
Disable flaky posix test
2021-06-17 15:12:59 -07:00
Ryan Houdek 5fccf5f741 Disable flaky posix test
Test sleeps for 1 second and expects to come back within 10ms of the time.
When the CPU is doing other things then it can end up missing that timeframe. Thus flake.

Just disable it
2021-06-17 07:51:37 -07:00
Ryan Houdek 42f1dfeede Merge pull request #1094 from Sonicadvance1/fix_fexbash_squashfs
Fixes FEXBash with squashfs
2021-06-17 07:47:04 -07:00
Ryan Houdek 21fa93f3b3 Fixes FEXBash with squashfs
FEXBash wasn't setting up a squashfs on its end. This meant that it couldn't find any
x86-64 libraries and would fail
2021-06-17 06:41:07 -07:00
Ryan Houdek f088d97060 Merge pull request #1093 from lioncash/cast
General: Resolve -Wcast-qual warnings
2021-06-16 23:28:52 -07:00
Lioncash e369626929 BitUtils: Add BitCast
libc++ doesn't implement std::bit_cast, so we can provide our own for
the time being.
2021-06-17 01:30:07 -04:00
Lioncash cba4ca7d01 General: Resolve -Wcast-qual warnings
Ensures that qualifiers are preserved on references and pointers to
prevent undefined behavior.
2021-06-17 01:25:39 -04:00
Ryan Houdek fb7964e6b1 Merge pull request #1092 from lioncash/ignored-qual
General: Resolve -Wignored-qualifiers warnings
2021-06-16 19:52:39 -07:00
Lioncash ef7aff796f General: Resolve -Wignored-qualifiers warnings
Removes const qualifiers that don't do anything to the interface.
2021-06-16 21:51:45 -04:00
Ryan Houdek d2771669e6 Merge pull request #1091 from lioncash/unused
Core: Remove unused DefaultFallbackCore
2021-06-16 17:32:56 -07:00
Lioncash e6e170805e Core: Remove unused DefaultFallbackCore
This doesn't seem to be hooked up to anything.
2021-06-16 19:58:05 -04:00
Ryan Houdek 1f89ca7218 Merge pull request #1090 from lioncash/array
Frontend: Make lookup tables static in MapModRMToReg
2021-06-16 14:44:35 -07:00
Ryan Houdek adcee99625 Merge pull request #1089 from lioncash/gprsize
Context: Move GPR size retrieval to its own function
2021-06-16 14:03:20 -07:00
Lioncash a9623f0e2a Frontend: Shrink MapModRMToReg array size from uint64_t to uint32_t
This function only returns a 32-bit value, so we can save some space by
using uint32_t instead.
2021-06-16 16:59:56 -04:00
Lioncash c40ca14b5b Frontend: Make lookup tables static in MapModRMToReg
Allows clang to emit better code, since it doesn't need to push all the
values onto the stack and off again for every invocation.
2021-06-16 16:47:12 -04:00
Lioncash 100bc4c833 Context: Move GPR size retrieval to its own function
This is repeated in quite a few spots, so we can place it in a utility
function and just call it instead.
2021-06-16 16:14:26 -04:00
Ryan Houdek a8855a330a Merge pull request #1088 from lioncash/table
OpcodeDispatcher: Make GPR indices in SyscallOp constexpr
2021-06-16 12:17:18 -07:00
Lioncash 72a625da4a OpcodeDispatcher: Make gpr indices in SyscallOp constexpr
Places them in RO where they can't be modified.

While we're in the area, we can use an alias to prevent duplicated array
types, and also add a static assert to ensure the arrays are always the
same size.

This allows us to avoid needing to bounds check several accesses in a
row that we know will always be successful.
2021-06-16 14:24:45 -04:00
Stefanos Kornilios Mitsis Poiitidis a67bdddbdc Merge pull request #1087 from Sonicadvance1/fix_typoe
Fixes typo on extension check for squashfs
2021-06-16 10:03:48 +03:00
Ryan Houdek 8274058895 Fixes typo on extension check for squashfs
Accidentally deleted the period on this
2021-06-15 23:03:46 -07:00
Stefanos Kornilios Mitsis Poiitidis 52426ae9f3 Merge pull request #1086 from Sonicadvance1/support_squashfs
Support Squashfs based rootfs files
2021-06-16 08:50:29 +03:00
Ryan Houdek 4ff3705ff0 Update fusermount location
fuse3 package installs this to /usr/fusermount
Try both locations on failure
2021-06-15 19:50:39 -07:00
Ryan Houdek e5d3699b26 FEXConfig: Adds support for squashfs files in FEXConfig
Searches the RootFS path in the fex-emu config folder for squashfs files and displays them
Allows easy configuration of squashfs
2021-06-15 19:34:42 -07:00
Ryan Houdek a5f07f1d01 FEXLoader: Set up the squashfs rootfs in FEXLoader 2021-06-15 19:34:42 -07:00
Ryan Houdek d4789895a0 Adds a helper function for setting up a squashfs rootfs
This function does everything required for setting up a squashfs as a rootfs.
- Checks if the file is a valid squashfs
- Executes the FEXMountDaemon
- Error checks to ensure it was mounted correctly
- Updates CONFIG_ROOTFS to point to the mounted location
- Takes 200-400ms more startup time
2021-06-15 19:34:42 -07:00
Ryan Houdek 86cbf06778 Adds a FileFormatCheck for checking if a file format is squashfs
This will be used to ensure a squashfs file that is selected is sane
2021-06-15 19:20:19 -07:00
Ryan Houdek d77a503510 Adds new FEXMountDaemon tool
This is a tool that communicates with FEXLoader/FEXInterpreter to automatically mount
squashfs based rootfs files on execve.

This tool will launch automatically if you have a squashfs based rootfs selected.
Once the rootfs is mounted, it will watch for the parent FEX processe to completely exit.
Once the parent FEX exits it will unmount and cleanup after itself.

This tool has a dependency on your host having FUSE, fusermount, and squashfuse applications.
If anything goes wrong in the bringup process then it propagates the erro up the chain and will
let FEX know that it couldn't mount.
2021-06-15 19:08:54 -07:00
Ryan Houdek 9b4b136121 Move Config.h to ConfigDefines.h
We already have a Config.h which conflicts with this file.
In issue cropped up if you need to include Config.h (the generated one)
from inside Common/ then it would only pull the Common/Config.h file.

Just change the name to not be confusing
2021-06-15 19:06:49 -07:00
Stefanos Kornilios Mitsis Poiitidis 779aca7e95 Merge pull request #1085 from Sonicadvance1/transparent_huge_pages
Enables transparent huge pages in our 64-bit VA allocator
2021-06-14 17:04:40 +03:00
Stefanos Kornilios Mitsis Poiitidis 3252f793df Merge pull request #1084 from Sonicadvance1/cleanup_stacks_after_fork
Core: After fork make sure to cleanup stacks
2021-06-14 17:03:49 +03:00
Stefanos Kornilios Mitsis Poiitidis 9635b34450 Merge pull request #1083 from Sonicadvance1/more_cpuid_fixes
CPUID: Improvements to have a more sane configuration
2021-06-14 10:14:34 +03:00
Stefanos Kornilios Mitsis Poiitidis afd35be91f Merge pull request #1082 from Sonicadvance1/remove_numa_really
Removes more libnuma references
2021-06-14 10:12:35 +03:00
Stefanos Kornilios Mitsis Poiitidis 50b5ad5762 Merge pull request #1070 from Sonicadvance1/remove_warnings
Remove most warnings in FEX again
2021-06-14 10:12:03 +03:00
Ryan Houdek 10ac1518e1 Enables transparent huge pages in our 64-bit VA allocator
Transparent huge pages is a feature that the linux kernel opportunistically uses.
Depending on kernel configuration this feature is either enabled always, or when you madvise the region.

To ensure we hit both cases, madvise the regions we allocate in the 64-bit VMA allocator always

Can reduce kernel bookkeeping memory usage for our abusive allocator
2021-06-12 18:52:51 -07:00
Ryan Houdek ff1c59b6af CPUID: Improvements to have a more sane configuration
When running the cpuid application (http://www.etallen.com/cpuid.html) I noticed
that we were returning some garbage data here.

After initially implementing support for leaf functions, it still didn't resolve the issue.
So I had to fix those in the x86-64 JIT.

I then went through and solved more issues with the function results.

- We now return a more sane CPU family that is near the feature set we support
- APICID now understands how to fill out the data correctly depending on emulated core counts
- Disabled some CPU features that we don't actually support
- Found two more cache functions that we weren't populating
  - Filled with generic data cache size data
  - Only thing that matters is that we ensure that cacheline size is reported as 64bytes
  - L1D: 32KB, L1I: 32KB, L2: 512KB, L3: 8MB claimed for caches
- Implemented Leafs for functions
  - 7h - Only has leaf 0
  - Dh - Extended CPU features support
    - Another register that lets you claim support for x87, SSE, and AVX
    - Leaf 1 & 2 has some additional data
  - 4h & 8000'0001Dh - Extended cache properties
    - Almost the same as each other. One reports slightly less data though
2021-06-12 18:49:43 -07:00
Ryan Houdek 085fca01bc Core: After fork make sure to cleanup stacks
After FEX has forked, there aren't any other threads in the process but their stacks remain.
We need to have some book keeping in place to have the stack ranges available to clean up
after fork.

We now keep both live stacks and dead stacks in a dequeue and on fork we will walk both to
clean up all stack objects that aren't our current thread.
2021-06-12 18:45:13 -07:00
Ryan Houdek 3d759a91ca Jit64: Fixes register overwrite in CPUID
rsi is a SSA argument, so we need to make sure to move the leaf argument first.
the leaf argument was getting corrupted when moving to the ABI.

Will be necessary once CPUID supports leafs
2021-06-12 18:29:59 -07:00
Ryan Houdek b54385162e Removes more libnuma references
We don't need libnuma, stop trying to link to it
2021-06-12 18:28:08 -07:00
Ryan Houdek 98714a4971 Merge pull request #1079 from lioncash/iostream
Passes: Replace <iostream> header with narrower equivalents
2021-06-11 21:00:29 -07:00
Lioncash d08189c3ed Passes: Log out errors and warnings through a format specifier
Passing in the string directly through the format string input can
unintentionally cause the output string to be interpreted as a format
string.

We can specify a separate format string to ensure it always prints
without any potential mangling.
2021-06-11 22:27:02 -04:00
Lioncash 2ad9ced80a Passes: Replace <iostream> header with narrower equivalents
<iostream> injects a static constructor in translation units that
include it, even if its facilities aren't used.

We can make use of <sstream> to avoid needing to execute those on
startup.
2021-06-11 22:24:45 -04:00
Ryan Houdek 12efcdc98d Merge pull request #1078 from lioncash/passes
Passes: Return by unique_ptr where applicable
2021-06-11 19:15:22 -07:00
Lioncash 120ba3d171 PassManager: std::move exit handler function
Avoids allocations if the internal std::function buffer ever happens to
be large enough to warrant it.
2021-06-11 22:05:17 -04:00
Lioncash 6084bdf982 Passes: Return by unique_ptr where applicable
Same behavior, but makes the ownership intentions explicit in the
interface.
2021-06-11 22:05:15 -04:00
Ryan Houdek 38d25d97f5 Merge pull request #1077 from lioncash/vtab
HostAllocator: Make Create64BitAllocator() return a unique_ptr
2021-06-11 16:36:30 -07:00
Lioncash 6fc117a67c HostAllocator: Make Create64BitAllocator() return a unique_ptr
Communicates the ownership intent in the interface.

Also removes the prototype for CreateBasicAllocator, as it isn't
implemented anywhere.
2021-06-11 19:11:46 -04:00
Lioncash 1b72f73224 GlobalAllocator: Add virtual destructor to interface
Prevents any potential destruction issues from occurring in the
interface.
2021-06-11 19:04:25 -04:00
Ryan Houdek 74f63c904e Merge pull request #1076 from lioncash/leak
RegisterAllocationPass: Prevent leaks in BucketList
2021-06-11 15:08:01 -07:00
Lioncash b6edc8dc53 RegisterAllocationPass: Prevent leaks in BucketList
release() relinquishes control of the allocated memory, but doesn't
deallocate it. We need reset() for this.
2021-06-11 17:49:56 -04:00
Ryan Houdek af112f4c47 Merge pull request #1074 from lioncash/args
Syscalls/Signals: Add missing arguments to pidfd_send_signal syscall
2021-06-11 09:28:59 -07:00
Ryan Houdek 58c66b736d Merge pull request #1073 from lioncash/version
Syscalls: Add helper for version testing
2021-06-11 09:17:57 -07:00
Lioncash 061f393186 Syscalls/Signals: Add missing arguments to pidfd_send_signal syscall
These were accidentally omitted from the call.
2021-06-11 12:14:44 -04:00
Lioncash f1fd197096 Syscalls: Add helper for version testing
Shortens up the version testing code in a few places to make for quicker
reading.
2021-06-11 11:11:59 -04:00
Ryan Houdek dc664b1e2e Merge pull request #1072 from lioncash/fm
FileManager: Minor changes
2021-06-10 19:08:33 -07:00
Lioncash c971d3e56b FileManager: Construct fstream in place
While we're at it, we can make use of .data() to avoid a bounds check
that we know will never fail.
2021-06-10 21:56:08 -04:00
Ryan Houdek 0a03fbdfc6 Merge pull request #1071 from Sonicadvance1/fix_tmpfile_crash
Fixes a crash in 32-bit applications with tmpfile
2021-06-10 18:49:12 -07:00
Lioncash 10ec1d5d15 FileManager: Make use of insert_or_assign
Avoids some default constructions that get assigned over immediately.
2021-06-10 21:38:39 -04:00
Lioncash 340919d0fb FileManager: Make use of heterogenous lookup
Allows lookups with const char* and other non-allocating string types to
be performed without constructing a std::string instance.

Reduces heap usage a little.
2021-06-10 21:04:52 -04:00
Ryan Houdek 999443b898 Fixes a crash in 32-bit applications with tmpfile
If a 32-bit application leaves a temporary file dangling from one of the EmulatedFiles
then glibc tries to clean up the FILE object on shutdown.
This results in a crash on 32-bit applications because our memory allocator will have already cleaned up
at that point

Use raw FDs in this instance which matches better with the syscall hooking expecting raw FDs anyway.
2021-06-10 17:16:45 -07:00
Ryan Houdek adc5e4d6b9 JitArm64: Remove warnings in JIT.cpp 2021-06-10 15:06:53 -07:00
Ryan Houdek 903704ad48 JitArm64: Remove warning in ALUOps 2021-06-10 15:06:53 -07:00
Ryan Houdek 1d2f6c3ba8 JitArm64: Remove warnings in VectorOps.cpp 2021-06-10 15:06:53 -07:00
Ryan Houdek a07c57c52c SignalDelegator: Remove warning 2021-06-10 14:58:09 -07:00
Ryan Houdek 9118285b60 Syscalls: Remove warnings 2021-06-10 14:57:58 -07:00
Ryan Houdek 64acc9ed3c x32 FD Syscalls: Remove warning 2021-06-10 14:57:42 -07:00
Ryan Houdek d10fd4ddd4 Config: Remove warnings 2021-06-10 14:57:14 -07:00
Ryan Houdek a67b0c151e IREmitter: Remove warning 2021-06-10 14:57:02 -07:00
Ryan Houdek ba18b5dca9 ELFSymbolDatabase: Remove warning 2021-06-10 14:56:51 -07:00
Ryan Houdek 0f1a41154d RAPass: Remove warning 2021-06-10 14:56:39 -07:00
Ryan Houdek 97dfe9b26e IRValidation: Remove warning 2021-06-10 14:56:29 -07:00
Ryan Houdek f94ce4c95b IRCompaction: Remove warning 2021-06-10 14:56:17 -07:00
Ryan Houdek 0e1892622c OpcodeDispatcher: Remove warning 2021-06-10 14:56:06 -07:00
Ryan Houdek 17977ab4ac LookupCache: Remove warning 2021-06-10 14:55:53 -07:00
Ryan Houdek 78e207f837 Jit64: Remove warnings in JIT.cpp 2021-06-10 14:55:39 -07:00
Ryan Houdek e244142665 Jit64: Remove warning in ALUOps.cpp 2021-06-10 14:55:25 -07:00
Ryan Houdek 448c4ec797 Docs: Update for release FEX-2106 2021-06-10 12:59:56 -07:00
Ryan Houdek 08fad8fb0d Merge pull request #1067 from Sonicadvance1/fix_32bit_crash
Fixes 32-bit applications crashing
2021-06-10 12:57:21 -07:00
Ryan Houdek 29083a0b41 Makes sure the TestHarnessRunner also hits the shutdown path 2021-06-10 12:47:05 -07:00
Ryan Houdek a1f4ca873c Have FEXLoader use the new freeing functions
It now calls the new functions for handling a cleaner shutdown
2021-06-10 12:47:05 -07:00
Ryan Houdek c3b0e4820b Allocates the signal handlers altstack using FEX's allocator
Using malloc and free was causing a bad crash
2021-06-10 12:47:05 -07:00
Ryan Houdek bb96955e05 Switches from FILE to raw int for Logging message
Due to our current allocation strategy. This variable was ending up in a weird state
where jemalloc allocated it using the glibc allocator.
On shutdown this was causing it to try and deallocate through FEX's allocator...Which
is very broken.

This lets linux clean up in this case, at least until the allocator lines are more
strongly written
2021-06-10 12:47:05 -07:00
Ryan Houdek 16bb64aaab Merge pull request #1063 from Sonicadvance1/fix_crash
Fixes crash in 32-bit pselect6 and pselect6_time64
2021-06-10 12:41:08 -07:00
Ryan Houdek b9e53b6c46 Merge pull request #1069 from lioncash/unex
Syscalls/Thread: Replace std::unexpected with std::terminate
2021-06-09 20:11:58 -07:00
Lioncash 616aa46c6b Syscalls/Thread: Replace std::unexpected with std::terminate
std::unexpected was deprecated in C++11 and removed from the standard in
C++17. The default unexpected_handler calls std::terminate, so this is
identical behavior.
2021-06-09 22:59:21 -04:00
Ryan Houdek c98fb7ae45 Merge pull request #1068 from lioncash/unique
Core: return unique_ptr by default for cores
2021-06-09 19:19:22 -07:00
Lioncash 6c5abc8819 Core: Make CustomCPUFactory instances return unique_ptr
Makes the ownership intentions of the function explicit.
2021-06-09 21:53:07 -04:00
Lioncash f36726ecf6 Core: return unique_ptr by default for cores
Communicates ownership semantics in the API.
2021-06-09 21:41:43 -04:00
Ryan Houdek 6d20ae5dc6 Merge pull request #1062 from lioncash/parser
IRParser: Minor cleanup
2021-06-09 18:34:27 -07:00
Ryan Houdek e469031c40 Merge pull request #1064 from lioncash/syscall
x32/FD: Construct vectors in place
2021-06-09 18:26:08 -07:00
Ryan Houdek b9a8c383fd Merge pull request #1065 from lioncash/poll
{x32, x64}/EPoll: Prevent edge-case out-of-bounds access scenarios
2021-06-09 18:19:25 -07:00
Ryan Houdek 57df4c6d61 Merge pull request #1066 from lioncash/signal
SignalDelegator: Minor changes
2021-06-09 18:17:55 -07:00
Ryan Houdek c5cfb45f48 In ELFCodeLoader2, changes Sections variable over to a unique_ptr
This class is partially allocated on either side of the allocator fence.
Moves `Sections` over to a unique_ptr so we can clear this variable prior to shutdown.
The rest of the class is allocated prior to switching to FEX allocator
2021-06-09 18:05:57 -07:00
Ryan Houdek b6417153d7 Hook more glibc hooks and allow clearing hooks in Allocator
Due to a disjoint mechanism inside of glibc we need to override both the glibc
publicly visible allocation functions AND the hooks.

We had broken the overriding when we changed the default visibility. So first fix that.
Then only override to FEX's allocators once we are ready for it.
Then also replace the hooks.

Additionally have a way to clear the hooks back to default.
2021-06-09 18:03:27 -07:00
Ryan Houdek 599b579192 Adds a ShutdownStaticTables context function
Due to a mixture of allocators this needs to be shutdown in the correct location.
A little bit dirty but is necessary without a full refactor
2021-06-09 18:00:54 -07:00
Lioncash 9ed0572fa4 IRParser: Make use of unique_ptr for Parse()
Communicates ownership semantics to the user of the API a little better,
and makes it harder to accidentally leak memory.
2021-06-09 21:00:30 -04:00
Lioncash 00b0222129 IRParser: Make use of find character overloads
Results in a tiny bit better codegen. May as well, since the changes are
essentially 'free'.
2021-06-09 21:00:30 -04:00
Lioncash 32cc98856f IRParser: Add missing default return in DecodeErrorToString()
Prevents undefined behavior in the event anything actually reaches this.
2021-06-09 21:00:30 -04:00
Lioncash dee7441795 IRParser: Make input string to DecodeValue a const reference
No specializations modify this, and even if they did, it would be a
little confusing to statefully modify the string this way.

Instead, we can make it read-only.
2021-06-09 21:00:29 -04:00
Lioncash f6a6f2eac8 x32/FD: Construct vectors in place
The syscall is defined as taking in an array, so we can construct the
std::vector in place and surround it.

This also fixes an edge case where an out of bounds access on the
Host_iovec vectors could occur if these syscalls were called with an
iovcnt of zero (.at would cause an exception to be thrown).

Looking into the syscalls for readv and writev, these just return early
if a size of zero is passed in.
2021-06-09 21:00:01 -04:00
Lioncash c10e139964 x64/EPoll: Prevent edge-case out-of-bounds access scenarios
Similarly to #1064, an application could potentially pass in bad values
that could result in assertions/exceptions being thrown, so we can
guard against that.

Despite the manual saying that applications *must* not pass
maxevents less than or equal to zero, against all odds, this is
still a handled case in the kernel, so some application out there
may rely on this behavior.
2021-06-09 20:59:44 -04:00
Lioncash 379b200d05 x32/EPoll: Prevent edge-case out-of-bounds access scenarios
Similarly to #1064, an application could potentially pass in bad values
that could result in assertions/exceptions being thrown, so we can guard
against that.

Despite the manual saying that applications *must* not pass maxevents
less than or equal to zero, against all odds, this is still a handled
case in the kernel, so some application out there may rely on this behavior.
2021-06-09 20:59:44 -04:00
Lioncash b4cde28137 SignalDelegator: Move virtual destructor to base class
Ensures that destruction behavior will always handle the polymorphic
case, no matter where it occurs in the hierarchy.
2021-06-09 20:59:21 -04:00
Lioncash 3f88150581 SignalDelegator: Make std::vector into a constexpr array
Minor change, but gets rid of some heap usage.
2021-06-09 20:59:20 -04:00
Lioncash e95076f321 SignalDelegator: Migrate from std::unexpected() to std::terminate()
std::unexpected() was deprecated in C++11 and removed from the standard
in C++17. The default unexpected_handler would call std::terminate
anyway, and given we don't explicitly set a termination handler (as far
as I can tell), this retains identical behavior.
2021-06-09 20:59:20 -04:00
Lioncash e2a16c53b6 SignalDelegator: std::move std::function instances
std::function is allowed to allocate if the size of captures exceeds its
internal storage buffer. It's unlikely this is regularly going to be the
case, but we can allow for avoiding it where necessary.
2021-06-09 20:59:20 -04:00
Lioncash ee89b7a794 SignalDelegator: Leverage std::array
Allows for better detection of out of bounds accesses, as library
implementations generally allow conditional enabling of bounds checks
through preprocessor defines.
2021-06-09 20:59:20 -04:00
Ryan Houdek 5ac9b45528 Merge pull request #1059 from Sonicadvance1/fix_rounding
Fixes floating point conversion rounding bugs
2021-06-09 17:58:35 -07:00
Ryan Houdek 6a94011b00 Fixes crash in 32-bit pselect6 and pselect6_time64
The kernel can be provided a sigmaskpack pointer without a sigset inside of it.

This was causing a crash in some random applications
2021-06-08 21:20:16 -07:00
Ryan Houdek 104fabc11f Merge pull request #1061 from lioncash/dump
IRParser: Make use of fmt where applicable
2021-06-08 07:10:10 -07:00
Ryan Houdek 15b2eb48e5 Disable rounding tests that only fail on Nvidia Xavier
Spooky rounding behaviour changes.
2021-06-08 06:54:11 -07:00
Ryan Houdek 62c446a5f9 Adds more unit tests for conversion operations 2021-06-08 06:54:11 -07:00
Ryan Houdek d57a034220 Fixes floating point conversion rounding bugs
When floats and doubles were converting to integers we weren't doing the correct transformation.
AArch64 provides direct ops for all four of the op types. So lets use them
  - f32 -> int64
  - f32 -> int32
  - f64 -> int64
  - f64 -> int32

Doesn't fully fix the case of overflow for AArch64 since overflow behaviour is different.
x86 returns 0x8000'0000 or 0x8000'0000'0000'0000 while AArch64 saturates to the maximum signed
integers. Can't work around that without checking overflow flags.

This does solve the typical case though.
Fixes audio problems in all FMod games.
2021-06-08 06:54:10 -07:00
Ryan Houdek 18fe049976 Extends a couple tests to ensure correct zext 2021-06-08 06:54:08 -07:00
Stefanos Kornilios Mitsis Poiitidis c6e09ddec5 Merge pull request #1046 from Sonicadvance1/more_sse41
Implements SSE4.1
2021-06-08 16:52:12 +03:00
Lioncash 05ee4c363b IRParser: Make use of fmt where applicable 2021-06-08 09:47:13 -04:00
Lioncash 6f2f4a9bc0 IRDumper: Make use of std::string_view over std::string where applicable
Same behavior, but with a smaller footprint (and the ability to be
constant data).
2021-06-08 09:36:59 -04:00
Ryan Houdek f699726409 Merge pull request #1060 from lioncash/byte
Utils: Add header for bit-related utilities
2021-06-08 05:40:05 -07:00
Lioncash a2b8fedba5 Utils: Add header for bit-related utilities
Places a layer of separation around the remaining compiler builtins.
2021-06-08 08:25:32 -04:00
Ryan Houdek ed2336c4f2 Disables some new sse4.1 gcc tests
The SSE 4.1 path of these tests seemingly work, it's the setup code before the
test that is broken.

We will need to find out why these are failing later
2021-06-08 02:44:40 -07:00
Ryan Houdek 748c3182a7 Fixes a typo in interpreter LSHL
This was causing uint32_t pointers to sign extend on 32-bit
New tests being run were hitting this when we were expecting them to zero extend
2021-06-08 02:44:40 -07:00
Ryan Houdek dfc454873c Moves AOTIR cache queue object to be inside the context
This fixes a crash that occurs due to mixing memory allocators.
This PR has tickled the allocation just enough that it broke
2021-06-08 02:44:40 -07:00
Ryan Houdek 1b06a06a4e Makes RoundType a unique type
This allows us to have the IRDumper have unique output for the type
2021-06-08 02:44:40 -07:00
Ryan Houdek b2ad8d73ac Adds unit test for FEX frontend decoder
This could potentially show up as being ch instead of edi.
Ensure we don't regress behaviour in the future
2021-06-08 02:44:39 -07:00
Ryan Houdek fec078f924 Fixes bug in interpreter VInsGPR
If the source GPR had data that was larger than the element it would overwrite other elements
Mask it correctly. This then matches behaviour with the other CPU backends
2021-06-08 02:44:39 -07:00
Ryan Houdek 38f1cded2c Implements unit tests for all the SSE4.1 instructions 2021-06-08 02:44:39 -07:00
Ryan Houdek a313253510 Enables SSE 4.1 in CPUID 2021-06-08 02:44:39 -07:00
Ryan Houdek 7b2b2a3d07 Implements all the remaining SSE4.1 instructions
Theres a fair number of these so I won't describe them all.

A couple highlights are MPSADBW and PHMINPOSUW.
These don't really match with AArch64 very well so their IR is a bit ugly.
2021-06-08 02:44:39 -07:00
Ryan Houdek 910a25b0ed Cleans up some vector ops that don't need dedicated functions
All of these ops can use the VectorALUOp and VectorUnaryOp templated functions
2021-06-08 02:44:39 -07:00
Ryan Houdek 75bae81331 Describe the remaining SSE4.1 ops in the tables 2021-06-08 02:44:39 -07:00
Ryan Houdek 02abbd216a Fixes an issue with debug printing vectors
We need to save the vector registers otherwise we will corrupt them.
Also in the case of printing a vector register, fall down the specialized path

Only useful when debugging
2021-06-08 02:44:39 -07:00
Ryan Houdek 8b7a7e91c9 Implements new IR ops
Adds VBic, VUMinV, VPopcount, VUnZip, VUnZip2, VDupElement, Vector_FToI, and VUABDL

We will need these for the SSE4.1 ops
2021-06-08 02:44:39 -07:00
Ryan Houdek 130f82a310 Merge pull request #1058 from lioncash/builtins
General: Place more compiler specifics into CompilerDefs.h
2021-06-07 05:31:07 -07:00
Lioncash 18e0f2636f General: Make use of the <bit> header where applicable
Since C++20, a bunch of bit manipulation functions finally have a common
interface, so lets make use of those
2021-06-07 08:16:51 -04:00
Lioncash 2b9029623e General: Abstract trapping behind a define
Provides a layer of separation from direct usages of compiler builtins.
2021-06-07 06:18:44 -04:00
Lioncash 6798afe91f General: Abstract unreachable behind a define
Provides a layer of separation from direct use of compiler built-ins.
2021-06-07 06:13:26 -04:00
Ryan Houdek 4fa8522d9e Merge pull request #1057 from lioncash/cprop
ConstProp: Separate out constituent chunks of const prop
2021-06-05 14:08:40 -07:00
Lioncash 58272ffc47 ConstProp: Separate out constituent chunks of const prop
Makes it nicer to find where each part of the pass is, and also see how
the pass is operating at a high level.
2021-06-05 16:52:03 -04:00
Ryan Houdek 4c275ecc08 Merge pull request #1056 from lioncash/compiler-defs
Utils: Add CompilerDefs.h for compiler-specifics
2021-06-05 12:57:51 -07:00
Ryan Houdek 294ec80aaf Merge pull request #1055 from lioncash/enum-cls
InternalThreadState: Make SignalEvent an enum class
2021-06-05 12:55:10 -07:00
Ryan Houdek e78d3d1a80 Merge pull request #1054 from lioncash/fmt-1
GdbServer: Migrate logging/string handling to fmt where applicable
2021-06-05 12:54:11 -07:00
Lioncash 262389f4ea ConstProp: Mark internal functions as static
Allows them to have internal linkage.
2021-06-05 15:25:35 -04:00
Lioncash 341b811642 Utils: Add CompilerDefs.h for compiler-specifics
Puts a layer of separation around compiler specifics (and also makes
them nicer to write).
2021-06-05 13:55:54 -04:00
Lioncash 32744d074e InternalThreadState: Make SignalEvent an enum class
Makes the enumeration strongly typed, preventing implicit conversions,
minimizing the potential chances of an invalid value being used by
accident.
2021-06-05 10:52:56 -04:00
Lioncash 5f91bbe28d GdbServer: Migrate code to fmt
Simplifies a bunch of string manipulation code.

Much nicer to grok than stream formatting in many cases.
2021-06-05 10:34:59 -04:00
Ryan Houdek 69035348f5 Merge pull request #1053 from lioncash/alias
CodeLoader: Add type aliases for mapper and unmapper functions
2021-06-05 06:16:27 -07:00
Lioncash be46db10e4 CodeLoader: Add virtual destructor
Ensures that no matter the context the hierarchy tree is used
polymorphically, that the deallocation will always be well-defined.

Gets rid of a potential bug vector.
2021-06-05 08:54:09 -04:00
Lioncash aa2c18d8cc CodeLoader: Add type aliases for mapper and unmapper functions
Centralizes the long types in one place for less reading.
2021-06-05 08:51:40 -04:00
Ryan Houdek 0a7f0a3441 Merge pull request #1052 from lioncash/fmtimpl
LogManager: Add fmt-capable logging functions
2021-06-05 05:46:56 -07:00
Lioncash ca2b04b309 LogManager: Add fmt-capable logging functions
Addresses #146 a little more by providing an interface to perform
fmt-compatible logging.

No more, will people on the project be tormented by classic printf
features like:

- Accidentally passing in a non-trivial type
- PRI macros
- Not being able to add support for custom types
- Mixing up signed/unsigned printf formatting specifiers accidentally

fmt-capable versions of the logging functions are named the same as the
existing functions, just with a "Fmt" or _FMT suffix (depending on
whether or not it's a function being used or a macro, respectively).
2021-06-05 08:29:48 -04:00
Ryan Houdek 376892793f Merge pull request #1051 from lioncash/view
IRParser: Convert array of std::string over to array of std::string_view
2021-06-05 04:05:28 -07:00
Stefanos Kornilios Mitsis Poiitidis 61f73cf0bf Merge pull request #1050 from Sonicadvance1/fix_semctl_shmctl
Fixes semctl and msgctl
2021-06-05 13:54:30 +03:00
Stefanos Kornilios Mitsis Poiitidis 9716a4f0cf Merge pull request #1049 from Sonicadvance1/drm_headers
Adds drm headers to an external repository
2021-06-05 13:53:16 +03:00
Lioncash 35f3777797 IRParser: Make use of insert_or_assign where applicable
Avoids some minor potential default constructions that get overwritten
immediately.
2021-06-05 06:43:17 -04:00
Lioncash d73b79470e IRParser: Mark parameter of CheckPrintError as a const reference
Def is only ever accessed to read members, so we can signify to the
reader to not expect it to be modified.
2021-06-05 06:43:16 -04:00
Lioncash 676d9cb665 IRParser: std::move elements where advantageous
e.g. LineDefinitions are moderately beefy, they contain
two strings and a vector of strings among other things,
so we can move instances into their containing vector to avoid some
allocation churn.
2021-06-05 06:43:13 -04:00
Lioncash 993b4513d9 IRParser: Convert array of std::string over to array of std::string_view
Same behavior, but allows the arrays to be constexpr (and use less
space; 16 bytes vs 32 bytes per element).
2021-06-05 06:16:42 -04:00
Ryan Houdek 2958744777 Fixes semctl and msgctl
There were some problems in both of these implementations.

Fixes #744
Fixes #745
2021-06-04 23:40:24 -07:00
Ryan Houdek 2e8aacffe6 Fixes compat_ptr to return reference instead of copy
The expectation was that a reference would be returned rather than a copy.
Oops
2021-06-04 23:39:13 -07:00
Ryan Houdek 2101914c9d Adds drm headers to an external repository
It's highly likely that the host system won't have these headers installed.
Carry them in an external repository to ensure they are available.

Fixes #1047
2021-06-04 21:26:45 -07:00
Ryan Houdek 97c4ba018b Merge pull request #1048 from lioncash/lookup
Config: Avoid a few minor string copies where trivially possible
2021-06-04 17:13:10 -07:00
Lioncash 525d50f07e config: Pass strings by const reference where applicable
In a few cases, the strings aren't ever directly modified, so they can
be passed by reference to eliminate a few trivial copies.
2021-06-04 12:46:34 -04:00
Lioncash 5cec8ddca0 config: Move input strings in Loader constructors
Eliminates a copy, minor, but basically a "free" change.
2021-06-04 12:40:38 -04:00
Lioncash 39b2be15ab config: Make use of heterogenous map lookup
Same behavior, but allows lookups without constructing a std::string
(in most cases find() inputs use const char*, so this gets rid of some
string churn).
2021-06-04 12:36:54 -04:00
Ryan Houdek 83bf79ab42 Merge pull request #1045 from lioncash/default
IR: Make use of defaulted operator==
2021-06-03 22:01:58 -07:00
Lioncash 1f97a2baa9 IR: Make use of defaulted operator==
Same behavior, but allows both operator== and operator!= to be
automatically generated with a single declaration.
2021-06-04 00:52:25 -04:00
Ryan Houdek 0c2344166c Merge pull request #1044 from lioncash/moves
Config: Move strings where applicable
2021-06-03 18:33:00 -07:00
Lioncash 0bf30e2024 Config: Simplify qualifiers
These functions are part of the same class, so we can use the function
names directly without qualifiers.
2021-06-03 21:06:14 -04:00
Lioncash 933cdf76b4 Config: Move strings where applicable
Noticed when adding amending missing const qualifiers on interfaces. We
can make use of std::move here to avoid potential allocation churn a
little.

While we're at it, we can implement EraseSet in terms of, well, Erase()
and Set().
2021-06-03 21:03:19 -04:00
Ryan Houdek 36a77bb396 Merge pull request #1043 from lioncash/const
General: Add missing const specifiers where applicable
2021-06-03 17:40:13 -07:00
Lioncash 1936ebc59c General: Add missing const specifiers where applicable
Minor change that adds missing const specifiers to getters that don't
modify internal class state.
2021-06-03 20:24:27 -04:00
Ryan Houdek 9838309560 Merge pull request #1042 from lioncash/strong
DecodedOperand: Convert operand type into an enum class
2021-06-02 05:01:56 -07:00
Lioncash 37e2210f64 DecodedOperand: Convert operand type into an enum class
Now possible in a less messy way, since the type is now centralized in
one location.

Makes it strongly typed and prevents any potential accidental implicit
assignments to Type instead of something intended for the Data members.
2021-06-02 06:51:15 -04:00
Ryan Houdek 71c32c0135 Merge pull request #1041 from lioncash/info
DecodedOperand: Add helper functions for type testing
2021-06-02 03:10:49 -07:00
Lioncash 1f0bc49e54 DecodedOperand: Add helper functions for type testing
Significantly shortens the amount of code necessary for testing the type
of a decoded operand.

Also relocates the type field out of all the union types to have it in a
central location.
2021-06-02 05:59:12 -04:00
Lioncash 718f7ef8e6 X86InstInfo: Add operator!=
Provides logical symmetry.
2021-06-02 05:44:31 -04:00
Ryan Houdek edd1dfdfe8 Merge pull request #1040 from Sonicadvance1/more_syscalls_5_12
Implements more syscalls for supporting a higher guest kernel version
2021-06-01 00:27:50 -07:00
Ryan Houdek 4243ed19a4 Merge pull request #1037 from Sonicadvance1/fexconfig_downgrade
FEXConfig: Allow lower GL versions
2021-06-01 00:14:04 -07:00
Scott Mansell 16467c0fb7 Merge pull request #1038 from Sonicadvance1/implement_insertps
Implements SSE4.1 insertps
2021-06-01 19:04:53 +12:00
Ryan Houdek 0b7587768d Unifies 32bit and 64bit clone implementation
Syscall entry points still have different argument orders,
Moves the arguments to the clone3 argument structure and passes to generic handler.
Also implements clone3 while doing this
2021-05-27 23:00:00 -07:00
Ryan Houdek d115a57fe6 Implements support for execveat 2021-05-27 22:59:57 -07:00
Ryan Houdek 4c74478610 Implements iouring syscalls
This is a very simple initial implementation.
io_uring allows some things that are hard to capture like setting personalities.

Assume sane usage for now
2021-05-27 22:59:55 -07:00
Ryan Houdek 77a111cf19 Implements process_madvise 64-bit syscall 2021-05-27 22:59:52 -07:00
Ryan Houdek 7e551e5f88 Implements epoll_pwait2 syscall 2021-05-27 22:59:49 -07:00
Ryan Houdek 9d44dfdca6 Actually updated the hardcoded guest kernel locations
Just a couple of files and uname syscall
2021-05-27 22:59:47 -07:00
Ryan Houdek a7858f4d37 Calculates a guest kernel version for FEX instead
Takes the host kernel version and makes sure it fits in our supported kernel range
Minimum kernel version FEX reports to the guest is 5.0
Maximum kernel version FEX reports to the guest is currently 5.12
2021-05-27 22:59:44 -07:00
Ryan Houdek 6b3b10f8ae Implements pidfd_send_signal syscall
Now that we know when to forward a siginfo_t to the guest, we can allow this syscall
2021-05-27 22:59:42 -07:00
Ryan Houdek 9d5fa64f68 Implements pidfd_open syscall 2021-05-27 22:59:39 -07:00
Ryan Houdek 8b3bbd0ea1 Implements openat2 2021-05-27 22:59:37 -07:00
Ryan Houdek cc477b6964 Implements close_range syscall 2021-05-27 22:59:34 -07:00
Ryan Houdek 7a64bba8c4 Removes check for ThreadState being standard layout
Latest clang and libstdc++ makes unique_ptr not be standard layout.
Results in a compile error
2021-05-27 22:59:31 -07:00
Ryan Houdek 2d342d4662 Handle user provided siginfo_t with user signal
If the guest has sent a signal and the si_code is SI_USER then
we need to pass that siginfo_t through without touching it.
User could be sticking whatever they want in to that struct
2021-05-27 22:59:27 -07:00
Stefanos Kornilios Mitsis Poiitidis 7b41808806 Merge pull request #1039 from Sonicadvance1/pidfd_getfd
Implements pidfd_getfd syscall
2021-05-28 07:30:30 +03:00
Ryan Houdek 912d019ad8 Implements pidfd_getfd syscall
Rise of the Tomb Raider launcher uses this without checking host kernel version.
Might be part of their crash handler.
2021-05-27 18:44:12 -07:00
Ryan Houdek 1f9405b880 Implements insertps unit test 2021-05-26 21:41:16 -07:00
Ryan Houdek 211e7bf0f0 Implements SSE4.1 insertps
Cheap Golf is using this instruction unconditionally.
With this implemented the game now runs
2021-05-26 21:40:39 -07:00
Ryan Houdek 822a08f271 FEXConfig: Allow lower GL versions
Automatically fall back through older GL versions

This allows us to freely support GL 3.0, 2.1 and ES 2.0.
Should fix an issue where Pi devices don't support GL 3.0 in all configs

External imgui had to be updated to fix an issue with ES 2.0

Fixes #1036
2021-05-26 18:45:03 -07:00
Ryan Houdek 6cba775d4e Merge pull request #1032 from FEX-Emu/skmp/aotir-gen-mt
Multi threaded AOTGen
2021-05-24 23:06:13 -07:00
Ryan Houdek 8b5873061a Merge pull request #1033 from lioncash/fmtlib
Externals: Add fmtlib as an external
2021-05-20 14:58:33 -07:00
Lioncash 37a33bf127 Externals: Add fmt as an external
Begins the process of addressing issue #146.

This is separated off, so that others can make use of fmt for other
purposes (general localized non-sucky string formatting), while the
logging rework is being tackled.
2021-05-20 13:18:38 -04:00
Stefanos Kornilios Mitsis Poiitidis d70c91ddc6 AOTIR: Multi-threaded AOTGen 2021-05-19 14:19:55 +03:00
Stefanos Kornilios Mitsis Poiitidis 370f36c8f7 Merge pull request #1027 from Sonicadvance1/fix_ppoll
ppoll fixes
2021-05-18 09:20:32 +03:00
Ryan Houdek 42bb27b1fb Merge pull request #1025 from Sonicadvance1/ignore_non_canonical
Ignore non-canonical addresses in FS/GS setting
2021-05-12 22:31:09 -07:00
Ryan Houdek f522303837 gvisor: arch_prctl now passes 2021-05-12 22:17:13 -07:00
Ryan Houdek 3d09a55715 Ignore non-canonical addresses in FS/GS setting 2021-05-12 22:17:13 -07:00
Ryan Houdek c103f54774 Merge pull request #1028 from Sonicadvance1/remove_syscall_forwards
Removes syscall forward errno preprocessor implementation
2021-05-12 22:04:30 -07:00
Ryan Houdek dc7437b8c6 Merge pull request #1018 from FEX-Emu/skmp/streamable-aotir
AOTIR: .aotir files are now streamed out
2021-05-12 22:04:17 -07:00
Ryan Houdek b45503f93c gvisor: chown and sync tests now pass 2021-05-12 21:54:08 -07:00
Ryan Houdek 8cf1b0263d Removes syscall forward errno preprocessor implementation
This was causing issues with syscalls returning errors.
Remove it and move on
2021-05-12 21:53:45 -07:00
Stefanos Kornilios Mitsis Poiitidis 7ecaf24e8b Merge pull request #1030 from Sonicadvance1/disable_userfaultfd
Disable userfaultfd until supported
2021-05-12 09:45:43 +03:00
Stefanos Kornilios Mitsis Poiitidis 52f7ea5433 Merge pull request #1029 from Sonicadvance1/fix_fadvise64
Fix fadvise64
2021-05-12 09:45:21 +03:00
Stefanos Kornilios Mitsis Poiitidis 023a32ea26 Merge pull request #1026 from Sonicadvance1/fix_flag_remapping
Handle FD flag remapping correctly
2021-05-12 09:39:39 +03:00
Stefanos Kornilios Mitsis Poiitidis 36980131e0 Merge pull request #1024 from Sonicadvance1/signal_fixes
Sigaction fixes
2021-05-12 09:34:51 +03:00
Stefanos Kornilios Mitsis Poiitidis b2eb13fa1c Merge pull request #1023 from Sonicadvance1/emulate_map_32bit
Emulate MAP_32BIT in mmap
2021-05-12 09:34:18 +03:00
Stefanos Kornilios Mitsis Poiitidis 27b6497bee Merge pull request #1031 from Sonicadvance1/fix_epoll
Use epoll syscalls directly
2021-05-12 09:33:59 +03:00
Stefanos Kornilios Mitsis Poiitidis 66ee89ec8a Merge pull request #1022 from Sonicadvance1/fix_uname
Fix uname
2021-05-12 09:22:58 +03:00
Stefanos Kornilios Mitsis Poiitidis bee73199e1 Merge pull request #1021 from Sonicadvance1/fix_thread_self
Support redirecting thread-self exe softlink
2021-05-12 09:22:36 +03:00
Stefanos Kornilios Mitsis Poiitidis 0e6db843ef Merge pull request #1020 from Sonicadvance1/invalid_syscall
Return ENOSYS on too large of syscall number
2021-05-12 09:21:17 +03:00
Stefanos Kornilios Mitsis Poiitidis b45538f01e Merge pull request #1019 from Sonicadvance1/fix_emufd
Fix two EmuFD issues
2021-05-12 09:20:53 +03:00
Ryan Houdek f8cf98d378 gvisor: fadvise64 test now passes 2021-05-11 18:55:48 -07:00
Ryan Houdek 796c5ccbc9 gvisor: sigaction_test now passes 2021-05-11 18:54:08 -07:00
Ryan Houdek b157a5a0fb gvisor: bad_test now passes 2021-05-11 18:53:22 -07:00
Ryan Houdek 783ceb2d56 Use epoll syscalls directly 2021-05-11 18:51:06 -07:00
Ryan Houdek d1fc65daaa Disable userfaultfd until supported
Until we properly wrap this we can't support it
2021-05-11 18:44:31 -07:00
Ryan Houdek 94f464e1a4 Fix fadvise64
posix variant still doesn't quite match the actual syscall implementation
2021-05-11 18:39:00 -07:00
Ryan Houdek dfb15aaeb9 ppoll fixes
glibc implementation makes a copy of the timeout and the kernel is expected to update it
Can't use the glibc implementation because of this.
2021-05-11 18:33:24 -07:00
Ryan Houdek 565f0e6af3 Handle FD flag remapping correctly
FD flag remapping was broken. It would remap one flag on to another and then the next check would remap it back.

Instead keep a mask of the flags to be remapped then remap them all at the end.
Also goes through the ops and fixes a few cases where it was remapping wrong flags.
2021-05-11 18:29:21 -07:00
Ryan Houdek b03554bb7d Sigaction fixes
Check for invalid sigsetsize

Since we are using SignalDelegator we don't use errno, so just return Result and check for Result == 0
2021-05-11 18:22:23 -07:00
Ryan Houdek 7b8cb5107a Emulate MAP_32BIT in mmap
If we are on AArch64 then MAP_32BIT doesn't exist.
Emulate it by setting the address as a hint to the kernel so it scans bottom up
2021-05-11 18:18:59 -07:00
Ryan Houdek cbb20ef6c6 Fix uname
There is a domainname variable that was missed
2021-05-11 18:16:35 -07:00
Ryan Houdek 298481f9af Support redirecting thread-self exe softlink 2021-05-11 18:15:14 -07:00
Ryan Houdek 0d108cd8fd Return ENOSYS on too large of syscall number 2021-05-11 18:14:02 -07:00
Ryan Houdek 490d7e57d7 Arguments file needs one additional null argument to finish the arguments. 2021-05-11 18:12:16 -07:00
Ryan Houdek fe77c3ef06 Fix cpuinfo needing tabs on its options and PM option
Depending on option it is split by anywhere from zero to two tabs
2021-05-11 18:12:16 -07:00
Ryan Houdek eb02afe952 Merge pull request #1013 from Sonicadvance1/fix_openat_symlinks
Handles symlinks in rootfs in openat
2021-05-11 17:52:07 -07:00
Stefanos Kornilios Mitsis Poiitidis b8dc63754b AOTIR: Don't keep IR,RA,Code data around when aotirgenerating 2021-05-11 15:15:39 +03:00
Stefanos Kornilios Mitsis Poiitidis df8f1850ae AOTIR: .aotir files are now stream-written 2021-05-11 14:37:12 +03:00
Ryan Houdek e282fd2221 Merge pull request #1016 from Sonicadvance1/ioctl32_micro_optimization
Microoptimization for the DRM ioctls
2021-05-10 21:14:16 -07:00
Stefanos Kornilios Mitsis Poiitidis 3147f0d84f Merge pull request #1017 from Sonicadvance1/fix_x86_jitsymbols
Fixes JITSymbols for x86-64 JIT
2021-05-10 00:36:30 +03:00
Stefanos Kornilios Mitsis Poiitidis 4c02c9f037 Merge pull request #1015 from Sonicadvance1/fix_inotify_thread
FEXConfig: Fixes INotify watcher never coming up
2021-05-10 00:34:27 +03:00
Ryan Houdek 83b4188569 Microoptimization for the DRM ioctls
For DRM applications have a three FD deep MRU cached for faster lookups of FD to DRM handlers.
In a completely DRM ioctl bound situation like es2gears or GL application without threaded context
Then this puts us /nearly/ at the performance of calling the ioctl32 handler directly.
Sadly there is overhead that can't be overcome so this is the best that can be done from userland
2021-05-06 02:55:22 -07:00
Ryan Houdek 82b04d9886 Fixes JITSymbols for x86-64 JIT 2021-05-06 01:12:51 -07:00
Ryan Houdek 36fdd7f6e2 FEXConfig: Fixes INotify watcher never coming up
Inverted this check on accident
2021-05-05 19:44:31 -07:00
Ryan Houdek 6a1ca654db Docs: Update for release FEX-2105 2021-05-05 17:53:55 -07:00
Ryan Houdek 623b29de8c Merge pull request #1014 from Sonicadvance1/fix_host_stacks
Fixes host thread stacks ending up in lower 32-bit VA
2021-05-05 15:51:40 -07:00
Ryan Houdek 9be67fb22a Merge pull request #1009 from Sonicadvance1/handle_self
Handle more cases of an application pinging self
2021-05-05 14:51:11 -07:00
Ryan Houdek cb9286c11e Merge pull request #1012 from Sonicadvance1/fix_ioctl_errors
Fixes ioctl returning error
2021-05-05 14:50:46 -07:00
Ryan Houdek 1e40148f60 Fixes host thread stacks ending up in lower 32-bit VA
Overriding glibc's mmap and munmap functions don't encapsulate that functions they use
for allocating stack data so it was falling down the system mmap/munmap path.
This was causing host side stacks to end up in the 32-bit VA space in 32-bit applications.
2021-05-04 22:15:46 -07:00
Ryan Houdek 19e45544fb Handles symlinks in rootfs in openat
This fixes an issue where rootfs has symlinks to other things in the rootfs so we need to track it through.

Relies on #1009 to be merged first.
Fixes any application that relies on libblas, mpv for example.
2021-05-04 22:13:52 -07:00
Ryan Houdek d9d5303aa0 Fixes ioctl returning error
Fixes evdev device description at the very least
2021-05-04 18:45:25 -07:00
Ryan Houdek 517b575783 Merge pull request #1010 from FEX-Emu/skmp/aotupdate-install-cmake
AOTIR: Add install rule for FEXUpdateAOTIRCache
2021-05-04 18:11:27 -07:00
Ryan Houdek 605994dca2 Merge pull request #1011 from FEX-Emu/skmp/fix-0xcd-breaks
JIT: Add reason 1 (0xCD) to OP_BREAK implementations
2021-05-04 18:11:17 -07:00
Ryan Houdek ac97c05271 Handle more cases of an application pinging self
AppImage uses this to inspect its own executable to ensure it is an AppImage file.
2021-05-04 18:09:15 -07:00
Stefanos Kornilios Mitsis Poiitidis c8bb02f3a6 JIT: Add reason 1 (0xCD) to OP_BREAK implementations 2021-05-04 13:22:00 +03:00
Stefanos Kornilios Mitsis Poiitidis 6b12c1768f AOTIR: Add install rule for FEXUpdateAOTIRCache 2021-05-04 12:33:59 +03:00
Stefanos Kornilios Mitsis Poiitidis 693097c5b1 Merge pull request #1008 from Sonicadvance1/fix_binfmt_misc
Updates binfmt_misc files to support AppImage
2021-05-04 11:14:49 +03:00
Ryan Houdek 47e823b773 Updates binfmt_misc files to support AppImage
AppImage files stick additional data in the ABI Version and PAD areas that it was getting blocked on.
2021-05-03 18:08:00 -07:00
Ryan Houdek fd6d33d197 Merge pull request #996 from FEX-Emu/skmp/aotirgen
AOTIR: Offline Generation
2021-05-03 15:58:57 -07:00
Stefanos Kornilios Mitsis Poiitidis d8b583bc51 Remove assert that was used in debugging 2021-05-04 01:47:39 +03:00
Stefanos Kornilios Mitsis Poiitidis be41f14452 Fix generation script 2021-05-03 18:16:43 +03:00
Stefanos Kornilios Mitsis Poiitidis cf5bd202d4 AOTIR: Update gen script to default to global fex installation 2021-05-03 06:15:25 +03:00
Stefanos Kornilios Mitsis Poiitidis 8893cc6a7d Undo xbyak update 2021-05-03 05:48:00 +03:00
Stefanos Kornilios Mitsis Poiitidis 1d82e9e59e JIT: Stub Break reasons 2, 3 2021-05-03 05:42:46 +03:00
Stefanos Kornilios Mitsis Poiitidis 5b017f9a66 Merge pull request #1007 from Sonicadvance1/more_ioctl32_emulation2
Adds Nouveau and Lima ioctl32 handlers
2021-05-03 05:10:21 +03:00
Ryan Houdek 3399052eaa Make sure to setup ioctl32 handler table 2021-05-02 14:30:23 -07:00
Ryan Houdek 3df951695e Adds Nouveau and Lima ioctl32 handlers 2021-05-02 13:05:46 -07:00
Ryan Houdek 8d1f9a9081 Merge pull request #1006 from Sonicadvance1/implement_clflush
Implements support for CLFLUSH
2021-05-02 12:27:54 -07:00
Ryan Houdek 68adb1ae12 Basic clflush test 2021-05-02 12:18:35 -07:00
Ryan Houdek 4961604807 Implements support for CLFLUSH
JRE requires this to run
2021-05-02 12:18:35 -07:00
Stefanos Kornilios Mitsis Poiitidis d6b7f32d33 AOTGen: Cleanup unwind symbol logic, add to x86-32 binaries 2021-05-02 11:08:12 +03:00
Stefanos Kornilios Mitsis Poiitidis 6dc69063f3 Merge pull request #1005 from Sonicadvance1/remove_numa
Remove reliance on librt and libnuma
2021-05-02 09:50:10 +03:00
Stefanos Kornilios Mitsis Poiitidis 2916c4f34c Merge pull request #1004 from Sonicadvance1/redirect_self
Redirect applications using execve with self
2021-05-02 09:49:55 +03:00
Stefanos Kornilios Mitsis Poiitidis 04137d0da5 Merge pull request #1002 from Sonicadvance1/sigtimedwait
Initial base implementation of sigtimedwait
2021-05-02 09:49:49 +03:00
Stefanos Kornilios Mitsis Poiitidis afa81294c9 Merge pull request #1001 from Sonicadvance1/more_ioctl32_emulation
More ioctl32 emulation
2021-05-02 09:49:44 +03:00
Ryan Houdek 86bdbe65e5 Implements missed x86-32 message queue syscalls
Nothing hit these but good to have them implemented
2021-05-01 17:40:33 -07:00
Ryan Houdek 63dee0e132 Remove reliance on librt and libnuma
These can fall down the raw syscall path
2021-05-01 17:40:33 -07:00
Ryan Houdek 6618bb809b Redirect applications using execve with self
Fixes a step in the Java JRE and shapez.io
2021-05-01 11:09:44 -07:00
Ryan Houdek d2b19c0b1c Disable gvisor sigtimedwait test 2021-05-01 10:19:13 -07:00
Ryan Houdek 90a8c4f114 Shuffle timed sigwait unit tests that have changed behaviour 2021-05-01 10:11:20 -07:00
Ryan Houdek 866baf66db Merge pull request #1003 from Sonicadvance1/cleaned_sendrecvmsg
Workaround fixes in sendmsg/recvmsg
2021-05-01 10:06:41 -07:00
Ryan Houdek 9498a41f9b More ioctl32 emulation
More DRM and i915 things, then wireless for some reason renderdoc relies on it
2021-05-01 09:40:02 -07:00
Ryan Houdek 1340ffabe4 Workaround fixes in sendmsg/recvmsg
Rebased and cleaned up #933
Needs to be larger allocations than what was originally proposed by #933
2021-05-01 09:37:01 -07:00
Ryan Houdek b5115e096f Initial base implementation of sigtimedwait 2021-05-01 09:35:29 -07:00
Ryan Houdek 235e05e8b9 Merge pull request #978 from Sonicadvance1/wip_allocator
64-bit allocator and ioctl emulation
2021-05-01 09:29:46 -07:00
Ryan Houdek 26b4bd80bf Force 32-bit allocator for CI 2021-05-01 09:19:55 -07:00
Ryan Houdek a8881e8835 Adds option to force 32-bit allocator 2021-05-01 09:19:55 -07:00
Ryan Houdek d1da1b0be5 Disables 20080723-1.c.gcc-target-test-32 CI test
This test consistently fails on the solid run board.
This requires some deep dive investigation
2021-05-01 09:06:39 -07:00
Ryan Houdek a19c0184a6 Keep using the 32-bit allocator if the kernel is old
Works around the Xavier boards running kernel 4.9
2021-05-01 09:06:39 -07:00
Ryan Houdek 2d0dced9e7 Work around old asound headers 2021-05-01 09:06:39 -07:00
Ryan Houdek 52cb96d630 Work around old DRM headers for i915 2021-05-01 09:06:39 -07:00
Ryan Houdek 94017d2e65 On old kernels, steal the 32-bit address space before the 64-bit
Increases speed of startup
2021-05-01 09:06:39 -07:00
Ryan Houdek 13b93ef6b5 Allow memory allocation to go through if the base was above the lower bound 2021-05-01 09:06:39 -07:00
Ryan Houdek c83a5d1bc3 Allow some more clone flags through with info log 2021-05-01 09:06:39 -07:00
Ryan Houdek c20777d558 Change these log messages so we know which one failed 2021-05-01 09:06:38 -07:00
Ryan Houdek 9a0fc4e7e8 Workaround quick with memory allocator. Aligned 4GB regions are hard to allocate 2021-05-01 07:39:14 -07:00
Ryan Houdek 417e8836a4 Arm64: Fixes some wrong sign extensions 2021-05-01 07:39:14 -07:00
Ryan Houdek 237865e064 Adds ioctl verification to struct verifier 2021-05-01 07:39:14 -07:00
Ryan Houdek 45223cc7e7 Adds initial ioctl emulation code 2021-05-01 07:39:14 -07:00
Ryan Houdek 8d34222556 Add vardecl type matching for ioctl verification 2021-05-01 07:39:14 -07:00
Ryan Houdek b6344c4753 Removes old 32-bit allocator usage
Replaces with using host side allocations since we always have a 64-bit allocator in this case
2021-05-01 07:39:14 -07:00
Ryan Houdek c5bda47b2a Adds more allocator hooking 2021-05-01 07:39:14 -07:00
Ryan Houdek 5067737052 Adds 64-bit allocator 2021-05-01 07:39:14 -07:00
Ryan Houdek 5143100a73 Switches more usages of memory allocation routines over to FEX internal 2021-05-01 07:39:14 -07:00
Stefanos Kornilios Mitsis Poiitidis 8b6e767742 Frontend: Slightly more resilient max instruction size handling 2021-05-01 09:40:50 +03:00
Stefanos Kornilios Mitsis Poiitidis 331549c64a RLSE: Use Constant, not inline constant, let ConstProp generate all inline consts 2021-05-01 09:40:50 +03:00
Stefanos Kornilios Mitsis Poiitidis be0b29bbc6 OpDisp: Convert more asserts to soft-errors 2021-05-01 09:40:50 +03:00
Stefanos Kornilios Mitsis Poiitidis ee8f8ed176 Improve error handling 2021-05-01 09:40:50 +03:00
Stefanos Kornilios Mitsis Poiitidis 5b8cbae1e1 Core/Frontend: Improve error handling 2021-05-01 09:40:50 +03:00
Stefanos Kornilios Mitsis Poiitidis b98ef9e48b Core/Frontend: Improve error handling 2021-05-01 09:40:50 +03:00
Stefanos Kornilios Mitsis Poiitidis fd724448ab Front End: Fix an error check 2021-05-01 09:40:50 +03:00
Stefanos Kornilios Mitsis Poiitidis f4083883ca Frontend: Fix ExternalBranches initialization 2021-05-01 09:40:50 +03:00
Stefanos Kornilios Mitsis Poiitidis 9a4fc758c1 AOTIR: FEXUpdateAOTIRCache.sh now passes compilation modifiers 2021-05-01 09:40:50 +03:00
Stefanos Kornilios Mitsis Poiitidis 40d2cd8992 Fix asserts 2021-05-01 09:40:50 +03:00
Stefanos Kornilios Mitsis Poiitidis ab44f800f7 AOTgen: Cleaner interfaces, use LogMan 2021-05-01 09:40:50 +03:00
Stefanos Kornilios Mitsis Poiitidis 1a5bc39d09 Rebase fixes 2021-05-01 09:40:50 +03:00
Stefanos Kornilios Mitsis Poiitidis 4bd997adaa AOT: Postfix ir files with .aotir 2021-05-01 09:40:50 +03:00
Stefanos Kornilios Mitsis Poiitidis 244ef94e9b AOTIR: Generate list of files with code, add Scripts/FEXUpdateAOTIRCache.sh to generate all files 2021-05-01 09:40:50 +03:00
Stefanos Kornilios Mitsis Poiitidis a85ef061d1 AOITR: Add AOTIRGenerate fo FEXLoader option that does generation 2021-05-01 09:40:50 +03:00
Stefanos Kornilios Mitsis Poiitidis 4e847d03b7 Remove more assertions for invalid code 2021-05-01 09:40:49 +03:00
Stefanos Kornilios Mitsis Poiitidis ce9324f50f Kinda-parse unwind table 2021-05-01 09:40:49 +03:00
Stefanos Kornilios Mitsis Poiitidis 0897fc4a0a pretrans wip 2021-05-01 09:40:49 +03:00
Ryan Houdek d256bc59fd Override glibc allocator functions 2021-04-30 18:28:45 -07:00
Ryan Houdek e0d6f4bc34 Make sure link jemalloc in FEXCore 2021-04-30 18:28:45 -07:00
Ryan Houdek 09284331a7 Adds pregen include directory for jemalloc 2021-04-30 18:28:45 -07:00
Ryan Houdek 73e61b0c8e Switches default logging output to stderr
stdout is too likely to break something
2021-04-30 18:28:45 -07:00
Ryan Houdek a30866a886 Merge pull request #976 from FEX-Emu/skmp/aotir-mmap
AOTIR: Switch over to mmap-based loading
2021-04-30 18:24:54 -07:00
Stefanos Kornilios Mitsis Poiitidis 3e1da51cae Merge pull request #998 from Sonicadvance1/remove_xxhash_check
Removes xxhash version check
2021-04-30 12:05:18 +03:00
Stefanos Kornilios Mitsis Poiitidis 4efe973b65 Merge pull request #999 from Sonicadvance1/fix_sdl2_interface_Again
Fixes SDL2 interface check...again.
2021-04-30 12:04:50 +03:00
Ryan Houdek f6cc75de87 Fixes SDL2 interface check...again. 2021-04-30 01:55:08 -07:00
Ryan Houdek 6b082f4ef7 Removes xxhash version check
xxhash doesn't yet have a version of the library with pkg-config giving
a version.
Debian has backported a change for this but Arch hasn't.
https://github.com/Cyan4973/xxHash/issues/524
2021-04-30 01:35:03 -07:00
Stefanos Kornilios Mitsis Poiitidis 8a73783a07 AOTIR: Use FEXCore::Allocator 2021-04-29 08:16:21 +03:00
Stefanos Kornilios Mitsis Poiitidis 32b4b02302 AOTIR: mmap based loading 2021-04-28 13:08:43 +03:00
Stefanos Kornilios Mitsis Poiitidis 528b01ad7a Merge pull request #988 from Sonicadvance1/memory_layout_optimizations
Memory layout optimizations
2021-04-28 12:31:53 +03:00
Stefanos Kornilios Mitsis Poiitidis cd0492a143 Merge pull request #994 from Sonicadvance1/enable_armv8.4
Enables ARMv8.4 CI runner
2021-04-28 09:53:47 +03:00
Ryan Houdek 26bf903b93 Enables ARMv8.4 CI runner
Requires #990 and #993 to be merged first
2021-04-27 23:16:39 -07:00
Stefanos Kornilios Mitsis Poiitidis aae4de106a Merge pull request #992 from Sonicadvance1/optimize_cpuid
Optimizes CPUID generation a bit
2021-04-28 09:12:09 +03:00
Stefanos Kornilios Mitsis Poiitidis c495d8120d Merge pull request #993 from Sonicadvance1/fix_sdl_missed_line
Oops: Missed line on FEXConfig SDL2
2021-04-28 09:10:52 +03:00
Stefanos Kornilios Mitsis Poiitidis a6d9425091 Merge pull request #990 from Sonicadvance1/fix_native_m1
Work around M1 Parallels hypervisor not showing CPU type
2021-04-28 09:09:08 +03:00
Ryan Houdek b06ebf486f Work around M1 Parallels hypervisor not showing CPU type
Parallels claims that the CPU is of implementor 0x41 and part number 0.
Easy enough to work around before real numbers get exposed
2021-04-27 22:26:35 -07:00
Ryan Houdek b17464d81a Oops: Missed line on FEXConfig SDL2 2021-04-27 16:51:12 -07:00
Ryan Houdek 2ab8a97055 Merge pull request #989 from Sonicadvance1/sdl2_fix
Only use SDL2 import target if it exists
2021-04-27 01:11:28 -07:00
Ryan Houdek 34a5eb6d6f Optimizes CPUID generation a bit
The flags calculation is fairly intensive and we were doing it for every core.
Do it once then duplicate it per core instead

Improves generation time quite significantly. A couple of milliseconds converted to half a millisecond
Slight improvement to startup time
2021-04-27 01:09:28 -07:00
Ryan Houdek 2a1a863c58 Only use SDL2 import target if it exists
Should fix Arch and Ubuntu 20.04 together
2021-04-27 00:45:20 -07:00
Ryan Houdek bab5927931 Merge pull request #981 from Sonicadvance1/optimize_long_divide
Adds Long Divide removal pass
2021-04-26 23:48:50 -07:00
Ryan Houdek 6255eba524 Memory layout optimizations
Some of these memory layouts weren't optimal for ARM loading.
This is now significantly improved
2021-04-26 23:48:22 -07:00
Ryan Houdek 37108058d1 Removes some unnecessary branching checks
It doesn't actually matter if the invalid node has has its linked list nodes changed
Faster to just set than to check while in a tight loop
2021-04-26 23:48:22 -07:00
Stefanos Kornilios Mitsis Poiitidis c3ad571062 Merge pull request #987 from Sonicadvance1/fix_assert_dumb_code
Changes LogMan assert to defines
2021-04-27 09:41:12 +03:00
Ryan Houdek 04e0baadc6 Changes LogMan assert to defines
C++ no-op functions can't optimize out the predicate arguments in all cases.
This was causing a problem where zero cost assertions weren't actually zero cost.

The only way to resolve this is to actually use macros sadly enough.

This will give a fairly hefty performance uplift with anything operating on IR.
2021-04-26 13:49:35 -07:00
Ryan Houdek 5ffbd97f01 IROp types can be marked const
Allows the compiler to know it can be optimized out without LTO
2021-04-26 13:45:37 -07:00
Ryan Houdek 7d4380fe6d Merge pull request #984 from FEX-Emu/skmp/relocation-fixes
IR: Fix RIP relocation edge-cases
2021-04-26 10:11:24 -07:00
Ryan Houdek 7d5157b602 Merge pull request #983 from FEX-Emu/skmp/faster-l1
LookupCache: Add new entries to L1C
2021-04-26 10:10:51 -07:00
Ryan Houdek e9a32da997 Merge pull request #986 from FEX-Emu/skmp/fix-20.04-build
Deps: Require xxhash 0.7.3, ubuntu 20.04 ships with that version
2021-04-26 10:05:35 -07:00
Stefanos Kornilios Mitsis Poiitidis b3564a4a48 IR: Fix RIP relocation edge-cases 2021-04-26 19:20:01 +03:00
Stefanos Kornilios Mitsis Poiitidis 2294419353 Deps: Require xxhash 0.7.3, ubuntu 20.04 ships with that version 2021-04-26 19:07:23 +03:00
Stefanos Kornilios Mitsis Poiitidis df8b78b327 Dispatcher: Use Also lookup L1C when dispatching interpreter 2021-04-26 19:06:11 +03:00
Stefanos Kornilios Mitsis Poiitidis 2ea6a2a141 LookupCache: Fix L1C to actually cache entries 2021-04-26 18:38:03 +03:00
Ryan Houdek db776fae4e Merge pull request #985 from FEX-Emu/skmp/rip-relative-compilation
IR: Remove Entry from OP_HEADER, pass as parameter to CompileCode
2021-04-25 18:27:37 -07:00
Stefanos Kornilios Mitsis Poiitidis b45b7c3441 IR: Remove Entry from OP_HEADER, pass as parameter to CompileCode 2021-04-25 13:15:23 +03:00
Stefanos Kornilios Mitsis Poiitidis 7805552edb LookupCache: Add new entries to L1C 2021-04-25 13:03:43 +03:00
Ryan Houdek 520dfd7edf Adds Long Divide removal pass
If the long divides don't have anything in the upper bits then they can be optimized away.
Teeworlds, SuperTuxKart, and RRootage were between 50%-55% getting removed
2021-04-24 23:10:59 -07:00
Stefanos Kornilios Mitsis Poiitidis 60671ee6cb Merge pull request #975 from Sonicadvance1/use_xxhash
Switch from fasthash64 to xxhash's XXH3
2021-04-23 09:49:06 +03:00
Stefanos Kornilios Mitsis Poiitidis 6635765ea9 Merge pull request #972 from Sonicadvance1/naked_mman
Naked mman usage removal
2021-04-22 09:47:41 +03:00
Stefanos Kornilios Mitsis Poiitidis 3e4ff23ab6 Merge pull request #977 from Sonicadvance1/fix_relative_elf
Fixes ELFCodeLoader2 loading relative ELF files
2021-04-22 09:40:08 +03:00
Ryan Houdek 93ed411899 Fixes ELFCodeLoader2 loading relative ELF files 2021-04-20 01:58:37 -07:00
Ryan Houdek 0b0db08ebf Switch from fasthash64 to xxhash's XXH3
Fixes #797
2021-04-18 18:13:27 -07:00
Stefanos Kornilios Mitsis Poiitidis 3419f40c00 Merge pull request #971 from Sonicadvance1/template_specialization
Declare missing template specialization for GetListIfExists
2021-04-19 00:03:25 +03:00
Stefanos Kornilios Mitsis Poiitidis 8260ddda13 Merge pull request #970 from Sonicadvance1/sdl2_target_imports
Use cmake SDL2 target properties
2021-04-19 00:02:59 +03:00
Stefanos Kornilios Mitsis Poiitidis 872e49f02a Merge pull request #969 from Sonicadvance1/mcpu_native
Use mcpu=native with Clang 12
2021-04-19 00:02:22 +03:00
Stefanos Kornilios Mitsis Poiitidis f0f6bc7a73 Merge pull request #973 from Sonicadvance1/fix_openat
Fixes OpenAt breaking with anonymous objects
2021-04-17 15:51:24 +03:00
Ryan Houdek 5820c251e8 Fixes OpenAt breaking with anonymous objects
Fixes #753
2021-04-16 21:21:08 -07:00
Ryan Houdek eff97509c0 Replaces naked mman usage in TestHarnessRunner 2021-04-16 21:01:59 -07:00
Ryan Houdek 3128d0c148 Replaces naked mman usage in SyscallHandler 2021-04-16 21:01:39 -07:00
Ryan Houdek 0216d7b552 Replaces naked mman usage in x86-64 syscalls 2021-04-16 21:01:07 -07:00
Ryan Houdek 87b473c7ee Replaces naked mman usage in IRLoader 2021-04-16 21:00:51 -07:00
Ryan Houdek f9d647c852 Replaces naked mman usage in HarnessHelpers 2021-04-16 21:00:19 -07:00
Ryan Houdek 2c175f1e0b Replaces naked mman usage in FEXLoader 2021-04-16 21:00:01 -07:00
Ryan Houdek 96813b0d6c Replaces naked mman usage in ELFCodeLoader 2021-04-16 20:59:27 -07:00
Ryan Houdek 967ac863be Replaces naked mman usage in X86Dispatcher 2021-04-16 20:50:14 -07:00
Ryan Houdek fa546fb492 Replaces naked mman usage in ELFSymbolDatabase 2021-04-16 20:50:09 -07:00
Ryan Houdek 99ab9864aa Replaces naked mman usage in X86HelperGen 2021-04-16 20:50:03 -07:00
Ryan Houdek 11246355e2 Replaces naked mman usage in LookupCache 2021-04-16 20:49:58 -07:00
Ryan Houdek 0d236940e0 Replaces naked mman usage in x86-64 JIT 2021-04-16 20:49:52 -07:00
Ryan Houdek a0afb4be5a Replaces naked mman usage in Arm64 JIT 2021-04-16 20:49:47 -07:00
Ryan Houdek ca45485d48 Adds new Allocator namespace for replace mmap and munmap
This is necessary for overriding these functions
2021-04-16 20:49:35 -07:00
Ryan Houdek c8bb0e2d51 Declare missing template specialization for GetListIfExists
Clang 12 picks up that this was missing
2021-04-16 18:54:16 -07:00
Ryan Houdek e9aeebb3a1 Use cmake SDL2 target properties
Some distros include generated cmake files instead of the two cmake files from the source.
Distros that use the generated cmake files only have SDL import properties, so we are required
to use those.
2021-04-16 18:45:06 -07:00
Ryan Houdek 1ad46c5af1 Use mcpu=native with Clang 12
Clang 12 fixes the bug with big.little configurations so we
can keep using mcpu=native past this point
2021-04-16 18:44:34 -07:00
Ryan Houdek 08bc7a8364 Merge pull request #967 from FEX-Emu/skmp/new-elf-loading
ELF: Simpler loader, uses mmap
2021-04-16 18:35:56 -07:00
Ryan Houdek e7179ff84b Merge pull request #968 from FEX-Emu/skmp/fix-self-exe-after-fork
Emulated Files: Don't precalculate /proc/pid path, it changes after fork
2021-04-15 13:01:41 -07:00
Stefanos Kornilios Mitsis Poiitidis 13bfdcad06 Emulated Files: Don't precalculate /proc/pid path, it changes after fork 2021-04-15 15:33:10 +03:00
Stefanos Kornilios Mitsis Poiitidis 38a707b38e Add missing functions to elfcodeloader2 2021-04-15 13:51:22 +03:00
Stefanos Kornilios Mitsis Poiitidis ada7915b01 Convert an assert to log 2021-04-15 13:40:28 +03:00
Stefanos Kornilios Mitsis Poiitidis 9775a2148e Fix rootfs interpreter loading 2021-04-15 12:16:33 +03:00
Stefanos Kornilios Mitsis Poiitidis bc8696828b More logging 2021-04-15 12:14:55 +03:00
Stefanos Kornilios Mitsis Poiitidis 7d792e65f0 Rootfs, assert fixes 2021-04-15 12:04:42 +03:00
Stefanos Kornilios Mitsis Poiitidis 14b5b881c5 Add more logging 2021-04-15 11:47:42 +03:00
Stefanos Kornilios Mitsis Poiitidis 830e41459c Fix build for test harness, irloader 2021-04-15 11:05:07 +03:00
Stefanos Kornilios Mitsis Poiitidis 0db99ea4f6 Use allocator for x32 2021-04-15 10:51:16 +03:00
Stefanos Kornilios Mitsis Poiitidis 22f1cf0e2a More wip 2021-04-15 10:05:59 +03:00
Stefanos Kornilios Mitsis Poiitidis 80493487db ELF Loading: Cleanups all around 2021-04-14 10:59:23 +03:00
Stefanos Kornilios Mitsis Poiitidis 59ce750af6 Elf: Add ELFCodeLoader2, with simplified loading logic 2021-04-13 15:59:36 +03:00
Stefanos Kornilios Mitsis Poiitidis 4ac81e22de Move some more files around 2021-04-13 15:59:36 +03:00
Stefanos Kornilios Mitsis Poiitidis a0a382cc5c Clear up file naming a big 2021-04-13 15:59:36 +03:00
Ryan Houdek 6cc7d065aa Merge pull request #964 from Sonicadvance1/paranoid_tso
First step in paranoid TSO mode
2021-04-12 23:34:54 -07:00
Ryan Houdek 93b9ca4bdb Merge pull request #965 from Sonicadvance1/add_jemalloc
Switches FEX over to using jemalloc
2021-04-08 14:04:12 -07:00
Ryan Houdek cc0c45cf47 Switches FEX over to using jemalloc
This is the first step that we need to do for correct 32-bit memory allocations.
2021-04-08 12:42:37 -07:00
Ryan Houdek 4a504bdd00 Merge pull request #939 from Sonicadvance1/hidden_visibility
Default hidden visibility and strip symbols
2021-04-06 18:56:28 -07:00
Ryan Houdek 66c8dd2013 First step in paranoid TSO mode
This moves vector ops down the GPR atomic backpatch path.
Next step after this is to add SIGBUS based handlers for these rather than backpatching.
Needs more work to treat aligned versus unaligned vector loadstores differently
2021-04-06 18:48:36 -07:00
Ryan Houdek 91ac4ccdd8 Adds LDAXP and STLXP backpatch handlers
This will be used in the next commit
2021-04-06 18:47:50 -07:00
Ryan Houdek 849b83bd02 Adds new ParanoidTSO option 2021-04-06 18:46:52 -07:00
Ryan Houdek 286f103dab Adds visibility attributes to everything we need to expose 2021-04-06 01:01:31 -07:00
Ryan Houdek 2a49037789 update external vixl 2021-04-06 00:39:03 -07:00
Ryan Houdek 235f6fbe39 Strip symbols and dead sections
These are unnecessary and just add some file bloat. Strip them out
2021-04-06 00:39:03 -07:00
Ryan Houdek b43db294ec Change our compilation visibility to hidden
This bloats our compilation with symbols that don't need to be visible
2021-04-06 00:39:03 -07:00
Stefanos Kornilios Mitsis Poiitidis 5e6c6adf8b Merge pull request #956 from Sonicadvance1/enable_gcc_32_tests
Enables GCC 32-bit unit tests
2021-04-06 09:48:38 +03:00
Stefanos Kornilios Mitsis Poiitidis 50c3ee047c Merge pull request #961 from Sonicadvance1/fix_segment_override
Fixes segment register override on string instructions.
2021-04-06 09:24:23 +03:00
Stefanos Kornilios Mitsis Poiitidis c0ef6da6a8 Merge pull request #960 from Sonicadvance1/more_cpuid
Adds some more CPUID functions
2021-04-06 09:23:13 +03:00
Stefanos Kornilios Mitsis Poiitidis 2063411443 Merge pull request #959 from Sonicadvance1/fix_tag_order
Fixes ordering problem of tag generation in release script
2021-04-06 09:22:12 +03:00
Stefanos Kornilios Mitsis Poiitidis 4a5eefec25 Merge pull request #958 from Sonicadvance1/fix_atomic_zext
Fixes Zext semantics on atomic ops with x64 JIT
2021-04-06 09:21:44 +03:00
Stefanos Kornilios Mitsis Poiitidis eaad9e99a9 Merge pull request #954 from Sonicadvance1/posix_tests_docs
Updates a couple POSIX test case failures
2021-04-06 09:18:18 +03:00
Stefanos Kornilios Mitsis Poiitidis 62406a45be Merge pull request #953 from Sonicadvance1/env_var_override
Allows overriding FEX folder locations with environment variables
2021-04-06 09:16:12 +03:00
Stefanos Kornilios Mitsis Poiitidis 03537d351d Merge pull request #952 from Sonicadvance1/add_asan_use_after_scope
Adds use-after-scope to the ASAN options
2021-04-06 09:09:09 +03:00
Ryan Houdek 0cb53fff89 Change kernel version error to not be fatal 2021-04-05 23:09:06 -07:00
Ryan Houdek 3e250fa071 Enables GCC 32-bit unit tests
Relies on #950 getting merged first
2021-04-05 23:09:06 -07:00
Stefanos Kornilios Mitsis Poiitidis 1d4b81f556 Merge pull request #951 from Sonicadvance1/fix_getenv_aotir
Removes getenv usage for aotir
2021-04-06 09:08:46 +03:00
Stefanos Kornilios Mitsis Poiitidis c26f19ed4d Merge pull request #950 from Sonicadvance1/fix_gcc_tests
Fixes GCC tests
2021-04-06 09:05:45 +03:00
Stefanos Kornilios Mitsis Poiitidis 9619cea097 Merge pull request #949 from Sonicadvance1/arm64_print
Adds support for Arm64 JIT to Print
2021-04-06 08:58:13 +03:00
Stefanos Kornilios Mitsis Poiitidis 44dbecc2ac Merge pull request #948 from Sonicadvance1/fix_leak
Fixes memory leak in CPU backends
2021-04-06 08:56:00 +03:00
Stefanos Kornilios Mitsis Poiitidis 415632209f Merge pull request #947 from Sonicadvance1/minimal_siginfo
Implements a minimal siginfo_t in the dispatcher's signal handler
2021-04-06 08:54:01 +03:00
Ryan Houdek b7e81e867d Fixes segment register override on string instructions.
The segment prefix only overrides one side of the string instruction.
Passing in the flags was causing the instruction to have the segment override on both sides of the copy.
So something like `movsb es:[rdi], fs:[rsi]` was being interpreted as `movsb fs:[rdi], fs:[rsi]`
Same for all the string ops which is now fixed.

We can't unit test these currently since we can't set host segment register for confirmed host behaviour.
2021-04-05 21:28:31 -07:00
Ryan Houdek 4f276aae82 Adds comment for LEA and segment registers
LEA doesn't give you the result with segment register applied to it.
Adds a comment to OpDispatcher to make it more clear
2021-04-05 21:28:25 -07:00
Ryan Houdek c80254f6f4 Adds some more CPUID functions
Noticed a game poking at these
2021-04-05 18:37:05 -07:00
Ryan Houdek 8adb1ca3a2 Fixes ordering problem of tag generation in release script
Wasn't actually creating an annotated tag with this order
2021-04-05 18:35:09 -07:00
Ryan Houdek 0dd1959d65 Fixes Zext semantics on atomic ops with x64 JIT
Fixes Factorio 1.1.30 on x86-64 hosts

Fixes #957
2021-04-05 18:19:15 -07:00
Ryan Houdek f4ab7cceab Updates a couple POSIX test case failures
The behaviour of these is broken but now they are documented in how they are broken
2021-04-05 04:27:52 -07:00
Ryan Houdek 8cc799ab65 Allows overriding FEX folder locations with environment variables
Introduces three new environment variables that need to live outside of the scope
of the regular argument loader path.
This will allow external applications to adjust FEX parameters in interesting ways.

FEX_APP_CONFIG: Allows you to override where Config.json lives
FEX_APP_CONFIG_LOCATION: Allows you to provide an entire folder of app profiles
FEX_APP_DATA_LOCATION: Allows you to override where the data gets stored and loaded from

Fixes #572
2021-04-05 04:09:41 -07:00
Ryan Houdek acb329b8e7 Adds use-after-scope to the ASAN options
Useful for seeing stack scoping problems that can crop up
2021-04-04 22:07:40 -07:00
Ryan Houdek 3020e39428 Remove DestSize from Print IR Op 2021-04-04 22:07:03 -07:00
Ryan Houdek 4a094c5dee Reenables new gcc unit tests that are no longer failing
Documents the few remaining tests with details as to why the test fails.
2021-04-04 22:05:55 -07:00
Ryan Houdek f8eabb4f33 Implements 64bit scalar VUMin on x86 JIT 2021-04-04 22:05:55 -07:00
Ryan Houdek 4da492d361 Fixes vector integer min and max on AArch64
These instructions don't have a 64bit element variant.
Switch the 64bit element variant over to a coded variant similar to float min/max
2021-04-04 22:05:55 -07:00
Ryan Houdek a200d2c91c Fixes packed shift ops
In the case of a shift value larger than the element size then the shift will zero the element.
Incoming shift amount is always a 64bit value which means that the value can become quite large on shifts.
Checks the incoming source and sets the maximum to be the element size, which works for AArch64's vector shifts
Which only operate on 8bit incoming shift amounts
2021-04-04 22:05:55 -07:00
Ryan Houdek f1663abe81 Removes getenv usage for aotir
We can't rely on `getenv("HOME")` always working.
Have the AOTIR code use the helper method for getting the data directory instead.
2021-04-04 21:43:14 -07:00
Ryan Houdek af205e9f5f Fixes PEXTRW
In the case of 16bit PEXTRW it will zext to 32bits which wasn't being done
2021-04-04 21:32:12 -07:00
Ryan Houdek 032d1fbba2 Fixes VFMIN/VFMax NaN behaviour
In a world with NaNs we were hitting some undefined behaviour with fmin/fmax
Needed to change these over to something more complex to handle the NaN case.

ARMv8.7 alternative FP mode would solve this for the scalar case, but it still doesn't work for the vecto case sadly.
2021-04-04 21:29:46 -07:00
Ryan Houdek 313c4bb784 Fixes bug in VectorImm
If passing in a 64bit immediate then movi doesn't behaviour how you would expect.

Instead to a tmp move + dup to implement it the way we want
2021-04-04 20:24:23 -07:00
Ryan Houdek ceb484fd3a Adds support for Arm64 JIT to Print
Useful for debugging, supports both vector and GPR
2021-04-04 20:22:44 -07:00
Ryan Houdek b98c7bd853 Moves AArch64's GetPhys to be a class function
No need for it to be static
2021-04-04 20:21:06 -07:00
Ryan Houdek 4c045148b6 Fixes memory leak in CPU backends
Dispatcher wasn't ever getting cleaned up.
Also ensures sharing it with the CompileService otherwise it crashes on use.
2021-04-04 20:19:35 -07:00
Ryan Houdek 9f29bba4aa Fixes missing virtual destructor on Dispatcher class 2021-04-04 20:10:58 -07:00
Ryan Houdek 2c592bd8a8 Implements a minimal siginfo_t in the dispatcher's signal handler
This lets Carrion get farther in game.
Seems like one of its threads hits sigsegv and it just...lets it die after checking siginfo?
2021-04-03 17:03:19 -07:00
Ryan Houdek 8319a85524 Merge pull request #946 from Sonicadvance1/fix_eventfd
Fixes eventfd and eventfd2
2021-04-03 06:52:32 -07:00
Ryan Houdek 8ad45071e4 Merge pull request #945 from Sonicadvance1/fix_truncate_creat
Fixes truncate and creat
2021-04-03 06:47:31 -07:00
Ryan Houdek 3379f3a8f9 Fixes eventfd and eventfd2
glibc function signature doesn't match kernel
2021-04-03 06:46:28 -07:00
Stefanos Kornilios Mitsis Poiitidis c76205040e Merge pull request #944 from Sonicadvance1/fix_clock_nanosleep
Fixes clock_nanosleep
2021-04-03 11:09:46 +03:00
Stefanos Kornilios Mitsis Poiitidis 648ef617cf Merge pull request #943 from Sonicadvance1/fix_fexconfig_unnamed_options
Fixes FEXConfig filling in unnamed options
2021-04-03 11:09:21 +03:00
Stefanos Kornilios Mitsis Poiitidis 5fac51f659 Merge pull request #942 from Sonicadvance1/implement_cmdline
Fixes wrapping cmdline arguments
2021-04-03 11:08:50 +03:00
Ryan Houdek 5d63dc179c Fixes truncate and creat
These were using the passthrough helper which doesn't was breaking the result value
2021-04-02 20:45:33 -07:00
Ryan Houdek 858566c356 Fixes clock_nanosleep
glibc version behaves slightly differently than kernel
2021-04-02 20:41:04 -07:00
Ryan Houdek fb94e62df1 Fixes FEXConfig filling in unnamed options
These aren't meant to be set by the user
2021-04-02 20:28:12 -07:00
Ryan Houdek 51e4fe7f0a Fixes wrapping cmdline arguments
In case an application checks this
2021-04-02 20:26:28 -07:00
Stefanos Kornilios Mitsis Poiitidis be03767c0b Merge pull request #941 from Sonicadvance1/pselect6_fix
Fixes pselect6 signal arg pack argument
2021-04-02 23:42:02 +03:00
Stefanos Kornilios Mitsis Poiitidis adeea56ea8 Merge pull request #940 from Sonicadvance1/update_release_script
Updates release script to match what I want it to be
2021-04-02 23:41:17 +03:00
Ryan Houdek 6923c47a70 Fixes pselect6 signal arg pack argument
This doesn't exactly match the glibc call so do the syscall directly
2021-04-02 13:23:07 -07:00
Ryan Houdek 26d9ddef9f Updates release script to match what I want it to be
Generates the current and previous tag automatically.
Checks to ensure the previous tag exists and that the current tag does not
2021-04-02 11:49:19 -07:00
353 changed files with 26539 additions and 9766 deletions

No files matched your search

+14 -1
View File
@@ -13,13 +13,14 @@ env:
BUILD_TYPE: Release
CC: clang
CXX: clang++
FEX_FORCE32BITALLOCATOR: 1
jobs:
build:
runs-on: ${{ matrix.arch }}
strategy:
matrix:
arch: [[self-hosted, x64], [self-hosted, ARMv8.0], [self-hosted, ARMv8.2]]
arch: [[self-hosted, x64], [self-hosted, ARMv8.0], [self-hosted, ARMv8.2], [self-hosted, ARMv8.4]]
fail-fast: false
steps:
@@ -116,6 +117,18 @@ jobs:
working-directory: ${{runner.workspace}}/build
run: mv ${{runner.workspace}}/build/Testing/Temporary/LastTest.log ${{runner.workspace}}/build/Testing/Temporary/LastTest_GCC64.log || true
- name: gcc target tests 32
working-directory: ${{runner.workspace}}/build
shell: bash
# Execute the gvisor tests
run: cmake --build . --config $BUILD_TYPE --target gcc_target_tests_32
- name: GCC32 Test Results move
if: ${{ always() }}
shell: bash
working-directory: ${{runner.workspace}}/build
run: mv ${{runner.workspace}}/build/Testing/Temporary/LastTest.log ${{runner.workspace}}/build/Testing/Temporary/LastTest_GCC32.log || true
- name: Struct verifier tests
working-directory: ${{runner.workspace}}/build
shell: bash
+12
View File
@@ -30,3 +30,15 @@
shallow = true
path = External/fex-gcc-target-tests-bins
url = https://github.com/FEX-Emu/fex-gcc-target-tests-bins.git
[submodule "External/jemalloc"]
path = External/jemalloc
url = https://github.com/FEX-Emu/jemalloc.git
[submodule "External/fmt"]
path = External/fmt
url = https://github.com/fmtlib/fmt.git
[submodule "External/drm-headers"]
path = External/drm-headers
url = https://github.com/FEX-Emu/drm-headers.git
[submodule "External/xxhash"]
path = External/xxhash
url = https://github.com/FEX-Emu/xxHash.git
+267 -43
View File
@@ -14,6 +14,8 @@ option(ENABLE_ASSERTIONS "Enables assertions in build" FALSE)
option(ENABLE_VISUAL_DEBUGGER "Enables the visual debugger for compiling" FALSE)
option(ENABLE_STRICT_WERROR "Enables stricter -Werror for CI" FALSE)
option(ENABLE_WERROR "Enables -Werror" FALSE)
option(ENABLE_STATIC_PIE "Enables static-pie build" FALSE)
option(ENABLE_JEMALLOC "Enables jemalloc allocator" TRUE)
set (X86_C_COMPILER "x86_64-linux-gnu-gcc" CACHE STRING "c compiler for compiling x86 guest libs")
set (X86_CXX_COMPILER "x86_64-linux-gnu-g++" CACHE STRING "c++ compiler for compiling x86 guest libs")
@@ -44,38 +46,6 @@ else()
set(CMAKE_INTERPROCEDURAL_OPTIMIZATION FALSE)
endif()
find_program(CCACHE_PROGRAM ccache)
if(CCACHE_PROGRAM)
message(STATUS "CCache enabled")
set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE "${CCACHE_PROGRAM}")
endif()
if (ENABLE_XRAY)
add_compile_options(-fxray-instrument)
link_libraries(-fxray-instrument)
endif()
if (ENABLE_LLD)
link_libraries(-fuse-ld=lld)
endif()
if (ENABLE_ASAN)
add_definitions(-DENABLE_ASAN=1)
add_compile_options(-fno-omit-frame-pointer -fsanitize=address)
link_libraries(-fno-omit-frame-pointer -fsanitize=address)
endif()
if (ENABLE_TSAN)
add_compile_options(-fno-omit-frame-pointer -fsanitize=thread)
link_libraries(-fno-omit-frame-pointer -fsanitize=thread)
endif()
set (CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} -fno-omit-frame-pointer")
set (CMAKE_LINKER_FLAGS_RELWITHDEBINFO "${CMAKE_LINKER_FLAGS_RELWITHDEBINFO} -fno-omit-frame-pointer")
set (CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -fomit-frame-pointer")
set (CMAKE_LINKER_FLAGS_RELEASE "${CMAKE_LINKER_FLAGS_RELEASE} -fomit-frame-pointer")
if (CMAKE_SYSTEM_PROCESSOR MATCHES "x86_64")
option(ENABLE_X86_HOST_DEBUG "Enables compiling on x86_64 host" FALSE)
if (NOT ENABLE_X86_HOST_DEBUG)
@@ -95,6 +65,174 @@ if (CMAKE_SYSTEM_PROCESSOR MATCHES "aarch64")
add_definitions(-D_M_ARM_64=1)
endif()
find_program(CCACHE_PROGRAM ccache)
if(CCACHE_PROGRAM)
message(STATUS "CCache enabled")
set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE "${CCACHE_PROGRAM}")
endif()
if (ENABLE_XRAY)
add_compile_options(-fxray-instrument)
link_libraries(-fxray-instrument)
endif()
set (PTHREAD_LIB pthread)
if (ENABLE_LLD)
set (LD_OVERRIDE "-fuse-ld=lld")
link_libraries(${LD_OVERRIDE})
endif()
if (ENABLE_STATIC_PIE)
if (_M_ARM_64 AND ENABLE_LLD)
message (FATAL_ERROR "Static linking does not currently work with AArch64+LLD. Use GNU ld for now.")
endif()
file(WRITE ${PROJECT_BINARY_DIR}/CMakeFiles/CMakeTmp/Determine_iplt.c
"int main(int argc, char* argv[])
{
return 0;
}")
# Compile the test application with our LD_OVERRIDE and static-pie options
try_compile(
COMPILE_RESULT
${PROJECT_BINARY_DIR}/CMakeFiles/CMakeTmp
${PROJECT_BINARY_DIR}/CMakeFiles/CMakeTmp/Determine_iplt.c
COMPILE_DEFINITIONS "-fPIE ${LD_OVERRIDE}"
LINK_LIBRARIES "-static-pie ${LD_OVERRIDE}"
COPY_FILE ${PROJECT_BINARY_DIR}/CMakeFiles/CMakeTmp/Determine_iplt
)
if (${COMPILE_RESULT})
# Read the symbols from the elf
execute_process(COMMAND
readelf -s ${PROJECT_BINARY_DIR}/CMakeFiles/CMakeTmp/Determine_iplt
OUTPUT_FILE ${PROJECT_BINARY_DIR}/CMakeFiles/CMakeTmp/plt_out.txt
OUTPUT_VARIABLE PLT_SYMBOLS)
# Pull out the __rela_iplt_{start,end} symbols if they exist
execute_process(COMMAND
"grep" "__rela_iplt" ${PROJECT_BINARY_DIR}/CMakeFiles/CMakeTmp/plt_out.txt
OUTPUT_VARIABLE PLT_SYMBOLS)
set (SYMBOLS_FINE TRUE)
set (HAS_IPLT -1)
# Check if we have any symbols in our grep output
# The symbols must either not exist at all OR the symbols are zero
if (PLT_SYMBOLS)
string(FIND ${PLT_SYMBOLS} "__rela_iplt_start" HAS_IPLT)
endif()
if (NOT HAS_IPLT EQUAL -1)
# We have some symbols from readelf. Let's parse the results to check if they are zero
# Format: '35: 0000000000000000 0 NOTYPE LOCAL HIDDEN UND __rela_iplt_start'
string(REPLACE "\n" ";" SYMBOL_LIST ${PLT_SYMBOLS})
foreach (SYMBOL ${SYMBOL_LIST})
# strip any leading and trailing whitespace
string (STRIP ${SYMBOL} SYMBOL)
# Convert string to a list
string(REPLACE " " ";" SYMBOL_VALUES ${SYMBOL}})
# Pull out the address argument
list(GET SYMBOL_VALUES 1 OFFSET)
# Check against integer zero
if (NOT ${OFFSET} EQUAL 0)
# Symbol wasn't zero, this now fails
set (SYMBOLS_FINE FALSE)
endif()
endforeach()
endif()
if (SYMBOLS_FINE)
# We can now exnable static-pie
set (STATIC_PIE_OPTIONS "-static-pie")
# Pthreads has an issue with exposing symbols
# We need to make some concessions to the pthread gods
if (ENABLE_LLD)
set (PTHREAD_LIB
-Wl,--undefined-glob=pthread_*
-Wl,--undefined=__cxa_finalize
-Wl,--undefined=_pthread_cleanup_push_defer
-Wl,--undefined=_pthread_cleanup_pop_restore
-Wl,--undefined=__pthread_cleanup_upto
pthread)
else()
set (PTHREAD_LIB
-Wl,--undefined=pthread_join
-Wl,--undefined=pthread_attr_getdetachstate
-Wl,--undefined=pthread_sigmask
-Wl,--undefined=pthread_mutex_lock
-Wl,--undefined=pthread_cond_init
-Wl,--undefined=pthread_attr_init
-Wl,--undefined=pthread_mutex_unlock
-Wl,--undefined=pthread_mutexattr_destroy
-Wl,--undefined=pthread_detach
-Wl,--undefined=pthread_mutex_init
-Wl,--undefined=pthread_getattr_np
-Wl,--undefined=pthread_cond_timedwait
-Wl,--undefined=pthread_attr_destroy
-Wl,--undefined=pthread_mutexattr_settype
-Wl,--undefined=pthread_rwlock_unlock
-Wl,--undefined=pthread_rwlock_wrlock
-Wl,--undefined=pthread_setspecific
-Wl,--undefined=pthread_create
-Wl,--undefined=pthread_cond_clockwait
-Wl,--undefined=pthread_key_create
-Wl,--undefined=pthread_rwlock_rdlock
-Wl,--undefined=pthread_setname_np
-Wl,--undefined=pthread_cond_signal
-Wl,--undefined=pthread_mutexattr_init
-Wl,--undefined=pthread_attr_setstack
-Wl,--undefined=pthread_self
-Wl,--undefined=pthread_getaffinity_np
-Wl,--undefined=pthread_cond_wait
-Wl,--undefined=pthread_mutex_trylock
-Wl,--undefined=pthread_cond_broadcast
-Wl,--undefined=pthread_cond_destroy
-Wl,--undefined=pthread_getspecific
-Wl,--undefined=pthread_key_delete
-Wl,--undefined=pthread_once
-Wl,--undefined=__cxa_finalize
-Wl,--undefined=_pthread_cleanup_push_defer
-Wl,--undefined=_pthread_cleanup_pop_restore
-Wl,--undefined=__pthread_cleanup_upto
pthread)
endif()
else()
message (FATAL_ERROR "Application has __rela_iplt_{start,end} symbols. Which means static-pie can't be enabled")
endif()
else()
message (FATAL_ERROR "Couldn't compile static-pie test. Static-pie can't be enabled!")
endif()
endif()
if (ENABLE_ASAN)
add_definitions(-DENABLE_ASAN=1)
add_compile_options(-fno-omit-frame-pointer -fsanitize=address -fsanitize-address-use-after-scope)
link_libraries(-fno-omit-frame-pointer -fsanitize=address -fsanitize-address-use-after-scope)
endif()
if (ENABLE_TSAN)
add_compile_options(-fno-omit-frame-pointer -fsanitize=thread)
link_libraries(-fno-omit-frame-pointer -fsanitize=thread)
endif()
if (ENABLE_JEMALLOC)
add_definitions(-DENABLE_JEMALLOC=1)
else()
message (STATUS
" jemalloc disabled!\n"
" This is not a recommended configuration!\n"
" This will very explicitly break 32-bit application execution!\n"
" Use at your own risk!")
endif()
set (CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} -fno-omit-frame-pointer")
set (CMAKE_LINKER_FLAGS_RELWITHDEBINFO "${CMAKE_LINKER_FLAGS_RELWITHDEBINFO} -fno-omit-frame-pointer")
set (CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -fomit-frame-pointer")
set (CMAKE_LINKER_FLAGS_RELEASE "${CMAKE_LINKER_FLAGS_RELEASE} -fomit-frame-pointer")
add_subdirectory(External/vixl/)
include_directories(External/vixl/src/)
@@ -103,14 +241,27 @@ if (CMAKE_CXX_COMPILER_ID STREQUAL "GNU")
message(FATAL_ERROR "FEX doesn't support getting compiled with GCC!")
endif()
find_package(PkgConfig REQUIRED)
find_package(Python 3.0 REQUIRED COMPONENTS Interpreter)
pkg_check_modules(XXHASH libxxhash>=0.8.0 QUIET)
if (NOT XXHASH_FOUND)
message(STATUS "xxHash not found. Using Externals")
add_subdirectory(External/xxhash/)
include_directories(External/xxhash/)
endif()
add_definitions(-Wno-trigraphs)
add_definitions(-DGLOBAL_DATA_DIRECTORY="${DATA_DIRECTORY}/")
add_subdirectory(External/jemalloc/)
include_directories(External/jemalloc/pregen/include/)
add_subdirectory(External/cpp-optparse/)
include_directories(External/cpp-optparse/)
add_subdirectory(External/fmt/)
add_subdirectory(External/imgui/)
include_directories(External/imgui/)
@@ -158,18 +309,27 @@ if(ENABLE_WERROR OR ENABLE_STRICT_WERROR)
endif()
if(_M_ARM_64)
# Due to an oversight in llvm, it declares any reasonably new Kryo CPU to only be ARMv8.0
# Manually detect newer CPU revisions until clang and llvm fixes their bug
# This script will either provide a supported CPU or 'native'
# Additionally -march doesn't work under AArch64+Clang, so you have to use -mcpu or -mtune
execute_process(COMMAND python3 "${PROJECT_SOURCE_DIR}/Scripts/aarch64_fit_native.py" "/proc/cpuinfo"
OUTPUT_VARIABLE AARCH64_CPU)
if (CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 999999.0)
# Clang 12.0 fixed the -mcpu=native bug with mixed big.little implementers
# Clang can not currently check for native Apple M1 type in hypervisor. Currently disabled
check_cxx_compiler_flag("-mcpu=native" COMPILER_SUPPORTS_CPU_TYPE)
if(COMPILER_SUPPORTS_CPU_TYPE)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -mcpu=native")
endif()
else()
# Due to an oversight in llvm, it declares any reasonably new Kryo CPU to only be ARMv8.0
# Manually detect newer CPU revisions until clang and llvm fixes their bug
# This script will either provide a supported CPU or 'native'
# Additionally -march doesn't work under AArch64+Clang, so you have to use -mcpu or -mtune
execute_process(COMMAND python3 "${PROJECT_SOURCE_DIR}/Scripts/aarch64_fit_native.py" "/proc/cpuinfo" "${CMAKE_CXX_COMPILER_VERSION}"
OUTPUT_VARIABLE AARCH64_CPU)
string(STRIP ${AARCH64_CPU} AARCH64_CPU)
string(STRIP ${AARCH64_CPU} AARCH64_CPU)
check_cxx_compiler_flag("-mcpu=${AARCH64_CPU}" COMPILER_SUPPORTS_CPU_TYPE)
if(COMPILER_SUPPORTS_CPU_TYPE)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -mcpu=${AARCH64_CPU}")
check_cxx_compiler_flag("-mcpu=${AARCH64_CPU}" COMPILER_SUPPORTS_CPU_TYPE)
if(COMPILER_SUPPORTS_CPU_TYPE)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -mcpu=${AARCH64_CPU}")
endif()
endif()
endif()
@@ -236,7 +396,7 @@ add_compile_options(-Wall)
configure_file(
${CMAKE_CURRENT_SOURCE_DIR}/include/Config.h.in
${CMAKE_BINARY_DIR}/generated/Config.h)
${CMAKE_BINARY_DIR}/generated/ConfigDefines.h)
if (BUILD_TESTS)
include(CTest)
@@ -245,6 +405,9 @@ if (BUILD_TESTS)
endif()
add_subdirectory(External/FEXCore)
# Binfmt_misc files must be installed prior to Source/ installs
add_subdirectory(Data/binfmts/)
add_subdirectory(Source/)
add_subdirectory(Data/AppConfig/)
@@ -291,3 +454,64 @@ if (BUILD_THUNKS)
DEPENDS guest-libs
)
endif()
set(FEX_VERSION_MAJOR "0")
set(FEX_VERSION_MINOR "0")
set(FEX_VERSION_PATCH "0")
find_package(Git)
if (GIT_FOUND)
execute_process(
COMMAND ${GIT_EXECUTABLE} describe --abbrev=0
WORKING_DIRECTORY "${CMAKE_SOURCE_DIR}"
OUTPUT_VARIABLE GIT_DESCRIBE_STRING
RESULT_VARIABLE GIT_ERROR
ERROR_QUIET
OUTPUT_STRIP_TRAILING_WHITESPACE
)
if (NOT ${GIT_ERROR} EQUAL 0)
# Likely built in a way that doesn't have tags
# Setup a version tag that is unknown
set(GIT_DESCRIBE_STRING "FEX-0000")
endif()
# Change something like `FEX-2106.1-76-<hash>` in to a list
string(REPLACE "-" ";" DESCRIBE_LIST ${GIT_DESCRIBE_STRING})
# Extract the `2106.1` element
list(GET DESCRIBE_LIST 1 DESCRIBE_LIST)
# Change `2106.1` in to a list
string(REPLACE "." ";" DESCRIBE_LIST ${DESCRIBE_LIST})
# Calculate list size
list(LENGTH DESCRIBE_LIST LIST_SIZE)
# Pull out the major version
list(GET DESCRIBE_LIST 0 FEX_VERSION_MAJOR)
# Minor version only exists if there is a .1 at the end
# eg: 2106 versus 2106.1
if (LIST_SIZE GREATER 1)
list(GET DESCRIBE_LIST 1 FEX_VERSION_MINOR)
endif()
endif()
# Package creation
set (CPACK_GENERATOR "DEB")
set (CPACK_PACKAGE_NAME fex-emu)
set (CPACK_PACKAGE_CONTACT "team@fex-emu.org")
set (CPACK_PACKAGE_VERSION_MAJOR "${FEX_VERSION_MAJOR}")
set (CPACK_PACKAGE_VERSION_MINOR "${FEX_VERSION_MINOR}")
set (CPACK_PACKAGE_VERSION_PATCH "${FEX_VERSION_PATCH}")
# Debian defines
set (CPACK_DEBIAN_PACKAGE_DEPENDS "libstdc++6")
set (CPACK_DEBIAN_PACKAGE_CONTROL_EXTRA "${CMAKE_CURRENT_SOURCE_DIR}/CPack/postinst;${CMAKE_CURRENT_SOURCE_DIR}/CPack/prerm")
if (CMAKE_SYSTEM_PROCESSOR MATCHES "aarch64")
# binfmt_misc conflicts with qemu-user-static
# We also only install binfmt_misc on aarch64 hosts
set (CPACK_DEBIAN_PACKAGE_CONFLICTS "qemu-user-static")
endif()
include (CPack)
Executable
+18
View File
@@ -0,0 +1,18 @@
#!/bin/sh
set -e
update_binfmt() {
# Check for update-binfmts
command -v update-binfmts >/dev/null || return 0
# Setup binfmt_misc
update-binfmts --import FEX-x86
update-binfmts --import FEX-x86_64
}
# Install FEXInterpreter hardlink
# Needs to be done before setting up binfmt_misc
ln -f /usr/bin/FEXLoader /usr/bin/FEXInterpreter
if [ $(uname -m) = 'aarch64' ]; then
update_binfmt
fi
Executable
+17
View File
@@ -0,0 +1,17 @@
#!/bin/sh
set -e
update_binfmt() {
# Check for update-binfmts
command -v update-binfmts >/dev/null || return 0
# Uninstall
update-binfmts --unimport FEX-x86
update-binfmts --unimport FEX-x86_64
}
if [ $(uname -m) = 'aarch64' ]; then
update_binfmt
fi
# Remove FEXInterpreter hardlink
unlink /usr/bin/FEXInterpreter
+5
View File
@@ -0,0 +1,5 @@
{
"Config": {
"StallProcess": "1"
}
}
+17
View File
@@ -0,0 +1,17 @@
function(GenBinFmt Name)
# Get the filename only component
get_filename_component(FMT_NAME ${Name} NAME_WE)
# Configure it
configure_file(
${Name}
${CMAKE_BINARY_DIR}/Data/binfmts/${FMT_NAME})
# Then install the configured binfmt
install(
FILES ${CMAKE_BINARY_DIR}/Data/binfmts/${FMT_NAME}
DESTINATION ${CMAKE_INSTALL_PREFIX}/share/binfmts/)
endfunction()
GenBinFmt(FEX-x86.in)
GenBinFmt(FEX-x86_64.in)
+8
View File
@@ -0,0 +1,8 @@
package fex
interpreter @CMAKE_INSTALL_PREFIX@/bin/FEXInterpreter
magic \x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x03\x00
offset 0
mask \xff\xff\xff\xff\xff\xfe\xfe\x00\x00\x00\x00\xff\xff\xff\xff\xff\xfe\xff\xff\xff
credentials yes
fix_binary yes
preserve yes
+8
View File
@@ -0,0 +1,8 @@
package fex
interpreter @CMAKE_INSTALL_PREFIX@/bin/FEXInterpreter
magic \x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x3e\x00
offset 0
mask \xff\xff\xff\xff\xff\xfe\xfe\x00\x00\x00\x00\xff\xff\xff\xff\xff\xfe\xff\xff\xff
credentials yes
fix_binary yes
preserve yes
+2 -2
View File
@@ -3,7 +3,7 @@ FROM ubuntu:20.04 as builder
RUN DEBIAN_FRONTEND="noninteractive" apt-get update
RUN DEBIAN_FRONTEND="noninteractive" apt install -y cmake \
clang-10 llvm-10 nasm ninja-build libnuma-dev \
clang-10 llvm-10 nasm ninja-build \
libcap-dev libglfw3-dev libepoxy-dev python3-dev \
python3 linux-headers-generic
@@ -23,7 +23,7 @@ FROM ubuntu:20.04
RUN DEBIAN_FRONTEND="noninteractive" apt-get update
RUN DEBIAN_FRONTEND="noninteractive" apt install -y \
libnuma-dev libcap-dev libglfw3-dev libepoxy-dev
libcap-dev libglfw3-dev libepoxy-dev
COPY --from=builder /opt/FEX/build/Bin/* /usr/bin/
+45 -4
View File
@@ -98,14 +98,19 @@ def print_man_option(short, long, desc, default):
output_man.write("\\fBdefault:\\fR {0}\n".format(default))
output_man.write(".Pp\n\n")
def print_man_env_option(name, desc, default):
output_man.write("\\fBFEX_{0}\\fR\n".format(name))
def print_man_env_option(name, desc, default, no_json_key):
output_man.write("\\fBFEX_{0}\\fR\n".format(name.upper()))
# Print description
for line in desc:
output_man.write(".Pp\n")
output_man.write("{0}\n".format(line))
if (not no_json_key):
output_man.write(".Pp\n")
output_man.write("\\fBJSON key:\\fR '{0}'\n".format(name))
output_man.write(".Pp\n\n")
output_man.write(".Pp\n")
output_man.write("\\fBdefault:\\fR {0}\n".format(default))
output_man.write(".Pp\n\n")
@@ -154,12 +159,48 @@ def print_man_environment(options):
# Wrap the string argument in quotes
default = "'" + default + "'"
print_man_env_option(
op_key.upper(),
op_key,
op_vals["Desc"],
default
default,
False
)
print_man_environment_tail()
output_man.write(".El\n")
def print_man_environment_tail():
# Additional environment variables that live outside of the normal loop
print_man_env_option(
"FEX_APP_CONFIG_LOCATION",
[
"Allows the user to override where FEX looks for configuration files",
"By default FEX will look in {$HOME, $XDG_CONFIG_HOME}/.fex-emu/",
"This will override the full path",
],
"''", True)
print_man_env_option(
"FEX_APP_CONFIG",
[
"Allows the user to override where FEX looks for only the application config file",
"By default FEX will look in {$HOME, $XDG_CONFIG_HOME}/.fex-emu/Config.json",
"This will override this file location",
"One must be careful with this option as it will override any applications that load with execve as well"
"If you need to support applications that execve then use FEX_APP_CONFIG_LOCATION instead"
],
"''", True)
print_man_env_option(
"FEX_APP_DATA_LOCATION",
[
"Allows the user to override where FEX looks for data files",
"By default FEX will look in {$HOME, $XDG_DATA_HOME}/.fex-emu/",
"This will override the full path",
"This is the folder where FEX stores generated files like IR cache"
],
"''", True)
def print_man_header():
header ='''.Dd {0}
.Dt FEX
+25 -19
View File
@@ -108,10 +108,10 @@ def print_ir_sizes(ops, defines):
output_file.write("[[maybe_unused]] static size_t GetSize(IROps Op) { return IRSizes[Op]; }\n\n")
output_file.write("std::string_view const& GetName(IROps Op);\n")
output_file.write("uint8_t GetArgs(IROps Op);\n")
output_file.write("FEXCore::IR::RegisterClassType GetRegClass(IROps Op);\n\n")
output_file.write("bool HasSideEffects(IROps Op);\n")
output_file.write("__attribute__((const)) __attribute__((visibility(\"default\"))) std::string_view const& GetName(IROps Op);\n")
output_file.write("__attribute__((const)) __attribute__((visibility(\"default\"))) uint8_t GetArgs(IROps Op);\n")
output_file.write("__attribute__((const)) __attribute__((visibility(\"default\"))) FEXCore::IR::RegisterClassType GetRegClass(IROps Op);\n\n")
output_file.write("__attribute__((const)) __attribute__((visibility(\"default\"))) bool HasSideEffects(IROps Op);\n")
output_file.write("#undef IROP_SIZES\n")
output_file.write("#endif\n\n")
@@ -277,7 +277,7 @@ def print_ir_allocator_helpers(ops, defines):
output_file.write("\tusing IRPair = Wrapper<T>;\n\n")
output_file.write("\tIRPair<IROp_Header> AllocateRawOp(size_t HeaderSize) {\n")
output_file.write("\t\tauto Op = reinterpret_cast<IROp_Header*>(Data.Allocate(HeaderSize));\n")
output_file.write("\t\tauto Op = reinterpret_cast<IROp_Header*>(DualListData.DataAllocate(HeaderSize));\n")
output_file.write("\t\tmemset(Op, 0, HeaderSize);\n")
output_file.write("\t\tOp->Op = IROps::OP_DUMMY;\n")
output_file.write("\t\treturn IRPair<IROp_Header>{Op, CreateNode(Op)};\n")
@@ -286,7 +286,7 @@ def print_ir_allocator_helpers(ops, defines):
output_file.write("\ttemplate<class T, IROps T2>\n")
output_file.write("\tT *AllocateOrphanOp() {\n")
output_file.write("\t\tsize_t Size = FEXCore::IR::GetSize(T2);\n")
output_file.write("\t\tauto Op = reinterpret_cast<T*>(Data.Allocate(Size));\n")
output_file.write("\t\tauto Op = reinterpret_cast<T*>(DualListData.DataAllocate(Size));\n")
output_file.write("\t\tmemset(Op, 0, Size);\n")
output_file.write("\t\tOp->Header.Op = T2;\n")
output_file.write("\t\treturn Op;\n")
@@ -295,25 +295,25 @@ def print_ir_allocator_helpers(ops, defines):
output_file.write("\ttemplate<class T, IROps T2>\n")
output_file.write("\tIRPair<T> AllocateOp() {\n")
output_file.write("\t\tsize_t Size = FEXCore::IR::GetSize(T2);\n")
output_file.write("\t\tauto Op = reinterpret_cast<T*>(Data.Allocate(Size));\n")
output_file.write("\t\tauto Op = reinterpret_cast<T*>(DualListData.DataAllocate(Size));\n")
output_file.write("\t\tmemset(Op, 0, Size);\n")
output_file.write("\t\tOp->Header.Op = T2;\n")
output_file.write("\t\treturn IRPair<T>{Op, CreateNode(&Op->Header)};\n")
output_file.write("\t}\n\n")
output_file.write("\tuint8_t GetOpSize(OrderedNode *Op) const {\n")
output_file.write("\t\tauto HeaderOp = Op->Header.Value.GetNode(Data.Begin());\n")
output_file.write("\t\tauto HeaderOp = Op->Header.Value.GetNode(DualListData.DataBegin());\n")
output_file.write("\t\treturn HeaderOp->Size;\n")
output_file.write("\t}\n\n")
output_file.write("\tuint8_t GetOpElements(OrderedNode *Op) const {\n")
output_file.write("\t\tauto HeaderOp = Op->Header.Value.GetNode(Data.Begin());\n")
output_file.write("\t\tLogMan::Throw::A(HeaderOp->HasDest, \"Op %s has no dest\\n\", GetName(HeaderOp->Op));\n")
output_file.write("\t\tauto HeaderOp = Op->Header.Value.GetNode(DualListData.DataBegin());\n")
output_file.write("\t\tLOGMAN_THROW_A(HeaderOp->HasDest, \"Op %s has no dest\\n\", GetName(HeaderOp->Op));\n")
output_file.write("\t\treturn HeaderOp->Size / HeaderOp->ElementSize;\n")
output_file.write("\t}\n\n")
output_file.write("\tbool OpHasDest(OrderedNode *Op) const {\n")
output_file.write("\t\tauto HeaderOp = Op->Header.Value.GetNode(Data.Begin());\n")
output_file.write("\t\tauto HeaderOp = Op->Header.Value.GetNode(DualListData.DataBegin());\n")
output_file.write("\t\treturn HeaderOp->HasDest;\n")
output_file.write("\t}\n\n")
@@ -387,11 +387,14 @@ def print_ir_allocator_helpers(ops, defines):
output_file.write(") {\n")
output_file.write("\t\tauto Op = AllocateOp<IROp_%s, IROps::OP_%s>();\n" % (op_key, op_key.upper()))
output_file.write("\t\tOp.first->Header.NumArgs = %d;\n" % (SSAArgs))
if (SSAArgs != 0):
output_file.write("\t\tauto ListDataBegin = DualListData.ListBegin();\n")
for i in range(0, SSAArgs):
output_file.write("\t\tOp.first->Header.Args[%d] = ssa%d->Wrapped(ListDataBegin);\n" % (i, i))
if (SSAArgs != 0):
for i in range(0, SSAArgs):
output_file.write("\t\tOp.first->Header.Args[%d] = ssa%d->Wrapped(ListData.Begin());\n" % (i, i))
output_file.write("\t\tssa%d->AddUse();\n" % (i))
if (HasArgs):
@@ -399,11 +402,6 @@ def print_ir_allocator_helpers(ops, defines):
data_name = op_vals["Args"][i]
output_file.write("\t\tOp.first->%s = %s;\n" % (data_name, data_name))
if (HasFixedDestSize):
output_file.write("\t\tOp.first->Header.Size = %d;\n" % FixedDestSize)
if (HasDestSize):
output_file.write("\t\tOp.first->Header.Size = %s;\n" % DestSize)
if (HasDest):
# We can only infer a size if we have arguments
if not (HasFixedDestSize or HasDestSize):
@@ -412,10 +410,18 @@ def print_ir_allocator_helpers(ops, defines):
if (SSAArgs != 0):
for i in range(0, SSAArgs):
output_file.write("\t\tuint8_t Size%d = GetOpSize(ssa%s);\n" % (i, i))
for i in range(0, SSAArgs):
output_file.write("\t\tInferSize = std::max(InferSize, Size%d);\n" % (i))
output_file.write("\t\tOp.first->Header.Size = InferSize;\n")
output_file.write("\t\tOp.first->Header.NumArgs = %d;\n" % (SSAArgs))
if (HasFixedDestSize):
output_file.write("\t\tOp.first->Header.Size = %d;\n" % FixedDestSize)
if (HasDestSize):
output_file.write("\t\tOp.first->Header.Size = %s;\n" % DestSize)
output_file.write("\t\tOp.first->Header.ElementSize = Op.first->Header.Size / (%s);\n" % NumElements)
if (HasDest):
@@ -499,7 +505,7 @@ def print_ir_parser_allocator_helpers(ops, defines):
if (SSAArgs != 0):
for i in range(0, SSAArgs):
output_file.write("\t\tOp.first->Header.Args[%d] = ssa%d->Wrapped(ListData.Begin());\n" % (i, i))
output_file.write("\t\tOp.first->Header.Args[%d] = ssa%d->Wrapped(DualListData.ListBegin());\n" % (i, i))
output_file.write("\t\tssa%d->AddUse();\n" % (i))
if (HasArgs):
+28 -5
View File
@@ -81,6 +81,10 @@ set (SRCS
Interface/Core/Frontend.cpp
Interface/Core/GdbServer.cpp
Interface/Core/HostFeatures.cpp
Interface/Core/OpcodeDispatcher/Crypto.cpp
Interface/Core/OpcodeDispatcher/Flags.cpp
Interface/Core/OpcodeDispatcher/Vector.cpp
Interface/Core/OpcodeDispatcher/X87.cpp
Interface/Core/OpcodeDispatcher.cpp
Interface/Core/X86Tables.cpp
Interface/Core/X86DebugInfo.cpp
@@ -114,6 +118,7 @@ set (SRCS
Interface/IR/Passes/DeadContextStoreElimination.cpp
Interface/IR/Passes/IRCompaction.cpp
Interface/IR/Passes/IRValidation.cpp
Interface/IR/Passes/LongDivideRemovalPass.cpp
Interface/IR/Passes/ValueDominanceValidation.cpp
Interface/IR/Passes/PhiValidation.cpp
Interface/IR/Passes/RedundantFlagCalculationElimination.cpp
@@ -121,8 +126,8 @@ set (SRCS
Interface/IR/Passes/StaticRegisterAllocationPass.cpp
Interface/IR/Passes/RegisterAllocationPass.cpp
Interface/IR/Passes/SyscallOptimization.cpp
Utils/ELFLoader.cpp
Utils/ELFSymbolDatabase.cpp
Utils/Allocator.cpp
Utils/Allocator/64BitAllocator.cpp
Utils/LogManager.cpp
Utils/Threads.cpp
)
@@ -132,7 +137,7 @@ if(_M_ARM_64)
Interface/Core/ArchHelpers/Arm64.cpp)
endif()
set(DEFINES )
set(DEFINES -DTHREAD_LOCAL=_Thread_local)
if (_M_X86_64)
list(APPEND DEFINES -D_M_X86_64=1)
@@ -264,8 +269,11 @@ function(AddObject Name Type)
add_dependencies(${Name} IR_INC)
add_dependencies(${Name} CONFIG_INC)
target_link_libraries(${Name} pthread rt vixl ${LINUX_LIBS} dl)
target_link_libraries(${Name} vixl dl fmt::fmt xxhash)
set_target_properties(${Name} PROPERTIES OUTPUT_NAME FEXCore)
set_target_properties(${Name} PROPERTIES C_VISIBILITY_PRESET hidden)
set_target_properties(${Name} PROPERTIES CXX_VISIBILITY_PRESET hidden)
set_target_properties(${Name} PROPERTIES VISIBILITY_INLINES_HIDDEN TRUE)
target_include_directories(${Name} PUBLIC "${CMAKE_CURRENT_BINARY_DIR}")
@@ -280,9 +288,12 @@ function(AddObject Name Type)
target_compile_options(${Name}
PRIVATE
-Wall
-Werror=cast-qual
-Werror=ignored-qualifiers
-Werror=implicit-fallthrough
-Wno-trigraphs
-ffunction-sections
)
if (GCC_COLOR)
@@ -299,12 +310,24 @@ endfunction()
function(AddLibrary Name Type)
add_library(${Name} ${Type} $<TARGET_OBJECTS:${PROJECT_NAME}_object>)
target_link_libraries(${Name} pthread rt vixl ${LINUX_LIBS} dl)
target_link_libraries(${Name} vixl dl fmt::fmt xxhash)
set_target_properties(${Name} PROPERTIES OUTPUT_NAME FEXCore)
set_target_properties(${Name} PROPERTIES C_VISIBILITY_PRESET hidden)
set_target_properties(${Name} PROPERTIES CXX_VISIBILITY_PRESET hidden)
set_target_properties(${Name} PROPERTIES VISIBILITY_INLINES_HIDDEN TRUE)
target_include_directories(${Name} PUBLIC "${CMAKE_CURRENT_BINARY_DIR}")
target_include_directories(${Name} PUBLIC "${PROJECT_SOURCE_DIR}/include/")
target_include_directories(${Name} PUBLIC "${CMAKE_BINARY_DIR}/include/")
if (CMAKE_BUILD_TYPE MATCHES "RELEASE")
target_link_options(${Name}
PRIVATE
"LINKER:--gc-sections"
"LINKER:--strip-all"
"LINKER:--as-needed"
)
endif()
endfunction()
AddObject(${PROJECT_NAME}_object OBJECT)
+7 -6
View File
@@ -1,5 +1,6 @@
#pragma once
#include "Common/MathUtils.h"
#include <FEXCore/Utils/Allocator.h>
#include <FEXCore/Utils/LogManager.h>
#include <cstring>
@@ -16,16 +17,16 @@ struct BitSet final {
ElementType *Memory;
void Allocate(size_t Elements) {
size_t AllocateSize = AlignUp(Elements, MinimumSizeBits) / MinimumSize;
LogMan::Throw::A((AllocateSize * MinimumSize) >= Elements, "Fail");
Memory = static_cast<ElementType*>(malloc(AllocateSize));
LOGMAN_THROW_A((AllocateSize * MinimumSize) >= Elements, "Fail");
Memory = static_cast<ElementType*>(FEXCore::Allocator::malloc(AllocateSize));
}
void Realloc(size_t Elements) {
size_t AllocateSize = AlignUp(Elements, MinimumSizeBits) / MinimumSize;
LogMan::Throw::A((AllocateSize * MinimumSize) >= Elements, "Fail");
Memory = static_cast<ElementType*>(realloc(Memory, AllocateSize));
LOGMAN_THROW_A((AllocateSize * MinimumSize) >= Elements, "Fail");
Memory = static_cast<ElementType*>(FEXCore::Allocator::realloc(Memory, AllocateSize));
}
void Free() {
free(Memory);
FEXCore::Allocator::free(Memory);
Memory = nullptr;
}
bool Get(T Element) {
@@ -60,7 +61,7 @@ struct BitSetView final {
ElementType *Memory;
void GetView(BitSet<T> &Set, uint64_t ElementOffset) {
LogMan::Throw::A((ElementOffset % MinimumSize) == 0,
LOGMAN_THROW_A((ElementOffset % MinimumSize) == 0,
"Bitset view offset needs to be aligned to size of backing element");
Memory = &Set.Memory[ElementOffset / MinimumSizeBits];
}
+20 -11
View File
@@ -6,10 +6,13 @@
#include <sys/stat.h>
namespace FEXCore::Paths {
std::string CachePath;
std::string EntryCache;
std::unique_ptr<std::string> CachePath;
std::unique_ptr<std::string> EntryCache;
void InitializePaths() {
CachePath = std::make_unique<std::string>();
EntryCache = std::make_unique<std::string>();
char const *HomeDir = getenv("HOME");
if (!HomeDir) {
@@ -22,29 +25,35 @@ namespace FEXCore::Paths {
char *XDGDataDir = getenv("XDG_DATA_DIR");
if (XDGDataDir) {
CachePath = XDGDataDir;
*CachePath = XDGDataDir;
}
else {
if (HomeDir) {
CachePath = HomeDir;
*CachePath = HomeDir;
}
}
CachePath += "/.fex-emu/";
EntryCache = CachePath + "/EntryCache/";
*CachePath += "/.fex-emu/";
*EntryCache = *CachePath + "/EntryCache/";
std::error_code ec{};
// Ensure the folder structure is created for our Data
if (!std::filesystem::exists(EntryCache) &&
!std::filesystem::create_directories(EntryCache)) {
LogMan::Msg::D("Couldn't create EntryCache directory: '%s'", EntryCache.c_str());
if (!std::filesystem::exists(*EntryCache, ec) &&
!std::filesystem::create_directories(*EntryCache, ec)) {
LogMan::Msg::D("Couldn't create EntryCache directory: '%s'", EntryCache->c_str());
}
}
void ShutdownPaths() {
CachePath.reset();
EntryCache.reset();
}
std::string GetCachePath() {
return CachePath;
return *CachePath;
}
std::string GetEntryCachePath() {
return EntryCache;
return *EntryCache;
}
}
+1
View File
@@ -3,6 +3,7 @@
namespace FEXCore::Paths {
void InitializePaths();
void ShutdownPaths();
std::string GetCachePath();
std::string GetEntryCachePath();
}
+13 -11
View File
@@ -1,4 +1,6 @@
#pragma once
#include <FEXCore/Utils/BitUtils.h>
#include <FEXCore/Utils/LogManager.h>
#include <cmath>
@@ -158,18 +160,18 @@ struct X80SoftFloat {
}
operator float() const {
float32_t Result = extF80_to_f32(*this);
return *(float*)&Result;
const float32_t Result = extF80_to_f32(*this);
return FEXCore::BitCast<float>(Result);
}
operator double() const {
float64_t Result = extF80_to_f64(*this);
return *(double*)&Result;
const float64_t Result = extF80_to_f64(*this);
return FEXCore::BitCast<double>(Result);
}
operator BIGFLOAT() const {
float128_t Result = extF80_to_f128(*this);
return *(BIGFLOAT*)&Result;
const float128_t Result = extF80_to_f128(*this);
return FEXCore::BitCast<BIGFLOAT>(Result);
}
operator int16_t() const {
@@ -196,11 +198,11 @@ struct X80SoftFloat {
}
void operator=(const float rhs) {
*this = f32_to_extF80(*(float32_t*)&rhs);
*this = f32_to_extF80(FEXCore::BitCast<float32_t>(rhs));
}
void operator=(const double rhs) {
*this = f64_to_extF80(*(float64_t*)&rhs);
*this = f64_to_extF80(FEXCore::BitCast<float64_t>(rhs));
}
void operator=(const int16_t rhs) {
@@ -226,15 +228,15 @@ struct X80SoftFloat {
}
X80SoftFloat(const float rhs) {
*this = f32_to_extF80(*(float32_t*)&rhs);
*this = f32_to_extF80(FEXCore::BitCast<float32_t>(rhs));
}
X80SoftFloat(const double rhs) {
*this = f64_to_extF80(*(float64_t*)&rhs);
*this = f64_to_extF80(FEXCore::BitCast<float64_t>(rhs));
}
X80SoftFloat(BIGFLOAT rhs) {
*this = f128_to_extF80(*(float128_t*)&rhs);
*this = f128_to_extF80(FEXCore::BitCast<float128_t>(rhs));
}
X80SoftFloat(const int16_t rhs) {
+69 -16
View File
@@ -48,13 +48,20 @@ namespace FEXCore::Config {
else {
char const *HomeDir = GetHomeDirectory();
char const *ConfigXDG = getenv("XDG_CONFIG_HOME");
ConfigDir = ConfigXDG ? ConfigXDG : HomeDir;
ConfigDir += "/.fex-emu/";
char const *ConfigOverride = getenv("FEX_APP_CONFIG_LOCATION");
if (ConfigOverride) {
// Config override completely overrides the config directory
ConfigDir = ConfigOverride;
}
else {
ConfigDir = ConfigXDG ? ConfigXDG : HomeDir;
ConfigDir += "/.fex-emu/";
}
// Ensure the folder structure is created for our configuration
if (!std::filesystem::exists(ConfigDir) &&
!std::filesystem::create_directories(ConfigDir)) {
LogMan::Msg::D("Couldn't create config directory: '%s'", ConfigDir.c_str());
std::error_code ec{};
if (!std::filesystem::exists(ConfigDir, ec) &&
!std::filesystem::create_directories(ConfigDir, ec)) {
// Let's go local in this case
return "./";
}
@@ -64,21 +71,41 @@ namespace FEXCore::Config {
}
std::string GetConfigFileLocation() {
std::string ConfigFile = GetConfigDirectory(false) + "Config.json";
std::string ConfigFile{};
const char *AppConfig = getenv("FEX_APP_CONFIG");
if (AppConfig) {
// App config environment variable overwrites only the config file
ConfigFile = AppConfig;
}
else {
ConfigFile = GetConfigDirectory(false) + "Config.json";
}
return ConfigFile;
}
std::string GetApplicationConfig(std::string &Filename, bool Global) {
std::string GetApplicationConfig(const std::string &Filename, bool Global) {
std::string ConfigFile = GetConfigDirectory(Global);
std::error_code ec{};
if (!Global &&
!std::filesystem::exists(ConfigFile) &&
!std::filesystem::create_directories(ConfigFile)) {
!std::filesystem::exists(ConfigFile, ec) &&
!std::filesystem::create_directories(ConfigFile, ec)) {
LogMan::Msg::D("Couldn't create config directory: '%s'", ConfigFile.c_str());
// Let's go local in this case
return "./";
return "./" + Filename + ".json";
}
ConfigFile += "AppConfig/" + Filename + ".json";
ConfigFile += "AppConfig/";
// Attempt to create the local folder if it doesn't exist
if (!Global &&
!std::filesystem::exists(ConfigFile, ec) &&
!std::filesystem::create_directories(ConfigFile, ec)) {
// Let's go local in this case
return "./" + Filename + ".json";
}
ConfigFile += Filename + ".json";
return ConfigFile;
}
@@ -87,8 +114,15 @@ namespace FEXCore::Config {
char const *HomeDir = GetHomeDirectory();
char const *DataXDG = getenv("XDG_DATA_HOME");
DataDir = DataXDG ?: HomeDir;
DataDir += "/.fex-emu/";
char const *DataOverride = getenv("FEX_APP_DATA_LOCATION");
if (DataOverride) {
// Data override will override the complete directory
DataDir = DataOverride;
}
else {
DataDir = DataXDG ?: HomeDir;
DataDir += "/.fex-emu/";
}
return DataDir;
}
@@ -240,7 +274,8 @@ namespace FEXCore::Config {
Path = std::filesystem::absolute(Path);
// Only return if it exists
if (std::filesystem::exists(Path)) {
std::error_code ec{};
if (std::filesystem::exists(Path, ec)) {
return Path;
}
}
@@ -276,7 +311,8 @@ namespace FEXCore::Config {
else if (!PathName().empty()) {
// If the filesystem doesn't exist then let's see if it exists in the fex-emu folder
std::string NamedRootFS = GetDataDirectory() + "RootFS/" + PathName();
if (std::filesystem::exists(NamedRootFS)) {
std::error_code ec{};
if (std::filesystem::exists(NamedRootFS, ec)) {
FEXCore::Config::EraseSet(FEXCore::Config::CONFIG_ROOTFS, NamedRootFS);
}
}
@@ -291,7 +327,19 @@ namespace FEXCore::Config {
}
if (FEXCore::Config::Exists(FEXCore::Config::CONFIG_THUNKCONFIG)) {
FEX_CONFIG_OPT(PathName, THUNKCONFIG);
ExpandPathIfExists(FEXCore::Config::CONFIG_THUNKCONFIG, PathName());
auto ExpandedString = ExpandPath(PathName());
if (!ExpandedString.empty()) {
// Adjust the path if it ended up being relative
FEXCore::Config::EraseSet(FEXCore::Config::CONFIG_THUNKCONFIG, ExpandedString);
}
else if (!PathName().empty()) {
// If the filesystem doesn't exist then let's see if it exists in the fex-emu folder
std::string NamedConfig = GetDataDirectory() + "ThunkConfigs/" + PathName();
std::error_code ec{};
if (std::filesystem::exists(NamedConfig, ec)) {
FEXCore::Config::EraseSet(FEXCore::Config::CONFIG_THUNKCONFIG, NamedConfig);
}
}
}
if (FEXCore::Config::Exists(FEXCore::Config::CONFIG_OUTPUTLOG)) {
FEX_CONFIG_OPT(PathName, OUTPUTLOG);
@@ -326,6 +374,10 @@ namespace FEXCore::Config {
Meta->Set(Option, Data);
}
void Erase(ConfigOption Option) {
Meta->Erase(Option);
}
void EraseSet(ConfigOption Option, std::string Data) {
Meta->EraseSet(Option, Data);
}
@@ -389,5 +441,6 @@ namespace FEXCore::Config {
*List = **Value;
}
}
template void Value<std::string>::GetListIfExists(FEXCore::Config::ConfigOption Option, std::list<std::string> *List);
}
+44 -2
View File
@@ -77,7 +77,14 @@
"Default": "",
"ShortArg": "k",
"Desc": [
"A json file specifying where to overlay the thunks."
"A json file specifying where to overlay the thunks.",
"This can be a filesystem path",
"\teg: ~/MyThunkConfig.json",
"Or this can be a named of a Thunk config file",
"If the named config file exists in the FEX data folder folder the it will use that one",
"\teg: $HOME/.fex-emu/ThunkConfigs/<ThunkConfig name>",
"Or if you have XDG_DATA_HOME the config will search in that directory",
"\teg: $XDG_DATA_HOME/.fex-emu/ThunkConfigs/<ThunkConfig name>"
]
},
"Env": {
@@ -129,7 +136,18 @@
"Desc": [
"Disables optimizations passes for debugging."
]
},
"Force32BitAllocator": {
"Type": "bool",
"Default": "false",
"Desc": [
"Forces use of the 32-bit allocator on 32-bit applications",
"Used to work around ulimit problems of CI runner",
"Potentially useful for debugging memory problems",
"32-bit allocator is always used if your host kernel is older than 4.17"
]
}
},
"Logging": {
"SilentLog": {
@@ -142,7 +160,7 @@
},
"OutputLog": {
"Type": "str",
"Default": "stdout",
"Default": "stderr",
"ShortArg": "o",
"Desc": [
"File to write FEX output to.",
@@ -188,6 +206,22 @@
"Removes the calculation of the parity flag from GPR instructions.",
"Assuming no uses rely on it"
]
},
"ParanoidTSO": {
"Type": "bool",
"Default": "false",
"Desc": [
"Makes TSO operations even more strict.",
"Forces vector loadstores to also become atomic."
]
},
"StallProcess": {
"Type": "bool",
"Default": "false",
"Desc": [
"Forces a process to stall out on initialization",
"Useful for a process that keeps restarting and doesn't work"
]
}
},
"Misc": {
@@ -199,6 +233,14 @@
"Captures both the loaded executable and libraries it loads."
]
},
"AOTIRGenerate": {
"Type": "bool",
"Default": "false",
"Desc": [
"Scans file for executable code and generates an AOT IR cache.",
"Does not run the executable."
]
},
"AOTIRLoad": {
"Type": "bool",
"Default": "false",
+26 -11
View File
@@ -2,6 +2,7 @@
#include "Interface/Context/Context.h"
#include "Interface/Core/Core.h"
#include "Interface/Core/OpcodeDispatcher.h"
#include "Interface/Core/X86Tables/X86Tables.h"
#include <FEXCore/Config/Config.h>
#include <FEXCore/Core/CoreState.h>
@@ -14,6 +15,10 @@ namespace FEXCore::Context {
IR::InstallOpcodeHandlers(Mode);
}
void ShutdownStaticTables() {
FEXCore::Paths::ShutdownPaths();
}
FEXCore::Context::Context *CreateNewContext() {
return new FEXCore::Context::Context{};
}
@@ -33,12 +38,11 @@ namespace FEXCore::Context {
return CTX->InitCore(Loader);
}
void SetExitHandler(FEXCore::Context::Context *CTX,
std::function<void(uint64_t ThreadId, FEXCore::Context::ExitReason)> handler) {
CTX->CustomExitHandler = handler;
void SetExitHandler(FEXCore::Context::Context *CTX, ExitHandler handler) {
CTX->CustomExitHandler = std::move(handler);
}
std::function<void(uint64_t ThreadId, FEXCore::Context::ExitReason)> GetExitHandler(FEXCore::Context::Context *CTX) {
ExitHandler GetExitHandler(FEXCore::Context::Context *CTX) {
return CTX->CustomExitHandler;
}
@@ -50,6 +54,9 @@ namespace FEXCore::Context {
CTX->Step();
}
void CompileRIP(FEXCore::Core::InternalThreadState *Thread, uint64_t GuestRIP) {
Thread->CTX->CompileBlock(Thread->CurrentFrame, GuestRIP);
}
FEXCore::Context::ExitReason RunUntilExit(FEXCore::Context::Context *CTX) {
return CTX->RunUntilExit();
@@ -98,12 +105,12 @@ namespace FEXCore::Context {
CTX->HandleCallback(RIP);
}
void RegisterHostSignalHandler(FEXCore::Context::Context *CTX, int Signal, HostSignalDelegatorFunction Func) {
CTX->RegisterHostSignalHandler(Signal, Func);
void RegisterHostSignalHandler(FEXCore::Context::Context *CTX, int Signal, HostSignalDelegatorFunction Func, bool Required) {
CTX->RegisterHostSignalHandler(Signal, Func, Required);
}
void RegisterFrontendHostSignalHandler(FEXCore::Context::Context *CTX, int Signal, HostSignalDelegatorFunction Func) {
CTX->RegisterFrontendHostSignalHandler(Signal, Func);
void RegisterFrontendHostSignalHandler(FEXCore::Context::Context *CTX, int Signal, HostSignalDelegatorFunction Func, bool Required) {
CTX->RegisterFrontendHostSignalHandler(Signal, Func, Required);
}
FEXCore::Core::InternalThreadState* CreateThread(FEXCore::Context::Context *CTX, FEXCore::Core::CPUState *NewThreadState, uint64_t ParentTID) {
@@ -142,12 +149,20 @@ namespace FEXCore::Context {
return CTX->CPUID.RunFunction(Function, Leaf);
}
void SetAOTIRLoader(FEXCore::Context::Context *CTX, std::function<std::unique_ptr<std::istream>(const std::string&)> CacheReader) {
void SetAOTIRLoader(FEXCore::Context::Context *CTX, std::function<int(const std::string&)> CacheReader) {
CTX->AOTIRLoader = CacheReader;
}
bool WriteAOTIR(FEXCore::Context::Context *CTX, std::function<std::unique_ptr<std::ostream>(const std::string&)> CacheWriter) {
return CTX->WriteAOTIRCache(CacheWriter);
void SetAOTIRWriter(FEXCore::Context::Context *CTX, std::function<std::unique_ptr<std::ostream>(const std::string&)> CacheWriter) {
CTX->AOTIRWriter = CacheWriter;
}
void FinalizeAOTIRCache(FEXCore::Context::Context *CTX) {
CTX->FinalizeAOTIRCache();
}
void WriteFilesWithCode(FEXCore::Context::Context *CTX, std::function<void(const std::string& fileid, const std::string& filename)> Writer) {
CTX->WriteFilesWithCode(Writer);
}
void AddNamedRegion(FEXCore::Context::Context *CTX, uintptr_t Base, uintptr_t Length, uintptr_t Offset, const std::string& Name) {
+92 -30
View File
@@ -1,4 +1,5 @@
#pragma once
#include "Common/JitSymbols.h"
#include "Interface/Core/CPUID.h"
#include "Interface/Core/Frontend.h"
@@ -9,6 +10,7 @@
#include "Interface/IR/Passes/RegisterAllocationPass.h"
#include <FEXCore/Config/Config.h>
#include <FEXCore/Core/CPUBackend.h>
#include <FEXCore/Utils/CompilerDefs.h>
#include <FEXCore/Utils/Event.h>
#include <stdint.h>
@@ -20,7 +22,9 @@
#include <optional>
#include <ostream>
#include <set>
#include <shared_mutex>
#include <unordered_map>
#include <queue>
namespace FEXCore {
class ThunkHandler;
@@ -52,6 +56,38 @@ namespace FEXCore::Context {
MODE_SINGLESTEP = 1,
};
struct AOTIRInlineEntry {
uint64_t GuestHash;
uint64_t GuestLength;
/* RAData followed by IRData */
uint8_t InlineData[0];
IR::RegisterAllocationData *GetRAData();
IR::IRListView *GetIRData();
};
struct AOTIRInlineIndexEntry {
uint64_t GuestStart;
uint64_t DataOffset;
};
struct AOTIRInlineIndex {
uint64_t Count;
uint64_t DataBase;
AOTIRInlineIndexEntry Entries[0];
AOTIRInlineEntry *Find(uint64_t GuestStart);
AOTIRInlineEntry *GetInlineEntry(uint64_t DataOffset);
};
struct AOTIRCaptureCacheEntry {
std::unique_ptr<std::ostream> Stream;
std::map<uint64_t, uint64_t> Index;
void AppendAOTIRCaptureCache(uint64_t GuestRIP, uint64_t Start, uint64_t Length, uint64_t Hash, FEXCore::IR::IRListView *IRList, FEXCore::IR::RegisterAllocationData *RAData);
};
struct Context {
friend class FEXCore::HLE::SyscallHandler;
#ifdef JIT_ARM64
@@ -78,6 +114,7 @@ namespace FEXCore::Context {
FEX_CONFIG_OPT(ABILocalFlags, ABILOCALFLAGS);
FEX_CONFIG_OPT(ABINoPF, ABINOPF);
FEX_CONFIG_OPT(AOTIRCapture, AOTIRCAPTURE);
FEX_CONFIG_OPT(AOTIRGenerate, AOTIRGENERATE);
FEX_CONFIG_OPT(AOTIRLoad, AOTIRLOAD);
FEX_CONFIG_OPT(SMCChecks, SMCCHECKS);
FEX_CONFIG_OPT(Core, CORE);
@@ -87,7 +124,7 @@ namespace FEXCore::Context {
FEX_CONFIG_OPT(DumpIR, DUMPIR);
} Config;
using IntCallbackReturn = __attribute__((naked)) void(*)(FEXCore::Core::InternalThreadState *Thread, volatile void *Host_RSP);
using IntCallbackReturn = FEX_NAKED void(*)(FEXCore::Core::InternalThreadState *Thread, volatile void *Host_RSP);
IntCallbackReturn InterpreterCallbackReturn;
FEXCore::HostFeatures HostFeatures;
@@ -110,29 +147,31 @@ namespace FEXCore::Context {
std::unique_ptr<FEXCore::ThunkHandler> ThunkHandler;
CustomCPUFactoryType CustomCPUFactory;
std::function<void(uint64_t ThreadId, FEXCore::Context::ExitReason)> CustomExitHandler;
FEXCore::Context::ExitHandler CustomExitHandler;
struct AOTIRCacheEntry {
uint64_t start;
uint64_t len;
uint64_t crc;
IR::IRListView *IR;
IR::RegisterAllocationData *RAData;
AOTIRInlineIndex *Array;
void *mapping;
size_t size;
};
std::function<std::unique_ptr<std::istream>(const std::string&)> AOTIRLoader;
std::unordered_map<std::string, std::map<uint64_t, AOTIRCacheEntry>> AOTIRCache;
std::unordered_map<std::string, AOTIRCacheEntry> AOTIRCache;
std::function<int(const std::string&)> AOTIRLoader;
std::function<std::unique_ptr<std::ostream>(const std::string&)> AOTIRWriter;
std::unordered_map<std::string, AOTIRCaptureCacheEntry> AOTIRCaptureCache;
struct AddrToFileEntry {
uint64_t Start;
uint64_t Len;
uint64_t Offset;
std::string fileid;
std::string filename;
void *CachedFileEntry;
bool ContainsCode;
};
std::map<uint64_t, AddrToFileEntry> AddrToFile;
std::map<std::string, std::string> FilesWithCode;
#ifdef BLOCKSTATS
std::unique_ptr<FEXCore::BlockSamplingData> BlockData;
@@ -146,7 +185,7 @@ namespace FEXCore::Context {
bool InitCore(FEXCore::CodeLoader *Loader);
FEXCore::Context::ExitReason RunUntilExit();
int GetProgramStatus();
int GetProgramStatus() const;
bool IsPaused() const { return !Running; }
void Pause();
void Run();
@@ -157,12 +196,12 @@ namespace FEXCore::Context {
void StopThread(FEXCore::Core::InternalThreadState *Thread);
void SignalThread(FEXCore::Core::InternalThreadState *Thread, FEXCore::Core::SignalEvent Event);
bool GetGdbServerStatus() { return (bool)DebugServer; }
bool GetGdbServerStatus() const { return DebugServer != nullptr; }
void StartGdbServer();
void StopGdbServer();
void HandleCallback(uint64_t RIP);
void RegisterHostSignalHandler(int Signal, HostSignalDelegatorFunction Func);
void RegisterFrontendHostSignalHandler(int Signal, HostSignalDelegatorFunction Func);
void RegisterHostSignalHandler(int Signal, HostSignalDelegatorFunction Func, bool Required);
void RegisterFrontendHostSignalHandler(int Signal, HostSignalDelegatorFunction Func, bool Required);
static void RemoveCodeEntry(FEXCore::Core::InternalThreadState *Thread, uint64_t GuestRIP);
@@ -178,18 +217,37 @@ namespace FEXCore::Context {
bool GetDebugDataForRIP(uint64_t RIP, FEXCore::Core::DebugData *Data);
bool FindHostCodeForRIP(uint64_t RIP, uint8_t **Code);
// XXX:
// bool FindIRForRIP(uint64_t RIP, FEXCore::IR::IntrusiveIRList **ir);
// void SetIRForRIP(uint64_t RIP, FEXCore::IR::IntrusiveIRList *const ir);
void LoadEntryList();
struct GenerateIRResult {
FEXCore::IR::IRListView* IRList;
// User's responsibility to deallocate this.
FEXCore::IR::RegisterAllocationData* RAData;
uint64_t TotalInstructions;
uint64_t TotalInstructionsLength;
uint64_t StartAddr;
uint64_t Length;
};
[[nodiscard]] GenerateIRResult GenerateIR(FEXCore::Core::InternalThreadState *Thread, uint64_t GuestRIP);
std::tuple<FEXCore::IR::IRListView *, FEXCore::IR::RegisterAllocationData *, uint64_t, uint64_t, uint64_t, uint64_t> GenerateIR(FEXCore::Core::InternalThreadState *Thread, uint64_t GuestRIP);
std::tuple<void *, FEXCore::IR::IRListView *, FEXCore::Core::DebugData *, FEXCore::IR::RegisterAllocationData *, bool, uint64_t, uint64_t> CompileCode(FEXCore::Core::InternalThreadState *Thread, uint64_t GuestRIP);
struct CompileCodeResult {
void* CompiledCode;
FEXCore::IR::IRListView* IRData;
FEXCore::Core::DebugData* DebugData;
// User's responsibility to deallocate this.
FEXCore::IR::RegisterAllocationData* RAData;
bool GeneratedIR;
uint64_t StartAddr;
uint64_t Length;
};
[[nodiscard]] CompileCodeResult CompileCode(FEXCore::Core::InternalThreadState *Thread, uint64_t GuestRIP);
uintptr_t CompileBlock(FEXCore::Core::CpuStateFrame *Frame, uint64_t GuestRIP);
// same as CompileBlock, but aborts on failure
void CompileBlockJit(FEXCore::Core::CpuStateFrame *Frame, uint64_t GuestRIP);
bool LoadAOTIRCache(std::istream &stream);
bool WriteAOTIRCache(std::function<std::unique_ptr<std::ostream>(const std::string&)> CacheWriter);
bool LoadAOTIRCache(int streamfd);
void FinalizeAOTIRCache();
void WriteFilesWithCode(std::function<void(const std::string& fileid, const std::string& filename)> Writer);
// Used for thread creation from syscalls
void InitializeCompiler(FEXCore::Core::InternalThreadState* State, bool CompileThread);
FEXCore::Core::InternalThreadState* CreateThread(FEXCore::Core::CPUState *NewThreadState, uint64_t ParentTID);
@@ -201,7 +259,9 @@ namespace FEXCore::Context {
void DestroyThread(FEXCore::Core::InternalThreadState *Thread);
void CleanupAfterFork(FEXCore::Core::InternalThreadState *ExceptForThread);
std::vector<FEXCore::Core::InternalThreadState*> *const GetThreads() { return &Threads; }
std::vector<FEXCore::Core::InternalThreadState*>* GetThreads() { return &Threads; }
uint8_t GetGPRSize() const { return Config.Is64BitMode ? 8 : 4; }
void AddNamedRegion(uintptr_t Base, uintptr_t Size, uintptr_t Offset, const std::string &filename);
void RemoveNamedRegion(uintptr_t Base, uintptr_t Size);
@@ -222,19 +282,21 @@ namespace FEXCore::Context {
void NotifyPause();
void AddBlockMapping(FEXCore::Core::InternalThreadState *Thread, uint64_t Address, void *Ptr, uint64_t Start, uint64_t Length);
FEXCore::CodeLoader *LocalLoader{};
// Entry Cache
std::optional<std::string> GetFilenameHash(std::string const &Filename) const;
void AddThreadRIPsToEntryList(FEXCore::Core::InternalThreadState *Thread);
void SaveEntryList();
std::set<uint64_t> EntryList;
std::vector<uint64_t> InitLocations;
uint64_t StartingRIP;
std::mutex ExitMutex;
std::unique_ptr<GdbServer> DebugServer;
std::shared_mutex AOTIRCacheLock;
std::shared_mutex AOTIRCaptureCacheWriteoutLock;
std::atomic<bool> AOTIRCaptureCacheWriteoutFlusing;
std::queue<std::function<void()>> AOTIRCaptureCacheWriteoutQueue;
void AOTIRCaptureCacheWriteoutQueue_Flush();
void AOTIRCaptureCacheWriteoutQueue_Append(const std::function<void()> &fn);
bool StartPaused = false;
FEX_CONFIG_OPT(AppFilename, APP_FILENAME);
};
File diff suppressed because it is too large. Load diff
@@ -12,6 +12,37 @@ namespace FEXCore::ArchHelpers::Arm64 {
constexpr uint32_t ATOMIC_MEM_MASK = 0x3B200C00;
constexpr uint32_t ATOMIC_MEM_INST = 0x38200000;
constexpr uint32_t LDAXP_MASK = 0xBF'FF'80'00;
constexpr uint32_t LDAXP_INST = 0x88'7F'80'00;
constexpr uint32_t STLXP_MASK = 0xBF'E0'80'00;
constexpr uint32_t STLXP_INST = 0x88'20'80'00;
constexpr uint32_t LDAXR_MASK = 0x3F'FF'FC'00;
constexpr uint32_t LDAXR_INST = 0x08'5F'FC'00;
constexpr uint32_t STLXR_MASK = 0x3F'E0'FC'00;
constexpr uint32_t STLXR_INST = 0x08'00'FC'00;
constexpr uint32_t CBNZ_MASK = 0x7F'00'00'00;
constexpr uint32_t CBNZ_INST = 0x35'00'00'00;
constexpr uint32_t ALU_OP_MASK = 0x7F'00'00'00;
constexpr uint32_t ADD_INST = 0x0B'00'00'00;
constexpr uint32_t SUB_INST = 0x4B'00'00'00;
constexpr uint32_t AND_INST = 0x0A'00'00'00;
constexpr uint32_t OR_INST = 0x2A'00'00'00;
constexpr uint32_t EOR_INST = 0x4A'00'00'00;
enum ExclusiveAtomicPairType {
TYPE_SWAP,
TYPE_ADD,
TYPE_SUB,
TYPE_AND,
TYPE_OR,
TYPE_EOR,
TYPE_NEG, // This is just a sub with zero. Need to know the differences
};
// Load ops are 4 bits
// Acquire and release bits are independent on the instruction
constexpr uint32_t ATOMIC_ADD_OP = 0b0000;
@@ -24,6 +55,27 @@ namespace FEXCore::ArchHelpers::Arm64 {
constexpr uint32_t ATOMIC_UMIN_OP = 0b0111;
constexpr uint32_t ATOMIC_SWAP_OP = 0b1000;
constexpr uint32_t REGISTER_MASK = 0b11111;
constexpr uint32_t RD_OFFSET = 0;
constexpr uint32_t RN_OFFSET = 5;
constexpr uint32_t RM_OFFSET = 16;
inline uint32_t GetRdReg(uint32_t Instr) {
return (Instr >> RD_OFFSET) & REGISTER_MASK;
}
inline uint32_t GetRnReg(uint32_t Instr) {
return (Instr >> RN_OFFSET) & REGISTER_MASK;
}
inline uint32_t GetRmReg(uint32_t Instr) {
return (Instr >> RM_OFFSET) & REGISTER_MASK;
}
bool HandleAtomicLoad(void *_ucontext, void *_info, uint32_t Instr);
bool HandleAtomicStore(void *_ucontext, void *_info, uint32_t Instr);
bool HandleAtomicLoad128(void *_ucontext, void *_info, uint32_t Instr);
uint64_t HandleAtomicLoadstoreExclusive(void *_ucontext, void *_info);
bool HandleCASPAL(void *_ucontext, void *_info, uint32_t Instr);
bool HandleCASAL(void *_ucontext, void *_info, uint32_t Instr);
bool HandleAtomicMemOp(void *_ucontext, void *_info, uint32_t Instr);
@@ -29,6 +29,17 @@ Arm64Emitter::Arm64Emitter(size_t size) : vixl::aarch64::Assembler(size, vixl::a
if (!SupportsAtomics) {
WARN_ONCE("Host CPU doesn't support atomics. Expect bad performance");
}
#ifdef _M_ARM_64
// We need to get the CPU's cache line size
// We expect sane targets that have correct cacheline sizes across clusters
uint64_t CTR;
__asm volatile ("mrs %[ctr], ctr_el0"
: [ctr] "=r"(CTR));
DCacheLineSize = 4 << ((CTR >> 16) & 0xF);
ICacheLineSize = 4 << (CTR & 0xF);
#endif
}
void Arm64Emitter::LoadConstant(vixl::aarch64::Register Reg, uint64_t Constant) {
@@ -69,6 +69,9 @@ protected:
void Align16B();
uint32_t SpillSlots{};
uint32_t DCacheLineSize{};
uint32_t ICacheLineSize{};
};
}
@@ -111,7 +111,7 @@ static inline void BackupContext(void* ucontext, T *Backup) {
// Host FPR state starts at _mcontext->reserved[0];
HostFPRState *HostState = reinterpret_cast<HostFPRState*>(&_mcontext->__reserved[0]);
LogMan::Throw::A(HostState->Head.Magic == FPR_MAGIC, "Wrong FPR Magic: 0x%08x", HostState->Head.Magic);
LOGMAN_THROW_A(HostState->Head.Magic == FPR_MAGIC, "Wrong FPR Magic: 0x%08x", HostState->Head.Magic);
Backup->FPSR = HostState->FPSR;
Backup->FPCR = HostState->FPCR;
memcpy(&Backup->FPRs[0], &HostState->FPRs[0], 32 * sizeof(__uint128_t));
@@ -126,7 +126,7 @@ static inline void RestoreContext(void* ucontext, T *Backup) {
auto _mcontext = GetMContext(ucontext);
HostFPRState *HostState = reinterpret_cast<HostFPRState*>(&_mcontext->__reserved[0]);
LogMan::Throw::A(HostState->Head.Magic == FPR_MAGIC, "Wrong FPR Magic: 0x%08x", HostState->Head.Magic);
LOGMAN_THROW_A(HostState->Head.Magic == FPR_MAGIC, "Wrong FPR Magic: 0x%08x", HostState->Head.Magic);
memcpy(&HostState->FPRs[0], &Backup->FPRs[0], 32 * sizeof(__uint128_t));
HostState->FPCR = Backup->FPCR;
HostState->FPSR = Backup->FPSR;
+486 -162
View File
@@ -15,7 +15,26 @@ $end_info$
#endif
namespace FEXCore {
//#define CPUID_AMD
constexpr uint32_t SUPPORTS_AVX = 0;
// #define CPUID_AMD
#ifdef CPUID_AMD
constexpr uint32_t FAMILY_IDENTIFIER =
0 | // Stepping
(0xA << 4) | // Model
(0xF << 8) | // Family ID
(0 << 12) | // Processor type
(0 << 16) | // Extended model ID
(1 << 20); // Extended family ID
#else
constexpr uint32_t FAMILY_IDENTIFIER =
0 | // Stepping
(0x7 << 4) | // Model
(0x6 << 8) | // Family ID
(0 << 12) | // Processor type
(1 << 16) | // Extended model ID
(0x0 << 20); // Extended family ID
#endif
#ifdef _M_ARM_64
static uint32_t GetCycleCounterFrequency() {
uint64_t Result{};
@@ -38,7 +57,7 @@ static uint32_t GetCycleCounterFrequency() {
}
#endif
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_0h() {
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_0h(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
// EBX, EDX, ECX become the manufacturer id string
@@ -57,29 +76,27 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_0h() {
}
// Processor Info and Features bits
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_01h() {
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_01h(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
uint32_t CoreCount = Cores();
Res.eax = FAMILY_IDENTIFIER;
Res.eax = 0 | // Stepping
(0 << 4) | // Model
(0xF << 8) | // Family ID
(0 << 12) | // Processor type
(0 << 16) | // Extended model ID
(0 << 20); // Extended family ID
Res.ebx = 0 | // Brand index
(8 << 8) | // Cache line size in bytes
(8 << 16) | // Number of addressable IDs for the logical cores in the physical CPU
(CoreCount << 16) | // Number of addressable IDs for the logical cores in the physical CPU
(0 << 24); // Local APIC ID
Res.ecx =
(1 << 0) | // SSE3
(0 << 1) | // PCLMULQDQ
(1 << 2) | // DS area supports 64bit layout
(1 << 3) | // MWait
(1 << 4) | // DS-CPL
(0 << 4) | // DS-CPL
(0 << 5) | // VMX
(0 << 6) | // SMX
(0 << 7) | // Intel SpeedStep
(0 << 8) | // Thermal Monitor 2
(1 << 8) | // Thermal Monitor 2
(1 << 9) | // SSSE3
(0 << 10) | // L1 context ID
(0 << 11) | // Silicon debug
@@ -89,8 +106,8 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_01h() {
(0 << 15) | // Perfmon and debug capability
(0 << 16) | // Reserved
(0 << 17) | // Process-context identifiers
(1 << 18) | // Prefetching from memory mapped device
(0 << 19) | // SSE4.1
(0 << 18) | // Prefetching from memory mapped device
(1 << 19) | // SSE4.1
(0 << 20) | // SSE4.2
(0 << 21) | // X2APIC
(1 << 22) | // MOVBE
@@ -99,7 +116,7 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_01h() {
(CTX->HostFeatures.SupportsAES << 25) | // AES
(0 << 26) | // XSAVE
(0 << 27) | // OSXSAVE
(0 << 28) | // AVX
(SUPPORTS_AVX << 28) | // AVX
(0 << 29) | // F16C
(0 << 30) | // RDRAND
(0 << 31); // Hypervisor always returns zero
@@ -124,7 +141,7 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_01h() {
(1 << 16) | // Page Attribute Table
(1 << 17) | // 36bit page size extension
(0 << 18) | // Processor serial number
(0 << 19) | // CLFLUSH
(1 << 19) | // CLFLUSH
(0 << 20) | // Reserved
(0 << 21) | // Debug store
(0 << 22) | // Thermal monitor and software controled clock
@@ -132,16 +149,16 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_01h() {
(1 << 24) | // FXSAVE/FXRSTOR
(1 << 25) | // SSE
(1 << 26) | // SSE2
(1 << 27) | // Self Snoop
(0 << 27) | // Self Snoop
(1 << 28) | // Max APIC IDs reserved field is valid
(1 << 29) | // Thermal monitor
(0 << 30) | // Reserved
(1 << 31); // Pending break enable
(0 << 31); // Pending break enable
return Res;
}
// 2: Cache and TLB information
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_02h() {
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_02h(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
// returns default values from i7 model 1Ah
@@ -165,124 +182,286 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_02h() {
return Res;
}
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_06h() {
// 4: Deterministic cache parameters for each level
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_04h(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
constexpr uint32_t CacheType_Data = 1;
constexpr uint32_t CacheType_Instruction = 2;
constexpr uint32_t CacheType_Unified = 3;
if (Leaf == 0) {
// Report L1D
uint32_t CoreCount = Cores() - 1;
Res.eax = CacheType_Data | // Cache type
(0b001 << 5) | // Cache level
(1 << 8) | // Self initializing cache level
(0 << 9) | // Fully associative
(0 << 14) | // Maximum number of addressable IDs for logical processors sharing this cache (With SMT this would be 1)
(CoreCount << 26); // Maximum number of addressable IDs for processor cores in the physical package
Res.ebx =
(63 << 0) | // Line Size - 1 : Claiming 64 byte
(0 << 12) | // Physical Line partitions
(7 << 22); // Associativity - 1 : Claiming 8 way
// 32KB
Res.ecx = 63; // Number of sets - 1 : Claiming 64 sets
Res.edx =
(0 << 0) | // Write-back invalidate
(0 << 1) | // Cache inclusiveness - Includes lower caches
(0 << 2); // Complex cache indexing - 0: Direct, 1: Complex
}
else if (Leaf == 1) {
// Report L1I
uint32_t CoreCount = Cores() - 1;
Res.eax = CacheType_Instruction | // Cache type
(0b001 << 5) | // Cache level
(1 << 8) | // Self initializing cache level
(0 << 9) | // Fully associative
(0 << 14) | // Maximum number of addressable IDs for logical processors sharing this cache (With SMT this would be 1)
(CoreCount << 26); // Maximum number of addressable IDs for processor cores in the physical package
Res.ebx =
(63 << 0) | // Line Size - 1 : Claiming 64 byte
(0 << 12) | // Physical Line partitions
(7 << 22); // Associativity - 1 : Claiming 8 way
// 32KB
Res.ecx = 63; // Number of sets - 1 : Claiming 64 sets
Res.edx =
(0 << 0) | // Write-back invalidate
(0 << 1) | // Cache inclusiveness - Includes lower caches
(0 << 2); // Complex cache indexing - 0: Direct, 1: Complex
}
else if (Leaf == 2) {
// Report L2
uint32_t CoreCount = Cores() - 1;
Res.eax = CacheType_Unified | // Cache type
(0b010 << 5) | // Cache level
(1 << 8) | // Self initializing cache level
(0 << 9) | // Fully associative
(0 << 14) | // Maximum number of addressable IDs for logical processors sharing this cache
(CoreCount << 26); // Maximum number of addressable IDs for processor cores in the physical package
Res.ebx =
(63 << 0) | // Line Size - 1 : Claiming 64 byte
(0 << 12) | // Physical Line partitions
(7 << 22); // Associativity - 1 : Claiming 8 way
// 512KB
Res.ecx = 0x3FF; // Number of sets - 1 : Claiming 1024 sets
Res.edx =
(0 << 0) | // Write-back invalidate
(0 << 1) | // Cache inclusiveness - Includes lower caches
(0 << 2); // Complex cache indexing - 0: Direct, 1: Complex
}
else if (Leaf == 3) {
// Report L3
uint32_t CoreCount = Cores() - 1;
Res.eax = CacheType_Unified | // Cache type
(0b011 << 5) | // Cache level
(1 << 8) | // Self initializing cache level
(0 << 9) | // Fully associative
(CoreCount << 14) | // Maximum number of addressable IDs for logical processors sharing this cache
(CoreCount << 26); // Maximum number of addressable IDs for processor cores in the physical package
Res.ebx =
(63 << 0) | // Line Size - 1 : Claiming 64 byte
(0 << 12) | // Physical Line partitions
(7 << 22); // Associativity - 1 : Claiming 8 way
// 8MB
Res.ecx = 0x4000; // Number of sets - 1 : Claiming 16384 sets
Res.edx =
(0 << 0) | // Write-back invalidate
(0 << 1) | // Cache inclusiveness - Includes lower caches
(1 << 2); // Complex cache indexing - 0: Direct, 1: Complex
}
return Res;
}
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_06h(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
Res.eax = (1 << 2); // Always running APIC
Res.ecx = (0 << 3); // Intel performance energy bias preference (EPB)
return Res;
}
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_07h() {
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_07h(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
if (Leaf == 0) {
// Number of subfunctions
Res.eax = 0x0;
Res.ebx =
(1 << 0) | // FS/GS support
(0 << 1) | // TSC adjust MSR
(0 << 2) | // SGX
(0 << 3) | // BMI1
(0 << 4) | // Intel Hardware Lock Elison
(0 << 5) | // AVX2 support
(1 << 6) | // FPU data pointer updated only on exception
(1 << 7) | // SMEP support
(0 << 8) | // BMI2
(0 << 9) | // Enhanced REP MOVSB/STOSB
(1 << 10) | // INVPCID for system software control of process-context
(0 << 11) | // Restricted transactional memory
(0 << 12) | // Intel resource directory technology Monitoring
(1 << 13) | // Deprecates FPU CS and DS
(0 << 14) | // Intel MPX
(0 << 15) | // Intel Resource Directory Technology Allocation
(0 << 16) | // Reserved
(0 << 17) | // Reserved
(0 << 18) | // RDSEED
(0 << 19) | // ADCX and ADOX instructions
(0 << 20) | // SMAP Supervisor mode access prevention and CLAC/STAC instructions
(0 << 21) | // Reserved
(0 << 22) | // Reserved
(0 << 23) | // CLFLUSHOPT instruction
(0 << 24) | // CLWB instruction
(0 << 25) | // Intel processor trace
(0 << 26) | // Reserved
(0 << 27) | // Reserved
(0 << 28) | // Reserved
(0 << 29) | // SHA instructions
(0 << 30) | // Reserved
(0 << 31); // Reserved
// Number of subfunctions
Res.eax = 0x0;
Res.ebx =
(1 << 0) | // FS/GS support
(0 << 1) | // TSC adjust MSR
(0 << 2) | // SGX
(0 << 3) | // BMI1
(0 << 4) | // Intel Hardware Lock Elison
(0 << 5) | // AVX2 support
(1 << 6) | // FPU data pointer updated only on exception
(1 << 7) | // SMEP support
(0 << 8) | // BMI2
(0 << 9) | // Enhanced REP MOVSB/STOSB
(1 << 10) | // INVPCID for system software control of process-context
(0 << 11) | // Restricted transactional memory
(0 << 12) | // Intel resource directory technology Monitoring
(1 << 13) | // Deprecates FPU CS and DS
(0 << 14) | // Intel MPX
(0 << 15) | // Intel Resource Directory Technology Allocation
(0 << 16) | // Reserved
(0 << 17) | // Reserved
(0 << 18) | // RDSEED
(0 << 19) | // ADCX and ADOX instructions
(0 << 20) | // SMAP Supervisor mode access prevention and CLAC/STAC instructions
(0 << 21) | // Reserved
(0 << 22) | // Reserved
(0 << 23) | // CLFLUSHOPT instruction
(0 << 24) | // CLWB instruction
(0 << 25) | // Intel processor trace
(0 << 26) | // Reserved
(0 << 27) | // Reserved
(0 << 28) | // Reserved
(0 << 29) | // SHA instructions
(0 << 30) | // Reserved
(0 << 31); // Reserved
Res.ecx =
(1 << 0) | // PREFETCHWT1
(0 << 1) | // AVX512VBMI
(0 << 2) | // Usermode instruction prevention
(0 << 3) | // Protection keys for user mode pages
(0 << 4) | // OS protection keys
(0 << 5) | // waitpkg
(0 << 6) | // AVX512_VBMI2
(0 << 7) | // CET shadow stack
(0 << 8) | // GFNI
(0 << 9) | // VAES
(0 << 10) | // VPCLMULQDQ
(0 << 11) | // AVX512_VNNI
(0 << 12) | // AVX512_BITALG
(0 << 13) | // Intel Total Memory Encryption
(0 << 14) | // AVX512_VPOPCNTDQ
(0 << 15) | // Reserved
(0 << 16) | // 5 Level page tables
(0 << 17) | // MPX MAWAU
(0 << 18) | // MPX MAWAU
(0 << 19) | // MPX MAWAU
(0 << 20) | // MPX MAWAU
(0 << 21) | // MPX MAWAU
(0 << 22) | // RDPID Read Processor ID
(0 << 23) | // Reserved
(0 << 24) | // Reserved
(0 << 25) | // CLDEMOTE
(0 << 26) | // Reserved
(0 << 27) | // MOVDIRI
(0 << 28) | // MOVDIR64B
(0 << 29) | // Reserved
(0 << 30) | // SGX Launch configuration
(0 << 31); // Reserved
Res.ecx =
(1 << 0) | // PREFETCHWT1
(0 << 1) | // AVX512VBMI
(0 << 2) | // Usermode instruction prevention
(0 << 3) | // Protection keys for user mode pages
(1 << 4) | // OS protection keys
(0 << 5) | // waitpkg
(0 << 6) | // AVX512_VBMI2
(0 << 7) | // CET shadow stack
(0 << 8) | // GFNI
(0 << 9) | // VAES
(0 << 10) | // VPCLMULQDQ
(0 << 11) | // AVX512_VNNI
(0 << 12) | // AVX512_BITALG
(0 << 13) | // Intel Total Memory Encryption
(0 << 14) | // AVX512_VPOPCNTDQ
(0 << 15) | // Reserved
(0 << 16) | // 5 Level page tables
(0 << 17) | // MPX MAWAU
(0 << 18) | // MPX MAWAU
(0 << 19) | // MPX MAWAU
(0 << 20) | // MPX MAWAU
(0 << 21) | // MPX MAWAU
(0 << 22) | // RDPID Read Processor ID
(0 << 23) | // Reserved
(0 << 24) | // Reserved
(0 << 25) | // CLDEMOTE
(0 << 26) | // Reserved
(0 << 27) | // MOVDIRI
(0 << 28) | // MOVDIR64B
(0 << 29) | // Reserved
(0 << 30) | // SGX Launch configuration
(0 << 31); // Reserved
Res.edx =
(0 << 0) | // Reserved
(0 << 1) | // Reserved
(0 << 2) | // AVX512_4VNNIW
(0 << 3) | // AVX512_4FMAPS
(0 << 4) | // Fast Short Rep Mov
(0 << 5) | // Reserved
(0 << 6) | // Reserved
(0 << 7) | // Reserved
(0 << 8) | // AVX512_VP2INTERSECT
(0 << 9) | // Reserved
(0 << 10) | // VERW clears CPU buffers
(0 << 11) | // Reserved
(0 << 12) | // Reserved
(0 << 13) | // Reserved
(0 << 14) | // SERIALIZE instruction
(0 << 15) | // Reserved
(0 << 16) | // Reserved
(0 << 17) | // Reserved
(0 << 18) | // Intel PCONFIG
(0 << 19) | // Intel Architectural LBR
(0 << 20) | // Intel CET
(0 << 21) | // Reserved
(0 << 22) | // Reserved
(0 << 23) | // Reserved
(0 << 24) | // Reserved
(0 << 25) | // Reserved
(0 << 26) | // Reserved
(0 << 27) | // Reserved
(0 << 28) | // L1D Flush
(0 << 29) | // Arch capabilities
(0 << 30) | // Reserved
(0 << 31); // Reserved
Res.edx =
(0 << 0) | // Reserved
(0 << 1) | // Reserved
(0 << 2) | // AVX512_4VNNIW
(0 << 3) | // AVX512_4FMAPS
(0 << 4) | // Fast Short Rep Mov
(0 << 5) | // Reserved
(0 << 6) | // Reserved
(0 << 7) | // Reserved
(0 << 8) | // AVX512_VP2INTERSECT
(0 << 9) | // Reserved
(0 << 10) | // VERW clears CPU buffers
(0 << 11) | // Reserved
(0 << 12) | // Reserved
(0 << 13) | // Reserved
(0 << 14) | // SERIALIZE instruction
(0 << 15) | // Reserved
(0 << 16) | // Reserved
(0 << 17) | // Reserved
(0 << 18) | // Intel PCONFIG
(0 << 19) | // Intel Architectural LBR
(0 << 20) | // Intel CET
(0 << 21) | // Reserved
(0 << 22) | // Reserved
(0 << 23) | // Reserved
(0 << 24) | // Reserved
(0 << 25) | // Reserved
(0 << 26) | // Reserved
(0 << 27) | // Reserved
(0 << 28) | // L1D Flush
(0 << 29) | // Arch capabilities
(0 << 30) | // Reserved
(0 << 31); // Reserved
}
return Res;
}
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_15h() {
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_0Dh(uint32_t Leaf) {
// Leaf 0
FEXCore::CPUID::FunctionResults Res{};
uint32_t XFeatureSupportedSizeMax = SUPPORTS_AVX ? 0x0000'0340 : 0x0000'0240; // XFeatureEnabledSizeMax: Legacy Header + FPU/SSE + AVX
if (Leaf == 0) {
// XFeatureSupportedMask[31:0]
Res.eax =
(1 << 0) | // X87 support
(1 << 1) | // 128-bit SSE support
(SUPPORTS_AVX << 2) | // 256-bit AVX support
(0b00 << 3) | // MPX State
(0b000 << 5) | // AVX-512 state
(0 << 8) | // "Used for IA32_XSS" ... Used for what?
(0 << 9); // PKRU state
// EBX and ECX doesn't need to match if a feature is supported but not enabled
Res.ebx = XFeatureSupportedSizeMax;
Res.ecx = XFeatureSupportedSizeMax; // XFeatureSupportedSizeMax: Size in bytes of XSAVE/XRSTOR area
// XFeatureSupportedMask[63:32]
Res.edx = 0; // Upper 32-bits of XFeatureSupportedMask
}
else if (Leaf == 1) {
Res.eax =
(0 << 0) | // XSAVEOPT
(0 << 1) | // XSAVEC (and XRSTOR)
(0 << 2) | // XGETBV - XGETBV with ECX=1 supported
(0 << 3); // XSAVES - XSAVES, XRSTORS, and IA32_XSS supported
// Same information as Leaf 0 for ebx
Res.ebx = XFeatureSupportedSizeMax;
// Lower supported 32bits of IA32_XSS MSR. IA32_XSS[n] can only be set to 1 if ECX[n] is 1
Res.ecx =
(0b0000'0000 << 0) | // Used for XCR0
(0 << 8) | // PT state
(0 << 9); // Used for XCR0
// Upper supported 32bits of IA32_XSS MSR. IA32_XSS[n+32] can only be set to 1 if EDX[n] is 1
// Entirely reserved atm
Res.edx = 0;
}
else if (Leaf == 2) {
Res.eax = SUPPORTS_AVX ? 0x0000'0100 : 0; // YmmSaveStateSize
Res.ebx = SUPPORTS_AVX ? 0x0000'0240 : 0; // YmmSaveStateOffset
// Reserved
Res.ecx = 0;
Res.edx = 0;
}
return Res;
}
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_15h(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
// TSC frequency = ECX * EBX / EAX
uint32_t FrequencyHz = GetCycleCounterFrequency();
@@ -295,7 +474,7 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_15h() {
}
// Highest extended function implemented
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0000h() {
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0000h(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
Res.eax = 0x8000001F;
@@ -314,15 +493,10 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0000h() {
}
// Extended processor and feature bits
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0001h() {
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0001h(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
Res.eax = 0 | // Stepping
(0 << 4) | // Model
(0 << 8) | // Family ID
(0 << 12) | // Processor type
(0 << 16) | // Extended model ID
(0 << 20); // Extended family ID
Res.eax = FAMILY_IDENTIFIER;
Res.ecx =
(1 << 0) | // LAHF/SAHF
@@ -347,13 +521,13 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0001h() {
(0 << 19) | // Reserved
(0 << 20) | // Reserved
(0 << 21) | // Reserved
(1 << 22) | // Topology extensions support
(1 << 23) | // Core performance counter extensions
(1 << 24) | // NB performance counter extensions
(0 << 22) | // Topology extensions support
(0 << 23) | // Core performance counter extensions
(0 << 24) | // NB performance counter extensions
(0 << 25) | // Reserved
(0 << 26) | // Data breakpoints extensions
(1 << 27) | // Performance TSC
(1 << 28) | // L2 perf counter extensions
(0 << 27) | // Performance TSC
(0 << 28) | // L2 perf counter extensions
(0 << 29) | // Reserved
(0 << 30) | // Reserved
(0 << 31); // Reserved
@@ -385,7 +559,7 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0001h() {
(1 << 23) | // MMX
(1 << 24) | // FXSAVE/FXRSTOR
(1 << 25) | // FXSAVE/FXRSTOR Optimizations
(1 << 26) | // 1 gigabit pages
(0 << 26) | // 1 gigabit pages
(0 << 27) | // RDTSCP
(0 << 28) | // Reserved
(1 << 29) | // Long Mode
@@ -400,26 +574,26 @@ constexpr char ProcessorBrand[48] = {
};
//Processor brand string
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0002h() {
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0002h(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
memcpy(&Res, &ProcessorBrand[0], sizeof(FEXCore::CPUID::FunctionResults));
return Res;
}
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0003h() {
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0003h(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
memcpy(&Res, &ProcessorBrand[16], sizeof(FEXCore::CPUID::FunctionResults));
return Res;
}
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0004h() {
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0004h(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
memcpy(&Res, &ProcessorBrand[32], sizeof(FEXCore::CPUID::FunctionResults));
return Res;
}
// L1 Cache and TLB identifiers
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0005h() {
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0005h(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
// L1 TLB Information for 2MB and 4MB pages
@@ -454,7 +628,7 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0005h() {
}
// L2 Cache identifiers
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0006h() {
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0006h(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
// L2 TLB Information for 2MB and 4MB pages
@@ -488,7 +662,7 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0006h() {
}
// Advanced power management
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0007h() {
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0007h(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
Res.eax = (1 << 2); // APIC timer not affected by p-state
Res.edx =
@@ -496,27 +670,167 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0007h() {
return Res;
}
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_Reserved() {
// Virtual and physical address sizes
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0008h(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
Res.eax =
(48 << 0) | // PhysAddrSize = 48-bit
(48 << 8) | // LinAddrSize = 48-bit
(0 << 16); // GuestPhysAddrSize == PhysAddrSize
Res.ebx =
(0 << 2) | // XSaveErPtr: Saving and restoring error pointers
(0 << 1) | // IRPerf: Instructions retired count support
(0 << 0); // CLZERO support
uint32_t CoreCount = Cores() - 1;
Res.ecx =
(0 << 16) | // PerfTscSize: Performance timestamp count size
((uint32_t)std::log2(CoreCount + 1) << 12) | // ApicIdSize: Number of bits in ApicID
(CoreCount << 0); // Count count subtract one
return Res;
}
// TLB 1GB page identifiers
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0019h(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
Res.eax =
(0xF << 28) | // L1 DTLB associativity for 1GB pages
(64 << 16) | // L1 DTLB entry count for 1GB pages
(0xF << 12) | // L1 ITLB associativity for 1GB pages
(64 << 0); // L1 ITLB entry count for 1GB pages
Res.ebx =
(0 << 28) | // L2 DTLB associativity for 1GB pages
(0 << 16) | // L2 DTLB entry count for 1GB pages
(0 << 12) | // L2 ITLB associativity for 1GB pages
(0 << 0); // L2 ITLB entry count for 1GB pages
return Res;
}
// Deterministic cache parameters for each level
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_001Dh(uint32_t Leaf) {
// This is nearly a copy of CPUID function 4h
// There are some minor changes though
FEXCore::CPUID::FunctionResults Res{};
constexpr uint32_t CacheType_Data = 1;
constexpr uint32_t CacheType_Instruction = 2;
constexpr uint32_t CacheType_Unified = 3;
if (Leaf == 0) {
// Report L1D
Res.eax = CacheType_Data | // Cache type
(0b001 << 5) | // Cache level
(1 << 8) | // Self initializing cache level
(0 << 9) | // Fully associative
(0 << 14); // Maximum number of addressable IDs for logical processors sharing this cache (With SMT this would be 1)
Res.ebx =
(63 << 0) | // Line Size - 1 : Claiming 64 byte
(0 << 12) | // Physical Line partitions
(7 << 22); // Associativity - 1 : Claiming 8 way
// 32KB
Res.ecx = 63; // Number of sets - 1 : Claiming 64 sets
Res.edx =
(0 << 0) | // Write-back invalidate
(0 << 1); // Cache inclusiveness - Includes lower caches
}
else if (Leaf == 1) {
// Report L1I
Res.eax = CacheType_Instruction | // Cache type
(0b001 << 5) | // Cache level
(1 << 8) | // Self initializing cache level
(0 << 9) | // Fully associative
(0 << 14); // Maximum number of addressable IDs for logical processors sharing this cache (With SMT this would be 1)
Res.ebx =
(63 << 0) | // Line Size - 1 : Claiming 64 byte
(0 << 12) | // Physical Line partitions
(7 << 22); // Associativity - 1 : Claiming 8 way
// 32KB
Res.ecx = 63; // Number of sets - 1 : Claiming 64 sets
Res.edx =
(0 << 0) | // Write-back invalidate
(0 << 1); // Cache inclusiveness - Includes lower caches
}
else if (Leaf == 2) {
// Report L2
Res.eax = CacheType_Unified | // Cache type
(0b010 << 5) | // Cache level
(1 << 8) | // Self initializing cache level
(0 << 9) | // Fully associative
(0 << 14); // Maximum number of addressable IDs for logical processors sharing this cache
Res.ebx =
(63 << 0) | // Line Size - 1 : Claiming 64 byte
(0 << 12) | // Physical Line partitions
(7 << 22); // Associativity - 1 : Claiming 8 way
// 512KB
Res.ecx = 0x3FF; // Number of sets - 1 : Claiming 1024 sets
Res.edx =
(0 << 0) | // Write-back invalidate
(0 << 1); // Cache inclusiveness - Includes lower caches
}
else if (Leaf == 3) {
// Report L3
uint32_t CoreCount = Cores() - 1;
Res.eax = CacheType_Unified | // Cache type
(0b011 << 5) | // Cache level
(1 << 8) | // Self initializing cache level
(0 << 9) | // Fully associative
(CoreCount << 14); // Maximum number of addressable IDs for logical processors sharing this cache
Res.ebx =
(63 << 0) | // Line Size - 1 : Claiming 64 byte
(0 << 12) | // Physical Line partitions
(7 << 22); // Associativity - 1 : Claiming 8 way
// 8MB
Res.ecx = 0x4000; // Number of sets - 1 : Claiming 16384 sets
Res.edx =
(0 << 0) | // Write-back invalidate
(0 << 1); // Cache inclusiveness - Includes lower caches
}
return Res;
}
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_Reserved(uint32_t Leaf) {
FEXCore::CPUID::FunctionResults Res{};
return Res;
}
void CPUIDEmu::Init(FEXCore::Context::Context *ctx) {
CTX = ctx;
RegisterFunction(0, std::bind(&CPUIDEmu::Function_0h, this));
RegisterFunction(1, std::bind(&CPUIDEmu::Function_01h, this));
RegisterFunction(2, std::bind(&CPUIDEmu::Function_02h, this));
using namespace std::placeholders;
RegisterFunction(0, std::bind(&CPUIDEmu::Function_0h, this, _1));
RegisterFunction(1, std::bind(&CPUIDEmu::Function_01h, this, _1));
RegisterFunction(2, std::bind(&CPUIDEmu::Function_02h, this, _1));
// 3: Serial Number(previously), now reserved
// 4: Deterministic cache parameters for each level
#ifndef CPUID_AMD
// Deterministic cache parameters for each level
RegisterFunction(0x4, std::bind(&CPUIDEmu::Function_04h, this, _1));
#endif
// 5: Monitor/mwait
// Thermal and power management
RegisterFunction(6, std::bind(&CPUIDEmu::Function_06h, this));
RegisterFunction(6, std::bind(&CPUIDEmu::Function_06h, this, _1));
// Extended feature flags
RegisterFunction(7, std::bind(&CPUIDEmu::Function_07h, this));
RegisterFunction(7, std::bind(&CPUIDEmu::Function_07h, this, _1));
// 9: Direct Cache Access information
// 0x0A: Architectural performance monitoring
// 0x0B: Extended topology enumeration
// 0x0D: Processor extended state enumeration
RegisterFunction(0x0D, std::bind(&CPUIDEmu::Function_0Dh, this, _1));
// 0x0F: Intel RDT monitoring
// 0x10: Intel RDT allocation enumeration
// 0x12: Intel SGX capability enumeration
@@ -525,37 +839,47 @@ void CPUIDEmu::Init(FEXCore::Context::Context *ctx) {
#ifndef CPUID_AMD
// Timestamp counter information
// Doesn't exist on AMD hardware
RegisterFunction(0x15, std::bind(&CPUIDEmu::Function_15h, this));
RegisterFunction(0x15, std::bind(&CPUIDEmu::Function_15h, this, _1));
#endif
// 0x16: Processor frequency information
// 0x17: SoC vendor attribute enumeration
// Largest extended function number
RegisterFunction(0x8000'0000, std::bind(&CPUIDEmu::Function_8000_0000h, this));
RegisterFunction(0x8000'0000, std::bind(&CPUIDEmu::Function_8000_0000h, this, _1));
// Processor vendor
RegisterFunction(0x8000'0001, std::bind(&CPUIDEmu::Function_8000_0001h, this));
RegisterFunction(0x8000'0001, std::bind(&CPUIDEmu::Function_8000_0001h, this, _1));
// Processor brand string
RegisterFunction(0x8000'0002, std::bind(&CPUIDEmu::Function_8000_0002h, this));
RegisterFunction(0x8000'0002, std::bind(&CPUIDEmu::Function_8000_0002h, this, _1));
// Processor brand string continued
RegisterFunction(0x8000'0003, std::bind(&CPUIDEmu::Function_8000_0003h, this));
RegisterFunction(0x8000'0003, std::bind(&CPUIDEmu::Function_8000_0003h, this, _1));
// Processor brand string continued
RegisterFunction(0x8000'0004, std::bind(&CPUIDEmu::Function_8000_0004h, this));
RegisterFunction(0x8000'0004, std::bind(&CPUIDEmu::Function_8000_0004h, this, _1));
// 0x8000'0005: L1 Cache and TLB identifiers
#ifdef CPUID_AMD
RegisterFunction(0x8000'0005, std::bind(&CPUIDEmu::Function_8000_0005h, this, _1));
#else
// This is full reserved on Intel platforms
RegisterFunction(0x8000'0005, std::bind(&CPUIDEmu::Function_8000_0005h, this));
RegisterFunction(0x8000'0005, std::bind(&CPUIDEmu::Function_Reserved, this, _1));
#endif
// 0x8000'0006: L2 Cache identifiers
RegisterFunction(0x8000'0006, std::bind(&CPUIDEmu::Function_8000_0006h, this));
RegisterFunction(0x8000'0006, std::bind(&CPUIDEmu::Function_8000_0006h, this, _1));
// Advanced power management information
RegisterFunction(0x8000'0007, std::bind(&CPUIDEmu::Function_8000_0007h, this));
// 0x8000'0008: Virtual and physical address sizes
RegisterFunction(0x8000'0007, std::bind(&CPUIDEmu::Function_8000_0007h, this, _1));
// Virtual and physical address sizes
RegisterFunction(0x8000'0008, std::bind(&CPUIDEmu::Function_8000_0008h, this, _1));
// 0x8000'000A: SVM Revision
// 0x8000'0019: TLB 1GB page identifiers
// TLB 1GB page identifiers
RegisterFunction(0x8000'0019, std::bind(&CPUIDEmu::Function_8000_0019h, this, _1));
// 0x8000'001A: Performance optimization identifiers
// 0x8000'001B: Instruction based sampling identifiers
// 0x8000'001C: Lightweight profiling capabilities
// 0x8000'001D: Cache properties
#ifdef CPUID_AMD
// Deterministic cache parameters for each level
RegisterFunction(0x8000'001D, std::bind(&CPUIDEmu::Function_8000_001Dh, this, _1));
#endif
// 0x8000'001E: Extended APIC ID
// 0x8000'001F: AMD Secure Encryption
}
+27 -23
View File
@@ -3,6 +3,7 @@
#include <unordered_map>
#include <FEXCore/Core/CPUID.h>
#include <FEXCore/Config/Config.h>
#include <FEXCore/Utils/LogManager.h>
namespace FEXCore {
@@ -23,22 +24,20 @@ private:
public:
void Init(FEXCore::Context::Context *ctx);
FEXCore::CPUID::FunctionResults RunFunction(uint32_t Function, [[maybe_unused]] uint32_t Leaf) {
FEXCore::CPUID::FunctionResults RunFunction(uint32_t Function, uint32_t Leaf) {
auto Handler = FunctionHandlers.find(Function);
if (Handler == FunctionHandlers.end()) {
#ifndef NDEBUG
LogMan::Msg::E("Unhandled CPU ID function, 0x%x", Function);
#endif
return Function_Reserved();
return Function_Reserved(Leaf);
}
return Handler->second();
return Handler->second(Leaf);
}
private:
FEXCore::Context::Context *CTX;
FEX_CONFIG_OPT(Cores, THREADS);
using FunctionHandler = std::function<FEXCore::CPUID::FunctionResults()>;
using FunctionHandler = std::function<FEXCore::CPUID::FunctionResults(uint32_t Leaf)>;
void RegisterFunction(uint32_t Function, FunctionHandler Handler) {
FunctionHandlers[Function] = Handler;
}
@@ -46,21 +45,26 @@ private:
std::unordered_map<uint32_t, FunctionHandler> FunctionHandlers;
// Functions
FEXCore::CPUID::FunctionResults Function_0h();
FEXCore::CPUID::FunctionResults Function_01h();
FEXCore::CPUID::FunctionResults Function_02h();
FEXCore::CPUID::FunctionResults Function_06h();
FEXCore::CPUID::FunctionResults Function_07h();
FEXCore::CPUID::FunctionResults Function_15h();
FEXCore::CPUID::FunctionResults Function_8000_0000h();
FEXCore::CPUID::FunctionResults Function_8000_0001h();
FEXCore::CPUID::FunctionResults Function_8000_0002h();
FEXCore::CPUID::FunctionResults Function_8000_0003h();
FEXCore::CPUID::FunctionResults Function_8000_0004h();
FEXCore::CPUID::FunctionResults Function_8000_0005h();
FEXCore::CPUID::FunctionResults Function_8000_0006h();
FEXCore::CPUID::FunctionResults Function_8000_0007h();
FEXCore::CPUID::FunctionResults Function_Reserved();
FEXCore::CPUID::FunctionResults Function_0h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_01h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_02h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_04h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_06h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_07h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_0Dh(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_15h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_8000_0000h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_8000_0001h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_8000_0002h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_8000_0003h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_8000_0004h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_8000_0005h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_8000_0006h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_8000_0007h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_8000_0008h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_8000_0009h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_8000_0019h(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_8000_001Dh(uint32_t Leaf);
FEXCore::CPUID::FunctionResults Function_Reserved(uint32_t Leaf);
};
}
+3 -3
View File
@@ -61,7 +61,7 @@ namespace FEXCore {
}
}
LogMan::Throw::A(CompileThreadData->LocalIRCache.size() == 0, "Compile service must never have LocalIRCache");
LOGMAN_THROW_A(CompileThreadData->LocalIRCache.size() == 0, "Compile service must never have LocalIRCache");
CompileMutex.unlock();
}
@@ -124,7 +124,7 @@ namespace FEXCore {
// If we had a work item then work on it
if (Item) {
// Make sure it's not in lookup cache by accident
LogMan::Throw::A(CompileThreadData->LookupCache->FindBlock(Item->RIP) == 0, "Compile Service must never have entries in the LookupCache");
LOGMAN_THROW_A(CompileThreadData->LookupCache->FindBlock(Item->RIP) == 0, "Compile Service must never have entries in the LookupCache");
// Code isn't in cache, compile now
// Set our thread state's RIP
@@ -132,7 +132,7 @@ namespace FEXCore {
auto [CodePtr, IRList, DebugData, RAData, Generated, StartAddr, Length] = CTX->CompileCode(CompileThreadData.get(), Item->RIP);
LogMan::Throw::A(Generated == true, "Compile Service doesn't have IR Cache");
LOGMAN_THROW_A(Generated == true, "Compile Service doesn't have IR Cache");
if (!CodePtr) {
// XXX: We currently have the expectation that compile service code will be significantly smaller than regular thread's code
File diff suppressed because it is too large. Load diff
@@ -1,3 +1,5 @@
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/ArchHelpers/MContext.h"
#include "Interface/Core/Dispatcher/Arm64Dispatcher.h"
@@ -5,6 +7,7 @@
#include "Interface/Context/Context.h"
#include <FEXCore/Core/X86Enums.h>
#include <bit>
#include <cmath>
#include "aarch64/assembler-aarch64.h"
@@ -67,6 +70,7 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::
// We want to ensure that we are 16 byte aligned at the top of this loop
Align16B();
aarch64::Label FullLookup{};
aarch64::Label CallBlock{};
aarch64::Label LoopTop{};
aarch64::Label ExitSpillSRA{};
aarch64::Label ThreadPauseHandler{};
@@ -79,16 +83,19 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::
ldr(x2, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.rip)));
auto RipReg = x2;
if (!config.ExecuteBlocksWithCall) {
// L1 Cache
ldr(x0, &l_L1Ptr);
// L1 Cache
ldr(x0, &l_L1Ptr);
and_(x3, RipReg, LookupCache::L1_ENTRIES_MASK);
add(x0, x0, Operand(x3, Shift::LSL, 4));
ldp(x1, x0, MemOperand(x0));
cmp(x0, RipReg);
b(&FullLookup, Condition::ne);
br(x1);
and_(x3, RipReg, LookupCache::L1_ENTRIES_MASK);
add(x0, x0, Operand(x3, Shift::LSL, 4));
ldp(x3, x0, MemOperand(x0));
cmp(x0, RipReg);
b(&FullLookup, Condition::ne);
if (!config.ExecuteBlocksWithCall) {
br(x3);
} else {
b(&CallBlock);
}
// L1C check failed, do a full lookup
@@ -99,7 +106,7 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::
ldr(x0, &l_PagePtr);
// Mask the address by the virtual address size so we can check for aliases
if (__builtin_popcountl(VirtualMemorySize) == 1) {
if (std::popcount(VirtualMemorySize) == 1) {
and_(x3, RipReg, Thread->LookupCache->GetVirtualMemorySize() - 1);
}
else {
@@ -136,40 +143,37 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::
// If we've made it here then we have a real compiled block
{
// update L1 cache
ldr(x0, &l_L1Ptr);
and_(x1, RipReg, LookupCache::L1_ENTRIES_MASK);
add(x0, x0, Operand(x1, Shift::LSL, 4));
stp(x3, x2, MemOperand(x0));
// Jump to the block
if (!config.ExecuteBlocksWithCall) {
// update L1 cache
ldr(x0, &l_L1Ptr);
and_(x1, RipReg, LookupCache::L1_ENTRIES_MASK);
add(x0, x0, Operand(x1, Shift::LSL, 4));
stp(x3, x2, MemOperand(x0));
br(x3);
} else {
bind(&CallBlock);
mov(x0, STATE);
blr(x3);
}
}
if (config.ExecuteBlocksWithCall) {
// Interpreter continues execution here
if (CTX->GetGdbServerStatus()) {
// If we have a gdb server running then run in a less efficient mode that checks if we need to exit
// This happens when single stepping
if (CTX->GetGdbServerStatus()) {
// If we have a gdb server running then run in a less efficient mode that checks if we need to exit
// This happens when single stepping
static_assert(sizeof(CTX->Config.RunningMode) == 4, "This is expected to be size of 4");
ldr(x0, &l_CTX);
ldr(w0, MemOperand(x0, offsetof(FEXCore::Context::Context, Config.RunningMode)));
// If the value == 0 then branch to the top
cbz(x0, &LoopTop);
// Else we need to pause now
b(&ThreadPauseHandler);
}
else {
// Unconditionally loop to the top
// We will only stop on error when compiling a block or signal
b(&LoopTop);
static_assert(sizeof(CTX->Config.RunningMode) == 4, "This is expected to be size of 4");
ldr(x0, &l_CTX);
ldr(w0, MemOperand(x0, offsetof(FEXCore::Context::Context, Config.RunningMode)));
// If the value == 0 then branch to the top
cbz(x0, &LoopTop);
// Else we need to pause now
b(&ThreadPauseHandler);
} else {
// Unconditionally loop to the top
// We will only stop on error when compiling a block or signal
b(&LoopTop);
}
}
}
}
@@ -350,7 +354,7 @@ void InterpreterCore::CreateAsmDispatch(FEXCore::Context::Context *ctx, FEXCore:
DispatcherConfig config;
config.ExecuteBlocksWithCall = true;
Dispatcher = new Arm64Dispatcher(ctx, Thread, config);
Dispatcher = std::make_unique<Arm64Dispatcher>(ctx, Thread, config);
DispatchPtr = Dispatcher->DispatchPtr;
CallbackPtr = Dispatcher->CallbackPtr;
@@ -54,6 +54,7 @@ void Dispatcher::StoreThreadState(int Signal, void *ucontext) {
}
void Dispatcher::RestoreThreadState(void *ucontext) {
LOGMAN_THROW_A(!SignalFrames.empty(), "Trying to restore a signal frame when we don't have any");
uint64_t OldSP = SignalFrames.top();
SignalFrames.pop();
uintptr_t NewSP = OldSP;
@@ -105,10 +106,15 @@ bool Dispatcher::HandleGuestSignal(int Signal, void *info, void *ucontext, Guest
// Don't need this offset if we aren't going to be putting siginfo in to it
NewGuestSP -= 128;
if (GuestAction->sa_flags & SA_SIGINFO) {
// siginfo_t
siginfo_t *HostSigInfo = reinterpret_cast<siginfo_t*>(info);
if (GuestAction->sa_flags & SA_SIGINFO &&
!(HostSigInfo->si_code == SI_QUEUE || // If the siginfo comes from sigqueue or user then we don't need to check
HostSigInfo->si_code == SI_USER)) {
if (SRAEnabled) {
if (!IsAddressInJITCode(ArchHelpers::Context::GetPc(ucontext), false)) {
LogMan::Throw::A(!IsAddressInJITCode(ArchHelpers::Context::GetPc(ucontext), true), "Signals in dispatcher have unsynchronized context");
LOGMAN_THROW_A(!IsAddressInJITCode(ArchHelpers::Context::GetPc(ucontext), true), "Signals in dispatcher have unsynchronized context");
} else {
// We are in jit, SRA must be spilled
SpillSRA(ucontext);
@@ -117,11 +123,14 @@ bool Dispatcher::HandleGuestSignal(int Signal, void *info, void *ucontext, Guest
// Setup ucontext a bit
if (CTX->Config.Is64BitMode) {
NewGuestSP -= sizeof(FEXCore::x86_64::ucontext_t);
uint64_t UContextLocation = NewGuestSP;
NewGuestSP -= sizeof(siginfo_t);
uint64_t SigInfoLocation = NewGuestSP;
FEXCore::x86_64::ucontext_t *guest_uctx = reinterpret_cast<FEXCore::x86_64::ucontext_t*>(UContextLocation);
siginfo_t *guest_siginfo = reinterpret_cast<siginfo_t*>(SigInfoLocation);
// We have extended float information
guest_uctx->uc_flags |= FEXCore::x86_64::UC_FP_XSTATE;
@@ -129,6 +138,14 @@ bool Dispatcher::HandleGuestSignal(int Signal, void *info, void *ucontext, Guest
// Pointer to where the fpreg memory is
guest_uctx->uc_mcontext.fpregs = &guest_uctx->__fpregs_mem;
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_RIP] = Frame->State.rip;
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_EFL] = 0;
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_CSGSFS] = 0;
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_ERR] = 0;
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_TRAPNO] = Signal;
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_OLDMASK] = 0;
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_CR2] = 0;
#define COPY_REG(x) \
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_##x] = Frame->State.gregs[X86State::REG_##x];
COPY_REG(R8);
@@ -155,6 +172,7 @@ bool Dispatcher::HandleGuestSignal(int Signal, void *info, void *ucontext, Guest
// FCW store default
guest_uctx->__fpregs_mem.fcw = Frame->State.FCW;
guest_uctx->__fpregs_mem.ftw = Frame->State.FTW;
// Reconstruct FSW
guest_uctx->__fpregs_mem.fsw =
@@ -169,26 +187,121 @@ bool Dispatcher::HandleGuestSignal(int Signal, void *info, void *ucontext, Guest
guest_uctx->uc_stack.ss_sp = GuestStack->ss_sp;
guest_uctx->uc_stack.ss_size = GuestStack->ss_size;
// XXX: siginfo_t(RSI)
Frame->State.gregs[X86State::REG_RSI] = 0x4142434445460000;
// aarch64 and x86_64 siginfo_t matches. We can just copy this over
// SI_USER could also potentially have random data in it, needs to be bit perfect
// For guest faults we don't have a real way to reconstruct state to a real guest RIP
*guest_siginfo = *HostSigInfo;
Frame->State.gregs[X86State::REG_RSI] = SigInfoLocation;
Frame->State.gregs[X86State::REG_RDX] = UContextLocation;
}
else {
// XXX: 32bit Support
NewGuestSP -= sizeof(FEXCore::x86::ucontext_t);
uint64_t UContextLocation = 0; // NewGuestSP;
uint64_t UContextLocation = NewGuestSP;
NewGuestSP -= sizeof(FEXCore::x86::siginfo_t);
uint64_t SigInfoLocation = 0; // NewGuestSP;
uint64_t SigInfoLocation = NewGuestSP;
FEXCore::x86::ucontext_t *guest_uctx = reinterpret_cast<FEXCore::x86::ucontext_t*>(UContextLocation);
FEXCore::x86::siginfo_t *guest_siginfo = reinterpret_cast<FEXCore::x86::siginfo_t*>(SigInfoLocation);
// We have extended float information
guest_uctx->uc_flags |= FEXCore::x86::UC_FP_XSTATE;
// Pointer to where the fpreg memory is
guest_uctx->uc_mcontext.fpregs = static_cast<uint32_t>(reinterpret_cast<uint64_t>(&guest_uctx->__fpregs_mem));
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_GS] = Frame->State.gs;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_FS] = Frame->State.fs;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_ES] = Frame->State.es;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_DS] = Frame->State.ds;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_TRAPNO] = Signal;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_ERR] = 0;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_EIP] = Frame->State.rip;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_CS] = Frame->State.cs;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_EFL] = 0;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_UESP] = 0;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_SS] = Frame->State.ss;
#define COPY_REG(x) \
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_##x] = Frame->State.gregs[X86State::REG_##x];
COPY_REG(RDI);
COPY_REG(RSI);
COPY_REG(RBP);
COPY_REG(RBX);
COPY_REG(RDX);
COPY_REG(RAX);
COPY_REG(RCX);
COPY_REG(RSP);
#undef COPY_REG
// Copy float registers
memcpy(guest_uctx->__fpregs_mem._st, Frame->State.mm, sizeof(Frame->State.mm));
if (0) {
// XXX: Handle XMM
// memcpy(guest_uctx->__fpregs_mem._xmm, Frame->State.xmm, sizeof(Frame->State.xmm));
guest_uctx->__fpregs_mem.status = FEXCore::x86::fpstate_magic::MAGIC_XFPSTATE;
}
else {
guest_uctx->__fpregs_mem.status = FEXCore::x86::fpstate_magic::MAGIC_FPU;
}
// FCW store default
guest_uctx->__fpregs_mem.fcw = Frame->State.FCW;
guest_uctx->__fpregs_mem.ftw = Frame->State.FTW;
// Reconstruct FSW
guest_uctx->__fpregs_mem.fsw =
(Frame->State.flags[FEXCore::X86State::X87FLAG_TOP_LOC] << 11) |
(Frame->State.flags[FEXCore::X86State::X87FLAG_C0_LOC] << 8) |
(Frame->State.flags[FEXCore::X86State::X87FLAG_C1_LOC] << 9) |
(Frame->State.flags[FEXCore::X86State::X87FLAG_C2_LOC] << 10) |
(Frame->State.flags[FEXCore::X86State::X87FLAG_C3_LOC] << 14);
// Copy over signal stack information
guest_uctx->uc_stack.ss_flags = GuestStack->ss_flags;
guest_uctx->uc_stack.ss_sp = static_cast<uint32_t>(reinterpret_cast<uint64_t>(GuestStack->ss_sp));
guest_uctx->uc_stack.ss_size = GuestStack->ss_size;
// These three elements are in every siginfo
guest_siginfo->si_signo = HostSigInfo->si_signo;
guest_siginfo->si_errno = HostSigInfo->si_errno;
guest_siginfo->si_code = HostSigInfo->si_code;
switch (Signal) {
case SIGSEGV:
case SIGBUS:
// Macro expansion to get the si_addr
// Can't really give a real result here. Pull from the context for now
guest_siginfo->_sifields._sigfault.addr = Frame->State.rip;
break;
case SIGCHLD:
guest_siginfo->_sifields._sigchld.pid = HostSigInfo->si_pid;
guest_siginfo->_sifields._sigchld.uid = HostSigInfo->si_uid;
guest_siginfo->_sifields._sigchld.status = HostSigInfo->si_status;
guest_siginfo->_sifields._sigchld.utime = HostSigInfo->si_utime;
guest_siginfo->_sifields._sigchld.stime = HostSigInfo->si_stime;
break;
default:
// Hope for the best, most things just copy over
memcpy(&guest_siginfo->_sifields, &HostSigInfo->_sifields, sizeof(siginfo_t));
break;
}
NewGuestSP -= 4;
*(uint32_t*)NewGuestSP = UContextLocation;
NewGuestSP -= 4;
*(uint32_t*)NewGuestSP = SigInfoLocation;
NewGuestSP -= 4;
*(uint32_t*)NewGuestSP = Signal;
}
Frame->State.rip = reinterpret_cast<uint64_t>(GuestAction->sigaction_handler.sigaction);
}
else {
if (!CTX->Config.Is64BitMode) {
NewGuestSP -= 4;
*(uint32_t*)NewGuestSP = Signal;
}
Frame->State.rip = reinterpret_cast<uint64_t>(GuestAction->sigaction_handler.handler);
}
@@ -203,7 +316,7 @@ bool Dispatcher::HandleGuestSignal(int Signal, void *info, void *ucontext, Guest
else {
NewGuestSP -= 4;
*(uint32_t*)NewGuestSP = CTX->X86CodeGen.SignalReturn;
LogMan::Throw::A(CTX->X86CodeGen.SignalReturn < 0x1'0000'0000ULL, "This needs to be below 4GB");
LOGMAN_THROW_A(CTX->X86CodeGen.SignalReturn < 0x1'0000'0000ULL, "This needs to be below 4GB");
Frame->State.gregs[X86State::REG_RSP] = NewGuestSP;
}
@@ -237,7 +350,7 @@ bool Dispatcher::HandleSignalPause(int Signal, void *info, void *ucontext) {
FEXCore::Core::SignalEvent SignalReason = ThreadState->SignalReason.load();
auto Frame = ThreadState->CurrentFrame;
if (SignalReason == FEXCore::Core::SignalEvent::SIGNALEVENT_PAUSE) {
if (SignalReason == FEXCore::Core::SignalEvent::Pause) {
// Store our thread state so we can come back to this
StoreThreadState(Signal, ucontext);
@@ -247,7 +360,7 @@ bool Dispatcher::HandleSignalPause(int Signal, void *info, void *ucontext) {
} else {
if (SRAEnabled) {
// We are in non-jit, SRA is already spilled
LogMan::Throw::A(!IsAddressInJITCode(ArchHelpers::Context::GetPc(ucontext), true), "Signals in dispatcher have unsynchronized context");
LOGMAN_THROW_A(!IsAddressInJITCode(ArchHelpers::Context::GetPc(ucontext), true), "Signals in dispatcher have unsynchronized context");
}
ArchHelpers::Context::SetPc(ucontext, ThreadPauseHandlerAddress);
}
@@ -262,11 +375,11 @@ bool Dispatcher::HandleSignalPause(int Signal, void *info, void *ucontext) {
// We use this to track if it is safe to clear cache
++SignalHandlerRefCounter;
ThreadState->SignalReason.store(FEXCore::Core::SIGNALEVENT_NONE);
ThreadState->SignalReason.store(FEXCore::Core::SignalEvent::Nothing);
return true;
}
if (SignalReason == FEXCore::Core::SignalEvent::SIGNALEVENT_STOP) {
if (SignalReason == FEXCore::Core::SignalEvent::Stop) {
// Our thread is stopping
// We don't care about anything at this point
// Set the stack to our starting location when we entered the core and get out safely
@@ -282,23 +395,23 @@ bool Dispatcher::HandleSignalPause(int Signal, void *info, void *ucontext) {
} else {
if (SRAEnabled) {
// We are in non-jit, SRA is already spilled
LogMan::Throw::A(!IsAddressInJITCode(ArchHelpers::Context::GetPc(ucontext), true), "Signals in dispatcher have unsynchronized context");
LOGMAN_THROW_A(!IsAddressInJITCode(ArchHelpers::Context::GetPc(ucontext), true), "Signals in dispatcher have unsynchronized context");
}
ArchHelpers::Context::SetPc(ucontext, ThreadStopHandlerAddress);
}
ThreadState->SignalReason.store(FEXCore::Core::SIGNALEVENT_NONE);
ThreadState->SignalReason.store(FEXCore::Core::SignalEvent::Nothing);
return true;
}
if (SignalReason == FEXCore::Core::SignalEvent::SIGNALEVENT_RETURN) {
if (SignalReason == FEXCore::Core::SignalEvent::Return) {
RestoreThreadState(ucontext);
// Ref count our faults
// We use this to track if it is safe to clear cache
--SignalHandlerRefCounter;
ThreadState->SignalReason.store(FEXCore::Core::SIGNALEVENT_NONE);
ThreadState->SignalReason.store(FEXCore::Core::SignalEvent::Nothing);
return true;
}
@@ -306,14 +419,14 @@ bool Dispatcher::HandleSignalPause(int Signal, void *info, void *ucontext) {
}
uint64_t Dispatcher::GetCompileBlockPtr() {
using ClassPtrType = uintptr_t (FEXCore::Context::Context::*)(FEXCore::Core::CpuStateFrame *, uint64_t);
using ClassPtrType = void (FEXCore::Context::Context::*)(FEXCore::Core::CpuStateFrame *, uint64_t);
union PtrCast {
ClassPtrType ClassPtr;
uintptr_t Data;
};
PtrCast CompileBlockPtr;
CompileBlockPtr.ClassPtr = &FEXCore::Context::Context::CompileBlock;
CompileBlockPtr.ClassPtr = &FEXCore::Context::Context::CompileBlockJit;
return CompileBlockPtr.Data;
}
@@ -327,7 +440,7 @@ void Dispatcher::RemoveCodeBuffer(uint8_t* start_to_remove) {
}
}
bool Dispatcher::IsAddressInJITCode(uint64_t Address, bool IncludeDispatcher) {
bool Dispatcher::IsAddressInJITCode(uint64_t Address, bool IncludeDispatcher) const {
for (auto [start, end] : CodeBuffers) {
if (Address >= start && Address < end) {
return true;
@@ -18,6 +18,7 @@ struct DispatcherConfig {
class Dispatcher {
public:
virtual ~Dispatcher() = default;
CPUBackend::AsmDispatch DispatchPtr;
CPUBackend::JITCallback CallbackPtr;
FEXCore::Context::Context::IntCallbackReturn ReturnPtr;
@@ -53,8 +54,8 @@ public:
void RemoveCodeBuffer(uint8_t* start);
bool IsAddressInJITCode(uint64_t Address, bool IncludeDispatcher = true);
bool IsAddressInDispatcher(uint64_t Address) {
bool IsAddressInJITCode(uint64_t Address, bool IncludeDispatcher = true) const;
bool IsAddressInDispatcher(uint64_t Address) const {
return Address >= Start && Address < End;
}
@@ -81,4 +82,4 @@ private:
std::vector<std::tuple<uint64_t, uint64_t>> CodeBuffers; // Start, End
};
}
}
@@ -1,3 +1,5 @@
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/Dispatcher/X86Dispatcher.h"
#include "Interface/Core/Interpreter/InterpreterClass.h"
@@ -12,7 +14,9 @@ static constexpr size_t MAX_DISPATCHER_CODE_SIZE = 4096;
X86Dispatcher::X86Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::InternalThreadState *Thread, DispatcherConfig &config)
: Dispatcher(ctx, Thread)
, Xbyak::CodeGenerator(MAX_DISPATCHER_CODE_SIZE) {
, Xbyak::CodeGenerator(MAX_DISPATCHER_CODE_SIZE,
FEXCore::Allocator::mmap(nullptr, MAX_DISPATCHER_CODE_SIZE, PROT_READ | PROT_WRITE | PROT_EXEC, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0),
nullptr) {
using namespace Xbyak;
using namespace Xbyak::util;
@@ -66,6 +70,7 @@ X86Dispatcher::X86Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::Inte
Label LoopTop;
Label FullLookup;
Label CallBlock;
Label NoBlock;
Label ExitBlock;
Label ThreadPauseHandler;
@@ -77,17 +82,20 @@ X86Dispatcher::X86Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::Inte
// Load our RIP
mov(rdx, qword [STATE + offsetof(FEXCore::Core::CPUState, rip)]);
if (!config.ExecuteBlocksWithCall)
{
// L1 Cache
mov(r13, Thread->LookupCache->GetL1Pointer());
mov(rax, rdx);
// L1 Cache
mov(r13, Thread->LookupCache->GetL1Pointer());
mov(rax, rdx);
and_(rax, LookupCache::L1_ENTRIES_MASK);
shl(rax, 4);
cmp(qword[r13 + rax + 8], rdx);
jne(FullLookup);
and_(rax, LookupCache::L1_ENTRIES_MASK);
shl(rax, 4);
cmp(qword[r13 + rax + 8], rdx);
jne(FullLookup);
if (!config.ExecuteBlocksWithCall) {
jmp(qword[r13 + rax + 0]);
} else {
mov(rax, qword[r13 + rax + 0]);
jmp(CallBlock);
}
L(FullLookup);
@@ -122,19 +130,19 @@ X86Dispatcher::X86Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::Inte
je(NoBlock);
// Update L1
if (config.ExecuteBlocksWithCall) {
mov(r13, Thread->LookupCache->GetL1Pointer());
mov(rcx, rdx);
and_(rcx, LookupCache::L1_ENTRIES_MASK);
shl(rcx, 1);
mov(qword[r13 + rcx*8 + 8], rdx);
mov(qword[r13 + rcx*8 + 0], rax);
}
mov(r13, Thread->LookupCache->GetL1Pointer());
mov(rcx, rdx);
and_(rcx, LookupCache::L1_ENTRIES_MASK);
shl(rcx, 1);
mov(qword[r13 + rcx*8 + 8], rdx);
mov(qword[r13 + rcx*8 + 0], rax);
// Real block if we made it here
if (!config.ExecuteBlocksWithCall) {
jmp(rax);
} else {
L(CallBlock);
mov(rdi, STATE);
call(rax);
@@ -177,19 +185,10 @@ X86Dispatcher::X86Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::Inte
{
L(NoBlock);
using ClassPtrType = uintptr_t (FEXCore::Context::Context::*)(FEXCore::Core::CpuStateFrame *, uint64_t);
union PtrCast {
ClassPtrType ClassPtr;
uintptr_t Data;
};
PtrCast Ptr;
Ptr.ClassPtr = &FEXCore::Context::Context::CompileBlock;
// {rdi, rsi, rdx}
mov(rdi, reinterpret_cast<uint64_t>(CTX));
mov(rsi, STATE);
mov(rax, Ptr.Data);
mov(rax, GetCompileBlockPtr());
call(rax);
@@ -271,7 +270,7 @@ X86Dispatcher::X86Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::Inte
{
ReturnPtr = getCurr<FEXCore::Context::Context::IntCallbackReturn>();
// using CallbackReturn = __attribute__((naked)) void(*)(FEXCore::Core::InternalThreadState *Thread, volatile void *Host_RSP);
// using CallbackReturn = FEX_NAKED void(*)(FEXCore::Core::InternalThreadState *Thread, volatile void *Host_RSP);
// rdi = thread
// rsi = rsp
@@ -298,12 +297,12 @@ X86Dispatcher::X86Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::Inte
#if ENABLE_JITSYMBOLS
std::string Name = "Dispatch_" + std::to_string(::gettid());
CTX->Symbols.Register(Start, End-Start, Name);
CTX->Symbols.Register(reinterpret_cast<void*>(Start), End-Start, Name);
#endif
}
X86Dispatcher::~X86Dispatcher() {
FEXCore::Allocator::munmap(top_, MAX_DISPATCHER_CODE_SIZE);
}
#ifdef _M_X86_64
@@ -312,7 +311,7 @@ void InterpreterCore::CreateAsmDispatch(FEXCore::Context::Context *ctx, FEXCore:
DispatcherConfig config;
config.ExecuteBlocksWithCall = true;
Dispatcher = new X86Dispatcher(ctx, Thread, config);
Dispatcher = std::make_unique<X86Dispatcher>(ctx, Thread, config);
DispatchPtr = Dispatcher->DispatchPtr;
CallbackPtr = Dispatcher->CallbackPtr;
@@ -2,6 +2,8 @@
#include "Interface/Core/Dispatcher/Dispatcher.h"
#include <FEXCore/Utils/Allocator.h>
#define XBYAK64
#include <xbyak/xbyak.h>
@@ -14,4 +16,4 @@ class X86Dispatcher final : public Dispatcher, public Xbyak::CodeGenerator {
virtual ~X86Dispatcher() override;
};
}
}
+151 -85
View File
@@ -15,12 +15,18 @@ $end_info$
#include <FEXCore/Core/X86Enums.h>
#include <FEXCore/Debug/X86Tables.h>
#include <FEXCore/Utils/LogManager.h>
#include <set>
#include <sys/mman.h>
namespace FEXCore::Frontend {
#include "Interface/Core/VSyscall/VSyscall.inc"
using namespace FEXCore::X86Tables;
static uint32_t MapModRMToReg(uint8_t REX, uint8_t bits, bool HighBits, bool HasREX, bool HasXMM, bool HasMM, uint8_t InvalidOffset = 16) {
constexpr std::array<uint64_t, 16> GPRIndexes = {
using GPRArray = std::array<uint32_t, 16>;
static constexpr GPRArray GPRIndexes = {
// Classical ordering?
FEXCore::X86State::REG_RAX,
FEXCore::X86State::REG_RCX,
@@ -40,7 +46,7 @@ static uint32_t MapModRMToReg(uint8_t REX, uint8_t bits, bool HighBits, bool Has
FEXCore::X86State::REG_R15,
};
constexpr std::array<uint64_t, 16> GPR8BitHighIndexes = {
static constexpr GPRArray GPR8BitHighIndexes = {
// Classical ordering?
FEXCore::X86State::REG_RAX,
FEXCore::X86State::REG_RCX,
@@ -60,7 +66,7 @@ static uint32_t MapModRMToReg(uint8_t REX, uint8_t bits, bool HighBits, bool Has
FEXCore::X86State::REG_R15,
};
constexpr std::array<uint64_t, 16> XMMIndexes = {
static constexpr GPRArray XMMIndexes = {
FEXCore::X86State::REG_XMM_0,
FEXCore::X86State::REG_XMM_1,
FEXCore::X86State::REG_XMM_2,
@@ -79,7 +85,7 @@ static uint32_t MapModRMToReg(uint8_t REX, uint8_t bits, bool HighBits, bool Has
FEXCore::X86State::REG_XMM_15,
};
constexpr std::array<uint64_t, 16> MMIndexes = {
static constexpr GPRArray MMIndexes = {
FEXCore::X86State::REG_MM_0,
FEXCore::X86State::REG_MM_1,
FEXCore::X86State::REG_MM_2,
@@ -98,7 +104,7 @@ static uint32_t MapModRMToReg(uint8_t REX, uint8_t bits, bool HighBits, bool Has
FEXCore::X86State::REG_INVALID
};
const std::array<uint64_t, 16> *GPRs = &GPRIndexes;
const GPRArray *GPRs = &GPRIndexes;
if (HasXMM) {
GPRs = &XMMIndexes;
}
@@ -118,19 +124,29 @@ static uint32_t MapModRMToReg(uint8_t REX, uint8_t bits, bool HighBits, bool Has
}
Decoder::Decoder(FEXCore::Context::Context *ctx)
: CTX {ctx} {
DecodedBuffer.resize(DefaultDecodedBufferSize);
: CTX {ctx}
, OSABI { ctx->SyscallHandler ? ctx->SyscallHandler->GetOSABI() : FEXCore::HLE::SyscallOSABI::OS_UNKNOWN } {
// Using mmap is a start-up time optimization
// Take advantage of page faulting to reduce startup time for minimal runtime cost
DecodedBuffer =
reinterpret_cast<FEXCore::X86Tables::DecodedInst *>(
FEXCore::Allocator::mmap(0, sizeof(FEXCore::X86Tables::DecodedInst) * DefaultDecodedBufferSize,
PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0));
}
Decoder::~Decoder() {
FEXCore::Allocator::munmap(DecodedBuffer, sizeof(FEXCore::X86Tables::DecodedInst) * DefaultDecodedBufferSize);
}
uint8_t Decoder::ReadByte() {
uint8_t Byte = InstStream[InstructionSize];
LogMan::Throw::A(InstructionSize < MAX_INST_SIZE, "Max instruction size exceeded!");
LOGMAN_THROW_A(InstructionSize < MAX_INST_SIZE, "Max instruction size exceeded!");
Instruction[InstructionSize] = Byte;
InstructionSize++;
return Byte;
}
uint8_t Decoder::PeekByte(uint8_t Offset) {
uint8_t Decoder::PeekByte(uint8_t Offset) const {
uint8_t Byte = InstStream[InstructionSize + Offset];
return Byte;
}
@@ -141,7 +157,7 @@ uint64_t Decoder::ReadData(uint8_t Size) {
}
if (Size > sizeof(uint64_t)) {
LogMan::Msg::A("Unknown data size to read");
LOGMAN_MSG_A("Unknown data size to read");
return 0;
}
@@ -196,9 +212,9 @@ void Decoder::DecodeModRM_16(X86Tables::DecodedOperand *Operand, X86Tables::ModR
}
}
Operand->TypeSIB.Type = DecodedOperand::TYPE_SIB;
Operand->TypeSIB.Scale = 1;
Operand->TypeSIB.Offset = Literal;
Operand->Type = DecodedOperand::OpType::SIB;
Operand->Data.SIB.Scale = 1;
Operand->Data.SIB.Offset = Literal;
// Only called when ModRM.mod != 0b11
struct Encodings {
@@ -211,34 +227,34 @@ void Decoder::DecodeModRM_16(X86Tables::DecodedOperand *Operand, X86Tables::ModR
{FEXCore::X86State::REG_RBX, FEXCore::X86State::REG_RDI},
{FEXCore::X86State::REG_RBP, FEXCore::X86State::REG_RSI},
{FEXCore::X86State::REG_RBP, FEXCore::X86State::REG_RDI},
{FEXCore::X86State::REG_RSI, 255},
{FEXCore::X86State::REG_RDI, 255},
{255, 255},
{FEXCore::X86State::REG_RBX, 255},
{FEXCore::X86State::REG_RSI, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_RDI, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_INVALID, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_RBX, FEXCore::X86State::REG_INVALID},
// Mod = 0b01
{FEXCore::X86State::REG_RBX, FEXCore::X86State::REG_RSI},
{FEXCore::X86State::REG_RBX, FEXCore::X86State::REG_RDI},
{FEXCore::X86State::REG_RBP, FEXCore::X86State::REG_RSI},
{FEXCore::X86State::REG_RBP, FEXCore::X86State::REG_RDI},
{FEXCore::X86State::REG_RSI, 255},
{FEXCore::X86State::REG_RDI, 255},
{FEXCore::X86State::REG_RBP, 255},
{FEXCore::X86State::REG_RBX, 255},
{FEXCore::X86State::REG_RSI, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_RDI, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_RBP, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_RBX, FEXCore::X86State::REG_INVALID},
// Mod = 0b10
{FEXCore::X86State::REG_RBX, FEXCore::X86State::REG_RSI},
{FEXCore::X86State::REG_RBX, FEXCore::X86State::REG_RDI},
{FEXCore::X86State::REG_RBP, FEXCore::X86State::REG_RSI},
{FEXCore::X86State::REG_RBP, FEXCore::X86State::REG_RDI},
{FEXCore::X86State::REG_RSI, 255},
{FEXCore::X86State::REG_RDI, 255},
{FEXCore::X86State::REG_RBP, 255},
{FEXCore::X86State::REG_RBX, 255},
{FEXCore::X86State::REG_RSI, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_RDI, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_RBP, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_RBX, FEXCore::X86State::REG_INVALID},
}};
uint8_t LookupIndex = ModRM.mod << 3 | ModRM.rm;
auto it = Lookup[LookupIndex];
Operand->TypeSIB.Base = it.Base;
Operand->TypeSIB.Index = it.Index;
Operand->Data.SIB.Base = it.Base;
Operand->Data.SIB.Index = it.Index;
}
void Decoder::DecodeModRM_64(X86Tables::DecodedOperand *Operand, X86Tables::ModRMDecoded ModRM) {
@@ -276,21 +292,21 @@ void Decoder::DecodeModRM_64(X86Tables::DecodedOperand *Operand, X86Tables::ModR
}
// SIB
Operand->TypeSIB.Type = DecodedOperand::TYPE_SIB;
Operand->TypeSIB.Scale = 1 << SIB.scale;
Operand->Type = DecodedOperand::OpType::SIB;
Operand->Data.SIB.Scale = 1 << SIB.scale;
// The invalid encoding types are described at Table 1-12. "promoted nsigned is always non-zero"
Operand->TypeSIB.Index = MapModRMToReg(DecodeInst->Flags & DecodeFlags::FLAG_REX_XGPR_X ? 1 : 0, SIB.index, false, false, false, false, 0b100);
Operand->TypeSIB.Base = MapModRMToReg(DecodeInst->Flags & DecodeFlags::FLAG_REX_XGPR_B ? 1 : 0, SIB.base, false, false, false, false, ModRM.mod == 0 ? 0b101 : 16);
Operand->Data.SIB.Index = MapModRMToReg(DecodeInst->Flags & DecodeFlags::FLAG_REX_XGPR_X ? 1 : 0, SIB.index, false, false, false, false, 0b100);
Operand->Data.SIB.Base = MapModRMToReg(DecodeInst->Flags & DecodeFlags::FLAG_REX_XGPR_B ? 1 : 0, SIB.base, false, false, false, false, ModRM.mod == 0 ? 0b101 : 16);
uint64_t Literal {0};
LogMan::Throw::A(Displacement <= 4, "Number of bytes should be <= 4 for literal src");
LOGMAN_THROW_A(Displacement <= 4, "Number of bytes should be <= 4 for literal src");
Literal = ReadData(Displacement);
if (Displacement == 1) {
Literal = static_cast<int8_t>(Literal);
}
Operand->TypeSIB.Offset = Literal;
Operand->Data.SIB.Offset = Literal;
}
else if (ModRM.mod == 0) {
// Explained in Table 1-14. "Operand Addressing Using ModRM and SIB Bytes"
@@ -299,13 +315,13 @@ void Decoder::DecodeModRM_64(X86Tables::DecodedOperand *Operand, X86Tables::ModR
uint32_t Literal;
Literal = ReadData(4);
Operand->TypeRIPLiteral.Type = DecodedOperand::TYPE_RIP_RELATIVE;
Operand->TypeRIPLiteral.Literal.u = Literal;
Operand->Type = DecodedOperand::OpType::RIPRelative;
Operand->Data.RIPLiteral.Value.u = Literal;
}
else {
// Register-direct addressing
Operand->TypeGPR.Type = DecodedOperand::TYPE_GPR_DIRECT;
Operand->TypeGPR.GPR = MapModRMToReg(DecodeInst->Flags & DecodeFlags::FLAG_REX_XGPR_B ? 1 : 0, ModRM.rm, false, false, false, false);
Operand->Type = DecodedOperand::OpType::GPRDirect;
Operand->Data.GPR.GPR = MapModRMToReg(DecodeInst->Flags & DecodeFlags::FLAG_REX_XGPR_B ? 1 : 0, ModRM.rm, false, false, false, false);
}
}
else {
@@ -317,9 +333,9 @@ void Decoder::DecodeModRM_64(X86Tables::DecodedOperand *Operand, X86Tables::ModR
}
Displacement = DisplacementSize;
Operand->TypeGPRIndirect.Type = DecodedOperand::TYPE_GPR_INDIRECT;
Operand->TypeGPRIndirect.GPR = MapModRMToReg(DecodeInst->Flags & DecodeFlags::FLAG_REX_XGPR_B ? 1 : 0, ModRM.rm, false, false, false, false);
Operand->TypeGPRIndirect.Displacement = Literal;
Operand->Type = DecodedOperand::OpType::GPRIndirect;
Operand->Data.GPRIndirect.GPR = MapModRMToReg(DecodeInst->Flags & DecodeFlags::FLAG_REX_XGPR_B ? 1 : 0, ModRM.rm, false, false, false, false);
Operand->Data.GPRIndirect.Displacement = Literal;
}
}
@@ -343,7 +359,7 @@ bool Decoder::NormalOp(FEXCore::X86Tables::X86InstInfo const *Info, uint16_t Op)
return false;
}
LogMan::Throw::A(!(Info->Type >= FEXCore::X86Tables::TYPE_GROUP_1 && Info->Type <= FEXCore::X86Tables::TYPE_GROUP_P),
LOGMAN_THROW_A(!(Info->Type >= FEXCore::X86Tables::TYPE_GROUP_1 && Info->Type <= FEXCore::X86Tables::TYPE_GROUP_P),
"Group Ops should have been decoded before this!");
uint8_t DestSize{};
@@ -459,22 +475,25 @@ bool Decoder::NormalOp(FEXCore::X86Tables::X86InstInfo const *Info, uint16_t Op)
if (HAS_NON_XMM_SUBFLAG(Info->Flags, FEXCore::X86Tables::InstFlags::FLAGS_SF_DST_RAX) ||
HAS_NON_XMM_SUBFLAG(Info->Flags, FEXCore::X86Tables::InstFlags::FLAGS_SF_DST_RDX)) {
// Some instructions hardcode their destination as RAX
CurrentDest->TypeGPR.Type = DecodedOperand::TYPE_GPR;
CurrentDest->TypeGPR.HighBits = false;
CurrentDest->TypeGPR.GPR = HAS_NON_XMM_SUBFLAG(Info->Flags, FEXCore::X86Tables::InstFlags::FLAGS_SF_DST_RAX) ? FEXCore::X86State::REG_RAX : FEXCore::X86State::REG_RDX;
CurrentDest->Type = DecodedOperand::OpType::GPR;
CurrentDest->Data.GPR.HighBits = false;
CurrentDest->Data.GPR.GPR = HAS_NON_XMM_SUBFLAG(Info->Flags, FEXCore::X86Tables::InstFlags::FLAGS_SF_DST_RAX) ? FEXCore::X86State::REG_RAX : FEXCore::X86State::REG_RDX;
CurrentDest = &DecodeInst->Src[0];
}
if (HAS_NON_XMM_SUBFLAG(Info->Flags, FEXCore::X86Tables::InstFlags::FLAGS_SF_REX_IN_BYTE)) {
LogMan::Throw::A(!HasMODRM, "This instruction shouldn't have ModRM!");
LOGMAN_THROW_A(!HasMODRM, "This instruction shouldn't have ModRM!");
// If the REX is in the byte that means the lower nibble of the OP contains the destination GPR
// This also means that the destination is always a GPR on these ones
// ADDITIONALLY:
// If there is a REX prefix then that allows extended GPR usage
CurrentDest->TypeGPR.Type = DecodedOperand::TYPE_GPR;
DecodeInst->Dest.TypeGPR.HighBits = (Is8BitDest && !HasREX && (Op & 0b111) >= 0b100) || HasHighXMM;
CurrentDest->TypeGPR.GPR = MapModRMToReg(DecodeInst->Flags & DecodeFlags::FLAG_REX_XGPR_B ? 1 : 0, Op & 0b111, Is8BitDest, HasREX, false, false);
CurrentDest->Type = DecodedOperand::OpType::GPR;
DecodeInst->Dest.Data.GPR.HighBits = (Is8BitDest && !HasREX && (Op & 0b111) >= 0b100) || HasHighXMM;
CurrentDest->Data.GPR.GPR = MapModRMToReg(DecodeInst->Flags & DecodeFlags::FLAG_REX_XGPR_B ? 1 : 0, Op & 0b111, Is8BitDest, HasREX, false, false);
if (CurrentDest->Data.GPR.GPR == FEXCore::X86State::REG_INVALID)
return false;
}
uint8_t Bytes = Info->MoreBytes;
@@ -497,55 +516,63 @@ bool Decoder::NormalOp(FEXCore::X86Tables::X86InstInfo const *Info, uint16_t Op)
ModRM.Hex = DecodeInst->ModRM;
// Decode the GPR source first
GPR.TypeGPR.Type = DecodedOperand::TYPE_GPR;
GPR.TypeGPR.HighBits = (GPR8Bit && ModRM.reg >= 0b100 && !HasREX) || HasHighXMM;
GPR.TypeGPR.GPR = MapModRMToReg(DecodeInst->Flags & DecodeFlags::FLAG_REX_XGPR_R ? 1 : 0, ModRM.reg, GPR8Bit, HasREX, HasXMMGPR, HasMMGPR);
GPR.Type = DecodedOperand::OpType::GPR;
GPR.Data.GPR.HighBits = (GPR8Bit && ModRM.reg >= 0b100 && !HasREX) || HasHighXMM;
GPR.Data.GPR.GPR = MapModRMToReg(DecodeInst->Flags & DecodeFlags::FLAG_REX_XGPR_R ? 1 : 0, ModRM.reg, GPR8Bit, HasREX, HasXMMGPR, HasMMGPR);
if (GPR.Data.GPR.GPR == FEXCore::X86State::REG_INVALID)
return false;
// ModRM.mod == 0b11 == Register
// ModRM.Mod != 0b11 == Register-direct addressing
if (ModRM.mod == 0b11) {
NonGPR.TypeGPR.Type = DecodedOperand::TYPE_GPR;
NonGPR.TypeGPR.HighBits = (NonGPR8Bit && ModRM.rm >= 0b100 && !HasREX) || HasHighXMM;
NonGPR.TypeGPR.GPR = MapModRMToReg(DecodeInst->Flags & DecodeFlags::FLAG_REX_XGPR_B ? 1 : 0, ModRM.rm, NonGPR8Bit, HasREX, HasXMMNonGPR, HasMMNonGPR);
NonGPR.Type = DecodedOperand::OpType::GPR;
NonGPR.Data.GPR.HighBits = (NonGPR8Bit && ModRM.rm >= 0b100 && !HasREX) || HasHighXMM;
NonGPR.Data.GPR.GPR = MapModRMToReg(DecodeInst->Flags & DecodeFlags::FLAG_REX_XGPR_B ? 1 : 0, ModRM.rm, NonGPR8Bit, HasREX, HasXMMNonGPR, HasMMNonGPR);
if (NonGPR.Data.GPR.GPR == FEXCore::X86State::REG_INVALID)
return false;
}
else {
auto Disp = DecodeModRMs_Disp[Has16BitAddressing];
(this->*Disp)(&NonGPR, ModRM);
}
return true;
};
size_t CurrentSrc = 0;
if (Info->Flags & FEXCore::X86Tables::InstFlags::FLAGS_MODRM) {
if (Info->Flags & FEXCore::X86Tables::InstFlags::FLAGS_SF_MOD_DST) {
ModRMOperand(DecodeInst->Src[CurrentSrc], DecodeInst->Dest, HasXMMSrc, HasXMMDst, HasMMSrc, HasMMDst, Is8BitSrc, Is8BitDest);
if (!ModRMOperand(DecodeInst->Src[CurrentSrc], DecodeInst->Dest, HasXMMSrc, HasXMMDst, HasMMSrc, HasMMDst, Is8BitSrc, Is8BitDest))
return false;
}
else {
ModRMOperand(DecodeInst->Dest, DecodeInst->Src[CurrentSrc], HasXMMDst, HasXMMSrc, HasMMDst, HasMMSrc, Is8BitDest, Is8BitSrc);
if (!ModRMOperand(DecodeInst->Dest, DecodeInst->Src[CurrentSrc], HasXMMDst, HasXMMSrc, HasMMDst, HasMMSrc, Is8BitDest, Is8BitSrc))
return false;
}
++CurrentSrc;
}
if (HAS_NON_XMM_SUBFLAG(Info->Flags, FEXCore::X86Tables::InstFlags::FLAGS_SF_SRC_RAX)) {
DecodeInst->Src[CurrentSrc].TypeGPR.Type = DecodedOperand::TYPE_GPR;
DecodeInst->Src[CurrentSrc].TypeGPR.HighBits = false;
DecodeInst->Src[CurrentSrc].TypeGPR.GPR = FEXCore::X86State::REG_RAX;
DecodeInst->Src[CurrentSrc].Type = DecodedOperand::OpType::GPR;
DecodeInst->Src[CurrentSrc].Data.GPR.HighBits = false;
DecodeInst->Src[CurrentSrc].Data.GPR.GPR = FEXCore::X86State::REG_RAX;
++CurrentSrc;
}
else if (HAS_NON_XMM_SUBFLAG(Info->Flags, FEXCore::X86Tables::InstFlags::FLAGS_SF_SRC_RCX)) {
DecodeInst->Src[CurrentSrc].TypeGPR.Type = DecodedOperand::TYPE_GPR;
DecodeInst->Src[CurrentSrc].TypeGPR.HighBits = false;
DecodeInst->Src[CurrentSrc].TypeGPR.GPR = FEXCore::X86State::REG_RCX;
DecodeInst->Src[CurrentSrc].Type = DecodedOperand::OpType::GPR;
DecodeInst->Src[CurrentSrc].Data.GPR.HighBits = false;
DecodeInst->Src[CurrentSrc].Data.GPR.GPR = FEXCore::X86State::REG_RCX;
++CurrentSrc;
}
if (Bytes != 0) {
LogMan::Throw::A(Bytes <= 8, "Number of bytes should be <= 8 for literal src");
LOGMAN_THROW_A(Bytes <= 8, "Number of bytes should be <= 8 for literal src");
DecodeInst->Src[CurrentSrc].TypeLiteral.Size = Bytes;
DecodeInst->Src[CurrentSrc].Data.Literal.Size = Bytes;
uint64_t Literal {0};
Literal = ReadData(Bytes);
uint64_t Literal = ReadData(Bytes);
if ((Info->Flags & FEXCore::X86Tables::InstFlags::FLAGS_SRC_SEXT) ||
(DecodeFlags::GetSizeDstFlags(DecodeInst->Flags) == DecodeFlags::SIZE_64BIT && Info->Flags & FEXCore::X86Tables::InstFlags::FLAGS_SRC_SEXT64BIT)) {
@@ -558,15 +585,15 @@ bool Decoder::NormalOp(FEXCore::X86Tables::X86InstInfo const *Info, uint16_t Op)
else {
Literal = static_cast<int32_t>(Literal);
}
DecodeInst->Src[CurrentSrc].TypeLiteral.Size = DestSize;
DecodeInst->Src[CurrentSrc].Data.Literal.Size = DestSize;
}
Bytes = 0;
DecodeInst->Src[CurrentSrc].TypeLiteral.Type = DecodedOperand::TYPE_LITERAL;
DecodeInst->Src[CurrentSrc].TypeLiteral.Literal = Literal;
DecodeInst->Src[CurrentSrc].Type = DecodedOperand::OpType::Literal;
DecodeInst->Src[CurrentSrc].Data.Literal.Value = Literal;
}
LogMan::Throw::A(Bytes == 0, "Inst at 0x%lx: 0x%04x '%s' Had an instruction of size %d with %d remaining", DecodeInst->PC, DecodeInst->OP, DecodeInst->TableInfo->Name, InstructionSize, Bytes);
LOGMAN_THROW_A(Bytes == 0, "Inst at 0x%lx: 0x%04x '%s' Had an instruction of size %d with %d remaining", DecodeInst->PC, DecodeInst->OP, DecodeInst->TableInfo->Name, InstructionSize, Bytes);
DecodeInst->InstSize = InstructionSize;
return true;
}
@@ -591,7 +618,7 @@ bool Decoder::NormalOpHeader(FEXCore::X86Tables::X86InstInfo const *Info, uint16
return false;
}
LogMan::Throw::A(Info->Type != FEXCore::X86Tables::TYPE_REX_PREFIX, "REX PREFIX should have been decoded before this!");
LOGMAN_THROW_A(Info->Type != FEXCore::X86Tables::TYPE_REX_PREFIX, "REX PREFIX should have been decoded before this!");
if (Info->Type >= FEXCore::X86Tables::TYPE_GROUP_1 &&
Info->Type <= FEXCore::X86Tables::TYPE_GROUP_11) {
@@ -647,7 +674,7 @@ bool Decoder::NormalOpHeader(FEXCore::X86Tables::X86InstInfo const *Info, uint16
3,
};
uint8_t Field = RegToField[ModRM.reg];
LogMan::Throw::A(Field != 255, "Invalid field selected!");
LOGMAN_THROW_A(Field != 255, "Invalid field selected!");
LocalOp = (Field << 3) | ModRM.rm;
return NormalOp(&SecondModRMTableOps[LocalOp], LocalOp);
@@ -681,7 +708,10 @@ bool Decoder::NormalOpHeader(FEXCore::X86Tables::X86InstInfo const *Info, uint16
uint8_t Byte2 = ReadByte();
pp = Byte2 & 0b11;
map_select = Byte1 & 0b11111;
LogMan::Throw::A(map_select >= 1 && map_select <= 3, "We don't understand a map_select of: %d", map_select);
if (!(map_select >= 1 && map_select <= 3)) {
LogMan::Msg::E("We don't understand a map_select of: %d", map_select);
return false;
}
}
uint16_t VEXOp = ReadByte();
@@ -729,6 +759,8 @@ bool Decoder::DecodeInstruction(uint64_t PC) {
DecodeInst->PC = PC;
for(;;) {
if (InstructionSize >= MAX_INST_SIZE)
return false;
uint8_t Op = ReadByte();
switch (Op) {
case 0x0F: {// Escape Op
@@ -879,7 +911,7 @@ bool Decoder::DecodeInstruction(uint64_t PC) {
auto Info = &FEXCore::X86Tables::BaseOps[Op];
if (Info->Type == FEXCore::X86Tables::TYPE_REX_PREFIX) {
LogMan::Throw::A(CTX->Config.Is64BitMode, "Got REX prefix in 32bit mode");
LOGMAN_THROW_A(CTX->Config.Is64BitMode, "Got REX prefix in 32bit mode");
DecodeInst->Flags |= DecodeFlags::FLAG_REX_PREFIX;
// Widening displacement
@@ -909,6 +941,10 @@ bool Decoder::DecodeInstruction(uint64_t PC) {
}
if (DecodeInst->Dest.IsGPR()) {
assert(DecodeInst->Dest.Data.GPR.GPR != FEXCore::X86State::REG_INVALID);
}
return true;
}
@@ -918,7 +954,7 @@ void Decoder::BranchTargetInMultiblockRange() {
// If the RIP setting is conditional AND within our symbol range then it can be considered for multiblock
uint64_t TargetRIP = 0;
uint8_t GPRSize = CTX->Config.Is64BitMode ? 8 : 4;
const uint8_t GPRSize = CTX->GetGPRSize();
bool Conditional = true;
switch (DecodeInst->OP) {
@@ -928,19 +964,23 @@ void Decoder::BranchTargetInMultiblockRange() {
// auto RIPOffset = LoadSource(Op, Op->Src[0], Op->Flags);
// auto RIPTargetConst = _Constant(Op->PC + Op->InstSize);
// Target offset is PC + InstSize + Literal
LogMan::Throw::A(DecodeInst->Src[0].TypeNone.Type == DecodedOperand::TYPE_LITERAL, "Had wrong operand type");
TargetRIP = DecodeInst->PC + DecodeInst->InstSize + DecodeInst->Src[0].TypeLiteral.Literal;
LOGMAN_THROW_A(DecodeInst->Src[0].IsLiteral(), "Had wrong operand type");
TargetRIP = DecodeInst->PC + DecodeInst->InstSize + DecodeInst->Src[0].Data.Literal.Value;
break;
}
case 0xE9:
case 0xEB: // Both are unconditional JMP instructions
LogMan::Throw::A(DecodeInst->Src[0].TypeNone.Type == DecodedOperand::TYPE_LITERAL, "Had wrong operand type");
TargetRIP = DecodeInst->PC + DecodeInst->InstSize + DecodeInst->Src[0].TypeLiteral.Literal;
LOGMAN_THROW_A(DecodeInst->Src[0].IsLiteral(), "Had wrong operand type");
TargetRIP = DecodeInst->PC + DecodeInst->InstSize + DecodeInst->Src[0].Data.Literal.Value;
Conditional = false;
break;
case 0xE8: // Call - Immediate target, We don't want to inline calls
if (ExternalBranches) {
ExternalBranches->insert(DecodeInst->PC + DecodeInst->InstSize);
}
[[fallthrough]];
case 0xC2: // RET imm
case 0xC3: // RET
case 0xE8: // Call - Immediate target, We don't want to inline calls
default:
return;
break;
@@ -970,9 +1010,32 @@ void Decoder::BranchTargetInMultiblockRange() {
BlocksToDecode.find(TargetRIP) == BlocksToDecode.end()) {
BlocksToDecode.emplace(TargetRIP);
}
} else {
if (ExternalBranches) {
ExternalBranches->insert(TargetRIP);
}
}
}
const uint8_t *Decoder::AdjustAddrForSpecialRegion(uint8_t const* _InstStream, uint64_t EntryPoint, uint64_t RIP) {
constexpr uint64_t VSyscall_Base = 0xFFFF'FFFF'FF60'0000ULL;
constexpr uint64_t VSyscall_End = VSyscall_Base + 0x1000;
if (OSABI == FEXCore::HLE::SyscallOSABI::OS_LINUX64 &&
RIP >= VSyscall_Base &&
RIP < VSyscall_End) {
// VSyscall
// This doesn't exist on AArch64 and on x86_64 hosts this is emulated with faults to a region mapped with --xp permissions
// Offset 0: vgettimeofday
// Offset 0x400: vtime
// Offset 0x800: vgetcpu
uint64_t Offset = RIP - VSyscall_Base;
return VSyscallData + Offset;
}
return _InstStream - EntryPoint + RIP;
}
bool Decoder::DecodeInstructionsAtEntry(uint8_t const* _InstStream, uint64_t PC) {
Blocks.clear();
BlocksToDecode.clear();
@@ -993,7 +1056,7 @@ bool Decoder::DecodeInstructionsAtEntry(uint8_t const* _InstStream, uint64_t PC)
// If we don't have symbols available then we become a bit optimistic about multiblock ranges
if (!SymbolAvailable) {
// If we don't have a symbol available then assume all branches are valid for multiblock
SymbolMaxAddress = ~0ULL;
SymbolMaxAddress = SectionMaxAddress;
SymbolMinAddress = EntryPoint;
}
@@ -1016,14 +1079,17 @@ bool Decoder::DecodeInstructionsAtEntry(uint8_t const* _InstStream, uint64_t PC)
uint64_t BlockStartOffset = DecodedSize;
// Do a bit of pointer math to figure out where we are in code
InstStream = _InstStream - EntryPoint + RIPToDecode;
InstStream = AdjustAddrForSpecialRegion(_InstStream, EntryPoint, RIPToDecode);
while (1) {
ErrorDuringDecoding = !DecodeInstruction(RIPToDecode + PCOffset);
if (ErrorDuringDecoding) {
LogMan::Msg::D("Couldn't Decode something at 0x%lx, Started at 0x%lx", PC + PCOffset, PC);
LogMan::Throw::A(Blocks.size() != 1, "Decode Error in entry block");
if (Blocks.size() == 1) {
return false;
}
LOGMAN_THROW_A(Blocks.size() != 1, "Decode Error in entry block");
CurrentBlockDecoding.HasInvalidInstruction = true;
if (ErrorDuringDecoding && Blocks.size() != 1) {
@@ -1057,7 +1123,7 @@ bool Decoder::DecodeInstructionsAtEntry(uint8_t const* _InstStream, uint64_t PC)
}
if (DecodedSize >= CTX->Config.MaxInstPerBlock ||
DecodedSize >= DecodedBuffer.size()) {
DecodedSize >= DefaultDecodedBufferSize) {
break;
}
@@ -1074,7 +1140,7 @@ bool Decoder::DecodeInstructionsAtEntry(uint8_t const* _InstStream, uint64_t PC)
// Copy over only the number of instructions we decoded
CurrentBlockDecoding.NumInstructions = BlockNumberOfInstructions;
CurrentBlockDecoding.DecodedInstructions = &DecodedBuffer.at(BlockStartOffset);
CurrentBlockDecoding.DecodedInstructions = &DecodedBuffer[BlockStartOffset];
}
+13 -3
View File
@@ -1,6 +1,8 @@
#pragma once
#include <FEXCore/Debug/X86Tables.h>
#include <FEXCore/HLE/SyscallHandler.h>
#include <array>
#include <cstdint>
#include <utility>
@@ -24,31 +26,35 @@ public:
};
Decoder(FEXCore::Context::Context *ctx);
~Decoder();
bool DecodeInstructionsAtEntry(uint8_t const* InstStream, uint64_t PC);
std::vector<DecodedBlocks> const *GetDecodedBlocks() {
std::vector<DecodedBlocks> const *GetDecodedBlocks() const {
return &Blocks;
}
uint64_t DecodedMinAddress {};
uint64_t DecodedMaxAddress {~0ULL};
void SetSectionMaxAddress(uint64_t v) { SectionMaxAddress = v; }
void SetExternalBranches(std::set<uint64_t> *v) { ExternalBranches = v; }
private:
FEXCore::Context::Context *CTX;
const FEXCore::HLE::SyscallOSABI OSABI{};
bool DecodeInstruction(uint64_t PC);
void BranchTargetInMultiblockRange();
uint8_t ReadByte();
uint8_t PeekByte(uint8_t Offset);
uint8_t PeekByte(uint8_t Offset) const;
uint64_t ReadData(uint8_t Size);
void SkipBytes(uint8_t Size) { InstructionSize += Size; }
bool NormalOp(FEXCore::X86Tables::X86InstInfo const *Info, uint16_t Op);
bool NormalOpHeader(FEXCore::X86Tables::X86InstInfo const *Info, uint16_t Op);
static constexpr size_t DefaultDecodedBufferSize = 0x10000;
std::vector<FEXCore::X86Tables::DecodedInst> DecodedBuffer;
FEXCore::X86Tables::DecodedInst *DecodedBuffer{};
size_t DecodedSize {};
uint8_t const *InstStream;
@@ -65,10 +71,12 @@ private:
uint64_t MaxCondBranchBackwards {~0ULL};
uint64_t SymbolMaxAddress {};
uint64_t SymbolMinAddress {~0ULL};
uint64_t SectionMaxAddress {~0ULL};
std::vector<DecodedBlocks> Blocks;
std::set<uint64_t> BlocksToDecode;
std::set<uint64_t> HasBlocks;
std::set<uint64_t> *ExternalBranches {nullptr};
// ModRM rm decoding
using DecodeModRMPtr = void (FEXCore::Frontend::Decoder::*)(X86Tables::DecodedOperand *Operand, X86Tables::ModRMDecoded ModRM);
@@ -79,5 +87,7 @@ private:
&FEXCore::Frontend::Decoder::DecodeModRM_64,
&FEXCore::Frontend::Decoder::DecodeModRM_16,
};
const uint8_t *AdjustAddrForSpecialRegion(uint8_t const* _InstStream, uint64_t EntryPoint, uint64_t RIP);
};
}
+78 -94
View File
@@ -15,15 +15,19 @@ $end_info$
#include <optional>
#include "Common/NetStream.h"
#include "Common/SoftFloat.h"
#include <FEXCore/Utils/CompilerDefs.h>
#include <FEXCore/Utils/LogManager.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netdb.h>
#include <string.h>
#include <cstring>
#include <fcntl.h>
#include <unistd.h>
#include <fmt/format.h>
#include <fstream>
#include <netdb.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <unistd.h>
#include "GdbServer.h"
#include <FEXCore/Core/CodeLoader.h>
@@ -34,20 +38,20 @@ namespace FEXCore
void GdbServer::Break(int signal) {
std::lock_guard lk(sendMutex);
if (!CommsStream) {
return;
}
std::ostringstream ss;
ss << "S" << std::setfill('0') << std::setw(2) << std::hex << signal;
if (CommsStream)
SendPacket(*CommsStream, ss.str());
const auto str = fmt::format("S{:02x}", signal);
SendPacket(*CommsStream, str);
}
GdbServer::GdbServer(FEXCore::Context::Context *ctx) : CTX(ctx) {
ctx->CustomExitHandler = [this](uint64_t ThreadId, FEXCore::Context::ExitReason ExitReason) {
Context::SetExitHandler(ctx, [this](uint64_t ThreadId, FEXCore::Context::ExitReason ExitReason) {
if (ExitReason == FEXCore::Context::ExitReason::EXIT_DEBUG) {
this->Break(SIGTRAP);
}
};
});
// This is a total hack as there is currently no way to resume once hitting a segfault
// But it's semi-useful for debugging.
@@ -60,12 +64,12 @@ GdbServer::GdbServer(FEXCore::Context::Context *ctx) : CTX(ctx) {
usleep(100000);
return true;
});
}, true);
StartThread();
}
static int calculateChecksum(std::string &packet) {
static int calculateChecksum(const std::string &packet) {
unsigned char checksum = 0;
for (const char &c : packet) {
checksum += c;
@@ -99,11 +103,9 @@ static std::string encodeHex(unsigned char *data, size_t length) {
}
static std::string getThreadName(uint32_t ThreadID) {
std::fstream fs;
std::ostringstream ThreadFile;
ThreadFile << "/proc/" << getpid() << "/task/" << ThreadID << "/comm";
const auto ThreadFile = fmt::format("/proc/{}/task/{}/comm", getpid(), ThreadID);
std::fstream fs(ThreadFile, std::fstream::in | std::fstream::binary);
fs.open(ThreadFile.str(), std::fstream::in | std::fstream::binary);
if (fs.is_open()) {
std::string ThreadName;
fs >> ThreadName;
@@ -135,7 +137,7 @@ std::string GdbServer::ReadPacket(std::iostream &stream) {
switch(c) {
case '$': // start of packet
if (packet.size() != 0)
LogMan::Msg::E("Dropping unexpected data: \"%s\"", packet.c_str());
LogMan::Msg::EFmt("Dropping unexpected data: \"{}\"", packet);
// clear any existing data, must have been a mistake.
packet = std::string();
@@ -156,7 +158,7 @@ std::string GdbServer::ReadPacket(std::iostream &stream) {
if (calculateChecksum(packet) == expected_checksum) {
return packet;
} else {
LogMan::Msg::E("Received Invalid Packet: $%s#%02x %c%c", packet.c_str(), expected_checksum);
LogMan::Msg::EFmt("Received Invalid Packet: ${}#{:02x}", packet, expected_checksum);
}
break;
}
@@ -169,10 +171,10 @@ std::string GdbServer::ReadPacket(std::iostream &stream) {
return "";
}
static std::string escapePacket(std::string packet) {
static std::string escapePacket(const std::string& packet) {
std::ostringstream ss;
for(auto &c : packet) {
for(const auto &c : packet) {
switch (c) {
case '$':
case '#':
@@ -191,13 +193,11 @@ static std::string escapePacket(std::string packet) {
return ss.str();
}
void GdbServer::SendPacket(std::ostream &stream, std::string packet) {
auto escaped = escapePacket(packet);
std::ostringstream ss;
void GdbServer::SendPacket(std::ostream &stream, const std::string& packet) {
const auto escaped = escapePacket(packet);
const auto str = fmt::format("${}#{:02x}", escaped, calculateChecksum(escaped));
ss << '$' << escaped << '#';
ss << std::setfill('0') << std::setw(2) << std::hex << (int)calculateChecksum(escaped);
stream << ss.str() << std::flush;
stream << str << std::flush;
}
void GdbServer::SendACK(std::ostream &stream, bool NACK) {
@@ -218,7 +218,7 @@ void GdbServer::SendACK(std::ostream &stream, bool NACK) {
}
}
struct __attribute__((packed)) GDBContextDefinition {
struct FEX_PACKED GDBContextDefinition {
uint64_t gregs[16];
uint64_t rip;
uint32_t eflags;
@@ -279,7 +279,7 @@ std::string GdbServer::readRegs() {
return encodeHex((unsigned char *)&GDB, sizeof(GDBContextDefinition));
}
GdbServer::HandledPacketType GdbServer::readReg(std::string& packet) {
GdbServer::HandledPacketType GdbServer::readReg(const std::string& packet) {
size_t addr;
auto ss = std::istringstream(packet);
ss.get(); // Drop first letter
@@ -357,7 +357,7 @@ GdbServer::HandledPacketType GdbServer::readReg(std::string& packet) {
return {encodeHex((unsigned char *)(&Empty), sizeof(uint32_t)), HandledPacketType::TYPE_ACK};
}
LogMan::Msg::E("Unknown GDB register 0x%lx", addr);
LogMan::Msg::EFmt("Unknown GDB register 0x{:x}", addr);
return {"E00", HandledPacketType::TYPE_ACK};
}
@@ -462,7 +462,7 @@ std::string buildTargetXML() {
return xml.str();
}
GdbServer::HandledPacketType GdbServer::handleXfer(std::string &packet) {
GdbServer::HandledPacketType GdbServer::handleXfer(const std::string &packet) {
std::string object;
std::string rw;
std::string annex;
@@ -548,10 +548,9 @@ GdbServer::HandledPacketType GdbServer::handleXfer(std::string &packet) {
static size_t CheckMemMapping(uint64_t Address, size_t Size) {
uint64_t AddressEnd = Address + Size;
std::fstream fs;
fs.open("/proc/self/maps", std::fstream::in | std::fstream::binary);
std::fstream fs("/proc/self/maps", std::fstream::in | std::fstream::binary);
std::string Line;
while (std::getline(fs, Line)) {
if (fs.eof()) break;
uint64_t Begin, End;
@@ -568,32 +567,29 @@ static size_t CheckMemMapping(uint64_t Address, size_t Size) {
}
}
fs.close();
return 0;
}
GdbServer::HandledPacketType GdbServer::handleProgramOffsets() {
std::fstream fs;
fs.open("/proc/self/maps", std::fstream::in | std::fstream::binary);
std::fstream fs("/proc/self/maps", std::fstream::in | std::fstream::binary);
std::string Line;
std::string const &RuntimeExecutable = Filename();
while (std::getline(fs, Line)) {
uint64_t Begin, End;
char Filename[255];
if (sscanf(Line.c_str(), "%lx-%lx %*c%*c%*c%*c %*x %*x:%*x %*d%s", &Begin, &End, Filename) == 3) {
if (RuntimeExecutable == Filename) {
std::ostringstream ss;
ss << "Text=" << std::hex << Begin << ";Data=" << std::hex << Begin << ";Bss=" << std::hex << Begin;
ss << std::flush;
return {ss.str(), HandledPacketType::TYPE_ACK};
auto str = fmt::format("Text={:x};Data={:x};Bss={:x}", Begin, Begin, Begin);
return {std::move(str), HandledPacketType::TYPE_ACK};
}
}
}
fs.close();
return {"Text=0;Data=0;Bss=0", HandledPacketType::TYPE_ACK};
}
GdbServer::HandledPacketType GdbServer::handleMemory(std::string &packet) {
GdbServer::HandledPacketType GdbServer::handleMemory(const std::string &packet) {
bool write;
size_t addr;
size_t length;
@@ -634,8 +630,8 @@ GdbServer::HandledPacketType GdbServer::handleMemory(std::string &packet) {
}
GdbServer::HandledPacketType GdbServer::handleQuery(std::string &packet) {
auto match = [&](const char *str) -> bool { return packet.rfind(str, 0) == 0; };
GdbServer::HandledPacketType GdbServer::handleQuery(const std::string &packet) {
const auto match = [&](const char *str) -> bool { return packet.rfind(str, 0) == 0; };
if (match("qSupported")) {
return {"PacketSize=5000;xmlRegisters=i386;qXfer:exec-file:read+;qXfer:features:read+;", HandledPacketType::TYPE_ACK};
@@ -693,8 +689,8 @@ GdbServer::HandledPacketType GdbServer::handleQuery(std::string &packet) {
return {"", HandledPacketType::TYPE_UNKNOWN};
}
GdbServer::HandledPacketType GdbServer::handleV(std::string& packet) {
auto match = [&](std::string str) -> std::optional<std::istringstream> {
GdbServer::HandledPacketType GdbServer::handleV(const std::string& packet) {
const auto match = [&](const std::string& str) -> std::optional<std::istringstream> {
if (packet.rfind(str, 0) == 0) {
auto ss = std::istringstream(packet);
ss.seekg(str.size());
@@ -703,18 +699,11 @@ GdbServer::HandledPacketType GdbServer::handleV(std::string& packet) {
return std::nullopt;
};
auto F = [](int result) {
std::ostringstream ss;
ss << "F" << std::hex << result;
return ss.str(); };
auto F_error = [&]() {
std::ostringstream ss;
ss << "F-1," << std::hex << errno;
return ss.str(); };
auto F_data = [&](int result, std::string data) {
std::ostringstream ss;
ss << "F" << std::hex << result << ";" << data;
return ss.str(); };
const auto F = [](int result) { return fmt::format("F{:x}", result); };
const auto F_error = [] { return fmt::format("F-1,{:x}", errno); };
const auto F_data = [](int result, const std::string& data) {
return fmt::format("F{:x};{}", result, data);
};
std::optional<std::istringstream> ss;
if((ss = match("vFile:open:"))) {
@@ -736,11 +725,11 @@ GdbServer::HandledPacketType GdbServer::handleV(std::string& packet) {
return {F(pid == 0 ? 0 : -1), HandledPacketType::TYPE_ACK}; // Only support the common filesystem
}
if((ss = match("vFile:close:"))) {
int fd;
*ss >> std::hex >> fd;
close(fd);
return {F(0), HandledPacketType::TYPE_ACK};
}
int fd;
*ss >> std::hex >> fd;
close(fd);
return {F(0), HandledPacketType::TYPE_ACK};
}
if((ss = match("vFile:pread:"))) {
int fd, count, offset;
@@ -777,7 +766,7 @@ GdbServer::HandledPacketType GdbServer::handleV(std::string& packet) {
}
if (ss->fail()) {
return {"E00", HandledPacketType::TYPE_ACK};
return {"E00", HandledPacketType::TYPE_ACK};
}
switch (action) {
@@ -787,27 +776,25 @@ GdbServer::HandledPacketType GdbServer::handleV(std::string& packet) {
}
case 's': {
CTX->Step();
SendPacketPair({"OK", HandledPacketType::TYPE_ACK});
std::ostringstream ss;
ss << "T05thread:" << std::setfill('0') << std::setw(2) << std::hex << getpid() << ";core:2c;";
SendPacketPair({ss.str(), HandledPacketType::TYPE_ACK});
SendPacketPair({"OK", HandledPacketType::TYPE_ACK});
auto str = fmt::format("T05thread:{:02x};core:2c;", getpid());
SendPacketPair({std::move(str), HandledPacketType::TYPE_ACK});
return {"OK", HandledPacketType::TYPE_ACK};
}
case 't':
// This thread isn't part of the thread pool
CTX->Stop(false /* Ignore current thread */);
return {"OK", HandledPacketType::TYPE_ACK};
return {"OK", HandledPacketType::TYPE_ACK};
default:
return {"E00", HandledPacketType::TYPE_ACK};
return {"E00", HandledPacketType::TYPE_ACK};
}
}
return {"", HandledPacketType::TYPE_ACK};
return {"", HandledPacketType::TYPE_ACK};
}
GdbServer::HandledPacketType GdbServer::handleThreadOp(std::string &packet) {
auto match = [&](const char *str) -> bool { return packet.rfind(str, 0) == 0; };
GdbServer::HandledPacketType GdbServer::handleThreadOp(const std::string &packet) {
const auto match = [&](const char *str) -> bool { return packet.rfind(str, 0) == 0; };
if (match("Hc")) {
// Sets thread to this ID for stepping
@@ -823,7 +810,7 @@ GdbServer::HandledPacketType GdbServer::handleThreadOp(std::string &packet) {
if (match("Hg")) {
// Sets thread for "other" operations
auto ss = std::istringstream(packet);
ss.seekg(std::string("Hg").size());
ss.seekg(std::string_view("Hg").size());
ss >> std::hex >> CurrentDebuggingThread;
// This must return quick otherwise IDA complains
@@ -834,7 +821,7 @@ GdbServer::HandledPacketType GdbServer::handleThreadOp(std::string &packet) {
return {"", HandledPacketType::TYPE_UNKNOWN};
}
GdbServer::HandledPacketType GdbServer::handleBreakpoint(std::string &packet) {
GdbServer::HandledPacketType GdbServer::handleBreakpoint(const std::string &packet) {
auto ss = std::istringstream(packet);
bool Set{};
@@ -850,17 +837,15 @@ GdbServer::HandledPacketType GdbServer::handleBreakpoint(std::string &packet) {
return {"OK", HandledPacketType::TYPE_ACK};
}
GdbServer::HandledPacketType GdbServer::ProcessPacket(std::string &packet) {
GdbServer::HandledPacketType GdbServer::ProcessPacket(const std::string &packet) {
switch (packet[0]) {
case '?': {
// Indicates the reason that the thread has stopped
// Behaviour changes if the target is in non-stop mode
// Binja doesn't support S response here
//return {"S00", HandledPacketType::TYPE_ACK};
std::ostringstream ss;
ss << "T00thread:" << std::setfill('0') << std::setw(2) << std::hex << getpid() << ";core:2c;";
return {ss.str(), HandledPacketType::TYPE_ACK};
auto str = fmt::format("T00thread:{:02x};core:2c;", getpid());
return {std::move(str), HandledPacketType::TYPE_ACK};
}
case 'g':
return {readRegs(), HandledPacketType::TYPE_ACK};
@@ -890,14 +875,14 @@ GdbServer::HandledPacketType GdbServer::ProcessPacket(std::string &packet) {
}
}
void GdbServer::SendPacketPair(HandledPacketType response) {
void GdbServer::SendPacketPair(const HandledPacketType& response) {
std::lock_guard lk(sendMutex);
if (response.TypeResponse == HandledPacketType::TYPE_ACK ||
response.TypeResponse == HandledPacketType::TYPE_ONLYACK) {
SendACK(*CommsStream, false);
}
else if (response.TypeResponse == HandledPacketType::TYPE_NACK ||
response.TypeResponse == HandledPacketType::TYPE_ONLYNACK) {
response.TypeResponse == HandledPacketType::TYPE_ONLYNACK) {
SendACK(*CommsStream, true);
}
@@ -905,8 +890,8 @@ void GdbServer::SendPacketPair(HandledPacketType response) {
SendPacket(*CommsStream, "");
}
else if (response.TypeResponse != HandledPacketType::TYPE_ONLYNACK &&
response.TypeResponse != HandledPacketType::TYPE_ONLYACK &&
response.TypeResponse != HandledPacketType::TYPE_NONE) {
response.TypeResponse != HandledPacketType::TYPE_ONLYACK &&
response.TypeResponse != HandledPacketType::TYPE_NONE) {
SendPacket(*CommsStream, response.Response);
}
}
@@ -927,7 +912,7 @@ void GdbServer::GdbServerLoop() {
response = ProcessPacket(packet);
SendPacketPair(response);
if (response.TypeResponse == HandledPacketType::TYPE_UNKNOWN) {
LogMan::Msg::D("Unknown packet %s", packet.c_str());
LogMan::Msg::DFmt("Unknown packet {}", packet);
}
break;
}
@@ -943,19 +928,18 @@ void GdbServer::GdbServerLoop() {
break;
case '\x03': { // ASCII EOT
CTX->Pause();
std::ostringstream ss;
ss << "T02thread:" << std::setfill('0') << std::setw(2) << std::hex << getpid() << ";core:2c;";
SendPacketPair({ss.str(), HandledPacketType::TYPE_ACK});
auto str = fmt::format("T02thread:{:02x};core:2c;", getpid());
SendPacketPair({std::move(str), HandledPacketType::TYPE_ACK});
break;
}
default:
LogMan::Msg::D("GdbServer: Unexpected byte %c (%02x)", c, c);
LogMan::Msg::DFmt("GdbServer: Unexpected byte {} ({:02x})", static_cast<char>(c), c);
}
}
{
std::lock_guard lk(sendMutex);
CommsStream.release();
CommsStream.reset();
}
}
}
@@ -1003,7 +987,7 @@ std::unique_ptr<std::iostream> GdbServer::OpenSocket() {
// Block until a connection arrives
LogMan::Msg::I("GdbServer, waiting for connection on localhost:8086");
LogMan::Msg::IFmt("GdbServer, waiting for connection on localhost:8086");
listen(sockfd, 1);
new_fd = accept(sockfd, (struct sockaddr *)&their_addr, &addr_size);
+10 -10
View File
@@ -30,7 +30,7 @@ private:
std::unique_ptr<std::iostream> OpenSocket();
void StartThread();
std::string ReadPacket(std::iostream &stream);
void SendPacket(std::ostream &stream, std::string packet);
void SendPacket(std::ostream &stream, const std::string& packet);
void SendACK(std::ostream &stream, bool NACK);
@@ -47,18 +47,18 @@ private:
ResponseType TypeResponse{};
};
void SendPacketPair(HandledPacketType packetPair);
HandledPacketType ProcessPacket(std::string &packet);
HandledPacketType handleQuery(std::string &packet);
HandledPacketType handleXfer(std::string &packet);
HandledPacketType handleMemory(std::string &packet);
HandledPacketType handleV(std::string& packet);
HandledPacketType handleThreadOp(std::string &packet);
HandledPacketType handleBreakpoint(std::string &packet);
void SendPacketPair(const HandledPacketType& packetPair);
HandledPacketType ProcessPacket(const std::string &packet);
HandledPacketType handleQuery(const std::string &packet);
HandledPacketType handleXfer(const std::string &packet);
HandledPacketType handleMemory(const std::string &packet);
HandledPacketType handleV(const std::string& packet);
HandledPacketType handleThreadOp(const std::string &packet);
HandledPacketType handleBreakpoint(const std::string &packet);
HandledPacketType handleProgramOffsets();
std::string readRegs();
HandledPacketType readReg(std::string& packet);
HandledPacketType readReg(const std::string& packet);
FEXCore::Context::Context *CTX;
std::unique_ptr<FEXCore::Threads::Thread> gdbServerThread;
@@ -1,6 +1,5 @@
#pragma once
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/InternalThreadState.h"
#include "Interface/Core/Dispatcher/Dispatcher.h"
@@ -22,9 +21,8 @@ using DestMapType = std::vector<uint32_t>;
class InterpreterCore final : public CPUBackend {
public:
explicit InterpreterCore(FEXCore::Context::Context *ctx, FEXCore::Core::InternalThreadState *Thread, bool CompileThread);
~InterpreterCore() override;
std::string GetName() override { return "Interpreter"; }
void *CompileCode(FEXCore::IR::IRListView const *IR, FEXCore::Core::DebugData *DebugData, FEXCore::IR::RegisterAllocationData *RAData) override;
void *CompileCode(uint64_t Entry, FEXCore::IR::IRListView const *IR, FEXCore::Core::DebugData *DebugData, FEXCore::IR::RegisterAllocationData *RAData) override;
void *MapRegion(void* HostPtr, uint64_t, uint64_t) override { return HostPtr; }
@@ -46,7 +44,7 @@ private:
template<typename Res>
Res GetSrc(void* SSAData, IR::OrderedNodeWrapper Src);
Dispatcher *Dispatcher{};
std::unique_ptr<Dispatcher> Dispatcher{};
};
}
@@ -31,7 +31,7 @@ static void InterpreterExecution(FEXCore::Core::CpuStateFrame *Frame) {
auto LocalEntry = Thread->LocalIRCache.find(Thread->CurrentFrame->State.rip);
InterpreterOps::InterpretIR(Thread, LocalEntry->second.IR.get(), LocalEntry->second.DebugData.get());
InterpreterOps::InterpretIR(Thread, Thread->CurrentFrame->State.rip, LocalEntry->second.IR.get(), LocalEntry->second.DebugData.get());
}
bool InterpreterCore::HandleSIGBUS(int Signal, void *info, void *ucontext) {
@@ -78,6 +78,18 @@ bool InterpreterCore::HandleSIGBUS(int Signal, void *info, void *ucontext) {
return false;
}
}
else if ((Instr & FEXCore::ArchHelpers::Arm64::LDAXR_MASK) == FEXCore::ArchHelpers::Arm64::LDAXR_INST) { // LDAXR*
uint64_t BytesToSkip = FEXCore::ArchHelpers::Arm64::HandleAtomicLoadstoreExclusive(ucontext, info);
if (BytesToSkip) {
// Skip this instruction now
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) + BytesToSkip);
return true;
}
else {
LogMan::Msg::E("Unhandled JIT SIGBUS LDAXR: PC: %p Instruction: 0x%08x\n", PC, PC[0]);
return false;
}
}
}
return false;
}
@@ -93,12 +105,12 @@ InterpreterCore::InterpreterCore(FEXCore::Context::Context *ctx, FEXCore::Core::
CTX->SignalDelegation->RegisterHostSignalHandler(SignalDelegator::SIGNAL_FOR_PAUSE, [](FEXCore::Core::InternalThreadState *Thread, int Signal, void *info, void *ucontext) -> bool {
InterpreterCore *Core = reinterpret_cast<InterpreterCore*>(Thread->CPUBackend.get());
return Core->Dispatcher->HandleSignalPause(Signal, info, ucontext);
});
}, true);
CTX->SignalDelegation->RegisterHostSignalHandler(SIGBUS, [](FEXCore::Core::InternalThreadState *Thread, int Signal, void *info, void *ucontext) -> bool {
InterpreterCore *Core = reinterpret_cast<InterpreterCore*>(Thread->CPUBackend.get());
return Core->HandleSIGBUS(Signal, info, ucontext);
});
}, true);
auto GuestSignalHandler = [](FEXCore::Core::InternalThreadState *Thread, int Signal, void *info, void *ucontext, GuestSigAction *GuestAction, stack_t *GuestStack) -> bool {
InterpreterCore *Core = reinterpret_cast<InterpreterCore*>(Thread->CPUBackend.get());
@@ -111,18 +123,12 @@ InterpreterCore::InterpreterCore(FEXCore::Context::Context *ctx, FEXCore::Core::
}
}
InterpreterCore::~InterpreterCore() {
delete Dispatcher;
}
void *InterpreterCore::CompileCode([[maybe_unused]] FEXCore::IR::IRListView const *IR, [[maybe_unused]] FEXCore::Core::DebugData *DebugData, FEXCore::IR::RegisterAllocationData *RAData) {
void *InterpreterCore::CompileCode(uint64_t Entry, [[maybe_unused]] FEXCore::IR::IRListView const *IR, [[maybe_unused]] FEXCore::Core::DebugData *DebugData, FEXCore::IR::RegisterAllocationData *RAData) {
return reinterpret_cast<void*>(InterpreterExecution);
}
FEXCore::CPU::CPUBackend *CreateInterpreterCore(FEXCore::Context::Context *ctx, FEXCore::Core::InternalThreadState *Thread, bool CompileThread) {
return new InterpreterCore(ctx, Thread, CompileThread);
std::unique_ptr<CPUBackend> CreateInterpreterCore(FEXCore::Context::Context *ctx, FEXCore::Core::InternalThreadState *Thread, bool CompileThread) {
return std::make_unique<InterpreterCore>(ctx, Thread, CompileThread);
}
}
@@ -1,5 +1,7 @@
#pragma once
#include <memory>
namespace FEXCore::Context {
struct Context;
}
@@ -11,6 +13,6 @@ namespace FEXCore::Core {
namespace FEXCore::CPU {
class CPUBackend;
FEXCore::CPU::CPUBackend *CreateInterpreterCore(FEXCore::Context::Context *ctx, FEXCore::Core::InternalThreadState *Thread, bool CompileThread);
std::unique_ptr<CPUBackend> CreateInterpreterCore(FEXCore::Context::Context *ctx, FEXCore::Core::InternalThreadState *Thread, bool CompileThread);
}
File diff suppressed because it is too large. Load diff
@@ -36,7 +36,7 @@ namespace FEXCore::CPU {
class InterpreterOps {
public:
static void InterpretIR(FEXCore::Core::InternalThreadState *Thread, FEXCore::IR::IRListView *CurrentIR, FEXCore::Core::DebugData *DebugData);
static void InterpretIR(FEXCore::Core::InternalThreadState *Thread, uint64_t Entry, FEXCore::IR::IRListView *CurrentIR, FEXCore::Core::DebugData *DebugData);
static bool GetFallbackHandler(IR::IROp_Header *IROp, FallbackInfo *Info);
};
};
+66 -55
View File
@@ -46,7 +46,7 @@ DEF_OP(TruncElementPair) {
mov(Dst.second, Src.second);
break;
}
default: LogMan::Msg::A("Unhandled Truncation size: %d", Op->Size); break;
default: LOGMAN_MSG_A_FMT("Unhandled Truncation size: {}", Op->Size); break;
}
}
@@ -59,7 +59,7 @@ DEF_OP(Constant) {
DEF_OP(EntrypointOffset) {
auto Op = IROp->C<IR::IROp_EntrypointOffset>();
auto Constant = IR->GetHeader()->Entry + Op->Offset;
auto Constant = Entry + Op->Offset;
auto Dst = GetReg<RA_64>(Node);
LoadConstant(Dst, Constant);
}
@@ -95,7 +95,7 @@ DEF_OP(Add) {
case 8:
add(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()), Const);
break;
default: LogMan::Msg::A("Unsupported Add size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported Add size: {}", OpSize);
}
} else {
switch (OpSize) {
@@ -105,7 +105,7 @@ DEF_OP(Add) {
case 8:
add(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()), GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
default: LogMan::Msg::A("Unsupported Add size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported Add size: {}", OpSize);
}
}
}
@@ -121,7 +121,7 @@ DEF_OP(Sub) {
case 8:
sub(GRS(Node), GRS(Op->Header.Args[0].ID()), Const);
break;
default: LogMan::Msg::A("Unsupported Sub size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported Sub size: {}", OpSize);
}
} else {
switch (OpSize) {
@@ -131,7 +131,7 @@ DEF_OP(Sub) {
case 8:
sub(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()), GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
default: LogMan::Msg::A("Unsupported Sub size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported Sub size: {}", OpSize);
}
}
@@ -147,7 +147,7 @@ DEF_OP(Neg) {
case 8:
neg(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()));
break;
default: LogMan::Msg::A("Unsupported Not size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported Neg size: {}", OpSize);
}
}
@@ -159,12 +159,11 @@ DEF_OP(Mul) {
switch (OpSize) {
case 4:
mul(Dst.W(), GetReg<RA_32>(Op->Header.Args[0].ID()), GetReg<RA_32>(Op->Header.Args[1].ID()));
sxtw(Dst, Dst);
break;
case 8:
mul(Dst, GetReg<RA_64>(Op->Header.Args[0].ID()), GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
default: LogMan::Msg::A("Unknown Sext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown Mul size: %d", OpSize);
}
}
@@ -180,7 +179,7 @@ DEF_OP(UMul) {
case 8:
mul(Dst, GetReg<RA_64>(Op->Header.Args[0].ID()), GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
default: LogMan::Msg::A("Unknown Sext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown UMul size: {}", OpSize);
}
}
@@ -217,7 +216,7 @@ DEF_OP(Div) {
sdiv(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()), GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
}
default: LogMan::Msg::A("Unknown DIV Size: %d", Size); break;
default: LOGMAN_MSG_A_FMT("Unknown DIV Size: {}", Size); break;
}
}
@@ -244,7 +243,7 @@ DEF_OP(UDiv) {
udiv(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()), GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
}
default: LogMan::Msg::A("Unknown UDIV Size: %d", Size); break;
default: LOGMAN_MSG_A_FMT("Unknown UDIV Size: {}", Size); break;
}
}
@@ -291,7 +290,7 @@ DEF_OP(Rem) {
msub(GetReg<RA_64>(Node), TMP1, Divisor, Dividend);
break;
}
default: LogMan::Msg::A("Unknown REM Size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown REM Size: {}", OpSize); break;
}
}
@@ -333,7 +332,7 @@ DEF_OP(URem) {
msub(GetReg<RA_64>(Node), TMP1, Divisor, Dividend);
break;
}
default: LogMan::Msg::A("Unknown UREM Size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown UREM Size: {}", OpSize); break;
}
}
@@ -345,12 +344,12 @@ DEF_OP(MulH) {
sxtw(TMP1, GetReg<RA_64>(Op->Header.Args[0].ID()));
sxtw(TMP2, GetReg<RA_64>(Op->Header.Args[1].ID()));
mul(TMP1, TMP1, TMP2);
sbfx(GetReg<RA_64>(Node), TMP1, 32, 32);
ubfx(GetReg<RA_64>(Node), TMP1, 32, 32);
break;
case 8:
smulh(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()), GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
default: LogMan::Msg::A("Unknown Sext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown Sext size: {}", OpSize);
}
}
@@ -367,7 +366,7 @@ DEF_OP(UMulH) {
case 8:
umulh(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()), GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
default: LogMan::Msg::A("Unknown Sext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown Sext size: {}", OpSize);
}
}
@@ -463,7 +462,7 @@ DEF_OP(Ror) {
break;
}
default: LogMan::Msg::A("Unhandled ROR size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled ROR size: {}", OpSize);
}
} else {
switch (OpSize) {
@@ -476,7 +475,7 @@ DEF_OP(Ror) {
break;
}
default: LogMan::Msg::A("Unhandled ROR size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled ROR size: {}", OpSize);
}
}
}
@@ -495,7 +494,7 @@ DEF_OP(Extr) {
break;
}
default: LogMan::Msg::A("Unhandled EXTR size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled EXTR size: {}", OpSize);
}
}
@@ -540,7 +539,7 @@ DEF_OP(LDiv) {
mov(GetReg<RA_64>(Node), x0);
break;
}
default: LogMan::Msg::A("Unknown LDIV Size: %d", Size); break;
default: LOGMAN_MSG_A_FMT("Unknown LDIV Size: {}", Size); break;
}
}
@@ -583,7 +582,7 @@ DEF_OP(LUDiv) {
mov(GetReg<RA_64>(Node), x0);
break;
}
default: LogMan::Msg::A("Unknown LUDIV Size: %d", Size); break;
default: LOGMAN_MSG_A_FMT("Unknown LUDIV Size: {}", Size); break;
}
}
@@ -636,7 +635,7 @@ DEF_OP(LRem) {
mov(GetReg<RA_64>(Node), x0);
break;
}
default: LogMan::Msg::A("Unknown LREM Size: %d", Size); break;
default: LOGMAN_MSG_A_FMT("Unknown LREM Size: {}", Size); break;
}
}
@@ -686,7 +685,7 @@ DEF_OP(LURem) {
mov(GetReg<RA_64>(Node), x0);
break;
}
default: LogMan::Msg::A("Unknown LUREM Size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown LUREM Size: {}", OpSize); break;
}
}
@@ -700,7 +699,7 @@ DEF_OP(Not) {
case 8:
mvn(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()));
break;
default: LogMan::Msg::A("Unsupported Not size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported Not size: {}", OpSize);
}
}
@@ -731,7 +730,7 @@ DEF_OP(Popcount) {
// fmov has zero extended, unused bytes are zero
addv(VTMP1.B(), VTMP1.V8B());
break;
default: LogMan::Msg::A("Unsupported Popcount size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported Popcount size: {}", OpSize);
}
auto Dst = GetReg<RA_32>(Node);
@@ -780,7 +779,7 @@ DEF_OP(FindMSB) {
clz(Dst, GetReg<RA_64>(Op->Header.Args[0].ID()));
sub(Dst, TMP1, Dst);
break;
default: LogMan::Msg::A("Unknown REV size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown FindMSB size: {}", OpSize); break;
}
}
@@ -801,7 +800,7 @@ DEF_OP(FindTrailingZeros) {
rbit(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()));
clz(GetReg<RA_64>(Node), GetReg<RA_64>(Node));
break;
default: LogMan::Msg::A("Unknown size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown FindTrailingZeros size: {}", OpSize); break;
}
}
@@ -820,7 +819,7 @@ DEF_OP(CountLeadingZeroes) {
case 8:
clz(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()));
break;
default: LogMan::Msg::A("Unknown size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown CountLeadingZeroes size: {}", OpSize); break;
}
}
@@ -838,7 +837,7 @@ DEF_OP(Rev) {
case 8:
rev(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()));
break;
default: LogMan::Msg::A("Unknown REV size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown REV size: {}", OpSize); break;
}
}
@@ -860,15 +859,14 @@ DEF_OP(Bfi) {
bfi(TMP1, GetReg<RA_64>(Op->Header.Args[1].ID()), Op->lsb, Op->Width);
mov(GetReg<RA_64>(Node), TMP1);
break;
default: LogMan::Msg::A("Unknown BFI size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown BFI size: {}", OpSize); break;
}
}
DEF_OP(Bfe) {
auto Op = IROp->C<IR::IROp_Bfe>();
uint8_t OpSize = IROp->Size;
LogMan::Throw::A(OpSize <= 8, "OpSize is too large for BFE: %d", OpSize);
LogMan::Throw::A(Op->Width != 0, "Invalid BFE width of 0");
LOGMAN_THROW_A_FMT(IROp->Size <= 8, "OpSize is too large for BFE: {}", IROp->Size);
LOGMAN_THROW_A_FMT(Op->Width != 0, "Invalid BFE width of 0");
auto Dst = GetReg<RA_64>(Node);
ubfx(Dst, GetReg<RA_64>(Op->Header.Args[0].ID()), Op->lsb, Op->Width);
@@ -882,7 +880,7 @@ DEF_OP(Sbfe) {
if (OpSize == 8) {
sbfx(Dst, GetReg<RA_64>(Op->Header.Args[0].ID()), Op->lsb, Op->Width);
} else {
LogMan::Msg::D("Unimplemented Sbfe size");
LogMan::Msg::DFmt("Unimplemented Sbfe size");
}
}
@@ -913,7 +911,7 @@ Condition MapSelectCC(IR::CondClassType Cond) {
case FEXCore::IR::COND_MI:
case FEXCore::IR::COND_PL:
default:
LogMan::Msg::A("Unsupported compare type");
LOGMAN_MSG_A_FMT("Unsupported compare type");
return Condition::nv;
}
}
@@ -931,7 +929,7 @@ DEF_OP(Select) {
} else if (IsFPR(Op->Cmp1.ID())) {
fcmp(GRFCMP(Op->Cmp1.ID()), GRFCMP(Op->Cmp2.ID()));
} else {
LogMan::Msg::A("Select: Expected GPR or FPR");
LOGMAN_MSG_A_FMT("Select: Expected GPR or FPR");
}
auto cc = MapSelectCC(Op->Cond);
@@ -942,7 +940,7 @@ DEF_OP(Select) {
if (is_const_true || is_const_false) {
if (is_const_false != true || is_const_true != true || const_true != 1 || const_false != 0) {
LogMan::Msg::A("Select: Unsupported compare inline parameters");
LOGMAN_MSG_A_FMT("Select: Unsupported compare inline parameters");
}
cset(GRS(Node), cc);
} else {
@@ -966,38 +964,53 @@ DEF_OP(VExtractToGPR) {
case 8:
umov(GetReg<RA_64>(Node), GetSrc(Op->Header.Args[0].ID()).V2D(), Op->Idx);
break;
default: LogMan::Msg::A("Unhandled ExtractElementSize: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled ExtractElementSize: {}", OpSize);
}
}
DEF_OP(Float_ToGPR_ZU) {
LogMan::Msg::D("Unimplemented");
}
DEF_OP(Float_ToGPR_ZS) {
auto Op = IROp->C<IR::IROp_Float_ToGPR_ZS>();
if (Op->Header.ElementSize == 8) {
fcvtzs(GetReg<RA_64>(Node), GetSrc(Op->Header.Args[0].ID()).D());
aarch64::Register Dst{};
aarch64::VRegister Src{};
if (Op->SrcElementSize == 8) {
Src = GetSrc(Op->Header.Args[0].ID()).D();
}
else {
fcvtzs(GetReg<RA_32>(Node), GetSrc(Op->Header.Args[0].ID()).S());
Src = GetSrc(Op->Header.Args[0].ID()).S();
}
}
DEF_OP(Float_ToGPR_U) {
LogMan::Msg::D("Unimplemented");
if (IROp->Size == 8) {
Dst = GetReg<RA_64>(Node);
}
else {
Dst = GetReg<RA_32>(Node);
}
fcvtzs(Dst, Src);
}
DEF_OP(Float_ToGPR_S) {
auto Op = IROp->C<IR::IROp_Float_ToGPR_S>();
if (Op->Header.ElementSize == 8) {
aarch64::Register Dst{};
aarch64::VRegister Src{};
if (Op->SrcElementSize == 8) {
frinti(VTMP1.D(), GetSrc(Op->Header.Args[0].ID()).D());
fcvtzs(GetReg<RA_64>(Node), VTMP1.D());
Src = VTMP1.D();
}
else {
frinti(VTMP1.S(), GetSrc(Op->Header.Args[0].ID()).S());
fcvtzs(GetReg<RA_32>(Node), VTMP1.S());
Src = VTMP1.S();
}
if (IROp->Size == 8) {
Dst = GetReg<RA_64>(Node);
}
else {
Dst = GetReg<RA_32>(Node);
}
fcvtzs(Dst, Src);
}
DEF_OP(FCmp) {
@@ -1014,7 +1027,7 @@ DEF_OP(FCmp) {
bool set = false;
if (Op->Flags & (1 << IR::FCMP_FLAG_EQ)) {
LogMan::Throw::A(IR::FCMP_FLAG_EQ == 0, "IR::FCMP_FLAG_EQ must equal 0");
LOGMAN_THROW_A_FMT(IR::FCMP_FLAG_EQ == 0, "IR::FCMP_FLAG_EQ must equal 0");
// EQ or unordered
cset(Dst, Condition::eq); // Z = 1
csinc(Dst, Dst, xzr, Condition::vc); // IF !V ? Z : 1
@@ -1088,9 +1101,7 @@ void Arm64JITCore::RegisterALUHandlers() {
REGISTER_OP(SBFE, Sbfe);
REGISTER_OP(SELECT, Select);
REGISTER_OP(VEXTRACTTOGPR, VExtractToGPR);
REGISTER_OP(FLOAT_TOGPR_ZU, Float_ToGPR_ZU);
REGISTER_OP(FLOAT_TOGPR_ZS, Float_ToGPR_ZS);
REGISTER_OP(FLOAT_TOGPR_U, Float_ToGPR_U);
REGISTER_OP(FLOAT_TOGPR_S, Float_ToGPR_S);
REGISTER_OP(FCMP, FCmp);
@@ -34,7 +34,7 @@ DEF_OP(CASPair) {
mov(Dst.first, TMP3);
mov(Dst.second, TMP4);
break;
default: LogMan::Msg::A("Unsupported: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported: {}", OpSize);
}
}
else {
@@ -44,11 +44,15 @@ DEF_OP(CASPair) {
aarch64::Label LoopNotExpected;
aarch64::Label LoopExpected;
bind(&LoopTop);
nop();
ldaxp(TMP2.W(), TMP3.W(), MemOperand(MemSrc));
nop();
cmp(TMP2.W(), Expected.first.W());
ccmp(TMP3.W(), Expected.second.W(), NoFlag, Condition::eq);
b(&LoopNotExpected, Condition::ne);
nop();
stlxp(TMP2.W(), Desired.first.W(), Desired.second.W(), MemOperand(MemSrc));
nop();
cbnz(TMP2.W(), &LoopTop);
mov(Dst.first.W(), Expected.first.W());
mov(Dst.second.W(), Expected.second.W());
@@ -69,11 +73,15 @@ DEF_OP(CASPair) {
aarch64::Label LoopNotExpected;
aarch64::Label LoopExpected;
bind(&LoopTop);
nop();
ldaxp(TMP2.X(), TMP3.X(), MemOperand(MemSrc));
nop();
cmp(TMP2.X(), Expected.first.X());
ccmp(TMP3.X(), Expected.second.X(), NoFlag, Condition::eq);
b(&LoopNotExpected, Condition::ne);
nop();
stlxp(TMP2.X(), Desired.first.X(), Desired.second.X(), MemOperand(MemSrc));
nop();
cbnz(TMP2.X(), &LoopTop);
mov(Dst.first.X(), Expected.first.X());
mov(Dst.second.X(), Expected.second.X());
@@ -89,7 +97,7 @@ DEF_OP(CASPair) {
bind(&LoopExpected);
break;
}
default: LogMan::Msg::A("Unsupported: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported: {}", OpSize);
}
}
}
@@ -115,7 +123,7 @@ DEF_OP(CAS) {
case 2: casalh(TMP2.W(), Desired.W(), MemOperand(MemSrc)); break;
case 4: casal(TMP2.W(), Desired.W(), MemOperand(MemSrc)); break;
case 8: casal(TMP2.X(), Desired.X(), MemOperand(MemSrc)); break;
default: LogMan::Msg::A("Unsupported: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported: {}", OpSize);
}
mov(GetReg<RA_64>(Node), TMP2);
}
@@ -206,7 +214,7 @@ DEF_OP(CAS) {
break;
}
default: LogMan::Msg::A("Unhandled Atomic size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", OpSize);
}
}
}
@@ -222,7 +230,7 @@ DEF_OP(AtomicAdd) {
case 2: staddlh(GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
case 4: staddl(GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
case 8: staddl(GetReg<RA_64>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
default: LogMan::Msg::A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
else {
@@ -264,7 +272,7 @@ DEF_OP(AtomicAdd) {
cbnz(TMP2, &LoopTop);
break;
}
default: LogMan::Msg::A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
}
@@ -281,7 +289,7 @@ DEF_OP(AtomicSub) {
case 2: staddlh(TMP2.W(), MemOperand(MemSrc)); break;
case 4: staddl(TMP2.W(), MemOperand(MemSrc)); break;
case 8: staddl(TMP2.X(), MemOperand(MemSrc)); break;
default: LogMan::Msg::A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
else {
@@ -323,7 +331,7 @@ DEF_OP(AtomicSub) {
cbnz(TMP2, &LoopTop);
break;
}
default: LogMan::Msg::A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
}
@@ -340,7 +348,7 @@ DEF_OP(AtomicAnd) {
case 2: stclrlh(TMP2.W(), MemOperand(MemSrc)); break;
case 4: stclrl(TMP2.W(), MemOperand(MemSrc)); break;
case 8: stclrl(TMP2.X(), MemOperand(MemSrc)); break;
default: LogMan::Msg::A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
else {
@@ -382,7 +390,7 @@ DEF_OP(AtomicAnd) {
cbnz(TMP2, &LoopTop);
break;
}
default: LogMan::Msg::A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
}
@@ -398,7 +406,7 @@ DEF_OP(AtomicOr) {
case 2: stsetlh(GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
case 4: stsetl(GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
case 8: stsetl(GetReg<RA_64>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
default: LogMan::Msg::A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
else {
@@ -440,7 +448,7 @@ DEF_OP(AtomicOr) {
cbnz(TMP2, &LoopTop);
break;
}
default: LogMan::Msg::A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
}
@@ -456,7 +464,7 @@ DEF_OP(AtomicXor) {
case 2: steorlh(GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
case 4: steorl(GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
case 8: steorl(GetReg<RA_64>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
default: LogMan::Msg::A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
else {
@@ -498,7 +506,7 @@ DEF_OP(AtomicXor) {
cbnz(TMP2, &LoopTop);
break;
}
default: LogMan::Msg::A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
}
@@ -515,36 +523,35 @@ DEF_OP(AtomicSwap) {
case 2: swplh(TMP2.W(), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 4: swpl(TMP2.W(), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 8: swpl(TMP2.X(), GetReg<RA_64>(Node), MemOperand(MemSrc)); break;
default: LogMan::Msg::A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
else {
// TMP2-TMP3
mov(TMP3, GetReg<RA_64>(Op->Header.Args[1].ID()));
switch (Op->Size) {
case 1: {
aarch64::Label LoopTop;
bind(&LoopTop);
ldaxrb(TMP2.W(), MemOperand(MemSrc));
stlxrb(TMP4.W(), TMP3.W(), MemOperand(MemSrc));
stlxrb(TMP4.W(), GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc));
cbnz(TMP4.W(), &LoopTop);
uxtb(GetReg<RA_64>(Node), TMP2.W());
uxtb(GetReg<RA_32>(Node), TMP2.W());
break;
}
case 2: {
aarch64::Label LoopTop;
bind(&LoopTop);
ldaxrh(TMP2.W(), MemOperand(MemSrc));
stlxrh(TMP4.W(), TMP3.W(), MemOperand(MemSrc));
stlxrh(TMP4.W(), GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc));
cbnz(TMP4.W(), &LoopTop);
uxtw(GetReg<RA_64>(Node), TMP2.W());
uxtw(GetReg<RA_32>(Node), TMP2.W());
break;
}
case 4: {
aarch64::Label LoopTop;
bind(&LoopTop);
ldaxr(TMP2.W(), MemOperand(MemSrc));
stlxr(TMP4.W(), TMP3.W(), MemOperand(MemSrc));
stlxr(TMP4.W(), GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc));
cbnz(TMP4.W(), &LoopTop);
mov(GetReg<RA_32>(Node), TMP2.W());
break;
@@ -553,12 +560,12 @@ DEF_OP(AtomicSwap) {
aarch64::Label LoopTop;
bind(&LoopTop);
ldaxr(TMP2, MemOperand(MemSrc));
stlxr(TMP4, TMP3.X(), MemOperand(MemSrc));
stlxr(TMP4, GetReg<RA_64>(Op->Header.Args[1].ID()), MemOperand(MemSrc));
cbnz(TMP4, &LoopTop);
mov(GetReg<RA_64>(Node), TMP2.X());
break;
}
default: LogMan::Msg::A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
}
@@ -573,7 +580,7 @@ DEF_OP(AtomicFetchAdd) {
case 2: ldaddalh(GetReg<RA_32>(Op->Header.Args[1].ID()), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 4: ldaddal(GetReg<RA_32>(Op->Header.Args[1].ID()), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 8: ldaddal(GetReg<RA_64>(Op->Header.Args[1].ID()), GetReg<RA_64>(Node), MemOperand(MemSrc)); break;
default: LogMan::Msg::A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
else {
@@ -619,7 +626,7 @@ DEF_OP(AtomicFetchAdd) {
mov(GetReg<RA_64>(Node), TMP2);
break;
}
default: LogMan::Msg::A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
}
@@ -635,7 +642,7 @@ DEF_OP(AtomicFetchSub) {
case 2: ldaddalh(TMP2.W(), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 4: ldaddal(TMP2.W(), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 8: ldaddal(TMP2.X(), GetReg<RA_64>(Node), MemOperand(MemSrc)); break;
default: LogMan::Msg::A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
else {
@@ -681,7 +688,7 @@ DEF_OP(AtomicFetchSub) {
mov(GetReg<RA_64>(Node), TMP2);
break;
}
default: LogMan::Msg::A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
}
@@ -697,7 +704,7 @@ DEF_OP(AtomicFetchAnd) {
case 2: ldclralh(TMP2.W(), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 4: ldclral(TMP2.W(), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 8: ldclral(TMP2.X(), GetReg<RA_64>(Node), MemOperand(MemSrc)); break;
default: LogMan::Msg::A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
else {
@@ -743,7 +750,7 @@ DEF_OP(AtomicFetchAnd) {
mov(GetReg<RA_64>(Node), TMP2);
break;
}
default: LogMan::Msg::A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
}
@@ -758,7 +765,7 @@ DEF_OP(AtomicFetchOr) {
case 2: ldsetalh(GetReg<RA_32>(Op->Header.Args[1].ID()), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 4: ldsetal(GetReg<RA_32>(Op->Header.Args[1].ID()), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 8: ldsetal(GetReg<RA_64>(Op->Header.Args[1].ID()), GetReg<RA_64>(Node), MemOperand(MemSrc)); break;
default: LogMan::Msg::A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
else {
@@ -804,7 +811,7 @@ DEF_OP(AtomicFetchOr) {
mov(GetReg<RA_64>(Node), TMP2);
break;
}
default: LogMan::Msg::A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
}
@@ -819,7 +826,7 @@ DEF_OP(AtomicFetchXor) {
case 2: ldeoralh(GetReg<RA_32>(Op->Header.Args[1].ID()), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 4: ldeoral(GetReg<RA_32>(Op->Header.Args[1].ID()), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 8: ldeoral(GetReg<RA_64>(Op->Header.Args[1].ID()), GetReg<RA_64>(Node), MemOperand(MemSrc)); break;
default: LogMan::Msg::A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
else {
@@ -865,11 +872,61 @@ DEF_OP(AtomicFetchXor) {
mov(GetReg<RA_64>(Node), TMP2);
break;
}
default: LogMan::Msg::A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
}
DEF_OP(AtomicFetchNeg) {
auto Op = IROp->C<IR::IROp_AtomicFetchNeg>();
auto MemSrc = GetReg<RA_64>(Op->Header.Args[0].ID());
// TMP2-TMP3
switch (Op->Size) {
case 1: {
aarch64::Label LoopTop;
bind(&LoopTop);
ldaxrb(TMP2.W(), MemOperand(MemSrc));
neg(TMP3.W(), TMP2.W());
stlxrb(TMP4.W(), TMP3.W(), MemOperand(MemSrc));
cbnz(TMP4.W(), &LoopTop);
mov(GetReg<RA_32>(Node), TMP2.W());
break;
}
case 2: {
aarch64::Label LoopTop;
bind(&LoopTop);
ldaxrh(TMP2.W(), MemOperand(MemSrc));
neg(TMP3.W(), TMP2.W());
stlxrh(TMP4.W(), TMP3.W(), MemOperand(MemSrc));
cbnz(TMP4.W(), &LoopTop);
mov(GetReg<RA_32>(Node), TMP2.W());
break;
}
case 4: {
aarch64::Label LoopTop;
bind(&LoopTop);
ldaxr(TMP2.W(), MemOperand(MemSrc));
neg(TMP3.W(), TMP2.W());
stlxr(TMP4.W(), TMP3.W(), MemOperand(MemSrc));
cbnz(TMP4.W(), &LoopTop);
mov(GetReg<RA_32>(Node), TMP2.W());
break;
}
case 8: {
aarch64::Label LoopTop;
bind(&LoopTop);
ldaxr(TMP2, MemOperand(MemSrc));
neg(TMP3, TMP2);
stlxr(TMP4, TMP3, MemOperand(MemSrc));
cbnz(TMP4, &LoopTop);
mov(GetReg<RA_64>(Node), TMP2);
break;
}
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
#undef DEF_OP
void Arm64JITCore::RegisterAtomicHandlers() {
#define REGISTER_OP(op, x) OpHandlers[FEXCore::IR::IROps::OP_##op] = &Arm64JITCore::Op_##x
@@ -886,6 +943,7 @@ void Arm64JITCore::RegisterAtomicHandlers() {
REGISTER_OP(ATOMICFETCHAND, AtomicFetchAnd);
REGISTER_OP(ATOMICFETCHOR, AtomicFetchOr);
REGISTER_OP(ATOMICFETCHXOR, AtomicFetchXor);
REGISTER_OP(ATOMICFETCHNEG, AtomicFetchNeg);
#undef REGISTER_OP
}
}
@@ -4,6 +4,8 @@ tags: backend|arm64
$end_info$
*/
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/JIT/Arm64/JITClass.h"
#include "Interface/Core/InternalThreadState.h"
@@ -16,15 +18,15 @@ using namespace vixl;
using namespace vixl::aarch64;
#define DEF_OP(x) void Arm64JITCore::Op_##x(FEXCore::IR::IROp_Header *IROp, uint32_t Node)
DEF_OP(GuestCallDirect) {
LogMan::Msg::D("Unimplemented");
LogMan::Msg::DFmt("Unimplemented");
}
DEF_OP(GuestCallIndirect) {
LogMan::Msg::D("Unimplemented");
LogMan::Msg::DFmt("Unimplemented");
}
DEF_OP(GuestReturn) {
LogMan::Msg::D("Unimplemented");
LogMan::Msg::DFmt("Unimplemented");
}
DEF_OP(SignalReturn) {
@@ -73,7 +75,7 @@ DEF_OP(ExitFunction) {
uint64_t NewRIP;
if (IsInlineConstant(Op->NewRIP, &NewRIP) || IsInlineEntrypointOffset(Op->NewRIP, &NewRIP)) {
Literal l_BranchHost{Dispatcher->ExitFunctionLinkerAddress};
Literal l_BranchHost{ThreadSharedData.Dispatcher->ExitFunctionLinkerAddress};
Literal l_BranchGuest{NewRIP};
ldr(x0, &l_BranchHost);
@@ -96,7 +98,7 @@ DEF_OP(ExitFunction) {
br(x1);
bind(&FullLookup);
LoadConstant(TMP1, Dispatcher->AbsoluteLoopTopAddress);
LoadConstant(TMP1, ThreadSharedData.Dispatcher->AbsoluteLoopTopAddress);
str(RipReg, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.rip)));
br(TMP1);
}
@@ -142,7 +144,7 @@ Condition MapBranchCC(IR::CondClassType Cond) {
case FEXCore::IR::COND_MI:
case FEXCore::IR::COND_PL:
default:
LogMan::Msg::A("Unsupported compare type");
LOGMAN_MSG_A_FMT("Unsupported compare type");
return Condition::nv;
}
}
@@ -169,10 +171,10 @@ DEF_OP(CondJump) {
bool isConst = IsInlineConstant(Op->Cmp2, &Const);
if (isConst && Const == 0 && Op->Cond.Val == FEXCore::IR::COND_EQ) {
LogMan::Throw::A(IsGPR(Op->Cmp1.ID()), "CondJump: Expected GPR");
LOGMAN_THROW_A_FMT(IsGPR(Op->Cmp1.ID()), "CondJump: Expected GPR");
cbz(GRCMP(Op->Cmp1.ID()), TrueTargetLabel);
} else if (isConst && Const == 0 && Op->Cond.Val == FEXCore::IR::COND_NEQ) {
LogMan::Throw::A(IsGPR(Op->Cmp1.ID()), "CondJump: Expected GPR");
LOGMAN_THROW_A_FMT(IsGPR(Op->Cmp1.ID()), "CondJump: Expected GPR");
cbnz(GRCMP(Op->Cmp1.ID()), TrueTargetLabel);
} else {
if (IsGPR(Op->Cmp1.ID())) {
@@ -183,7 +185,7 @@ DEF_OP(CondJump) {
} else if (IsFPR(Op->Cmp1.ID())) {
fcmp(GRFCMP(Op->Cmp1.ID()), GRFCMP(Op->Cmp2.ID()));
} else {
LogMan::Msg::A("CondJump: Expected GPR or FPR");
LOGMAN_MSG_A_FMT("CondJump: Expected GPR or FPR");
}
b(TrueTargetLabel, MapBranchCC(Op->Cond));
@@ -257,18 +259,18 @@ DEF_OP(Thunk) {
DEF_OP(ValidateCode) {
auto Op = IROp->C<IR::IROp_ValidateCode>();
uint8_t *OldCode = (uint8_t *)&Op->CodeOriginalLow;
const auto *OldCode = (const uint8_t *)&Op->CodeOriginalLow;
int len = Op->CodeLength;
int idx = 0;
LoadConstant(GetReg<RA_64>(Node), 0);
LoadConstant(x0, IR->GetHeader()->Entry + Op->Offset);
LoadConstant(x0, Entry + Op->Offset);
LoadConstant(x1, 1);
while (len >= 8)
{
ldr(x2, MemOperand(x0, idx));
LoadConstant(x3, *(uint32_t *)(OldCode + idx));
LoadConstant(x3, *(const uint32_t *)(OldCode + idx));
cmp(x2, x3);
csel(GetReg<RA_64>(Node), GetReg<RA_64>(Node), x1, Condition::eq);
len -= 8;
@@ -277,7 +279,7 @@ DEF_OP(ValidateCode) {
while (len >= 4)
{
ldr(w2, MemOperand(x0, idx));
LoadConstant(w3, *(uint32_t *)(OldCode + idx));
LoadConstant(w3, *(const uint32_t *)(OldCode + idx));
cmp(w2, w3);
csel(GetReg<RA_64>(Node), GetReg<RA_64>(Node), x1, Condition::eq);
len -= 4;
@@ -286,7 +288,7 @@ DEF_OP(ValidateCode) {
while (len >= 2)
{
ldrh(w2, MemOperand(x0, idx));
LoadConstant(w3, *(uint16_t *)(OldCode + idx));
LoadConstant(w3, *(const uint16_t *)(OldCode + idx));
cmp(w2, w3);
csel(GetReg<RA_64>(Node), GetReg<RA_64>(Node), x1, Condition::eq);
len -= 2;
@@ -295,7 +297,7 @@ DEF_OP(ValidateCode) {
while (len >= 1)
{
ldrb(w2, MemOperand(x0, idx));
LoadConstant(w3, *(uint8_t *)(OldCode + idx));
LoadConstant(w3, *(const uint8_t *)(OldCode + idx));
cmp(w2, w3);
csel(GetReg<RA_64>(Node), GetReg<RA_64>(Node), x1, Condition::eq);
len -= 1;
@@ -311,7 +313,7 @@ DEF_OP(RemoveCodeEntry) {
PushDynamicRegsAndLR();
mov(x0, STATE);
LoadConstant(x1, IR->GetHeader()->Entry);
LoadConstant(x1, Entry);
LoadConstant(x2, reinterpret_cast<uintptr_t>(&Context::Context::RemoveCodeEntryFromJit));
SpillStaticRegs();
@@ -31,7 +31,7 @@ DEF_OP(VInsGPR) {
ins(GetDst(Node).V2D(), Op->Index, GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
}
default: LogMan::Msg::A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -52,14 +52,10 @@ DEF_OP(VCastFromGPR) {
case 8:
fmov(GetDst(Node).D(), GetReg<RA_64>(Op->Header.Args[0].ID()).X());
break;
default: LogMan::Msg::A("Unknown castGPR element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown castGPR element size: {}", Op->Header.ElementSize);
}
}
DEF_OP(Float_FromGPR_U) {
LogMan::Msg::A("Unimplemented");
}
DEF_OP(Float_FromGPR_S) {
auto Op = IROp->C<IR::IROp_Float_FromGPR_S>();
uint16_t Conv = (Op->Header.ElementSize << 8) | Op->SrcElementSize;
@@ -95,20 +91,7 @@ DEF_OP(Float_FToF) {
fcvt(GetDst(Node).S(), GetSrc(Op->Header.Args[0].ID()).D());
break;
}
default: LogMan::Msg::A("Unknown FCVT sizes: 0x%x", Conv);
}
}
DEF_OP(Vector_UToF) {
auto Op = IROp->C<IR::IROp_Vector_UToF>();
switch (Op->Header.ElementSize) {
case 4:
ucvtf(GetDst(Node).V4S(), GetSrc(Op->Header.Args[0].ID()).V4S());
break;
case 8:
ucvtf(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
default: LogMan::Msg::A("Unknown castGPR element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown FCVT sizes: 0x{:x}", Conv);
}
}
@@ -121,20 +104,7 @@ DEF_OP(Vector_SToF) {
case 8:
scvtf(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
default: LogMan::Msg::A("Unknown castGPR element size: %d", Op->Header.ElementSize);
}
}
DEF_OP(Vector_FToZU) {
auto Op = IROp->C<IR::IROp_Vector_FToZU>();
switch (Op->Header.ElementSize) {
case 4:
fcvtzu(GetDst(Node).V4S(), GetSrc(Op->Header.Args[0].ID()).V4S());
break;
case 8:
fcvtzu(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
default: LogMan::Msg::A("Unknown castGPR element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Vector_SToF element size: {}", Op->Header.ElementSize);
}
}
@@ -147,22 +117,7 @@ DEF_OP(Vector_FToZS) {
case 8:
fcvtzs(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
default: LogMan::Msg::A("Unknown castGPR element size: %d", Op->Header.ElementSize);
}
}
DEF_OP(Vector_FToU) {
auto Op = IROp->C<IR::IROp_Vector_FToU>();
switch (Op->Header.ElementSize) {
case 4:
frinti(GetDst(Node).V4S(), GetSrc(Op->Header.Args[0].ID()).V4S());
fcvtzu(GetDst(Node).V4S(), GetDst(Node).V4S());
break;
case 8:
frinti(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
fcvtzu(GetDst(Node).V2D(), GetDst(Node).V2D());
break;
default: LogMan::Msg::A("Unknown castGPR element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Vector_FToZS element size: {}", Op->Header.ElementSize);
}
}
@@ -177,7 +132,7 @@ DEF_OP(Vector_FToS) {
frinti(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
fcvtzs(GetDst(Node).V2D(), GetDst(Node).V2D());
break;
default: LogMan::Msg::A("Unknown castGPR element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Vector_FToS element size: {}", Op->Header.ElementSize);
}
}
@@ -194,7 +149,63 @@ DEF_OP(Vector_FToF) {
fcvtn(GetDst(Node).V2S(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
}
default: LogMan::Msg::A("Unknown Conversion Type : 0%04x", Conv); break;
default: LOGMAN_MSG_A_FMT("Unknown Vector_FToF Type : 0x{:04x}", Conv); break;
}
}
DEF_OP(Vector_FToI) {
auto Op = IROp->C<IR::IROp_Vector_FToI>();
switch (Op->Round) {
case FEXCore::IR::Round_Nearest.Val:
switch (Op->Header.ElementSize) {
case 4:
frintn(GetDst(Node).V4S(), GetSrc(Op->Header.Args[0].ID()).V4S());
break;
case 8:
frintn(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
}
break;
case FEXCore::IR::Round_Negative_Infinity.Val:
switch (Op->Header.ElementSize) {
case 4:
frintm(GetDst(Node).V4S(), GetSrc(Op->Header.Args[0].ID()).V4S());
break;
case 8:
frintm(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
}
break;
case FEXCore::IR::Round_Positive_Infinity.Val:
switch (Op->Header.ElementSize) {
case 4:
frintp(GetDst(Node).V4S(), GetSrc(Op->Header.Args[0].ID()).V4S());
break;
case 8:
frintp(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
}
break;
case FEXCore::IR::Round_Towards_Zero.Val:
switch (Op->Header.ElementSize) {
case 4:
frintz(GetDst(Node).V4S(), GetSrc(Op->Header.Args[0].ID()).V4S());
break;
case 8:
frintz(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
}
break;
case FEXCore::IR::Round_Host.Val:
switch (Op->Header.ElementSize) {
case 4:
frinti(GetDst(Node).V4S(), GetSrc(Op->Header.Args[0].ID()).V4S());
break;
case 8:
frinti(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
}
break;
}
}
@@ -203,16 +214,13 @@ void Arm64JITCore::RegisterConversionHandlers() {
#define REGISTER_OP(op, x) OpHandlers[FEXCore::IR::IROps::OP_##op] = &Arm64JITCore::Op_##x
REGISTER_OP(VINSGPR, VInsGPR);
REGISTER_OP(VCASTFROMGPR, VCastFromGPR);
REGISTER_OP(FLOAT_FROMGPR_U, Float_FromGPR_U);
REGISTER_OP(FLOAT_FROMGPR_S, Float_FromGPR_S);
REGISTER_OP(FLOAT_FTOF, Float_FToF);
REGISTER_OP(VECTOR_UTOF, Vector_UToF);
REGISTER_OP(VECTOR_STOF, Vector_SToF);
REGISTER_OP(VECTOR_FTOZU, Vector_FToZU);
REGISTER_OP(VECTOR_FTOZS, Vector_FToZS);
REGISTER_OP(VECTOR_FTOU, Vector_FToU);
REGISTER_OP(VECTOR_FTOS, Vector_FToS);
REGISTER_OP(VECTOR_FTOF, Vector_FToF);
REGISTER_OP(VECTOR_FTOI, Vector_FToI);
#undef REGISTER_OP
}
}
+154 -72
View File
@@ -11,6 +11,7 @@ $end_info$
*/
#include "Interface/Context/Context.h"
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/ArchHelpers/Arm64.h"
#include "Interface/Core/ArchHelpers/MContext.h"
@@ -22,6 +23,8 @@ $end_info$
#include <FEXCore/Core/X86Enums.h>
#include <FEXCore/Core/UContext.h>
#include <FEXCore/Utils/Allocator.h>
#include <FEXCore/Utils/CompilerDefs.h>
#include "Interface/Core/Interpreter/InterpreterOps.h"
#include <sys/mman.h>
@@ -42,8 +45,9 @@ using namespace vixl::aarch64;
void Arm64JITCore::Op_Unhandled(FEXCore::IR::IROp_Header *IROp, uint32_t Node) {
FallbackInfo Info;
if (!InterpreterOps::GetFallbackHandler(IROp, &Info)) {
auto Name = FEXCore::IR::GetName(IROp->Op);
LogMan::Msg::A("Unhandled IR Op: %s", std::string(Name).c_str());
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
LOGMAN_MSG_A_FMT("Unhandled IR Op: {}", FEXCore::IR::GetName(IROp->Op));
#endif
} else {
switch(Info.ABI) {
case FABI_VOID_U16:{
@@ -290,8 +294,10 @@ void Arm64JITCore::Op_Unhandled(FEXCore::IR::IROp_Header *IROp, uint32_t Node) {
case FABI_UNKNOWN:
default:
auto Name = FEXCore::IR::GetName(IROp->Op);
LogMan::Msg::A("Unhandled IR Fallback abi: %s %d", std::string(Name).c_str(), Info.ABI);
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
LOGMAN_MSG_A_FMT("Unhandled IR Fallback ABI: {} {}", FEXCore::IR::GetName(IROp->Op), Info.ABI);
#endif
break;
}
}
}
@@ -303,18 +309,18 @@ Arm64JITCore::CodeBuffer Arm64JITCore::AllocateNewCodeBuffer(size_t Size) {
CodeBuffer Buffer;
Buffer.Size = Size;
Buffer.Ptr = static_cast<uint8_t*>(
mmap(nullptr,
FEXCore::Allocator::mmap(nullptr,
Buffer.Size,
PROT_READ | PROT_WRITE | PROT_EXEC,
MAP_PRIVATE | MAP_ANONYMOUS,
-1, 0));
LogMan::Throw::A(!!Buffer.Ptr, "Couldn't allocate code buffer");
LOGMAN_THROW_A_FMT(!!Buffer.Ptr, "Couldn't allocate code buffer");
Dispatcher->RegisterCodeBuffer(Buffer.Ptr, Buffer.Size);
return Buffer;
}
void Arm64JITCore::FreeCodeBuffer(CodeBuffer Buffer) {
munmap(Buffer.Ptr, Buffer.Size);
FEXCore::Allocator::munmap(Buffer.Ptr, Buffer.Size);
Dispatcher->RemoveCodeBuffer(Buffer.Ptr);
}
@@ -338,23 +344,77 @@ bool Arm64JITCore::HandleSIGBUS(int Signal, void *info, void *ucontext) {
0b1011'0000'0000; // Inner shareable all
if ((Instr & 0x3F'FF'FC'00) == 0x08'DF'FC'00 || // LDAR*
(Instr & 0x3F'FF'FC'00) == 0x38'BF'C0'00) { // LDAPR*
uint32_t LDR = 0b0011'1000'0111'1111'0110'1000'0000'0000;
LDR |= Size << 30;
LDR |= AddrReg << 5;
LDR |= DataReg;
if (ParanoidTSO()) {
if (FEXCore::ArchHelpers::Arm64::HandleAtomicLoad(ucontext, info, Instr)) {
// Skip this instruction now
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) + 4);
return true;
}
else {
LogMan::Msg::EFmt("Unhandled JIT SIGBUS LDAR*: PC: {} Instruction: 0x{:08x}\n", fmt::ptr(PC), PC[0]);
return false;
}
}
else {
uint32_t LDR = 0b0011'1000'0111'1111'0110'1000'0000'0000;
LDR |= Size << 30;
LDR |= AddrReg << 5;
LDR |= DataReg;
PC[-1] = DMB;
PC[0] = LDR;
PC[1] = DMB;
// Back up one instruction and have another go
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) - 4);
}
}
else if ( (Instr & 0x3F'FF'FC'00) == 0x08'9F'FC'00) { // STLR*
if (ParanoidTSO()) {
if (FEXCore::ArchHelpers::Arm64::HandleAtomicStore(ucontext, info, Instr)) {
// Skip this instruction now
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) + 4);
return true;
}
else {
LogMan::Msg::EFmt("Unhandled JIT SIGBUS STLR*: PC: {} Instruction: 0x{:08x}\n", fmt::ptr(PC), PC[0]);
return false;
}
}
else {
uint32_t STR = 0b0011'1000'0011'1111'0110'1000'0000'0000;
STR |= Size << 30;
STR |= AddrReg << 5;
STR |= DataReg;
PC[-1] = DMB;
PC[0] = STR;
PC[1] = DMB;
// Back up one instruction and have another go
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) - 4);
}
}
else if ((Instr & FEXCore::ArchHelpers::Arm64::LDAXP_MASK) == FEXCore::ArchHelpers::Arm64::LDAXP_INST) { // LDAXP
uint32_t DataReg2 = (Instr >> 10) & 0x1F;
// Convert to LDP
uint32_t LDP = 0b0010'1001'0100'0000'0000'0000'0000'0000;
LDP |= Size << 31;
LDP |= DataReg2 << 10;
LDP |= AddrReg << 5;
LDP |= DataReg;
PC[-1] = DMB;
PC[0] = LDR;
PC[0] = LDP;
PC[1] = DMB;
// Back up one instruction and have another go
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) - 4);
}
else if ( (Instr & 0x3F'FF'FC'00) == 0x08'9F'FC'00) { // STLR*
uint32_t STR = 0b0011'1000'0011'1111'0110'1000'0000'0000;
STR |= Size << 30;
STR |= AddrReg << 5;
STR |= DataReg;
else if ((Instr & FEXCore::ArchHelpers::Arm64::STLXP_MASK) == FEXCore::ArchHelpers::Arm64::STLXP_INST) { // STLXP
uint32_t DataReg2 = (Instr >> 10) & 0x1F;
// Convert to STP
uint32_t STP = 0b0010'1001'0000'0000'0000'0000'0000'0000;
STP |= Size << 31;
STP |= DataReg2 << 10;
STP |= AddrReg << 5;
STP |= DataReg;
PC[-1] = DMB;
PC[0] = STR;
PC[0] = STP;
PC[1] = DMB;
// Back up one instruction and have another go
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) - 4);
@@ -366,7 +426,7 @@ bool Arm64JITCore::HandleSIGBUS(int Signal, void *info, void *ucontext) {
return true;
}
else {
LogMan::Msg::E("Unhandled JIT SIGBUS CASPAL: PC: %p Instruction: 0x%08x\n", PC, PC[0]);
LogMan::Msg::EFmt("Unhandled JIT SIGBUS CASPAL: PC: {} Instruction: 0x{:08x}\n", fmt::ptr(PC), PC[0]);
return false;
}
}
@@ -377,7 +437,7 @@ bool Arm64JITCore::HandleSIGBUS(int Signal, void *info, void *ucontext) {
return true;
}
else {
LogMan::Msg::E("Unhandled JIT SIGBUS CASAL: PC: %p Instruction: 0x%08x\n", PC, PC[0]);
LogMan::Msg::EFmt("Unhandled JIT SIGBUS CASAL: PC: {} Instruction: 0x{:08x}\n", fmt::ptr(PC), PC[0]);
return false;
}
}
@@ -389,12 +449,24 @@ bool Arm64JITCore::HandleSIGBUS(int Signal, void *info, void *ucontext) {
}
else {
uint8_t Op = (PC[0] >> 12) & 0xF;
LogMan::Msg::E("Unhandled JIT SIGBUS Atomic mem op 0x%02x: PC: %p Instruction: 0x%08x\n", Op, PC, PC[0]);
LogMan::Msg::EFmt("Unhandled JIT SIGBUS Atomic mem op 0x{:02x}: PC: {} Instruction: 0x{:08x}\n", Op, fmt::ptr(PC), PC[0]);
return false;
}
}
else if ((Instr & FEXCore::ArchHelpers::Arm64::LDAXR_MASK) == FEXCore::ArchHelpers::Arm64::LDAXR_INST) { // LDAXR*
uint64_t BytesToSkip = FEXCore::ArchHelpers::Arm64::HandleAtomicLoadstoreExclusive(ucontext, info);
if (BytesToSkip) {
// Skip this instruction now
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) + BytesToSkip);
return true;
}
else {
LogMan::Msg::EFmt("Unhandled JIT SIGBUS LDAXR: PC: {} Instruction: 0x{:08x}\n", fmt::ptr(PC), PC[0]);
return false;
}
}
else {
LogMan::Msg::E("Unhandled JIT SIGBUS: PC: %p Instruction: 0x%08x\n", PC, PC[0]);
LogMan::Msg::EFmt("Unhandled JIT SIGBUS: PC: {} Instruction: 0x{:08x}\n", fmt::ptr(PC), PC[0]);
return false;
}
@@ -412,7 +484,7 @@ Arm64JITCore::Arm64JITCore(FEXCore::Context::Context *ctx, FEXCore::Core::Intern
config.ExitFunctionLinkThis = reinterpret_cast<uintptr_t>(this);
config.StaticRegisterAssignment = true;
Dispatcher = new Arm64Dispatcher(CTX, ThreadState, config);
Dispatcher = std::make_unique<Arm64Dispatcher>(CTX, ThreadState, config);
DispatchPtr = Dispatcher->DispatchPtr;
CallbackPtr = Dispatcher->CallbackPtr;
}
@@ -466,22 +538,23 @@ Arm64JITCore::Arm64JITCore(FEXCore::Context::Context *ctx, FEXCore::Core::Intern
if (!CompileThread) {
ThreadSharedData.SignalHandlerRefCounterPtr = &Dispatcher->SignalHandlerRefCounter;
ThreadSharedData.SignalReturnInstruction = Dispatcher->SignalHandlerReturnAddress;
ThreadSharedData.Dispatcher = Dispatcher.get();
// This will register the host signal handler per thread, which is fine
CTX->SignalDelegation->RegisterHostSignalHandler(SIGILL, [](FEXCore::Core::InternalThreadState *Thread, int Signal, void *info, void *ucontext) -> bool {
Arm64JITCore *Core = reinterpret_cast<Arm64JITCore*>(Thread->CPUBackend.get());
return Core->Dispatcher->HandleSIGILL(Signal, info, ucontext);
});
}, true);
CTX->SignalDelegation->RegisterHostSignalHandler(SIGBUS, [](FEXCore::Core::InternalThreadState *Thread, int Signal, void *info, void *ucontext) -> bool {
Arm64JITCore *Core = reinterpret_cast<Arm64JITCore*>(Thread->CPUBackend.get());
return Core->HandleSIGBUS(Signal, info, ucontext);
});
}, true);
CTX->SignalDelegation->RegisterHostSignalHandler(SignalDelegator::SIGNAL_FOR_PAUSE, [](FEXCore::Core::InternalThreadState *Thread, int Signal, void *info, void *ucontext) -> bool {
Arm64JITCore *Core = reinterpret_cast<Arm64JITCore*>(Thread->CPUBackend.get());
return Core->Dispatcher->HandleSignalPause(Signal, info, ucontext);
});
}, true);
auto GuestSignalHandler = [](FEXCore::Core::InternalThreadState *Thread, int Signal, void *info, void *ucontext, GuestSigAction *GuestAction, stack_t *GuestStack) -> bool {
Arm64JITCore *Core = reinterpret_cast<Arm64JITCore*>(Thread->CPUBackend.get());
@@ -545,77 +618,85 @@ Arm64JITCore::~Arm64JITCore() {
FreeCodeBuffer(InitialCodeBuffer);
}
static IR::PhysicalRegister GetPhys(IR::RegisterAllocationData *RAData, uint32_t Node) {
IR::PhysicalRegister Arm64JITCore::GetPhys(uint32_t Node) const {
auto PhyReg = RAData->GetNodeRegister(Node);
LogMan::Throw::A(!PhyReg.IsInvalid(), "Couldn't Allocate register for node: ssa%d. Class: %d", Node, PhyReg.Class);
LOGMAN_THROW_A_FMT(!PhyReg.IsInvalid(), "Couldn't Allocate register for node: ssa{}. Class: {}", Node, PhyReg.Class);
return PhyReg;
}
template<>
aarch64::Register Arm64JITCore::GetReg<Arm64JITCore::RA_32>(uint32_t Node) {
auto Reg = GetPhys(RAData, Node);
aarch64::Register Arm64JITCore::GetReg<Arm64JITCore::RA_32>(uint32_t Node) const {
auto Reg = GetPhys(Node);
if (Reg.Class == IR::GPRFixedClass.Val) {
return SRA64[Reg.Reg].W();
} else if (Reg.Class == IR::GPRClass.Val) {
return RA64[Reg.Reg].W();
} else {
LogMan::Throw::A(false, "Unexpected Class: %d", Reg.Class);
LOGMAN_THROW_A_FMT(false, "Unexpected Class: {}", Reg.Class);
}
__builtin_unreachable();
FEX_UNREACHABLE;
}
template<>
aarch64::Register Arm64JITCore::GetReg<Arm64JITCore::RA_64>(uint32_t Node) {
auto Reg = GetPhys(RAData, Node);
aarch64::Register Arm64JITCore::GetReg<Arm64JITCore::RA_64>(uint32_t Node) const {
auto Reg = GetPhys(Node);
if (Reg.Class == IR::GPRFixedClass.Val) {
return SRA64[Reg.Reg];
} else if (Reg.Class == IR::GPRClass.Val) {
return RA64[Reg.Reg];
} else {
LogMan::Throw::A(false, "Unexpected Class: %d", Reg.Class);
LOGMAN_THROW_A_FMT(false, "Unexpected Class: {}", Reg.Class);
}
__builtin_unreachable();
FEX_UNREACHABLE;
}
template<>
std::pair<aarch64::Register, aarch64::Register> Arm64JITCore::GetSrcPair<Arm64JITCore::RA_32>(uint32_t Node) {
uint32_t Reg = GetPhys(RAData, Node).Reg;
std::pair<aarch64::Register, aarch64::Register> Arm64JITCore::GetSrcPair<Arm64JITCore::RA_32>(uint32_t Node) const {
uint32_t Reg = GetPhys(Node).Reg;
return RA32Pair[Reg];
}
template<>
std::pair<aarch64::Register, aarch64::Register> Arm64JITCore::GetSrcPair<Arm64JITCore::RA_64>(uint32_t Node) {
uint32_t Reg = GetPhys(RAData, Node).Reg;
std::pair<aarch64::Register, aarch64::Register> Arm64JITCore::GetSrcPair<Arm64JITCore::RA_64>(uint32_t Node) const {
uint32_t Reg = GetPhys(Node).Reg;
return RA64Pair[Reg];
}
aarch64::VRegister Arm64JITCore::GetSrc(uint32_t Node) {
auto Reg = GetPhys(RAData, Node);
aarch64::VRegister Arm64JITCore::GetSrc(uint32_t Node) const {
auto Reg = GetPhys(Node);
if (Reg.Class == IR::FPRFixedClass.Val) {
return SRAFPR[Reg.Reg];
} else if (Reg.Class == IR::FPRClass.Val) {
return RAFPR[Reg.Reg];
} else {
LogMan::Throw::A(false, "Unexpected Class: %d", Reg.Class);
LOGMAN_THROW_A_FMT(false, "Unexpected Class: {}", Reg.Class);
}
__builtin_unreachable();
FEX_UNREACHABLE;
}
aarch64::VRegister Arm64JITCore::GetDst(uint32_t Node) {
auto Reg = GetPhys(RAData, Node);
aarch64::VRegister Arm64JITCore::GetDst(uint32_t Node) const {
auto Reg = GetPhys(Node);
if (Reg.Class == IR::FPRFixedClass.Val) {
return SRAFPR[Reg.Reg];
} else if (Reg.Class == IR::FPRClass.Val) {
return RAFPR[Reg.Reg];
} else {
LogMan::Throw::A(false, "Unexpected Class: %d", Reg.Class);
LOGMAN_THROW_A_FMT(false, "Unexpected Class: {}", Reg.Class);
}
__builtin_unreachable();
FEX_UNREACHABLE;
}
bool Arm64JITCore::IsInlineConstant(const IR::OrderedNodeWrapper& WNode, uint64_t* Value) {
bool Arm64JITCore::IsInlineConstant(const IR::OrderedNodeWrapper& WNode, uint64_t* Value) const {
auto OpHeader = IR->GetOp<IR::IROp_Header>(WNode);
if (OpHeader->Op == IR::IROps::OP_INLINECONSTANT) {
@@ -629,13 +710,13 @@ bool Arm64JITCore::IsInlineConstant(const IR::OrderedNodeWrapper& WNode, uint64_
}
}
bool Arm64JITCore::IsInlineEntrypointOffset(const IR::OrderedNodeWrapper& WNode, uint64_t* Value) {
bool Arm64JITCore::IsInlineEntrypointOffset(const IR::OrderedNodeWrapper& WNode, uint64_t* Value) const {
auto OpHeader = IR->GetOp<IR::IROp_Header>(WNode);
if (OpHeader->Op == IR::IROps::OP_INLINEENTRYPOINTOFFSET) {
auto Op = OpHeader->C<IR::IROp_InlineEntrypointOffset>();
if (Value) {
*Value = IR->GetHeader()->Entry + Op->Offset;
*Value = Entry + Op->Offset;
}
return true;
} else {
@@ -643,34 +724,33 @@ bool Arm64JITCore::IsInlineEntrypointOffset(const IR::OrderedNodeWrapper& WNode,
}
}
FEXCore::IR::RegisterClassType Arm64JITCore::GetRegClass(uint32_t Node) {
return FEXCore::IR::RegisterClassType {GetPhys(RAData, Node).Class};
FEXCore::IR::RegisterClassType Arm64JITCore::GetRegClass(uint32_t Node) const {
return FEXCore::IR::RegisterClassType {GetPhys(Node).Class};
}
bool Arm64JITCore::IsFPR(uint32_t Node) {
bool Arm64JITCore::IsFPR(uint32_t Node) const {
auto Class = GetRegClass(Node);
return Class == IR::FPRClass || Class == IR::FPRFixedClass;
}
bool Arm64JITCore::IsGPR(uint32_t Node) {
bool Arm64JITCore::IsGPR(uint32_t Node) const {
auto Class = GetRegClass(Node);
return Class == IR::GPRClass || Class == IR::GPRFixedClass;
}
void *Arm64JITCore::CompileCode([[maybe_unused]] FEXCore::IR::IRListView const *IR, [[maybe_unused]] FEXCore::Core::DebugData *DebugData, FEXCore::IR::RegisterAllocationData *RAData) {
void *Arm64JITCore::CompileCode(uint64_t Entry, [[maybe_unused]] FEXCore::IR::IRListView const *IR, [[maybe_unused]] FEXCore::Core::DebugData *DebugData, FEXCore::IR::RegisterAllocationData *RAData) {
using namespace aarch64;
JumpTargets.clear();
uint32_t SSACount = IR->GetSSACount();
this->Entry = Entry;
this->RAData = RAData;
auto HeaderOp = IR->GetHeader();
#ifndef NDEBUG
LoadConstant(x0, HeaderOp->Entry);
LoadConstant(x0, Entry);
#endif
this->IR = IR;
@@ -702,7 +782,7 @@ void *Arm64JITCore::CompileCode([[maybe_unused]] FEXCore::IR::IRListView const *
// X4-r18 = RA
auto Buffer = GetBuffer();
auto Entry = Buffer->GetOffsetAddress<uint64_t>(GetCursorOffset());
auto GuestEntry = Buffer->GetOffsetAddress<uint64_t>(GetCursorOffset());
if (CTX->GetGdbServerStatus()) {
aarch64::Label RunBlock;
@@ -719,17 +799,17 @@ void *Arm64JITCore::CompileCode([[maybe_unused]] FEXCore::IR::IRListView const *
cbz(w0, &RunBlock);
{
// Make sure RIP is syncronized to the context
LoadConstant(x0, HeaderOp->Entry);
LoadConstant(x0, Entry);
str(x0, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.rip)));
// Stop the thread
LoadConstant(x0, Dispatcher->ThreadPauseHandlerAddressSpillSRA);
LoadConstant(x0, ThreadSharedData.Dispatcher->ThreadPauseHandlerAddressSpillSRA);
br(x0);
}
bind(&RunBlock);
}
//LogMan::Throw::A(RAData->HasFullRA(), "Arm64 JIT only works with RA");
//LOGMAN_THROW_A(RAData->HasFullRA(), "Arm64 JIT only works with RA");
SpillSlots = RAData->SpillSlots();
@@ -746,8 +826,10 @@ void *Arm64JITCore::CompileCode([[maybe_unused]] FEXCore::IR::IRListView const *
for (auto [BlockNode, BlockHeader] : IR->GetBlocks()) {
using namespace FEXCore::IR;
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
auto BlockIROp = BlockHeader->CW<FEXCore::IR::IROp_CodeBlock>();
LogMan::Throw::A(BlockIROp->Header.Op == IR::OP_CODEBLOCK, "IR type failed to be a code block");
LOGMAN_THROW_A_FMT(BlockIROp->Header.Op == IR::OP_CODEBLOCK, "IR type failed to be a code block");
#endif
{
uint32_t Node = IR->GetID(BlockNode);
@@ -793,15 +875,15 @@ void *Arm64JITCore::CompileCode([[maybe_unused]] FEXCore::IR::IRListView const *
FinalizeCode();
auto CodeEnd = Buffer->GetOffsetAddress<uint64_t>(GetCursorOffset());
CPU.EnsureIAndDCacheCoherency(reinterpret_cast<void*>(Entry), CodeEnd - reinterpret_cast<uint64_t>(Entry));
CPU.EnsureIAndDCacheCoherency(reinterpret_cast<void*>(GuestEntry), CodeEnd - reinterpret_cast<uint64_t>(GuestEntry));
if (DebugData) {
DebugData->HostCodeSize = reinterpret_cast<uintptr_t>(CodeEnd) - reinterpret_cast<uintptr_t>(Entry);
DebugData->HostCodeSize = reinterpret_cast<uintptr_t>(CodeEnd) - reinterpret_cast<uintptr_t>(GuestEntry);
}
this->IR = nullptr;
return reinterpret_cast<void*>(Entry);
return reinterpret_cast<void*>(GuestEntry);
}
uint64_t Arm64JITCore::ExitFunctionLink(Arm64JITCore *core, FEXCore::Core::CpuStateFrame *Frame, uint64_t *record) {
@@ -811,13 +893,13 @@ uint64_t Arm64JITCore::ExitFunctionLink(Arm64JITCore *core, FEXCore::Core::CpuSt
auto HostCode = Thread->LookupCache->FindBlock(GuestRip);
if (!HostCode) {
//printf("ExitFunctionLink: Aborting, %lX not in cache\n", GuestRip);
//fmt::print("ExitFunctionLink: Aborting, {:X} not in cache\n", GuestRip);
Frame->State.rip = GuestRip;
return core->Dispatcher->AbsoluteLoopTopAddress;
return core->ThreadSharedData.Dispatcher->AbsoluteLoopTopAddress;
}
uintptr_t branch = (uintptr_t)(record) - 8;
auto LinkerAddress = core->Dispatcher->ExitFunctionLinkerAddress;
auto LinkerAddress = core->ThreadSharedData.Dispatcher->ExitFunctionLinkerAddress;
auto offset = HostCode/4 - branch/4;
if (IsInt26(offset)) {
@@ -853,7 +935,7 @@ uint64_t Arm64JITCore::ExitFunctionLink(Arm64JITCore *core, FEXCore::Core::CpuSt
return HostCode;
}
FEXCore::CPU::CPUBackend *CreateArm64JITCore(FEXCore::Context::Context *ctx, FEXCore::Core::InternalThreadState *Thread, bool CompileThread) {
return new Arm64JITCore(ctx, Thread, CompileThread);
std::unique_ptr<CPUBackend> CreateArm64JITCore(FEXCore::Context::Context *ctx, FEXCore::Core::InternalThreadState *Thread, bool CompileThread) {
return std::make_unique<Arm64JITCore>(ctx, Thread, CompileThread);
}
}
+33 -21
View File
@@ -6,7 +6,6 @@ $end_info$
#pragma once
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/ArchHelpers/Arm64Emitter.h"
#include "Interface/Core/Dispatcher/Dispatcher.h"
@@ -47,7 +46,7 @@ public:
~Arm64JITCore() override;
std::string GetName() override { return "JIT"; }
void *CompileCode(FEXCore::IR::IRListView const *IR, FEXCore::Core::DebugData *DebugData, FEXCore::IR::RegisterAllocationData *RAData) override;
void *CompileCode(uint64_t Entry, FEXCore::IR::IRListView const *IR, FEXCore::Core::DebugData *DebugData, FEXCore::IR::RegisterAllocationData *RAData) override;
void *MapRegion(void* HostPtr, uint64_t, uint64_t) override { return HostPtr; }
@@ -63,11 +62,14 @@ public:
void CopyNecessaryDataForCompileThread(CPUBackend *Original) override;
private:
Dispatcher *Dispatcher;
FEX_CONFIG_OPT(ParanoidTSO, PARANOIDTSO);
std::unique_ptr<FEXCore::CPU::Dispatcher> Dispatcher;
Label *PendingTargetLabel;
FEXCore::Context::Context *CTX;
FEXCore::Core::InternalThreadState *ThreadState;
FEXCore::IR::IRListView const *IR;
uint64_t Entry;
std::map<IR::OrderedNodeWrapper::NodeOffsetType, aarch64::Label> JumpTargets;
@@ -93,33 +95,35 @@ private:
constexpr static uint8_t RA_FPR = 2;
template<uint8_t RAType>
aarch64::Register GetReg(uint32_t Node);
aarch64::Register GetReg(uint32_t Node) const;
template<>
aarch64::Register GetReg<RA_32>(uint32_t Node);
aarch64::Register GetReg<RA_32>(uint32_t Node) const;
template<>
aarch64::Register GetReg<RA_64>(uint32_t Node);
aarch64::Register GetReg<RA_64>(uint32_t Node) const;
template<uint8_t RAType>
std::pair<aarch64::Register, aarch64::Register> GetSrcPair(uint32_t Node);
std::pair<aarch64::Register, aarch64::Register> GetSrcPair(uint32_t Node) const;
template<>
std::pair<aarch64::Register, aarch64::Register> GetSrcPair<RA_32>(uint32_t Node);
std::pair<aarch64::Register, aarch64::Register> GetSrcPair<RA_32>(uint32_t Node) const;
template<>
std::pair<aarch64::Register, aarch64::Register> GetSrcPair<RA_64>(uint32_t Node);
std::pair<aarch64::Register, aarch64::Register> GetSrcPair<RA_64>(uint32_t Node) const;
aarch64::VRegister GetSrc(uint32_t Node);
aarch64::VRegister GetDst(uint32_t Node);
aarch64::VRegister GetSrc(uint32_t Node) const;
aarch64::VRegister GetDst(uint32_t Node) const;
FEXCore::IR::RegisterClassType GetRegClass(uint32_t Node);
FEXCore::IR::RegisterClassType GetRegClass(uint32_t Node) const;
bool IsFPR(uint32_t Node);
bool IsGPR(uint32_t Node);
IR::PhysicalRegister GetPhys(uint32_t Node) const;
bool IsFPR(uint32_t Node) const;
bool IsGPR(uint32_t Node) const;
MemOperand GenerateMemOperand(uint8_t AccessSize, aarch64::Register Base, IR::OrderedNodeWrapper Offset, IR::MemOffsetType OffsetType, uint8_t OffsetScale);
bool IsInlineConstant(const IR::OrderedNodeWrapper& Node, uint64_t* Value = nullptr);
bool IsInlineEntrypointOffset(const IR::OrderedNodeWrapper& WNode, uint64_t* Value);
bool IsInlineConstant(const IR::OrderedNodeWrapper& Node, uint64_t* Value = nullptr) const;
bool IsInlineEntrypointOffset(const IR::OrderedNodeWrapper& WNode, uint64_t* Value) const;
struct LiveRange {
uint32_t Begin;
@@ -162,6 +166,7 @@ private:
uint64_t SignalReturnInstruction{};
uint32_t *SignalHandlerRefCounterPtr{};
FEXCore::CPU::Dispatcher *Dispatcher{};
};
CompilerSharedData ThreadSharedData;
@@ -234,7 +239,6 @@ private:
DEF_OP(VExtractToGPR);
DEF_OP(Float_ToGPR_ZU);
DEF_OP(Float_ToGPR_ZS);
DEF_OP(Float_ToGPR_U);
DEF_OP(Float_ToGPR_S);
DEF_OP(FCmp);
@@ -252,6 +256,7 @@ private:
DEF_OP(AtomicFetchAnd);
DEF_OP(AtomicFetchOr);
DEF_OP(AtomicFetchXor);
DEF_OP(AtomicFetchNeg);
///< Branch ops
DEF_OP(GuestCallDirect);
@@ -271,16 +276,13 @@ private:
///< Conversion ops
DEF_OP(VInsGPR);
DEF_OP(VCastFromGPR);
DEF_OP(Float_FromGPR_U);
DEF_OP(Float_FromGPR_S);
DEF_OP(Float_FToF);
DEF_OP(Vector_UToF);
DEF_OP(Vector_SToF);
DEF_OP(Vector_FToZU);
DEF_OP(Vector_FToZS);
DEF_OP(Vector_FToU);
DEF_OP(Vector_FToS);
DEF_OP(Vector_FToF);
DEF_OP(Vector_FToI);
///< Flag ops
DEF_OP(GetHostFlag);
@@ -300,8 +302,11 @@ private:
DEF_OP(StoreMem);
DEF_OP(LoadMemTSO);
DEF_OP(StoreMemTSO);
DEF_OP(ParanoidLoadMemTSO);
DEF_OP(ParanoidStoreMemTSO);
DEF_OP(VLoadMemElement);
DEF_OP(VStoreMemElement);
DEF_OP(CacheLineClear);
///< Misc ops
DEF_OP(EndBlock);
@@ -327,6 +332,7 @@ private:
DEF_OP(SplatVector4);
DEF_OP(VMov);
DEF_OP(VAnd);
DEF_OP(VBic);
DEF_OP(VOr);
DEF_OP(VXor);
DEF_OP(VAdd);
@@ -337,8 +343,10 @@ private:
DEF_OP(VSQSub);
DEF_OP(VAddP);
DEF_OP(VAddV);
DEF_OP(VUMinV);
DEF_OP(VURAvg);
DEF_OP(VAbs);
DEF_OP(VPopcount);
DEF_OP(VFAdd);
DEF_OP(VFAddP);
DEF_OP(VFSub);
@@ -358,6 +366,8 @@ private:
DEF_OP(VSMax);
DEF_OP(VZip);
DEF_OP(VZip2);
DEF_OP(VUnZip);
DEF_OP(VUnZip2);
DEF_OP(VBSL);
DEF_OP(VCMPEQ);
DEF_OP(VCMPEQZ);
@@ -380,6 +390,7 @@ private:
DEF_OP(VInsElement);
DEF_OP(VInsScalarElement);
DEF_OP(VExtractElement);
DEF_OP(VDupElement);
DEF_OP(VExtr);
DEF_OP(VSLI);
DEF_OP(VSRI);
@@ -402,6 +413,7 @@ private:
DEF_OP(VSMull);
DEF_OP(VUMull2);
DEF_OP(VSMull2);
DEF_OP(VUABDL);
DEF_OP(VTBL1);
///< Encryption ops
+223 -63
View File
@@ -5,6 +5,7 @@ $end_info$
*/
#include "Interface/Core/JIT/Arm64/JITClass.h"
#include <FEXCore/Utils/CompilerDefs.h>
namespace FEXCore::CPU {
@@ -29,7 +30,7 @@ DEF_OP(LoadContext) {
case 8:
ldr(GetReg<RA_64>(Node), MemOperand(STATE, Op->Offset));
break;
default: LogMan::Msg::A("Unhandled LoadContext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled LoadContext size: {}", OpSize);
}
}
else {
@@ -50,7 +51,7 @@ DEF_OP(LoadContext) {
case 16:
ldr(Dst, MemOperand(STATE, Op->Offset));
break;
default: LogMan::Msg::A("Unhandled LoadContext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled LoadContext size: {}", OpSize);
}
}
}
@@ -72,7 +73,7 @@ DEF_OP(StoreContext) {
case 8:
str(GetReg<RA_64>(Op->Header.Args[0].ID()), MemOperand(STATE, Op->Offset));
break;
default: LogMan::Msg::A("Unhandled StoreContext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled StoreContext size: {}", OpSize);
}
}
else {
@@ -93,7 +94,7 @@ DEF_OP(StoreContext) {
case 16:
str(Src, MemOperand(STATE, Op->Offset));
break;
default: LogMan::Msg::A("Unhandled LoadContext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled StoreContext size: {}", OpSize);
}
}
}
@@ -106,29 +107,29 @@ DEF_OP(LoadRegister) {
auto regId = (Op->Offset - offsetof(FEXCore::Core::CpuStateFrame, State.gregs[0])) / 8;
auto regOffs = Op->Offset & 7;
LogMan::Throw::A(regId < SRA64.size(), "out of range regId");
LOGMAN_THROW_A_FMT(regId < SRA64.size(), "out of range regId");
auto reg = SRA64[regId];
switch(Op->Header.Size) {
case 1:
LogMan::Throw::A(regOffs == 0 || regOffs == 1, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0 || regOffs == 1, "unexpected regOffs");
ubfx(GetReg<RA_64>(Node), reg, regOffs * 8, 8);
break;
case 2:
LogMan::Throw::A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
ubfx(GetReg<RA_64>(Node), reg, 0, 16);
break;
case 4:
LogMan::Throw::A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
if (GetReg<RA_64>(Node).GetCode() != reg.GetCode())
mov(GetReg<RA_32>(Node), reg.W());
break;
case 8:
LogMan::Throw::A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
if (GetReg<RA_64>(Node).GetCode() != reg.GetCode())
mov(GetReg<RA_64>(Node), reg);
break;
@@ -137,24 +138,24 @@ DEF_OP(LoadRegister) {
auto regId = (Op->Offset - offsetof(FEXCore::Core::CpuStateFrame, State.xmm[0][0])) / 16;
auto regOffs = Op->Offset & 15;
LogMan::Throw::A(regId < SRAFPR.size(), "out of range regId");
LOGMAN_THROW_A_FMT(regId < SRAFPR.size(), "out of range regId");
auto guest = SRAFPR[regId];
auto host = GetSrc(Node);
switch(Op->Header.Size) {
case 1:
LogMan::Throw::A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
mov(host.B(), guest.B());
break;
case 2:
LogMan::Throw::A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
fmov(host.H(), guest.H());
break;
case 4:
LogMan::Throw::A((regOffs & 3) == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT((regOffs & 3) == 0, "unexpected regOffs");
if (regOffs == 0) {
if (host.GetCode() != guest.GetCode())
fmov(host.S(), guest.S());
@@ -164,7 +165,7 @@ DEF_OP(LoadRegister) {
break;
case 8:
LogMan::Throw::A((regOffs & 7) == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT((regOffs & 7) == 0, "unexpected regOffs");
if (regOffs == 0) {
if (host.GetCode() != guest.GetCode())
mov(host.D(), guest.D());
@@ -174,13 +175,13 @@ DEF_OP(LoadRegister) {
break;
case 16:
LogMan::Throw::A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
if (host.GetCode() != guest.GetCode())
mov(host.Q(), guest.Q());
break;
}
} else {
LogMan::Throw::A(false, "Unhandled Op->Class %d", Op->Class);
LOGMAN_THROW_A_FMT(false, "Unhandled Op->Class {}", Op->Class);
}
}
@@ -191,28 +192,28 @@ DEF_OP(StoreRegister) {
auto regId = Op->Offset / 8 - 1;
auto regOffs = Op->Offset & 7;
LogMan::Throw::A(regId < SRA64.size(), "out of range regId");
LOGMAN_THROW_A_FMT(regId < SRA64.size(), "out of range regId");
auto reg = SRA64[regId];
switch(Op->Header.Size) {
case 1:
LogMan::Throw::A(regOffs == 0 || regOffs == 1, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0 || regOffs == 1, "unexpected regOffs");
bfi(reg, GetReg<RA_64>(Op->Value.ID()), regOffs * 8, 8);
break;
case 2:
LogMan::Throw::A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
bfi(reg, GetReg<RA_64>(Op->Value.ID()), 0, 16);
break;
case 4:
LogMan::Throw::A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
bfi(reg, GetReg<RA_64>(Op->Value.ID()), 0, 32);
break;
case 8:
LogMan::Throw::A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
if (GetReg<RA_64>(Op->Value.ID()).GetCode() != reg.GetCode())
mov(reg, GetReg<RA_64>(Op->Value.ID()));
break;
@@ -221,7 +222,7 @@ DEF_OP(StoreRegister) {
auto regId = (Op->Offset - offsetof(FEXCore::Core::CpuStateFrame, State.xmm[0][0])) / 16;
auto regOffs = Op->Offset & 15;
LogMan::Throw::A(regId < SRAFPR.size(), "regId out of range");
LOGMAN_THROW_A_FMT(regId < SRAFPR.size(), "regId out of range");
auto guest = SRAFPR[regId];
auto host = GetSrc(Op->Value.ID());
@@ -232,28 +233,28 @@ DEF_OP(StoreRegister) {
break;
case 2:
LogMan::Throw::A((regOffs & 1) == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT((regOffs & 1) == 0, "unexpected regOffs");
ins(guest.V8H(), regOffs/2, host.V8H(), 0);
break;
case 4:
LogMan::Throw::A((regOffs & 3) == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT((regOffs & 3) == 0, "unexpected regOffs");
ins(guest.V4S(), regOffs/4, host.V4S(), 0);
break;
case 8:
LogMan::Throw::A((regOffs & 7) == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT((regOffs & 7) == 0, "unexpected regOffs");
ins(guest.V2D(), regOffs / 8, host.V2D(), 0);
break;
case 16:
LogMan::Throw::A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
if (guest.GetCode() != host.GetCode())
mov(guest.Q(), host.Q());
break;
}
} else {
LogMan::Throw::A(false, "Unhandled Op->Class %d", Op->Class);
LOGMAN_THROW_A_FMT(false, "Unhandled Op->Class {}", Op->Class);
}
}
@@ -287,15 +288,17 @@ DEF_OP(LoadContextIndexed) {
ldr(GetReg<RA_64>(Node), MemOperand(TMP1, Op->BaseOffset));
break;
default:
LogMan::Msg::A("Unhandled LoadContextIndexed size: %d", Op->Size);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed size: {}", Op->Size);
break;
}
break;
}
case 16:
LogMan::Msg::A("Invalid Class load of size 16");
LOGMAN_MSG_A_FMT("Invalid Class load of size 16");
break;
default:
LogMan::Msg::A("Unhandled LoadContextIndexed stride: %d", Op->Stride);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed stride: {}", Op->Stride);
break;
}
}
else {
@@ -332,12 +335,14 @@ DEF_OP(LoadContextIndexed) {
}
break;
default:
LogMan::Msg::A("Unhandled LoadContextIndexed size: %d", Op->Size);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed size: {}", Op->Size);
break;
}
break;
}
default:
LogMan::Msg::A("Unhandled LoadContextIndexed stride: %d", Op->Stride);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed stride: {}", Op->Stride);
break;
}
}
}
@@ -373,15 +378,17 @@ DEF_OP(StoreContextIndexed) {
str(value, MemOperand(TMP1, Op->BaseOffset));
break;
default:
LogMan::Msg::A("Unhandled LoadContextIndexed size: %d", Op->Size);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed size: {}", Op->Size);
break;
}
break;
}
case 16:
LogMan::Msg::A("Invalid Class load of size 16");
LOGMAN_MSG_A_FMT("Invalid Class store of size 16");
break;
default:
LogMan::Msg::A("Unhandled LoadContextIndexed stride: %d", Op->Stride);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed stride: {}", Op->Stride);
break;
}
}
else {
@@ -420,12 +427,14 @@ DEF_OP(StoreContextIndexed) {
}
break;
default:
LogMan::Msg::A("Unhandled LoadContextIndexed size: %d", Op->Size);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed size: {}", Op->Size);
break;
}
break;
}
default:
LogMan::Msg::A("Unhandled LoadContextIndexed stride: %d", Op->Stride);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed stride: {}", Op->Stride);
break;
}
}
}
@@ -433,7 +442,7 @@ DEF_OP(StoreContextIndexed) {
DEF_OP(SpillRegister) {
auto Op = IROp->C<IR::IROp_SpillRegister>();
uint8_t OpSize = IROp->Size;
uint32_t SlotOffset = Op->Slot * 16 + 16;
uint32_t SlotOffset = Op->Slot * 16;
if (Op->Class == FEXCore::IR::GPRClass) {
switch (OpSize) {
@@ -453,7 +462,7 @@ DEF_OP(SpillRegister) {
str(GetReg<RA_64>(Op->Header.Args[0].ID()), MemOperand(sp, SlotOffset));
break;
}
default: LogMan::Msg::A("Unhandled SpillRegister size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled SpillRegister size: {}", OpSize);
}
} else if (Op->Class == FEXCore::IR::FPRClass) {
switch (OpSize) {
@@ -469,17 +478,17 @@ DEF_OP(SpillRegister) {
str(GetSrc(Op->Header.Args[0].ID()), MemOperand(sp, SlotOffset));
break;
}
default: LogMan::Msg::A("Unhandled SpillRegister size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled SpillRegister size: {}", OpSize);
}
} else {
LogMan::Msg::A("Unhandled SpillRegister class: %d", Op->Class.Val);
LOGMAN_MSG_A_FMT("Unhandled SpillRegister class: {}", Op->Class.Val);
}
}
DEF_OP(FillRegister) {
auto Op = IROp->C<IR::IROp_FillRegister>();
uint8_t OpSize = IROp->Size;
uint32_t SlotOffset = Op->Slot * 16 + 16;
uint32_t SlotOffset = Op->Slot * 16;
if (Op->Class == FEXCore::IR::GPRClass) {
switch (OpSize) {
@@ -499,7 +508,7 @@ DEF_OP(FillRegister) {
ldr(GetReg<RA_64>(Node), MemOperand(sp, SlotOffset));
break;
}
default: LogMan::Msg::A("Unhandled SpillRegister size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled FillRegister size: {}", OpSize);
}
} else if (Op->Class == FEXCore::IR::FPRClass) {
switch (OpSize) {
@@ -515,10 +524,10 @@ DEF_OP(FillRegister) {
ldr(GetDst(Node), MemOperand(sp, SlotOffset));
break;
}
default: LogMan::Msg::A("Unhandled SpillRegister size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled FillRegister size: {}", OpSize);
}
} else {
LogMan::Msg::A("Unhandled FillRegister class: %d", Op->Class.Val);
LOGMAN_MSG_A_FMT("Unhandled FillRegister class: {}", Op->Class.Val);
}
}
@@ -538,7 +547,7 @@ MemOperand Arm64JITCore::GenerateMemOperand(uint8_t AccessSize, aarch64::Registe
return MemOperand(Base);
} else {
if (OffsetScale != 1 && OffsetScale != AccessSize) {
LogMan::Msg::A("Unhandled GenerateMemOperand OffsetScale: %d", OffsetScale);
LOGMAN_MSG_A_FMT("Unhandled GenerateMemOperand OffsetScale: {}", OffsetScale);
}
uint64_t Const;
if (IsInlineConstant(Offset, &Const)) {
@@ -550,11 +559,12 @@ MemOperand Arm64JITCore::GenerateMemOperand(uint8_t AccessSize, aarch64::Registe
case IR::MEM_OFFSET_UXTW.Val: return MemOperand(Base, RegOffset.W(), Extend::UXTW, (int)std::log2(OffsetScale) );
case IR::MEM_OFFSET_SXTW.Val: return MemOperand(Base, RegOffset.W(), Extend::SXTW, (int)std::log2(OffsetScale) );
default: LogMan::Msg::A("Unhandled GenerateMemOperand OffsetType: %d", OffsetType.Val); break;
default: LOGMAN_MSG_A_FMT("Unhandled GenerateMemOperand OffsetType: {}", OffsetType.Val); break;
}
}
}
__builtin_unreachable();
FEX_UNREACHABLE;
}
DEF_OP(LoadMem) {
@@ -578,7 +588,7 @@ DEF_OP(LoadMem) {
case 8:
ldr(Dst, MemSrc);
break;
default: LogMan::Msg::A("Unhandled LoadMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled LoadMem size: {}", Op->Size);
}
}
else {
@@ -599,7 +609,7 @@ DEF_OP(LoadMem) {
case 16:
ldr(Dst, MemSrc);
break;
default: LogMan::Msg::A("Unhandled LoadMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled LoadMem size: {}", Op->Size);
}
}
}
@@ -610,7 +620,7 @@ DEF_OP(LoadMemTSO) {
auto MemSrc = MemOperand(GetReg<RA_64>(Op->Header.Args[0].ID()));
if (!Op->Offset.IsInvalid()) {
LogMan::Msg::A("LoadMemTSO: No offset allowed");
LOGMAN_MSG_A_FMT("LoadMemTSO: No offset allowed");
}
if (SupportsRCPC && Op->Class == FEXCore::IR::GPRClass) {
@@ -633,7 +643,7 @@ DEF_OP(LoadMemTSO) {
case 8:
ldapr(Dst, MemSrc);
break;
default: LogMan::Msg::A("Unhandled LoadMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled LoadMemTSO size: {}", Op->Size);
}
nop();
}
@@ -658,7 +668,7 @@ DEF_OP(LoadMemTSO) {
case 8:
ldar(Dst, MemSrc);
break;
default: LogMan::Msg::A("Unhandled LoadMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled LoadMemTSO size: {}", Op->Size);
}
nop();
}
@@ -679,7 +689,7 @@ DEF_OP(LoadMemTSO) {
case 16:
ldr(Dst, MemSrc);
break;
default: LogMan::Msg::A("Unhandled LoadMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled LoadMemTSO size: {}", Op->Size);
}
dmb(InnerShareable, BarrierAll);
}
@@ -706,7 +716,7 @@ DEF_OP(StoreMem) {
case 8:
str(GetReg<RA_64>(Op->Header.Args[1].ID()), MemSrc);
break;
default: LogMan::Msg::A("Unhandled StoreMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled StoreMem size: {}", Op->Size);
}
}
else {
@@ -727,7 +737,7 @@ DEF_OP(StoreMem) {
case 16:
str(Src, MemSrc);
break;
default: LogMan::Msg::A("Unhandled StoreMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled StoreMem size: {}", Op->Size);
}
}
}
@@ -737,7 +747,7 @@ DEF_OP(StoreMemTSO) {
auto MemSrc = MemOperand(GetReg<RA_64>(Op->Header.Args[0].ID()));
if (!Op->Offset.IsInvalid()) {
LogMan::Msg::A("StoreMemTSO: No offset allowed");
LOGMAN_MSG_A_FMT("StoreMemTSO: No offset allowed");
}
if (Op->Class == FEXCore::IR::GPRClass) {
@@ -757,7 +767,7 @@ DEF_OP(StoreMemTSO) {
case 8:
stlr(GetReg<RA_64>(Op->Header.Args[1].ID()), MemSrc);
break;
default: LogMan::Msg::A("Unhandled StoreMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled StoreMemTSO size: {}", Op->Size);
}
nop();
}
@@ -781,18 +791,161 @@ DEF_OP(StoreMemTSO) {
case 16:
str(Src, MemSrc);
break;
default: LogMan::Msg::A("Unhandled StoreMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled StoreMemTSO size: {}", Op->Size);
}
dmb(InnerShareable, BarrierAll);
}
}
DEF_OP(ParanoidLoadMemTSO) {
auto Op = IROp->C<IR::IROp_LoadMemTSO>();
auto MemSrc = MemOperand(GetReg<RA_64>(Op->Header.Args[0].ID()));
if (!Op->Offset.IsInvalid()) {
LOGMAN_MSG_A_FMT("ParanoidLoadMemTSO: No offset allowed");
}
if (Op->Class == FEXCore::IR::GPRClass) {
if (Op->Size == 1) {
// 8bit load is always aligned to natural alignment
auto Dst = GetReg<RA_64>(Node);
ldarb(Dst, MemSrc);
}
else {
auto Dst = GetReg<RA_64>(Node);
switch (Op->Size) {
case 2:
ldarh(Dst, MemSrc);
break;
case 4:
ldar(Dst.W(), MemSrc);
break;
case 8:
ldar(Dst, MemSrc);
break;
default: LOGMAN_MSG_A_FMT("Unhandled ParanoidLoadMemTSO size: {}", Op->Size);
}
}
}
else {
auto Dst = GetDst(Node);
switch (Op->Size) {
case 2:
ldarh(TMP1.W(), MemSrc);
fmov(Dst.H(), TMP1.W());
break;
case 4:
ldar(TMP1.W(), MemSrc);
fmov(Dst.S(), TMP1.W());
break;
case 8:
ldar(TMP1, MemSrc);
fmov(Dst.D(), TMP1);
break;
case 16:
nop();
ldaxp(TMP1, TMP2, MemSrc);
clrex();
mov(Dst.V2D(), 0, TMP1);
mov(Dst.V2D(), 1, TMP2);
break;
default: LOGMAN_MSG_A_FMT("Unhandled ParanoidLoadMemTSO size: {}", Op->Size);
}
}
}
DEF_OP(ParanoidStoreMemTSO) {
auto Op = IROp->C<IR::IROp_StoreMemTSO>();
auto MemSrc = MemOperand(GetReg<RA_64>(Op->Header.Args[0].ID()));
if (!Op->Offset.IsInvalid()) {
LOGMAN_MSG_A_FMT("ParanoidStoreMemTSO: No offset allowed");
}
if (Op->Class == FEXCore::IR::GPRClass) {
if (Op->Size == 1) {
// 8bit load is always aligned to natural alignment
stlrb(GetReg<RA_64>(Op->Header.Args[1].ID()), MemSrc);
}
else {
switch (Op->Size) {
case 2:
stlrh(GetReg<RA_64>(Op->Header.Args[1].ID()), MemSrc);
break;
case 4:
stlr(GetReg<RA_32>(Op->Header.Args[1].ID()), MemSrc);
break;
case 8:
stlr(GetReg<RA_64>(Op->Header.Args[1].ID()), MemSrc);
break;
default: LOGMAN_MSG_A_FMT("Unhandled ParanoidStoreMemTSO size: {}", Op->Size);
}
}
}
else {
auto Src = GetSrc(Op->Header.Args[1].ID());
if (Op->Size == 1) {
// 8bit load is always aligned to natural alignment
mov(TMP1.W(), Src.V16B(), 0);
stlrb(TMP1, MemSrc);
}
else {
switch (Op->Size) {
case 2:
mov(TMP1.W(), Src.V8H(), 0);
stlrh(TMP1, MemSrc);
break;
case 4:
mov(TMP1.W(), Src.V4S(), 0);
stlr(TMP1.W(), MemSrc);
break;
case 8:
mov(TMP1, Src.V2D(), 0);
stlr(TMP1, MemSrc);
break;
case 16: {
// Move vector to GPRs
mov(TMP1, Src.V2D(), 0);
mov(TMP2, Src.V2D(), 1);
Label B;
bind(&B);
nop(); // < Overwritten with DMB
// ldaxp must not have both the destination registers be the same
ldaxp(xzr, TMP3, MemSrc); // <- Can hit SIGBUS
nop(); // < Overwritten with DMB
stlxp(TMP3, TMP1, TMP2, MemSrc); // <- Can also hit SIGBUS
cbnz(TMP3, &B); // < Overwritten with DMB
break;
}
default: LOGMAN_MSG_A_FMT("Unhandled ParanoidStoreMemTSO size: {}", Op->Size);
}
}
}
}
DEF_OP(VLoadMemElement) {
LogMan::Msg::A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(VStoreMemElement) {
LogMan::Msg::A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(CacheLineClear) {
auto Op = IROp->C<IR::IROp_CacheLineClear>();
auto MemReg = GetReg<RA_64>(Op->Header.Args[0].ID());
// Clear dcache only
// icache doesn't matter here since the guest application shouldn't be calling clflush on JIT code.
mov(TMP1, MemReg);
for (size_t i = 0; i < std::max(1U, DCacheLineSize / 64U); ++i) {
dc(DataCacheOp::CVAU, TMP1);
add(TMP1, TMP1, DCacheLineSize);
}
dsb(InnerShareable, BarrierAll);
}
#undef DEF_OP
@@ -810,10 +963,17 @@ void Arm64JITCore::RegisterMemoryHandlers() {
REGISTER_OP(STOREFLAG, StoreFlag);
REGISTER_OP(LOADMEM, LoadMem);
REGISTER_OP(STOREMEM, StoreMem);
REGISTER_OP(LOADMEMTSO, LoadMemTSO);
REGISTER_OP(STOREMEMTSO, StoreMemTSO);
if (ParanoidTSO()) {
REGISTER_OP(LOADMEMTSO, ParanoidLoadMemTSO);
REGISTER_OP(STOREMEMTSO, ParanoidStoreMemTSO);
}
else {
REGISTER_OP(LOADMEMTSO, LoadMemTSO);
REGISTER_OP(STOREMEMTSO, StoreMemTSO);
}
REGISTER_OP(VLOADMEMELEMENT, VLoadMemElement);
REGISTER_OP(VSTOREMEMELEMENT, VStoreMemElement);
REGISTER_OP(CACHELINECLEAR, CacheLineClear);
#undef REGISTER_OP
}
}
@@ -7,6 +7,13 @@ $end_info$
#include "Interface/Core/JIT/Arm64/JITClass.h"
namespace FEXCore::CPU {
static void PrintValue(uint64_t Value) {
LogMan::Msg::DFmt("Value: 0x{:x}", Value);
}
static void PrintVectorValue(uint64_t Value, uint64_t ValueUpper) {
LogMan::Msg::DFmt("Value: 0x{:016x}'{:016x}", ValueUpper, Value);
}
using namespace vixl;
using namespace vixl::aarch64;
@@ -24,7 +31,7 @@ DEF_OP(Fence) {
case IR::Fence_Store.Val:
dmb(FullSystem, BarrierWrites);
break;
default: LogMan::Msg::A("Unknown Fence: %d", Op->Fence); break;
default: LOGMAN_MSG_A_FMT("Unknown Fence: {}", Op->Fence); break;
}
}
@@ -35,6 +42,15 @@ DEF_OP(Break) {
case 5: // Guest ud2
hlt(4);
break;
case 1: // Int <imm8>
hlt(4);
break;
case 2: // overflow
hlt(4);
break;
case 3: // int 1
hlt(4);
break;
case 4: { // HLT
// Time to quit
// Set our stack to the starting stack location
@@ -42,18 +58,18 @@ DEF_OP(Break) {
add(sp, TMP1, 0);
// Now we need to jump to the thread stop handler
LoadConstant(TMP1, Dispatcher->ThreadStopHandlerAddressSpillSRA);
LoadConstant(TMP1, ThreadSharedData.Dispatcher->ThreadStopHandlerAddressSpillSRA);
br(TMP1);
break;
}
case 6: { // INT3
ResetStack();
LoadConstant(TMP1, Dispatcher->ThreadPauseHandlerAddressSpillSRA);
LoadConstant(TMP1, ThreadSharedData.Dispatcher->ThreadPauseHandlerAddressSpillSRA);
br(TMP1);
break;
}
default: LogMan::Msg::A("Unknown Break reason: %d", Op->Reason);
default: LOGMAN_MSG_A_FMT("Unknown Break reason: {}", Op->Reason);
}
}
@@ -115,6 +131,28 @@ DEF_OP(SetRoundingMode) {
msr(FPCR, TMP1);
}
DEF_OP(Print) {
auto Op = IROp->C<IR::IROp_Print>();
PushDynamicRegsAndLR();
if (IsGPR(Op->Header.Args[0].ID())) {
mov(x0, GetReg<RA_64>(Op->Header.Args[0].ID()));
LoadConstant(x3, reinterpret_cast<uint64_t>(PrintValue));
}
else {
fmov(x0, GetSrc(Op->Header.Args[0].ID()).V1D());
// Bug in vixl that source vector needs to b V1D rather than V2D?
fmov(x1, GetSrc(Op->Header.Args[0].ID()).V1D(), 1);
LoadConstant(x3, reinterpret_cast<uint64_t>(PrintVectorValue));
}
SpillStaticRegs();
blr(x3);
FillStaticRegs();
PopDynamicRegsAndLR();
}
#undef DEF_OP
void Arm64JITCore::RegisterMiscHandlers() {
#define REGISTER_OP(op, x) OpHandlers[FEXCore::IR::IROps::OP_##op] = &Arm64JITCore::Op_##x
@@ -127,7 +165,7 @@ void Arm64JITCore::RegisterMiscHandlers() {
REGISTER_OP(BREAK, Break);
REGISTER_OP(PHI, NoOp);
REGISTER_OP(PHIVALUE, NoOp);
REGISTER_OP(PRINT, Unhandled);
REGISTER_OP(PRINT, Print);
REGISTER_OP(GETROUNDINGMODE, GetRoundingMode);
REGISTER_OP(SETROUNDINGMODE, SetRoundingMode);
REGISTER_OP(INVALIDATEFLAGS, NoOp);
@@ -26,7 +26,7 @@ DEF_OP(ExtractElementPair) {
mov (GetReg<RA_64>(Node), Regs[Op->Element]);
break;
}
default: LogMan::Msg::A("Unknown Size"); break;
default: LOGMAN_MSG_A_FMT("Unknown Size"); break;
}
}
@@ -52,7 +52,7 @@ DEF_OP(CreateElementPair) {
RegTmp = x0;
break;
}
default: LogMan::Msg::A("Unknown Size"); break;
default: LOGMAN_MSG_A_FMT("Unknown Size"); break;
}
if (Dst.first.GetCode() != RegSecond.GetCode()) {
File diff suppressed because it is too large. Load diff
+4 -2
View File
@@ -1,5 +1,7 @@
#pragma once
#include <memory>
namespace FEXCore::Context {
struct Context;
}
@@ -11,6 +13,6 @@ struct InternalThreadState;
namespace FEXCore::CPU {
class CPUBackend;
FEXCore::CPU::CPUBackend *CreateX86JITCore(FEXCore::Context::Context *ctx, FEXCore::Core::InternalThreadState *Thread, bool CompileThread);
FEXCore::CPU::CPUBackend *CreateArm64JITCore(FEXCore::Context::Context *ctx, FEXCore::Core::InternalThreadState *Thread, bool CompileThread);
std::unique_ptr<CPUBackend> CreateX86JITCore(FEXCore::Context::Context *ctx, FEXCore::Core::InternalThreadState *Thread, bool CompileThread);
std::unique_ptr<CPUBackend> CreateArm64JITCore(FEXCore::Context::Context *ctx, FEXCore::Core::InternalThreadState *Thread, bool CompileThread);
}
+77 -60
View File
@@ -20,7 +20,7 @@ DEF_OP(TruncElementPair) {
mov(Dst.second, Src.second);
break;
}
default: LogMan::Msg::A("Unhandled Truncation size: %d", Op->Size); break;
default: LOGMAN_MSG_A_FMT("Unhandled Truncation size: {}", Op->Size); break;
}
}
@@ -32,7 +32,7 @@ DEF_OP(Constant) {
DEF_OP(EntrypointOffset) {
auto Op = IROp->C<IR::IROp_EntrypointOffset>();
auto Constant = IR->GetHeader()->Entry + Op->Offset;
auto Constant = Entry + Op->Offset;
mov(GetDst<RA_64>(Node), Constant);
}
@@ -70,7 +70,7 @@ DEF_OP(Add) {
case 8:
add(rax, Const);
break;
default: LogMan::Msg::A("Unhandled Add size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled Add size: {}", OpSize);
break;
}
} else {
@@ -81,7 +81,7 @@ DEF_OP(Add) {
case 8:
add(rax, GetSrc<RA_64>(Op->Header.Args[1].ID()));
break;
default: LogMan::Msg::A("Unhandled Add size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled Add size: {}", OpSize);
break;
}
}
@@ -103,7 +103,7 @@ DEF_OP(Sub) {
case 8:
sub(rax, Const);
break;
default: LogMan::Msg::A("Unhandled Sub size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled Sub size: {}", OpSize);
break;
}
} else {
@@ -114,7 +114,7 @@ DEF_OP(Sub) {
case 8:
sub(rax, GetSrc<RA_64>(Op->Header.Args[1].ID()));
break;
default: LogMan::Msg::A("Unhandled Sub size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled Sub size: {}", OpSize);
break;
}
}
@@ -136,7 +136,7 @@ DEF_OP(Neg) {
Src = GetSrc<RA_64>(Op->Header.Args[0].ID());
Dst = GetDst<RA_64>(Node);
break;
default: LogMan::Msg::A("Unhandled Neg size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled Neg size: {}", OpSize);
break;
}
mov(Dst, Src);
@@ -160,7 +160,7 @@ DEF_OP(Mul) {
imul(rax, GetSrc<RA_64>(Op->Header.Args[1].ID()));
mov(Dst, rax);
break;
default: LogMan::Msg::A("Unknown Sext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown Mul size: {}", OpSize);
}
}
@@ -179,7 +179,7 @@ DEF_OP(UMul) {
mul(GetSrc<RA_64>(Op->Header.Args[1].ID()));
mov(GetDst<RA_64>(Node), rax);
break;
default: LogMan::Msg::A("Unknown Sext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown UMul size: {}", OpSize);
}
}
@@ -218,7 +218,7 @@ DEF_OP(Div) {
mov(GetDst<RA_64>(Node), rax);
break;
}
default: LogMan::Msg::A("Unknown UDIV Size: %d", Size); break;
default: LOGMAN_MSG_A_FMT("Unknown DIV Size: {}", Size); break;
}
}
@@ -261,7 +261,7 @@ DEF_OP(UDiv) {
mov(GetDst<RA_64>(Node), rax);
break;
}
default: LogMan::Msg::A("Unknown UDIV OpSize: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown UDIV OpSize: {}", OpSize); break;
}
}
@@ -298,7 +298,7 @@ DEF_OP(Rem) {
mov(GetDst<RA_64>(Node), rdx);
break;
}
default: LogMan::Msg::A("Unknown UDIV Size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Rem Size: {}", OpSize); break;
}
}
@@ -341,7 +341,7 @@ DEF_OP(URem) {
mov(GetDst<RA_64>(Node), rdx);
break;
}
default: LogMan::Msg::A("Unknown UDIV OpSize: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown URem OpSize: {}", OpSize); break;
}
}
@@ -360,7 +360,7 @@ DEF_OP(MulH) {
imul(GetSrc<RA_64>(Op->Header.Args[1].ID()));
mov(GetDst<RA_64>(Node), rdx);
break;
default: LogMan::Msg::A("Unknown Sext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown MulH size: {}", OpSize);
}
}
@@ -379,7 +379,7 @@ DEF_OP(UMulH) {
mul(GetSrc<RA_64>(Op->Header.Args[1].ID()));
mov(GetDst<RA_64>(Node), rdx);
break;
default: LogMan::Msg::A("Unknown Sext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown UMulH size: {}", OpSize);
}
}
@@ -441,7 +441,7 @@ DEF_OP(Lshl) {
mov(GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
shl(GetDst<RA_64>(Node), Const);
break;
default: LogMan::Msg::A("Unknown LSHL Size: %d\n", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown LSHL Size: {}\n", OpSize); break;
};
} else {
mov(rcx, GetSrc<RA_64>(Op->Header.Args[1].ID()));
@@ -456,7 +456,7 @@ DEF_OP(Lshl) {
mov(GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
shl(GetDst<RA_64>(Node), cl);
break;
default: LogMan::Msg::A("Unknown LSHL Size: %d\n", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown LSHL Size: {}\n", OpSize); break;
};
}
}
@@ -488,7 +488,7 @@ DEF_OP(Lshr) {
mov(GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
shr(GetDst<RA_64>(Node), Const);
break;
default: LogMan::Msg::A("Unknown Size: %d\n", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Size: {}\n", OpSize); break;
};
} else {
@@ -512,7 +512,7 @@ DEF_OP(Lshr) {
mov(GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
shr(GetDst<RA_64>(Node), cl);
break;
default: LogMan::Msg::A("Unknown Size: %d\n", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Size: {}\n", OpSize); break;
};
}
}
@@ -546,7 +546,7 @@ DEF_OP(Ashr) {
mov(GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
sar(GetDst<RA_64>(Node), Const);
break;
default: LogMan::Msg::A("Unknown ASHR Size: %d\n", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown ASHR Size: {}\n", OpSize); break;
};
} else {
@@ -571,7 +571,7 @@ DEF_OP(Ashr) {
mov(GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
sar(GetDst<RA_64>(Node), cl);
break;
default: LogMan::Msg::A("Unknown ASHR Size: %d\n", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown ASHR Size: {}\n", OpSize); break;
};
}
}
@@ -596,7 +596,7 @@ DEF_OP(Ror) {
ror(rax, Const);
break;
}
default: LogMan::Msg::A("Unknown ROR Size: %d\n", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown ROR Size: {}\n", OpSize); break;
}
} else {
mov (rcx, GetSrc<RA_64>(Op->Header.Args[1].ID()));
@@ -612,7 +612,7 @@ DEF_OP(Ror) {
ror(rax, cl);
break;
}
default: LogMan::Msg::A("Unknown ROR Size: %d\n", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown ROR Size: {}\n", OpSize); break;
}
}
mov(GetDst<RA_64>(Node), rax);
@@ -668,7 +668,7 @@ DEF_OP(LDiv) {
mov(GetDst<RA_64>(Node), rax);
break;
}
default: LogMan::Msg::A("Unknown LDIV OpSize: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown LDIV OpSize: {}", OpSize); break;
}
}
@@ -700,7 +700,7 @@ DEF_OP(LUDiv) {
mov(GetDst<RA_64>(Node), rax);
break;
}
default: LogMan::Msg::A("Unknown LUDIV OpSize: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown LUDIV OpSize: {}", OpSize); break;
}
}
@@ -732,7 +732,7 @@ DEF_OP(LRem) {
mov(GetDst<RA_64>(Node), rdx);
break;
}
default: LogMan::Msg::A("Unknown LREM OpSize: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown LREM OpSize: {}", OpSize); break;
}
}
@@ -764,7 +764,7 @@ DEF_OP(LURem) {
mov(GetDst<RA_64>(Node), rdx);
break;
}
default: LogMan::Msg::A("Unknown LUDIV OpSize: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown LUREM OpSize: {}", OpSize); break;
}
}
@@ -829,7 +829,7 @@ DEF_OP(FindMSB) {
case 8:
bsr(GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
break;
default: LogMan::Msg::A("Unknown OpSize: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown FindMSB OpSize: {}", OpSize);
}
}
@@ -853,7 +853,7 @@ DEF_OP(FindTrailingZeros) {
mov(rax, 0x40);
cmovz(GetDst<RA_64>(Node), rax);
break;
default: LogMan::Msg::A("Unknown size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown FindTrailingZeros size: {}", OpSize); break;
}
}
@@ -876,7 +876,7 @@ DEF_OP(CountLeadingZeroes) {
lzcnt(GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
break;
}
default: LogMan::Msg::A("Unknown size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown CountLeadingZeros size: {}", OpSize); break;
}
}
else {
@@ -915,7 +915,7 @@ DEF_OP(CountLeadingZeroes) {
mov(GetDst<RA_64>(Node), rax);
break;
}
default: LogMan::Msg::A("Unknown size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown CountLeadingZeros size: {}", OpSize); break;
}
}
}
@@ -937,7 +937,7 @@ DEF_OP(Rev) {
mov (GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
bswap(GetDst<RA_64>(Node).cvt64());
break;
default: LogMan::Msg::A("Unknown REV size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown REV size: {}", OpSize); break;
}
}
@@ -970,9 +970,7 @@ DEF_OP(Bfi) {
DEF_OP(Bfe) {
auto Op = IROp->C<IR::IROp_Bfe>();
uint8_t OpSize = IROp->Size;
LogMan::Throw::A(OpSize <= 8, "OpSize is too large for BFE: %d", OpSize);
LOGMAN_THROW_A_FMT(IROp->Size <= 8, "OpSize is too large for BFE: {}", IROp->Size);
auto Dst = GetDst<RA_64>(Node);
@@ -1073,7 +1071,7 @@ DEF_OP(Select) {
if (is_const_true || is_const_false) {
if (is_const_false != true || is_const_true != true || const_true != 1 || const_false != 0) {
LogMan::Msg::A("Select: Unsupported compare inline parameters");
LOGMAN_MSG_A_FMT("Select: Unsupported compare inline parameters");
}
(this->*SetCC)(al);
movzx(Dst, al);
@@ -1104,46 +1102,67 @@ DEF_OP(VExtractToGPR) {
pextrq(GetDst<RA_64>(Node), GetSrc(Op->Header.Args[0].ID()), Op->Idx);
break;
}
default: LogMan::Msg::A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
DEF_OP(Float_ToGPR_ZU) {
LogMan::Msg::D("Unimplemented");
}
DEF_OP(Float_ToGPR_ZS) {
auto Op = IROp->C<IR::IROp_Float_ToGPR_ZS>();
if (Op->Header.ElementSize == 8) {
cvttsd2si(GetDst<RA_64>(Node), GetSrc(Op->Header.Args[0].ID()));
}
else {
cvttss2si(GetDst<RA_32>(Node), GetSrc(Op->Header.Args[0].ID()));
}
}
DEF_OP(Float_ToGPR_U) {
LogMan::Msg::D("Unimplemented");
uint16_t Conv = (IROp->Size << 8) | Op->SrcElementSize;
switch (Conv) {
case 0x0804: // int64_t <- float
cvttss2si(GetDst<RA_64>(Node), GetSrc(Op->Header.Args[0].ID()));
break;
case 0x0808: // int64_t <- double
cvttsd2si(GetDst<RA_64>(Node), GetSrc(Op->Header.Args[0].ID()));
break;
case 0x0404: // int32_t <- float
cvttss2si(GetDst<RA_32>(Node), GetSrc(Op->Header.Args[0].ID()));
break;
case 0x0408: // int32_t <- double
cvttsd2si(GetDst<RA_32>(Node), GetSrc(Op->Header.Args[0].ID()));
break;
}
}
DEF_OP(Float_ToGPR_S) {
auto Op = IROp->C<IR::IROp_Float_ToGPR_S>();
if (Op->Header.ElementSize == 8) {
cvtsd2si(GetDst<RA_64>(Node), GetSrc(Op->Header.Args[0].ID()));
}
else {
cvtss2si(GetDst<RA_32>(Node), GetSrc(Op->Header.Args[0].ID()));
uint16_t Conv = (IROp->Size << 8) | Op->SrcElementSize;
switch (Conv) {
case 0x0804: // int64_t <- float
cvtss2si(GetDst<RA_64>(Node), GetSrc(Op->Header.Args[0].ID()));
break;
case 0x0808: // int64_t <- double
cvtsd2si(GetDst<RA_64>(Node), GetSrc(Op->Header.Args[0].ID()));
break;
case 0x0404: // int32_t <- float
cvtss2si(GetDst<RA_32>(Node), GetSrc(Op->Header.Args[0].ID()));
break;
case 0x0408: // int32_t <- double
cvtsd2si(GetDst<RA_32>(Node), GetSrc(Op->Header.Args[0].ID()));
break;
}
}
DEF_OP(FCmp) {
auto Op = IROp->C<IR::IROp_FCmp>();
if (Op->ElementSize == 4) {
ucomiss(GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
if (Op->Flags & (1 << IR::FCMP_FLAG_UNORDERED)) {
if (Op->ElementSize == 4) {
ucomiss(GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
}
else {
ucomisd(GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
}
}
else {
ucomisd(GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
if (Op->ElementSize == 4) {
comiss(GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
}
else {
comisd(GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
}
}
mov (rdx, 0);
@@ -1217,9 +1236,7 @@ void X86JITCore::RegisterALUHandlers() {
REGISTER_OP(SBFE, Sbfe);
REGISTER_OP(SELECT, Select);
REGISTER_OP(VEXTRACTTOGPR, VExtractToGPR);
REGISTER_OP(FLOAT_TOGPR_ZU, Float_ToGPR_ZU);
REGISTER_OP(FLOAT_TOGPR_ZS, Float_ToGPR_ZS);
REGISTER_OP(FLOAT_TOGPR_U, Float_ToGPR_U);
REGISTER_OP(FLOAT_TOGPR_S, Float_ToGPR_S);
REGISTER_OP(FCMP, FCmp);
#undef REGISTER_OP
@@ -55,7 +55,7 @@ DEF_OP(CASPair) {
mov(Dst.second, rdx);
break;
}
default: LogMan::Msg::A("Unsupported: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported: {}", OpSize);
}
}
@@ -74,7 +74,6 @@ DEF_OP(CAS) {
Xbyak::Reg MemReg = GetSrc<RA_64>(Op->Header.Args[2].ID());
mov(rdx, GetSrc<RA_64>(Op->Header.Args[1].ID()));
mov(rax, GetSrc<RA_64>(Op->Header.Args[0].ID()));
// RCX now contains pointer
@@ -82,31 +81,31 @@ DEF_OP(CAS) {
// RDX contains our desired
lock();
switch (OpSize) {
case 1: {
cmpxchg(byte [MemReg], dl);
movzx(rax, al);
cmpxchg(byte [MemReg], GetSrc<RA_8>(Op->Header.Args[1].ID()));
movzx(GetDst<RA_64>(Node), al);
break;
}
case 2: {
cmpxchg(word [MemReg], dx);
movzx(rax, ax);
cmpxchg(word [MemReg], GetSrc<RA_16>(Op->Header.Args[1].ID()));
movzx(GetDst<RA_64>(Node), ax);
break;
}
case 4: {
cmpxchg(dword [MemReg], edx);
cmpxchg(dword [MemReg], GetSrc<RA_32>(Op->Header.Args[1].ID()));
// RAX now contains the result
mov (GetDst<RA_64>(Node), eax);
break;
}
case 8: {
cmpxchg(qword [MemReg], rdx);
cmpxchg(qword [MemReg], GetSrc<RA_64>(Op->Header.Args[1].ID()));
// RAX now contains the result
mov (GetDst<RA_64>(Node), rax);
break;
}
default: LogMan::Msg::A("Unsupported: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported: {}", OpSize);
}
// RAX now contains the result
mov (GetDst<RA_64>(Node), rax);
}
DEF_OP(AtomicAdd) {
@@ -128,7 +127,7 @@ DEF_OP(AtomicAdd) {
case 8:
add(qword [MemReg], GetSrc<RA_64>(Op->Header.Args[1].ID()));
break;
default: LogMan::Msg::A("Unhandled AtomicAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicAdd size: {}", Op->Size);
}
}
@@ -150,7 +149,7 @@ DEF_OP(AtomicSub) {
case 8:
sub(qword [MemReg], GetSrc<RA_64>(Op->Header.Args[1].ID()));
break;
default: LogMan::Msg::A("Unhandled AtomicAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicAdd size: {}", Op->Size);
}
}
@@ -172,7 +171,7 @@ DEF_OP(AtomicAnd) {
case 8:
and_(qword [MemReg], GetSrc<RA_64>(Op->Header.Args[1].ID()));
break;
default: LogMan::Msg::A("Unhandled AtomicAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicAdd size: {}", Op->Size);
}
}
@@ -194,7 +193,7 @@ DEF_OP(AtomicOr) {
case 8:
or_(qword [MemReg], GetSrc<RA_64>(Op->Header.Args[1].ID()));
break;
default: LogMan::Msg::A("Unhandled AtomicAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicAdd size: {}", Op->Size);
}
}
@@ -216,7 +215,7 @@ DEF_OP(AtomicXor) {
case 8:
xor_(qword [MemReg], GetSrc<RA_64>(Op->Header.Args[1].ID()));
break;
default: LogMan::Msg::A("Unhandled AtomicAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicAdd size: {}", Op->Size);
}
}
@@ -228,17 +227,17 @@ DEF_OP(AtomicSwap) {
switch (Op->Size) {
case 1:
mov(GetDst<RA_8>(Node), GetSrc<RA_8>(Op->Header.Args[1].ID()));
movzx(GetDst<RA_64>(Node), GetSrc<RA_8>(Op->Header.Args[1].ID()));
lock();
xchg(byte [MemReg], GetDst<RA_8>(Node));
break;
case 2:
mov(GetDst<RA_16>(Node), GetSrc<RA_16>(Op->Header.Args[1].ID()));
movzx(GetDst<RA_64>(Node), GetSrc<RA_16>(Op->Header.Args[1].ID()));
lock();
xchg(word [MemReg], GetDst<RA_16>(Node));
break;
case 4:
mov(GetDst<RA_32>(Node), GetSrc<RA_32>(Op->Header.Args[1].ID()));
mov(GetDst<RA_64>(Node), GetSrc<RA_32>(Op->Header.Args[1].ID()));
lock();
xchg(dword [MemReg], GetDst<RA_32>(Node));
break;
@@ -247,7 +246,7 @@ DEF_OP(AtomicSwap) {
lock();
xchg(qword [MemReg], GetDst<RA_64>(Node));
break;
default: LogMan::Msg::A("Unhandled AtomicAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicSwap size: {}", Op->Size);
}
}
@@ -257,13 +256,13 @@ DEF_OP(AtomicFetchAdd) {
Xbyak::Reg MemReg = GetSrc<RA_64>(Op->Header.Args[0].ID());
switch (Op->Size) {
case 1:
mov(cl, GetSrc<RA_8>(Op->Header.Args[1].ID()));
movzx(rcx, GetSrc<RA_8>(Op->Header.Args[1].ID()));
lock();
xadd(byte [MemReg], cl);
movzx(GetDst<RA_32>(Node), cl);
break;
case 2:
mov(cx, GetSrc<RA_16>(Op->Header.Args[1].ID()));
movzx(rcx, GetSrc<RA_16>(Op->Header.Args[1].ID()));
lock();
xadd(word [MemReg], cx);
movzx(GetDst<RA_32>(Node), cx);
@@ -272,7 +271,7 @@ DEF_OP(AtomicFetchAdd) {
mov(ecx, GetSrc<RA_32>(Op->Header.Args[1].ID()));
lock();
xadd(dword [MemReg], ecx);
mov(GetDst<RA_32>(Node), ecx);
mov(GetDst<RA_64>(Node), ecx);
break;
case 8:
mov(rcx, GetSrc<RA_64>(Op->Header.Args[1].ID()));
@@ -280,7 +279,7 @@ DEF_OP(AtomicFetchAdd) {
xadd(qword [MemReg], rcx);
mov(GetDst<RA_64>(Node), rcx);
break;
default: LogMan::Msg::A("Unhandled AtomicFetchAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicFetchAdd size: {}", Op->Size);
}
}
@@ -317,7 +316,7 @@ DEF_OP(AtomicFetchSub) {
xadd(qword [MemReg], rcx);
mov(GetDst<RA_64>(Node), rcx);
break;
default: LogMan::Msg::A("Unhandled AtomicFetchAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicFetchSub size: {}", Op->Size);
}
}
@@ -395,7 +394,7 @@ DEF_OP(AtomicFetchAnd) {
mov(GetDst<RA_64>(Node), TMP3.cvt64());
break;
}
default: LogMan::Msg::A("Unhandled AtomicFetchAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicFetchAnd size: {}", Op->Size);
}
}
@@ -472,7 +471,7 @@ DEF_OP(AtomicFetchOr) {
mov(GetDst<RA_64>(Node), TMP3.cvt64());
break;
}
default: LogMan::Msg::A("Unhandled AtomicFetchAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicFetchOr size: {}", Op->Size);
}
}
@@ -549,7 +548,83 @@ DEF_OP(AtomicFetchXor) {
mov(GetDst<RA_64>(Node), TMP3.cvt64());
break;
}
default: LogMan::Msg::A("Unhandled AtomicFetchAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicFetchXor size: {}", Op->Size);
}
}
DEF_OP(AtomicFetchNeg) {
auto Op = IROp->C<IR::IROp_AtomicFetchNeg>();
Xbyak::Reg MemReg = GetSrc<RA_64>(Op->Header.Args[0].ID());
switch (Op->Size) {
case 1: {
mov(TMP1.cvt8(), byte [MemReg]);
Label Loop;
L(Loop);
mov(TMP2.cvt8(), TMP1.cvt8());
mov(TMP3.cvt8(), TMP1.cvt8());
neg(TMP2.cvt8());
// Updates RAX with the value from memory
lock(); cmpxchg(byte [MemReg], TMP2.cvt8());
jne(Loop);
// Result is the previous value from memory, which is currently in TMP3
movzx(GetDst<RA_64>(Node), TMP3.cvt8());
break;
}
case 2: {
mov(TMP1.cvt16(), word [MemReg]);
Label Loop;
L(Loop);
mov(TMP2.cvt16(), TMP1.cvt16());
mov(TMP3.cvt16(), TMP1.cvt16());
neg(TMP2.cvt16());
// Updates RAX with the value from memory
lock(); cmpxchg(word [MemReg], TMP2.cvt16());
jne(Loop);
// Result is the previous value from memory, which is currently in TMP3
movzx(GetDst<RA_64>(Node), TMP3.cvt16());
break;
}
case 4: {
mov(TMP1.cvt32(), dword [MemReg]);
Label Loop;
L(Loop);
mov(TMP2.cvt32(), TMP1.cvt32());
mov(TMP3.cvt32(), TMP1.cvt32());
neg(TMP2.cvt32());
// Updates RAX with the value from memory
lock(); cmpxchg(dword [MemReg], TMP2.cvt32());
jne(Loop);
// Result is the previous value from memory, which is currently in TMP3
mov(GetDst<RA_32>(Node), TMP3.cvt32());
break;
}
case 8: {
mov(TMP1.cvt64(), qword [MemReg]);
Label Loop;
L(Loop);
mov(TMP2.cvt64(), TMP1.cvt64());
mov(TMP3.cvt64(), TMP1.cvt64());
neg(TMP2.cvt64());
// Updates RAX with the value from memory
lock(); cmpxchg(qword [MemReg], TMP2.cvt64());
jne(Loop);
// Result is the previous value from memory, which is currently in TMP3
mov(GetDst<RA_64>(Node), TMP3.cvt64());
break;
}
default: LOGMAN_MSG_A_FMT("Unhandled AtomicFetchNeg size: {}", Op->Size);
}
}
@@ -569,6 +644,7 @@ void X86JITCore::RegisterAtomicHandlers() {
REGISTER_OP(ATOMICFETCHAND, AtomicFetchAnd);
REGISTER_OP(ATOMICFETCHOR, AtomicFetchOr);
REGISTER_OP(ATOMICFETCHXOR, AtomicFetchXor);
REGISTER_OP(ATOMICFETCHNEG, AtomicFetchNeg);
#undef REGISTER_OP
}
}
@@ -4,6 +4,8 @@ tags: backend|x86-64
$end_info$
*/
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/JIT/x86_64/JITClass.h"
#include "Interface/IR/Passes/RegisterAllocationPass.h"
@@ -14,15 +16,15 @@ $end_info$
namespace FEXCore::CPU {
#define DEF_OP(x) void X86JITCore::Op_##x(FEXCore::IR::IROp_Header *IROp, uint32_t Node)
DEF_OP(GuestCallDirect) {
LogMan::Msg::D("Unimplemented");
LogMan::Msg::DFmt("Unimplemented");
}
DEF_OP(GuestCallIndirect) {
LogMan::Msg::D("Unimplemented");
LogMan::Msg::DFmt("Unimplemented");
}
DEF_OP(GuestReturn) {
LogMan::Msg::D("Unimplemented");
LogMan::Msg::DFmt("Unimplemented");
}
DEF_OP(SignalReturn) {
@@ -81,7 +83,7 @@ DEF_OP(ExitFunction) {
jmp(qword[rax]);
L(l_BranchHost);
dq(Dispatcher->ExitFunctionLinkerAddress);
dq(ThreadSharedData.Dispatcher->ExitFunctionLinkerAddress);
L(l_BranchGuest);
dq(NewRIP);
} else {
@@ -101,7 +103,7 @@ DEF_OP(ExitFunction) {
jmp(qword[LookupBase + 0]);
L(FullLookup);
mov(rax, Dispatcher->AbsoluteLoopTopAddress);
mov(rax, ThreadSharedData.Dispatcher->AbsoluteLoopTopAddress);
mov(qword [STATE + offsetof(FEXCore::Core::CpuStateFrame, State.rip)], RipReg);
jmp(rax);
}
@@ -248,28 +250,28 @@ DEF_OP(Thunk) {
DEF_OP(ValidateCode) {
auto Op = IROp->C<IR::IROp_ValidateCode>();
uint8_t* OldCode = (uint8_t*)&Op->CodeOriginalLow;
const auto* OldCode = (const uint8_t*)&Op->CodeOriginalLow;
int len = Op->CodeLength;
int idx = 0;
xor_(GetDst<RA_64>(Node), GetDst<RA_64>(Node));
mov(rax, IR->GetHeader()->Entry + Op->Offset);
mov(rax, Entry + Op->Offset);
mov(rbx, 1);
while (len >= 4) {
cmp(dword[rax + idx], *(uint32_t*)(OldCode + idx));
cmp(dword[rax + idx], *(const uint32_t*)(OldCode + idx));
cmovne(GetDst<RA_64>(Node), rbx);
len-=4;
idx+=4;
}
while (len >= 2) {
mov(rcx, *(uint16_t*)(OldCode + idx));
mov(rcx, *(const uint16_t*)(OldCode + idx));
cmp(word[rax + idx], cx);
cmovne(GetDst<RA_64>(Node), rbx);
len-=2;
idx+=2;
}
while (len >= 1) {
cmp(byte[rax + idx], *(uint8_t*)(OldCode + idx));
cmp(byte[rax + idx], *(const uint8_t*)(OldCode + idx));
cmovne(GetDst<RA_64>(Node), rbx);
len-=1;
idx+=1;
@@ -286,7 +288,7 @@ DEF_OP(RemoveCodeEntry) {
sub(rsp, 8); // Align
mov(rdi, STATE);
mov(rax, IR->GetHeader()->Entry); // imm64 move
mov(rax, Entry); // imm64 move
mov(rsi, rax);
@@ -319,8 +321,9 @@ DEF_OP(CPUID) {
//
// Result: RAX, RDX. 4xi32
mov (rsi, GetSrc<RA_64>(Op->Header.Args[0].ID()));
mov (rdx, GetSrc<RA_64>(Op->Header.Args[1].ID()));
// rsi can be in the source registers, so copy argument to edx first
mov (edx, GetSrc<RA_32>(Op->Header.Args[1].ID()));
mov (esi, GetSrc<RA_32>(Op->Header.Args[0].ID()));
mov (rdi, reinterpret_cast<uint64_t>(&CTX->CPUID));
auto NumPush = RA64.size();
@@ -31,7 +31,7 @@ DEF_OP(VInsGPR) {
pinsrq(GetDst(Node), GetSrc<RA_64>(Op->Header.Args[1].ID()), Op->Index);
break;
}
default: LogMan::Msg::A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -52,14 +52,10 @@ DEF_OP(VCastFromGPR) {
case 8:
vmovq(GetDst(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()).cvt64());
break;
default: LogMan::Msg::A("Unknown castGPR element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown VCastFromGPR element size: {}", Op->Header.ElementSize);
}
}
DEF_OP(Float_FromGPR_U) {
LogMan::Msg::A("Unimplemented");
}
DEF_OP(Float_FromGPR_S) {
auto Op = IROp->C<IR::IROp_Float_FromGPR_S>();
uint16_t Conv = (Op->Header.ElementSize << 8) | Op->SrcElementSize;
@@ -95,14 +91,10 @@ DEF_OP(Float_FToF) {
cvtsd2ss(GetDst(Node), GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LogMan::Msg::A("Unknown FCVT sizes: 0x%x", Conv);
default: LOGMAN_MSG_A_FMT("Unknown Float_FToF sizes: 0x{:x}", Conv);
}
}
DEF_OP(Vector_UToF) {
LogMan::Msg::A("Unimplemented");
}
DEF_OP(Vector_SToF) {
auto Op = IROp->C<IR::IROp_Vector_SToF>();
switch (Op->Header.ElementSize) {
@@ -121,14 +113,10 @@ DEF_OP(Vector_SToF) {
cvtsi2sd(xmm15, rax);
movlhps(GetDst(Node), xmm15);
break;
default: LogMan::Msg::A("Unknown castGPR element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Vector_SToF element size: {}", Op->Header.ElementSize);
}
}
DEF_OP(Vector_FToZU) {
LogMan::Msg::A("Unimplemented");
}
DEF_OP(Vector_FToZS) {
auto Op = IROp->C<IR::IROp_Vector_FToZS>();
switch (Op->Header.ElementSize) {
@@ -138,14 +126,10 @@ DEF_OP(Vector_FToZS) {
case 8:
cvttpd2dq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()));
break;
default: LogMan::Msg::A("Unknown castGPR element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Vector_FToZS element size: {}", Op->Header.ElementSize);
}
}
DEF_OP(Vector_FToU) {
LogMan::Msg::A("Unimplemented");
}
DEF_OP(Vector_FToS) {
auto Op = IROp->C<IR::IROp_Vector_FToS>();
switch (Op->Header.ElementSize) {
@@ -155,7 +139,7 @@ DEF_OP(Vector_FToS) {
case 8:
cvtpd2dq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()));
break;
default: LogMan::Msg::A("Unknown castGPR element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Vector_FToS element size: {}", Op->Header.ElementSize);
}
}
@@ -172,7 +156,39 @@ DEF_OP(Vector_FToF) {
cvtpd2ps(GetDst(Node), GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LogMan::Msg::A("Unknown Conversion Type : 0%04x", Conv); break;
default: LOGMAN_MSG_A_FMT("Unknown Vector_FToF conversion type : 0x{:04x}", Conv); break;
}
}
DEF_OP(Vector_FToI) {
auto Op = IROp->C<IR::IROp_Vector_FToI>();
uint8_t RoundMode{};
switch (Op->Round) {
case FEXCore::IR::Round_Nearest.Val:
RoundMode = 0b0000'0'0'00;
break;
case FEXCore::IR::Round_Negative_Infinity.Val:
RoundMode = 0b0000'0'0'01;
break;
case FEXCore::IR::Round_Positive_Infinity.Val:
RoundMode = 0b0000'0'0'10;
break;
case FEXCore::IR::Round_Towards_Zero.Val:
RoundMode = 0b0000'0'0'11;
break;
case FEXCore::IR::Round_Host.Val:
RoundMode = 0b0000'0'1'00;
break;
}
switch (Op->Header.ElementSize) {
case 4:
roundps(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), RoundMode);
break;
case 8:
roundpd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), RoundMode);
break;
}
}
@@ -181,16 +197,13 @@ void X86JITCore::RegisterConversionHandlers() {
#define REGISTER_OP(op, x) OpHandlers[FEXCore::IR::IROps::OP_##op] = &X86JITCore::Op_##x
REGISTER_OP(VINSGPR, VInsGPR);
REGISTER_OP(VCASTFROMGPR, VCastFromGPR);
REGISTER_OP(FLOAT_FROMGPR_U, Float_FromGPR_U);
REGISTER_OP(FLOAT_FROMGPR_S, Float_FromGPR_S);
REGISTER_OP(FLOAT_FTOF, Float_FToF);
REGISTER_OP(VECTOR_UTOF, Vector_UToF);
REGISTER_OP(VECTOR_STOF, Vector_SToF);
REGISTER_OP(VECTOR_FTOZU, Vector_FToZU);
REGISTER_OP(VECTOR_FTOZS, Vector_FToZS);
REGISTER_OP(VECTOR_FTOU, Vector_FToU);
REGISTER_OP(VECTOR_FTOS, Vector_FToS);
REGISTER_OP(VECTOR_FTOF, Vector_FToF);
REGISTER_OP(VECTOR_FTOI, Vector_FToI);
#undef REGISTER_OP
}
}
+75 -66
View File
@@ -6,6 +6,7 @@ $end_info$
*/
#include "Interface/Context/Context.h"
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/Dispatcher/X86Dispatcher.h"
#include "Interface/Core/JIT/x86_64/JITClass.h"
@@ -15,6 +16,7 @@ $end_info$
#include <FEXCore/Core/X86Enums.h>
#include <FEXCore/Core/UContext.h>
#include <FEXCore/Utils/Allocator.h>
#include <cmath>
#include <signal.h>
@@ -30,17 +32,17 @@ CodeBuffer AllocateNewCodeBuffer(size_t Size) {
CodeBuffer Buffer;
Buffer.Size = Size;
Buffer.Ptr = static_cast<uint8_t*>(
mmap(nullptr,
FEXCore::Allocator::mmap(nullptr,
Buffer.Size,
PROT_READ | PROT_WRITE | PROT_EXEC,
MAP_PRIVATE | MAP_ANONYMOUS,
-1, 0));
LogMan::Throw::A(Buffer.Ptr != reinterpret_cast<uint8_t*>(~0ULL), "Couldn't allocate code buffer");
LOGMAN_THROW_A_FMT(Buffer.Ptr != reinterpret_cast<uint8_t*>(~0ULL), "Couldn't allocate code buffer");
return Buffer;
}
void FreeCodeBuffer(CodeBuffer Buffer) {
munmap(Buffer.Ptr, Buffer.Size);
FEXCore::Allocator::munmap(Buffer.Ptr, Buffer.Size);
}
}
@@ -53,9 +55,9 @@ void X86JITCore::CopyNecessaryDataForCompileThread(CPUBackend *Original) {
}
void X86JITCore::PushRegs() {
for (auto &Xmm : RAXMM_x) {
sub(rsp, 16);
movaps(ptr[rsp], Xmm);
sub(rsp, 16 * RAXMM_x.size());
for (size_t i = 0; i < RAXMM_x.size(); ++i) {
movaps(ptr[rsp + i * 16], RAXMM_x[i]);
}
for (auto &Reg : RA64)
@@ -74,17 +76,19 @@ void X86JITCore::PopRegs() {
for (uint32_t i = RA64.size(); i > 0; --i)
pop(RA64[i - 1]);
for (uint32_t i = RAXMM_x.size(); i > 0; --i) {
movaps(RAXMM_x[i - 1], ptr[rsp]);
add(rsp, 16);
for (size_t i = 0; i < RAXMM_x.size(); ++i) {
movaps(RAXMM_x[i], ptr[rsp + i * 16]);
}
add(rsp, 16 * RAXMM_x.size());
}
void X86JITCore::Op_Unhandled(FEXCore::IR::IROp_Header *IROp, uint32_t Node) {
FallbackInfo Info;
if (!InterpreterOps::GetFallbackHandler(IROp, &Info)) {
auto Name = FEXCore::IR::GetName(IROp->Op);
LogMan::Msg::A("Unhandled IR Op: %s", std::string(Name).c_str());
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
LOGMAN_MSG_A_FMT("Unhandled IR Op: {}", FEXCore::IR::GetName(IROp->Op));
#endif
} else {
switch(Info.ABI) {
case FABI_VOID_U16: {
@@ -281,8 +285,10 @@ void X86JITCore::Op_Unhandled(FEXCore::IR::IROp_Header *IROp, uint32_t Node) {
case FABI_UNKNOWN:
default:
auto Name = FEXCore::IR::GetName(IROp->Op);
LogMan::Msg::A("Unhandled IR Fallback abi: %s %d", std::string(Name).c_str(), Info.ABI);
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
LOGMAN_MSG_A_FMT("Unhandled IR Fallback ABI: {} {}", FEXCore::IR::GetName(IROp->Op), Info.ABI);
#endif
break;
}
}
}
@@ -330,24 +336,24 @@ X86JITCore::X86JITCore(FEXCore::Context::Context *ctx, FEXCore::Core::InternalTh
config.ExitFunctionLink = reinterpret_cast<uintptr_t>(&ExitFunctionLink);
config.ExitFunctionLinkThis = reinterpret_cast<uintptr_t>(this);
Dispatcher = new X86Dispatcher(CTX, ThreadState, config);
Dispatcher = std::make_unique<X86Dispatcher>(CTX, ThreadState, config);
DispatchPtr = Dispatcher->DispatchPtr;
CallbackPtr = Dispatcher->CallbackPtr;
ThreadSharedData.SignalHandlerRefCounterPtr = &Dispatcher->SignalHandlerRefCounter;
ThreadSharedData.SignalHandlerReturnAddress = Dispatcher->SignalHandlerReturnAddress;
ThreadSharedData.Dispatcher = Dispatcher.get();
// This will register the host signal handler per thread, which is fine
CTX->SignalDelegation->RegisterHostSignalHandler(SIGILL, [](FEXCore::Core::InternalThreadState *Thread, int Signal, void *info, void *ucontext) -> bool {
X86JITCore *Core = reinterpret_cast<X86JITCore*>(Thread->CPUBackend.get());
return Core->Dispatcher->HandleSIGILL(Signal, info, ucontext);
});
}, true);
CTX->SignalDelegation->RegisterHostSignalHandler(SignalDelegator::SIGNAL_FOR_PAUSE, [](FEXCore::Core::InternalThreadState *Thread, int Signal, void *info, void *ucontext) -> bool {
X86JITCore *Core = reinterpret_cast<X86JITCore*>(Thread->CPUBackend.get());
return Core->Dispatcher->HandleSignalPause(Signal, info, ucontext);
});
}, true);
auto GuestSignalHandler = [](FEXCore::Core::InternalThreadState *Thread, int Signal, void *info, void *ucontext, GuestSigAction *GuestAction, stack_t *GuestStack) -> bool {
X86JITCore *Core = reinterpret_cast<X86JITCore*>(Thread->CPUBackend.get());
@@ -410,106 +416,106 @@ void X86JITCore::ClearCache() {
}
}
IR::PhysicalRegister X86JITCore::GetPhys(uint32_t Node) {
IR::PhysicalRegister X86JITCore::GetPhys(uint32_t Node) const {
auto PhyReg = RAData->GetNodeRegister(Node);
LogMan::Throw::A(PhyReg.Raw != 255, "Couldn't Allocate register for node: ssa%d. Class: %d", Node, PhyReg.Class);
LOGMAN_THROW_A_FMT(PhyReg.Raw != 255, "Couldn't Allocate register for node: ssa{}. Class: {}", Node, PhyReg.Class);
return PhyReg;
}
bool X86JITCore::IsFPR(uint32_t Node) {
bool X86JITCore::IsFPR(uint32_t Node) const {
return RAData->GetNodeRegister(Node).Class == IR::FPRClass.Val;
}
bool X86JITCore::IsGPR(uint32_t Node) {
bool X86JITCore::IsGPR(uint32_t Node) const {
return RAData->GetNodeRegister(Node).Class == IR::GPRClass.Val;
}
template<uint8_t RAType>
Xbyak::Reg X86JITCore::GetSrc(uint32_t Node) {
Xbyak::Reg X86JITCore::GetSrc(uint32_t Node) const {
// rax, rcx, rdx, rsi, r8, r9,
// r10
// Callee Saved
// rbx, rbp, r12, r13, r14, r15
auto PhyReg = GetPhys(Node);
if (RAType == RA_64)
if constexpr (RAType == RA_64)
return RA64[PhyReg.Reg].cvt64();
else if (RAType == RA_XMM)
else if constexpr (RAType == RA_XMM)
return RAXMM[PhyReg.Reg];
else if (RAType == RA_32)
else if constexpr (RAType == RA_32)
return RA64[PhyReg.Reg].cvt32();
else if (RAType == RA_16)
else if constexpr (RAType == RA_16)
return RA64[PhyReg.Reg].cvt16();
else if (RAType == RA_8)
else if constexpr (RAType == RA_8)
return RA64[PhyReg.Reg].cvt8();
}
template
Xbyak::Reg X86JITCore::GetSrc<X86JITCore::RA_64>(uint32_t Node);
Xbyak::Reg X86JITCore::GetSrc<X86JITCore::RA_64>(uint32_t Node) const;
template
Xbyak::Reg X86JITCore::GetSrc<X86JITCore::RA_32>(uint32_t Node);
Xbyak::Reg X86JITCore::GetSrc<X86JITCore::RA_32>(uint32_t Node) const;
template
Xbyak::Reg X86JITCore::GetSrc<X86JITCore::RA_16>(uint32_t Node);
Xbyak::Reg X86JITCore::GetSrc<X86JITCore::RA_16>(uint32_t Node) const;
template
Xbyak::Reg X86JITCore::GetSrc<X86JITCore::RA_8>(uint32_t Node);
Xbyak::Reg X86JITCore::GetSrc<X86JITCore::RA_8>(uint32_t Node) const;
Xbyak::Xmm X86JITCore::GetSrc(uint32_t Node) {
Xbyak::Xmm X86JITCore::GetSrc(uint32_t Node) const {
auto PhyReg = GetPhys(Node);
return RAXMM_x[PhyReg.Reg];
}
template<uint8_t RAType>
Xbyak::Reg X86JITCore::GetDst(uint32_t Node) {
Xbyak::Reg X86JITCore::GetDst(uint32_t Node) const {
auto PhyReg = GetPhys(Node);
if (RAType == RA_64)
if constexpr (RAType == RA_64)
return RA64[PhyReg.Reg].cvt64();
else if (RAType == RA_XMM)
else if constexpr (RAType == RA_XMM)
return RAXMM[PhyReg.Reg];
else if (RAType == RA_32)
else if constexpr (RAType == RA_32)
return RA64[PhyReg.Reg].cvt32();
else if (RAType == RA_16)
else if constexpr (RAType == RA_16)
return RA64[PhyReg.Reg].cvt16();
else if (RAType == RA_8)
else if constexpr (RAType == RA_8)
return RA64[PhyReg.Reg].cvt8();
}
template
Xbyak::Reg X86JITCore::GetDst<X86JITCore::RA_64>(uint32_t Node);
Xbyak::Reg X86JITCore::GetDst<X86JITCore::RA_64>(uint32_t Node) const;
template
Xbyak::Reg X86JITCore::GetDst<X86JITCore::RA_32>(uint32_t Node);
Xbyak::Reg X86JITCore::GetDst<X86JITCore::RA_32>(uint32_t Node) const;
template
Xbyak::Reg X86JITCore::GetDst<X86JITCore::RA_16>(uint32_t Node);
Xbyak::Reg X86JITCore::GetDst<X86JITCore::RA_16>(uint32_t Node) const;
template
Xbyak::Reg X86JITCore::GetDst<X86JITCore::RA_8>(uint32_t Node);
Xbyak::Reg X86JITCore::GetDst<X86JITCore::RA_8>(uint32_t Node) const;
template<uint8_t RAType>
std::pair<Xbyak::Reg, Xbyak::Reg> X86JITCore::GetSrcPair(uint32_t Node) {
std::pair<Xbyak::Reg, Xbyak::Reg> X86JITCore::GetSrcPair(uint32_t Node) const {
auto PhyReg = GetPhys(Node);
if (RAType == RA_64)
if constexpr (RAType == RA_64)
return RA64Pair[PhyReg.Reg];
else if (RAType == RA_32)
else if constexpr (RAType == RA_32)
return {RA64Pair[PhyReg.Reg].first.cvt32(), RA64Pair[PhyReg.Reg].second.cvt32()};
}
template
std::pair<Xbyak::Reg, Xbyak::Reg> X86JITCore::GetSrcPair<X86JITCore::RA_64>(uint32_t Node);
std::pair<Xbyak::Reg, Xbyak::Reg> X86JITCore::GetSrcPair<X86JITCore::RA_64>(uint32_t Node) const;
template
std::pair<Xbyak::Reg, Xbyak::Reg> X86JITCore::GetSrcPair<X86JITCore::RA_32>(uint32_t Node);
std::pair<Xbyak::Reg, Xbyak::Reg> X86JITCore::GetSrcPair<X86JITCore::RA_32>(uint32_t Node) const;
Xbyak::Xmm X86JITCore::GetDst(uint32_t Node) {
Xbyak::Xmm X86JITCore::GetDst(uint32_t Node) const {
auto PhyReg = GetPhys(Node);
return RAXMM_x[PhyReg.Reg];
}
bool X86JITCore::IsInlineConstant(const IR::OrderedNodeWrapper& WNode, uint64_t* Value) {
bool X86JITCore::IsInlineConstant(const IR::OrderedNodeWrapper& WNode, uint64_t* Value) const {
auto OpHeader = IR->GetOp<IR::IROp_Header>(WNode);
if (OpHeader->Op == IR::IROps::OP_INLINECONSTANT) {
@@ -523,13 +529,13 @@ bool X86JITCore::IsInlineConstant(const IR::OrderedNodeWrapper& WNode, uint64_t*
}
}
bool X86JITCore::IsInlineEntrypointOffset(const IR::OrderedNodeWrapper& WNode, uint64_t* Value) {
bool X86JITCore::IsInlineEntrypointOffset(const IR::OrderedNodeWrapper& WNode, uint64_t* Value) const {
auto OpHeader = IR->GetOp<IR::IROp_Header>(WNode);
if (OpHeader->Op == IR::IROps::OP_INLINEENTRYPOINTOFFSET) {
auto Op = OpHeader->C<IR::IROp_InlineEntrypointOffset>();
if (Value) {
*Value = IR->GetHeader()->Entry + Op->Offset;
*Value = Entry + Op->Offset;
}
return true;
} else {
@@ -562,7 +568,7 @@ std::tuple<X86JITCore::SetCC, X86JITCore::CMovCC, X86JITCore::JCC> X86JITCore::G
case FEXCore::IR::COND_VS:
case FEXCore::IR::COND_VC:
default:
LogMan::Msg::A("Unsupported compare type");
LOGMAN_MSG_A_FMT("Unsupported compare type");
break;
}
@@ -570,10 +576,11 @@ std::tuple<X86JITCore::SetCC, X86JITCore::CMovCC, X86JITCore::JCC> X86JITCore::G
return { &CodeGenerator::sete , &CodeGenerator::cmove , &CodeGenerator::je };
}
void *X86JITCore::CompileCode([[maybe_unused]] FEXCore::IR::IRListView const *IR, [[maybe_unused]] FEXCore::Core::DebugData *DebugData, FEXCore::IR::RegisterAllocationData *RAData) {
void *X86JITCore::CompileCode(uint64_t Entry, [[maybe_unused]] FEXCore::IR::IRListView const *IR, [[maybe_unused]] FEXCore::Core::DebugData *DebugData, FEXCore::IR::RegisterAllocationData *RAData) {
JumpTargets.clear();
uint32_t SSACount = IR->GetSSACount();
this->Entry = Entry;
this->RAData = RAData;
// Fairly excessive buffer range to make sure we don't overflow
@@ -582,7 +589,7 @@ void *X86JITCore::CompileCode([[maybe_unused]] FEXCore::IR::IRListView const *IR
ThreadState->CTX->ClearCodeCache(ThreadState, false);
}
void *Entry = getCurr<void*>();
void *GuestEntry = getCurr<void*>();
this->IR = IR;
if (CTX->GetGdbServerStatus()) {
@@ -597,14 +604,14 @@ void *X86JITCore::CompileCode([[maybe_unused]] FEXCore::IR::IRListView const *IR
cmp(dword [rax + (offsetof(FEXCore::Context::Context, Config.RunningMode))], 0);
je(RunBlock);
// Else we need to pause now
mov(rax, Dispatcher->ThreadPauseHandlerAddress);
mov(rax, ThreadSharedData.Dispatcher->ThreadPauseHandlerAddress);
jmp(rax);
ud2();
L(RunBlock);
}
LogMan::Throw::A(RAData != nullptr, "Needs RA");
LOGMAN_THROW_A_FMT(RAData != nullptr, "Needs RA");
SpillSlots = RAData->SpillSlots();
@@ -613,7 +620,7 @@ void *X86JITCore::CompileCode([[maybe_unused]] FEXCore::IR::IRListView const *IR
}
#ifdef BLOCKSTATS
BlockSamplingData::BlockData *SamplingData = CTX->BlockData->GetBlockData(HeaderOp->Entry);
BlockSamplingData::BlockData *SamplingData = CTX->BlockData->GetBlockData(Entry);
if (GetSamplingData) {
mov(rcx, reinterpret_cast<uintptr_t>(SamplingData));
rdtsc();
@@ -660,8 +667,10 @@ void *X86JITCore::CompileCode([[maybe_unused]] FEXCore::IR::IRListView const *IR
for (auto [BlockNode, BlockHeader] : IR->GetBlocks()) {
using namespace FEXCore::IR;
{
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
auto BlockIROp = BlockHeader->CW<IROp_CodeBlock>();
LogMan::Throw::A(BlockIROp->Header.Op == IR::OP_CODEBLOCK, "IR type failed to be a code block");
LOGMAN_THROW_A_FMT(BlockIROp->Header.Op == IR::OP_CODEBLOCK, "IR type failed to be a code block");
#endif
uint32_t Node = IR->GetID(BlockNode);
auto IsTarget = JumpTargets.find(Node);
@@ -712,7 +721,7 @@ void *X86JITCore::CompileCode([[maybe_unused]] FEXCore::IR::IRListView const *IR
Inst << "Reg" << GetPhys(ArgNode) << (i + 1 == NumArgs ? "" : ", ");
}
LogMan::Msg::D("%s", Inst.str().c_str());
LogMan::Msg::DFmt("{}", Inst.str());
}
#endif
uint32_t ID = IR->GetID(CodeNode);
@@ -730,15 +739,15 @@ void *X86JITCore::CompileCode([[maybe_unused]] FEXCore::IR::IRListView const *IR
}
PendingTargetLabel = nullptr;
void *Exit = getCurr<void*>();
void *GuestExit = getCurr<void*>();
this->IR = nullptr;
ready();
if (DebugData) {
DebugData->HostCodeSize = reinterpret_cast<uintptr_t>(Exit) - reinterpret_cast<uintptr_t>(Entry);
DebugData->HostCodeSize = reinterpret_cast<uintptr_t>(GuestExit) - reinterpret_cast<uintptr_t>(GuestEntry);
}
return Entry;
return GuestEntry;
}
uint64_t X86JITCore::ExitFunctionLink(X86JITCore *core, FEXCore::Core::CpuStateFrame *Frame, uint64_t *record) {
@@ -749,10 +758,10 @@ uint64_t X86JITCore::ExitFunctionLink(X86JITCore *core, FEXCore::Core::CpuStateF
if (!HostCode) {
Thread->CurrentFrame->State.rip = GuestRip;
return core->Dispatcher->AbsoluteLoopTopAddress;
return core->ThreadSharedData.Dispatcher->AbsoluteLoopTopAddress;
}
auto LinkerAddress = core->Dispatcher->ExitFunctionLinkerAddress;
auto LinkerAddress = core->ThreadSharedData.Dispatcher->ExitFunctionLinkerAddress;
Thread->LookupCache->AddBlockLink(GuestRip, (uintptr_t)record, [record, LinkerAddress]{
// undo the link
record[0] = LinkerAddress;
@@ -762,7 +771,7 @@ uint64_t X86JITCore::ExitFunctionLink(X86JITCore *core, FEXCore::Core::CpuStateF
return HostCode;
}
FEXCore::CPU::CPUBackend *CreateX86JITCore(FEXCore::Context::Context *ctx, FEXCore::Core::InternalThreadState *Thread, bool CompileThread) {
return new X86JITCore(ctx, Thread, AllocateNewCodeBuffer(CompileThread ? X86JITCore::MAX_CODE_SIZE : X86JITCore::INITIAL_CODE_SIZE), CompileThread);
std::unique_ptr<CPUBackend> CreateX86JITCore(FEXCore::Context::Context *ctx, FEXCore::Core::InternalThreadState *Thread, bool CompileThread) {
return std::make_unique<X86JITCore>(ctx, Thread, AllocateNewCodeBuffer(CompileThread ? X86JITCore::MAX_CODE_SIZE : X86JITCore::INITIAL_CODE_SIZE), CompileThread);
}
}
+25 -19
View File
@@ -6,7 +6,6 @@ $end_info$
#pragma once
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/BlockSamplingData.h"
#include "Interface/Core/Dispatcher/Dispatcher.h"
@@ -66,7 +65,7 @@ public:
explicit X86JITCore(FEXCore::Context::Context *ctx, FEXCore::Core::InternalThreadState *Thread, CodeBuffer Buffer, bool CompileThread);
~X86JITCore() override;
std::string GetName() override { return "JIT"; }
void *CompileCode(FEXCore::IR::IRListView const *IR, FEXCore::Core::DebugData *DebugData, FEXCore::IR::RegisterAllocationData *RAData) override;
void *CompileCode(uint64_t Entry, FEXCore::IR::IRListView const *IR, FEXCore::Core::DebugData *DebugData, FEXCore::IR::RegisterAllocationData *RAData) override;
void *MapRegion(void* HostPtr, uint64_t, uint64_t) override { return HostPtr; }
@@ -83,7 +82,8 @@ private:
FEXCore::Context::Context *CTX;
FEXCore::Core::InternalThreadState *ThreadState;
FEXCore::IR::IRListView const *IR;
FEXCore::CPU::Dispatcher *Dispatcher;
std::unique_ptr<FEXCore::CPU::Dispatcher> Dispatcher;
uint64_t Entry;
std::unordered_map<IR::OrderedNodeWrapper::NodeOffsetType, Label> JumpTargets;
Xbyak::util::Cpu Features{};
@@ -111,26 +111,26 @@ private:
constexpr static uint8_t RA_64 = 3;
constexpr static uint8_t RA_XMM = 4;
IR::PhysicalRegister GetPhys(uint32_t Node);
IR::PhysicalRegister GetPhys(uint32_t Node) const;
bool IsFPR(uint32_t Node);
bool IsGPR(uint32_t Node);
bool IsFPR(uint32_t Node) const;
bool IsGPR(uint32_t Node) const;
template<uint8_t RAType>
Xbyak::Reg GetSrc(uint32_t Node);
Xbyak::Reg GetSrc(uint32_t Node) const;
template<uint8_t RAType>
std::pair<Xbyak::Reg, Xbyak::Reg> GetSrcPair(uint32_t Node);
std::pair<Xbyak::Reg, Xbyak::Reg> GetSrcPair(uint32_t Node) const;
template<uint8_t RAType>
Xbyak::Reg GetDst(uint32_t Node);
Xbyak::Reg GetDst(uint32_t Node) const;
Xbyak::Xmm GetSrc(uint32_t Node);
Xbyak::Xmm GetDst(uint32_t Node);
Xbyak::Xmm GetSrc(uint32_t Node) const;
Xbyak::Xmm GetDst(uint32_t Node) const;
Xbyak::RegExp GenerateModRM(Xbyak::Reg Base, IR::OrderedNodeWrapper Offset, IR::MemOffsetType OffsetType, uint8_t OffsetScale);
Xbyak::RegExp GenerateModRM(Xbyak::Reg Base, IR::OrderedNodeWrapper Offset, IR::MemOffsetType OffsetType, uint8_t OffsetScale) const;
bool IsInlineConstant(const IR::OrderedNodeWrapper& Node, uint64_t* Value = nullptr);
bool IsInlineEntrypointOffset(const IR::OrderedNodeWrapper& WNode, uint64_t* Value);
bool IsInlineConstant(const IR::OrderedNodeWrapper& Node, uint64_t* Value = nullptr) const;
bool IsInlineEntrypointOffset(const IR::OrderedNodeWrapper& WNode, uint64_t* Value) const;
IR::RegisterAllocationPass *RAPass;
FEXCore::IR::RegisterAllocationData *RAData;
@@ -161,6 +161,7 @@ private:
uint64_t SignalHandlerReturnAddress{};
uint32_t *SignalHandlerRefCounterPtr{};
FEXCore::CPU::Dispatcher *Dispatcher{};
};
CompilerSharedData ThreadSharedData;
@@ -239,9 +240,7 @@ private:
DEF_OP(Sbfe);
DEF_OP(Select);
DEF_OP(VExtractToGPR);
DEF_OP(Float_ToGPR_ZU);
DEF_OP(Float_ToGPR_ZS);
DEF_OP(Float_ToGPR_U);
DEF_OP(Float_ToGPR_S);
DEF_OP(FCmp);
DEF_OP(F80Cmp);
@@ -260,6 +259,7 @@ private:
DEF_OP(AtomicFetchAnd);
DEF_OP(AtomicFetchOr);
DEF_OP(AtomicFetchXor);
DEF_OP(AtomicFetchNeg);
///< Branch ops
DEF_OP(GuestCallDirect);
@@ -279,16 +279,14 @@ private:
///< Conversion ops
DEF_OP(VInsGPR);
DEF_OP(VCastFromGPR);
DEF_OP(Float_FromGPR_U);
DEF_OP(Float_FromGPR_S);
DEF_OP(Float_FToF);
DEF_OP(Vector_UToF);
DEF_OP(Vector_SToF);
DEF_OP(Vector_FToZU);
DEF_OP(Vector_FToZS);
DEF_OP(Vector_FToU);
DEF_OP(Vector_FToS);
DEF_OP(Vector_FToF);
DEF_OP(Vector_FToI);
///< Flag ops
DEF_OP(GetHostFlag);
@@ -306,6 +304,7 @@ private:
DEF_OP(StoreMem);
DEF_OP(VLoadMemElement);
DEF_OP(VStoreMemElement);
DEF_OP(CacheLineClear);
///< Misc ops
DEF_OP(EndBlock);
@@ -330,6 +329,7 @@ private:
DEF_OP(SplatVector);
DEF_OP(VMov);
DEF_OP(VAnd);
DEF_OP(VBic);
DEF_OP(VOr);
DEF_OP(VXor);
DEF_OP(VAdd);
@@ -340,8 +340,10 @@ private:
DEF_OP(VSQSub);
DEF_OP(VAddP);
DEF_OP(VAddV);
DEF_OP(VUMinV);
DEF_OP(VURAvg);
DEF_OP(VAbs);
DEF_OP(VPopcount);
DEF_OP(VFAdd);
DEF_OP(VFAddP);
DEF_OP(VFSub);
@@ -361,6 +363,8 @@ private:
DEF_OP(VSMax);
DEF_OP(VZip);
DEF_OP(VZip2);
DEF_OP(VUnZip);
DEF_OP(VUnZip2);
DEF_OP(VBSL);
DEF_OP(VCMPEQ);
DEF_OP(VCMPEQZ);
@@ -383,6 +387,7 @@ private:
DEF_OP(VInsElement);
DEF_OP(VInsScalarElement);
DEF_OP(VExtractElement);
DEF_OP(VDupElement);
DEF_OP(VExtr);
DEF_OP(VSLI);
DEF_OP(VSRI);
@@ -405,6 +410,7 @@ private:
DEF_OP(VSMull);
DEF_OP(VUMull2);
DEF_OP(VSMull2);
DEF_OP(VUABDL);
DEF_OP(VTBL1);
///< Encryption ops
@@ -36,10 +36,10 @@ DEF_OP(LoadContext) {
}
break;
case 16: {
LogMan::Msg::A("Invalid GPR load of size 16");
LOGMAN_MSG_A_FMT("Invalid GPR load of size 16");
}
break;
default: LogMan::Msg::A("Unhandled LoadContext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled LoadContext size: {}", OpSize);
}
}
else {
@@ -69,7 +69,7 @@ DEF_OP(LoadContext) {
movups(GetDst(Node), xword [STATE + Op->Offset]);
}
break;
default: LogMan::Msg::A("Unhandled LoadContext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled LoadContext size: {}", OpSize);
}
}
}
@@ -98,9 +98,9 @@ DEF_OP(StoreContext) {
}
break;
case 16:
LogMan::Msg::D("Invalid store size of 16");
LogMan::Msg::DFmt("Invalid store size of 16");
break;
default: LogMan::Msg::A("Unhandled StoreContext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled StoreContext size: {}", OpSize);
}
}
else {
@@ -129,7 +129,7 @@ DEF_OP(StoreContext) {
movups(xword [STATE + Op->Offset], GetSrc(Op->Header.Args[0].ID()));
}
break;
default: LogMan::Msg::A("Unhandled StoreContext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled StoreContext size: {}", OpSize);
}
}
}
@@ -160,17 +160,18 @@ DEF_OP(LoadContextIndexed) {
mov(GetDst<RA_64>(Node), qword [rax + index * Op->Stride]);
break;
default:
LogMan::Msg::A("Unhandled LoadContextIndexed size: %d", Op->Size);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed size: {}", Op->Size);
break;
}
break;
}
case 16:
LogMan::Msg::A("Invalid Class load of size 16");
LOGMAN_MSG_A_FMT("Invalid Class load of size 16");
break;
default:
LogMan::Msg::A("Unhandled LoadContextIndexed stride: %d", Op->Stride);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed stride: {}", Op->Stride);
break;
}
}
else {
switch (Op->Stride) {
@@ -195,7 +196,8 @@ DEF_OP(LoadContextIndexed) {
vmovq(GetDst(Node), qword [rax + index * Op->Stride]);
break;
default:
LogMan::Msg::A("Unhandled LoadContextIndexed size: %d", Op->Size);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed size: {}", Op->Size);
break;
}
break;
}
@@ -223,12 +225,14 @@ DEF_OP(LoadContextIndexed) {
movups(GetDst(Node), xword [STATE + rax]);
break;
default:
LogMan::Msg::A("Unhandled LoadContextIndexed size: %d", Op->Size);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed size: {}", Op->Size);
break;
}
break;
}
default:
LogMan::Msg::A("Unhandled LoadContextIndexed stride: %d", Op->Stride);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed stride: {}", Op->Stride);
break;
}
}
}
@@ -248,13 +252,14 @@ DEF_OP(StoreContextIndexed) {
case 4:
case 8: {
if (!(size == 1 || size == 2 || size == 4 || size == 8)) {
LogMan::Msg::A("Unhandled StoreContextIndexed size: %d", Op->Size);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed size: {}", Op->Size);
}
mov(AddressFrame(Op->Size * 8) [rax + index * Op->Stride], value);
break;
}
default:
LogMan::Msg::A("Unhandled StoreContextIndexed stride: %d", Op->Stride);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed stride: {}", Op->Stride);
break;
}
}
else {
@@ -279,7 +284,8 @@ DEF_OP(StoreContextIndexed) {
vmovq(AddressFrame(Op->Size * 8) [rax + index * Op->Stride], value);
break;
default:
LogMan::Msg::A("Unhandled StoreContextIndexed size: %d", size);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed size: {}", size);
break;
}
break;
}
@@ -307,12 +313,14 @@ DEF_OP(StoreContextIndexed) {
movups(xword [STATE + rax], value);
break;
default:
LogMan::Msg::A("Unhandled StoreContextIndexed size: %d", size);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed size: {}", size);
break;
}
break;
}
default:
LogMan::Msg::A("Unhandled StoreContextIndexed stride: %d", Op->Stride);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed stride: {}", Op->Stride);
break;
}
}
}
@@ -340,7 +348,7 @@ DEF_OP(SpillRegister) {
mov(qword [rsp + SlotOffset], GetSrc<RA_64>(Op->Header.Args[0].ID()));
break;
}
default: LogMan::Msg::A("Unhandled SpillRegister size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled SpillRegister size: {}", OpSize);
}
} else if (Op->Class == FEXCore::IR::FPRClass) {
switch (OpSize) {
@@ -356,10 +364,10 @@ DEF_OP(SpillRegister) {
movaps(xword [rsp + SlotOffset], GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LogMan::Msg::A("Unhandled SpillRegister size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled SpillRegister size: {}", OpSize);
}
} else {
LogMan::Msg::A("Unhandled SpillRegister class: %d", Op->Class.Val);
LOGMAN_MSG_A_FMT("Unhandled SpillRegister class: {}", Op->Class.Val);
}
@@ -388,7 +396,7 @@ DEF_OP(FillRegister) {
mov(GetDst<RA_64>(Node), qword [rsp + SlotOffset]);
break;
}
default: LogMan::Msg::A("Unhandled FillRegister size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled FillRegister size: {}", OpSize);
}
} else if (Op->Class == FEXCore::IR::FPRClass) {
switch (OpSize) {
@@ -404,10 +412,10 @@ DEF_OP(FillRegister) {
movaps(GetDst(Node), xword [rsp + SlotOffset]);
break;
}
default: LogMan::Msg::A("Unhandled FillRegister size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled FillRegister size: {}", OpSize);
}
} else {
LogMan::Msg::A("Unhandled FillRegister class: %d", Op->Class.Val);
LOGMAN_MSG_A_FMT("Unhandled FillRegister class: {}", Op->Class.Val);
}
}
@@ -425,16 +433,16 @@ DEF_OP(StoreFlag) {
mov(byte [STATE + (offsetof(FEXCore::Core::CPUState, flags[0]) + Op->Flag)], al);
}
Xbyak::RegExp X86JITCore::GenerateModRM(Xbyak::Reg Base, IR::OrderedNodeWrapper Offset, IR::MemOffsetType OffsetType, uint8_t OffsetScale) {
Xbyak::RegExp X86JITCore::GenerateModRM(Xbyak::Reg Base, IR::OrderedNodeWrapper Offset, IR::MemOffsetType OffsetType, uint8_t OffsetScale) const {
if (Offset.IsInvalid()) {
return Base;
} else {
if (OffsetScale != 1 && OffsetScale != 2 && OffsetScale != 4 && OffsetScale != 8) {
LogMan::Msg::A("Unhandled GenerateModRM OffsetScale: %d", OffsetScale);
LOGMAN_MSG_A_FMT("Unhandled GenerateModRM OffsetScale: {}", OffsetScale);
}
if (OffsetType != IR::MEM_OFFSET_SXTX) {
LogMan::Msg::A("Unhandled GenerateModRM OffsetType: %d", OffsetType.Val);
LOGMAN_MSG_A_FMT("Unhandled GenerateModRM OffsetType: {}", OffsetType.Val);
}
uint64_t Const;
@@ -475,7 +483,7 @@ DEF_OP(LoadMem) {
mov(Dst, qword [MemPtr]);
}
break;
default: LogMan::Msg::A("Unhandled LoadMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled LoadMem size: {}", Op->Size);
}
}
else
@@ -511,7 +519,7 @@ DEF_OP(LoadMem) {
}
}
break;
default: LogMan::Msg::A("Unhandled LoadMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled LoadMem size: {}", Op->Size);
}
}
}
@@ -537,7 +545,7 @@ DEF_OP(StoreMem) {
case 8:
mov(qword [MemPtr], GetSrc<RA_64>(Op->Header.Args[1].ID()));
break;
default: LogMan::Msg::A("Unhandled StoreMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled StoreMem size: {}", Op->Size);
}
}
else {
@@ -560,17 +568,25 @@ DEF_OP(StoreMem) {
else
movups(xword [MemPtr], GetSrc(Op->Header.Args[1].ID()));
break;
default: LogMan::Msg::A("Unhandled StoreMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled StoreMem size: {}", Op->Size);
}
}
}
DEF_OP(VLoadMemElement) {
LogMan::Msg::A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(VStoreMemElement) {
LogMan::Msg::A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(CacheLineClear) {
auto Op = IROp->C<IR::IROp_CacheLineClear>();
Xbyak::Reg MemReg = GetSrc<RA_64>(Op->Addr.ID());
clflush(ptr [MemReg]);
}
#undef DEF_OP
@@ -592,6 +608,7 @@ void X86JITCore::RegisterMemoryHandlers() {
REGISTER_OP(STOREMEMTSO, StoreMem);
REGISTER_OP(VLOADMEMELEMENT, VLoadMemElement);
REGISTER_OP(VSTOREMEMELEMENT, VStoreMemElement);
REGISTER_OP(CACHELINECLEAR, CacheLineClear);
#undef REGISTER_OP
}
}
+30 -19
View File
@@ -9,7 +9,11 @@ $end_info$
namespace FEXCore::CPU {
static void PrintValue(uint64_t Value) {
LogMan::Msg::D("Value: 0x%lx", Value);
LogMan::Msg::DFmt("Value: 0x{:x}", Value);
}
static void PrintVectorValue(uint64_t Value, uint64_t ValueUpper) {
LogMan::Msg::DFmt("Value: 0x{:016x}'{:016x}", ValueUpper, Value);
}
#define DEF_OP(x) void X86JITCore::Op_##x(FEXCore::IR::IROp_Header *IROp, uint32_t Node)
@@ -26,7 +30,7 @@ DEF_OP(Fence) {
case IR::Fence_Store.Val:
sfence();
break;
default: LogMan::Msg::A("Unknown Fence: %d", Op->Fence); break;
default: LOGMAN_MSG_A_FMT("Unknown Fence: {}", Op->Fence); break;
}
}
@@ -37,13 +41,22 @@ DEF_OP(Break) {
case 5: // Guest ud2
ud2();
break;
case 1: // Int <imm8>
ud2();
break;
case 2: // overflow
ud2();
break;
case 3: // int 1
ud2();
break;
case 4: { // HLT
// Time to quit
// Set our stack to the starting stack location
mov(rsp, qword [STATE + offsetof(FEXCore::Core::CpuStateFrame, ReturningStackLocation)]);
// Now we need to jump to the thread stop handler
mov(TMP1, Dispatcher->ThreadStopHandlerAddress);
mov(TMP1, ThreadSharedData.Dispatcher->ThreadStopHandlerAddress);
jmp(TMP1);
break;
}
@@ -56,7 +69,7 @@ DEF_OP(Break) {
}
// This jump target needs to be a constant offset here
mov(TMP1, Dispatcher->ThreadPauseHandlerAddress);
mov(TMP1, ThreadSharedData.Dispatcher->ThreadPauseHandlerAddress);
jmp(TMP1);
}
else {
@@ -65,12 +78,12 @@ DEF_OP(Break) {
mov(rsp, qword [STATE + offsetof(FEXCore::Core::CpuStateFrame, ReturningStackLocation)]);
// Now we need to jump to the thread stop handler
mov(TMP1, Dispatcher->ThreadStopHandlerAddress);
mov(TMP1, ThreadSharedData.Dispatcher->ThreadStopHandlerAddress);
jmp(TMP1);
}
break;
}
default: LogMan::Msg::A("Unknown Break reason: %d", Op->Reason);
default: LOGMAN_MSG_A_FMT("Unknown Break reason: {}", Op->Reason);
}
}
@@ -110,24 +123,22 @@ DEF_OP(SetRoundingMode) {
DEF_OP(Print) {
auto Op = IROp->C<IR::IROp_Print>();
for (auto &Reg : RA64)
push(Reg);
PushRegs();
if (IsGPR(Op->Header.Args[0].ID())) {
mov (rdi, GetSrc<RA_64>(Op->Header.Args[0].ID()));
auto NumPush = RA64.size();
if (NumPush & 1)
sub(rsp, 8); // Align
mov(rax, reinterpret_cast<uintptr_t>(PrintValue));
}
else {
pextrq(rdi, GetSrc(Op->Header.Args[0].ID()), 0);
pextrq(rsi, GetSrc(Op->Header.Args[0].ID()), 1);
mov (rdi, GetSrc<RA_64>(Op->Header.Args[0].ID()));
mov(rax, reinterpret_cast<uintptr_t>(PrintValue));
mov(rax, reinterpret_cast<uintptr_t>(PrintVectorValue));
}
call(rax);
if (NumPush & 1)
add(rsp, 8); // Align
for (uint32_t i = RA64.size(); i > 0; --i)
pop(RA64[i - 1]);
PopRegs();
}
#undef DEF_OP
@@ -25,7 +25,7 @@ DEF_OP(ExtractElementPair) {
mov (GetDst<RA_64>(Node), Regs[Op->Element]);
break;
}
default: LogMan::Msg::A("Unknown Size"); break;
default: LOGMAN_MSG_A_FMT("Unknown Size"); break;
}
}
@@ -51,7 +51,7 @@ DEF_OP(CreateElementPair) {
RegTmp = rax;
break;
}
default: LogMan::Msg::A("Unknown Size"); break;
default: LOGMAN_MSG_A_FMT("Unknown Size"); break;
}
if (Dst.first != RegSecond) {
File diff suppressed because it is too large. Load diff
+10 -8
View File
@@ -8,6 +8,8 @@ $end_info$
#include "Interface/Context/Context.h"
#include "Interface/Core/Core.h"
#include "Interface/Core/LookupCache.h"
#include <FEXCore/Utils/Allocator.h>
#include <sys/mman.h>
namespace FEXCore {
@@ -26,27 +28,27 @@ LookupCache::LookupCache(FEXCore::Context::Context *CTX)
// Allocate a region of memory that we can use to back our block pointers
// We need one pointer per page of virtual memory
// At 64GB of virtual memory this will allocate 128MB of virtual memory space
PagePointer = reinterpret_cast<uintptr_t>(mmap(nullptr, ctx->Config.VirtualMemSize / 4096 * 8, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0));
PagePointer = reinterpret_cast<uintptr_t>(FEXCore::Allocator::mmap(nullptr, ctx->Config.VirtualMemSize / 4096 * 8, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0));
// Allocate our memory backing our pages
// We need 32KB per guest page (One pointer per byte)
// XXX: We can drop down to 16KB if we store 4byte offsets from the code base
// We currently limit to 128MB of real memory for caching for the total cache size.
// Can end up being inefficient if we compile a small number of blocks per page
PageMemory = reinterpret_cast<uintptr_t>(mmap(nullptr, CODE_SIZE, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0));
LogMan::Throw::A(PageMemory != -1ULL, "Failed to allocate page memory");
PageMemory = reinterpret_cast<uintptr_t>(FEXCore::Allocator::mmap(nullptr, CODE_SIZE, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0));
LOGMAN_THROW_A(PageMemory != -1ULL, "Failed to allocate page memory");
// L1 Cache
L1Pointer = reinterpret_cast<uintptr_t>(mmap(nullptr, L1_SIZE, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0));
LogMan::Throw::A(L1Pointer != -1ULL, "Failed to allocate L1Pointer");
L1Pointer = reinterpret_cast<uintptr_t>(FEXCore::Allocator::mmap(nullptr, L1_SIZE, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0));
LOGMAN_THROW_A(L1Pointer != -1ULL, "Failed to allocate L1Pointer");
VirtualMemSize = ctx->Config.VirtualMemSize;
}
LookupCache::~LookupCache() {
munmap(reinterpret_cast<void*>(PagePointer), ctx->Config.VirtualMemSize / 4096 * 8);
munmap(reinterpret_cast<void*>(PageMemory), CODE_SIZE);
munmap(reinterpret_cast<void*>(L1Pointer), L1_SIZE);
FEXCore::Allocator::munmap(reinterpret_cast<void*>(PagePointer), ctx->Config.VirtualMemSize / 4096 * 8);
FEXCore::Allocator::munmap(reinterpret_cast<void*>(PageMemory), CODE_SIZE);
FEXCore::Allocator::munmap(reinterpret_cast<void*>(L1Pointer), L1_SIZE);
}
void LookupCache::HintUsedRange(uint64_t Address, uint64_t Size) {
+14 -8
View File
@@ -38,14 +38,21 @@ public:
std::map<uint64_t, std::vector<uint64_t>> CodePages;
void AddBlockMapping(uint64_t Address, void *HostCode, uint64_t Start, uint64_t Length) {
auto InsertPoint = BlockList.emplace(Address, (uintptr_t)HostCode);
LogMan::Throw::A(InsertPoint.second == true, "Dupplicate block mapping added");
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
auto InsertPoint =
#endif
BlockList.emplace(Address, (uintptr_t)HostCode);
LOGMAN_THROW_A(InsertPoint.second == true, "Dupplicate block mapping added");
for (auto CurrentPage = Start >> 12, EndPage = (Start + Length) >> 12; CurrentPage <= EndPage; CurrentPage++) {
CodePages[CurrentPage].push_back(Address);
}
// no need to update L1 or L2, they will get updated on first lookup
// There is no need to update L1 or L2, they will get updated on first lookup
// However, adding to L1 here increases performance
auto &L1Entry = reinterpret_cast<LookupCacheEntry*>(L1Pointer)[Address & L1_ENTRIES_MASK];
L1Entry.GuestCode = Address;
L1Entry.HostCode = (uintptr_t)HostCode;
}
void Erase(uint64_t Address) {
@@ -94,8 +101,8 @@ public:
void HintUsedRange(uint64_t Address, uint64_t Size);
uintptr_t GetL1Pointer() { return L1Pointer; }
uintptr_t GetPagePointer() { return PagePointer; }
uintptr_t GetL1Pointer() const { return L1Pointer; }
uintptr_t GetPagePointer() const { return PagePointer; }
uintptr_t GetVirtualMemorySize() const { return VirtualMemSize; }
constexpr static size_t L1_ENTRIES = 1 * 1024 * 1024; // Must be a power of 2
@@ -105,9 +112,8 @@ private:
void CacheBlockMapping(uint64_t Address, uintptr_t HostCode) {
// Do L1
auto &L1Entry = reinterpret_cast<LookupCacheEntry*>(L1Pointer)[Address & L1_ENTRIES_MASK];
if (L1Entry.GuestCode == Address) {
L1Entry.GuestCode = L1Entry.HostCode = 0;
}
L1Entry.GuestCode = Address;
L1Entry.HostCode = HostCode;
// Do ful map
auto FullAddress = Address;
File diff suppressed because it is too large. Load diff
+66 -40
View File
@@ -49,7 +49,7 @@ public:
OrderedNode* GetNewJumpBlock(uint64_t RIP) {
auto it = JumpTargets.find(RIP);
LogMan::Throw::A(it != JumpTargets.end(), "Couldn't find block generated for 0x%lx", RIP);
LOGMAN_THROW_A(it != JumpTargets.end(), "Couldn't find block generated for 0x%lx", RIP);
return it->second.BlockEntry;
}
@@ -59,7 +59,7 @@ public:
it->second.HaveEmitted = true;
if (CurrentCodeBlock->Wrapped(ListData.Begin()).ID() == it->second.BlockEntry->Wrapped(ListData.Begin()).ID()) return;
if (CurrentCodeBlock->Wrapped(DualListData.ListBegin()).ID() == it->second.BlockEntry->Wrapped(DualListData.ListBegin()).ID()) return;
// We have hit a RIP that is a jump target
// Thus we need to end up in a new block
@@ -81,14 +81,14 @@ public:
// rdi, 0x8
// cmp qword [rdi-8], 0
// jne .label
if (!BlockSetRIP) {
if (LastOp && !BlockSetRIP) {
auto it = JumpTargets.find(NextRIP);
if (it == JumpTargets.end() && LastOp) {
if (it == JumpTargets.end()) {
uint8_t GPRSize = CTX->Config.Is64BitMode ? 8 : 4;
const uint8_t GPRSize = CTX->GetGPRSize();
// If we don't have a jump target to a new block then we have to leave
// Set the RIP to the next instruction and leave
auto RelocatedNextRIP = _EntrypointOffset(NextRIP - Current_Header->Entry, GPRSize);
auto RelocatedNextRIP = _EntrypointOffset(NextRIP - Entry, GPRSize);
_ExitFunction(RelocatedNextRIP);
}
else if (it != JumpTargets.end()) {
@@ -104,7 +104,8 @@ public:
OpDispatchBuilder(FEXCore::Context::Context *ctx);
void ResetWorkingList();
bool HadDecodeFailure() { return DecodeFailure; }
void ResetDecodeFailure() { DecodeFailure = false; }
bool HadDecodeFailure() const { return DecodeFailure; }
void BeginFunction(uint64_t RIP, std::vector<FEXCore::Frontend::Decoder::DecodedBlocks> const *Blocks);
void Finalize();
@@ -259,12 +260,6 @@ public:
template<size_t ElementSize>
void PSUBQOp(OpcodeArgs);
template<size_t ElementSize>
void PMINUOp(OpcodeArgs);
template<size_t ElementSize>
void PMAXUOp(OpcodeArgs);
void PMINSWOp(OpcodeArgs);
void PMAXSWOp(OpcodeArgs);
template<size_t ElementSize>
void MOVMSKOp(OpcodeArgs);
void MOVMSKOpOne(OpcodeArgs);
template<size_t ElementSize>
@@ -274,20 +269,16 @@ public:
void PSHUFBOp(OpcodeArgs);
template<size_t ElementSize, bool HalfSize, bool Low>
void PSHUFDOp(OpcodeArgs);
template<size_t ElementSize>
void PCMPEQOp(OpcodeArgs);
template<size_t ElementSize>
void PCMPGTOp(OpcodeArgs);
void MOVDOp(OpcodeArgs);
template<size_t ElementSize, bool Scalar, uint32_t SrcIndex>
template<size_t ElementSize>
void PSRLDOp(OpcodeArgs);
template<size_t ElementSize>
void PSRLI(OpcodeArgs);
template<size_t ElementSize>
void PSLLI(OpcodeArgs);
template<size_t ElementSize, bool Scalar, uint32_t SrcIndex>
template<size_t ElementSize>
void PSLL(OpcodeArgs);
template<size_t ElementSize, bool Scalar, uint32_t SrcIndex>
template<size_t ElementSize>
void PSRAOp(OpcodeArgs);
void PSRLDQ(OpcodeArgs);
void PSLLDQ(OpcodeArgs);
@@ -296,21 +287,21 @@ public:
template<size_t ElementSize>
void PAVGOp(OpcodeArgs);
void MOVDDUPOp(OpcodeArgs);
template<size_t DstElementSize, bool Signed>
template<size_t DstElementSize>
void CVTGPR_To_FPR(OpcodeArgs);
template<size_t SrcElementSize, bool Signed, bool HostRoundingMode>
template<size_t SrcElementSize, bool HostRoundingMode>
void CVTFPR_To_GPR(OpcodeArgs);
template<size_t SrcElementSize, bool Signed, bool Widen>
template<size_t SrcElementSize, bool Widen>
void Vector_CVT_Int_To_Float(OpcodeArgs);
template<size_t DstElementSize, size_t SrcElementSize>
void Scalar_CVT_Float_To_Float(OpcodeArgs);
template<size_t DstElementSize, size_t SrcElementSize>
void Vector_CVT_Float_To_Float(OpcodeArgs);
template<size_t SrcElementSize, bool Signed, bool Narrow, bool HostRoundingMode>
template<size_t SrcElementSize, bool Narrow, bool HostRoundingMode>
void Vector_CVT_Float_To_Int(OpcodeArgs);
template<size_t SrcElementSize, bool Signed, bool Widen>
template<size_t SrcElementSize, bool Widen>
void MMX_To_XMM_Vector_CVT_Int_To_Float(OpcodeArgs);
template<size_t SrcElementSize, bool Signed, bool Narrow, bool HostRoundingMode>
template<size_t SrcElementSize, bool HostRoundingMode>
void XMM_To_MMX_Vector_CVT_Float_To_Int(OpcodeArgs);
void MASKMOVOp(OpcodeArgs);
void MOVBetweenGPR_FPR(OpcodeArgs);
@@ -324,17 +315,13 @@ public:
void ANDNOp(OpcodeArgs);
template<size_t ElementSize>
void PINSROp(OpcodeArgs);
void InsertPSOp(OpcodeArgs);
template<size_t ElementSize>
void PExtrOp(OpcodeArgs);
template<size_t ElementSize, bool Signed>
void PMULOp(OpcodeArgs);
template<size_t ElementSize>
void PSIGN(OpcodeArgs);
template<size_t ElementSize>
void PABS(OpcodeArgs);
// X87 Ops
template<size_t width>
void FLD(OpcodeArgs);
@@ -393,6 +380,8 @@ public:
void X87FRSTOR(OpcodeArgs);
void X87FXAM(OpcodeArgs);
void X87FCMOV(OpcodeArgs);
void X87EMMS(OpcodeArgs);
void X87FFREE(OpcodeArgs);
void FXCH(OpcodeArgs);
@@ -458,6 +447,8 @@ public:
template<uint8_t FenceType>
void FenceOp(OpcodeArgs);
void StoreFenceOrCLFlush(OpcodeArgs);
void PSADBW(OpcodeArgs);
void AESImcOp(OpcodeArgs);
@@ -467,6 +458,23 @@ public:
void AESDecLastOp(OpcodeArgs);
void AESKeyGenAssist(OpcodeArgs);
template<size_t ElementSize, size_t DstElementSize, bool Signed>
void ExtendVectorElements(OpcodeArgs);
template<size_t ElementSize, bool Scalar>
void VectorRound(OpcodeArgs);
template<size_t ElementSize>
void VectorBlend(OpcodeArgs);
template<size_t ElementSize>
void VectorVariableBlend(OpcodeArgs);
void PTestOp(OpcodeArgs);
void PHMINPOSUWOp(OpcodeArgs);
template<size_t ElementSize>
void DPPOp(OpcodeArgs);
void MPSADBWOp(OpcodeArgs);
void UnimplementedOp(OpcodeArgs);
#undef OpcodeArgs
@@ -491,13 +499,23 @@ private:
void StoreResult(FEXCore::IR::RegisterClassType Class, FEXCore::X86Tables::DecodedOp Op, FEXCore::X86Tables::DecodedOperand const& Operand, OrderedNode *const Src, int8_t Align);
void StoreResult(FEXCore::IR::RegisterClassType Class, FEXCore::X86Tables::DecodedOp Op, OrderedNode *const Src, int8_t Align);
uint8_t GetDstSize(FEXCore::X86Tables::DecodedOp Op);
uint8_t GetSrcSize(FEXCore::X86Tables::DecodedOp Op);
uint8_t GetDstSize(FEXCore::X86Tables::DecodedOp Op) const;
uint8_t GetSrcSize(FEXCore::X86Tables::DecodedOp Op) const;
template<unsigned BitOffset>
void SetRFLAG(OrderedNode *Value);
void SetRFLAG(OrderedNode *Value, unsigned BitOffset);
OrderedNode *GetRFLAG(unsigned BitOffset);
void SetRFLAG(OrderedNode *Value) {
flagsOp = FLAGS_OP_NONE;
_StoreFlag(_Bfe(1, 0, Value), BitOffset);
}
void SetRFLAG(OrderedNode *Value, unsigned BitOffset) {
flagsOp = FLAGS_OP_NONE;
_StoreFlag(_Bfe(1, 0, Value), BitOffset);
}
OrderedNode *GetRFLAG(unsigned BitOffset) {
return _LoadFlag(BitOffset);
}
OrderedNode *SelectCC(uint8_t OP, OrderedNode *TrueValue, OrderedNode *FalseValue);
@@ -520,14 +538,22 @@ private:
void GenerateFlags_RotateLeftImmediate(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, uint64_t Shift);
OrderedNode * GetX87Top();
enum X87Tag {
TAG_VALID = 0b00,
TAG_ZERO = 0b01,
TAG_SPECIAL = 0b10,
TAG_EMPTY = 0b11
};
void SetX87TopTag(OrderedNode *Value, uint32_t Tag);
OrderedNode *GetX87FTW(OrderedNode *Value);
void SetX87Top(OrderedNode *Value);
bool DestIsLockedMem(FEXCore::X86Tables::DecodedOp Op) {
return Op->Dest.TypeNone.Type !=FEXCore::X86Tables::DecodedOperand::TYPE_GPR && (Op->Flags & FEXCore::X86Tables::DecodeFlags::FLAG_LOCK);
bool DestIsLockedMem(FEXCore::X86Tables::DecodedOp Op) const {
return DestIsMem(Op) && (Op->Flags & FEXCore::X86Tables::DecodeFlags::FLAG_LOCK) != 0;
}
bool DestIsMem(FEXCore::X86Tables::DecodedOp Op) {
return Op->Dest.TypeNone.Type !=FEXCore::X86Tables::DecodedOperand::TYPE_GPR;
bool DestIsMem(FEXCore::X86Tables::DecodedOp Op) const {
return !Op->Dest.IsGPR();
}
void CreateJumpBlocks(std::vector<FEXCore::Frontend::Decoder::DecodedBlocks> const *Blocks);
@@ -0,0 +1,58 @@
/*
$info$
tags: frontend|x86-to-ir, opcodes|dispatcher-implementations
desc: Handles x86/64 Crypto instructions to IR
$end_info$
*/
#include "Interface/Core/OpcodeDispatcher.h"
#include <FEXCore/Core/X86Enums.h>
namespace FEXCore::IR {
#define OpcodeArgs [[maybe_unused]] FEXCore::X86Tables::DecodedOp Op
void OpDispatchBuilder::AESImcOp(OpcodeArgs) {
OrderedNode *Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
auto Res = _VAESImc(Src);
StoreResult(FPRClass, Op, Res, -1);
}
void OpDispatchBuilder::AESEncOp(OpcodeArgs) {
OrderedNode *Dest = LoadSource(FPRClass, Op, Op->Dest, Op->Flags, -1);
OrderedNode *Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
auto Res = _VAESEnc(Dest, Src);
StoreResult(FPRClass, Op, Res, -1);
}
void OpDispatchBuilder::AESEncLastOp(OpcodeArgs) {
OrderedNode *Dest = LoadSource(FPRClass, Op, Op->Dest, Op->Flags, -1);
OrderedNode *Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
auto Res = _VAESEncLast(Dest, Src);
StoreResult(FPRClass, Op, Res, -1);
}
void OpDispatchBuilder::AESDecOp(OpcodeArgs) {
OrderedNode *Dest = LoadSource(FPRClass, Op, Op->Dest, Op->Flags, -1);
OrderedNode *Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
auto Res = _VAESDec(Dest, Src);
StoreResult(FPRClass, Op, Res, -1);
}
void OpDispatchBuilder::AESDecLastOp(OpcodeArgs) {
OrderedNode *Dest = LoadSource(FPRClass, Op, Op->Dest, Op->Flags, -1);
OrderedNode *Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
auto Res = _VAESDecLast(Dest, Src);
StoreResult(FPRClass, Op, Res, -1);
}
void OpDispatchBuilder::AESKeyGenAssist(OpcodeArgs) {
OrderedNode *Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
LOGMAN_THROW_A(Op->Src[1].IsLiteral(), "Src1 needs to be literal here");
uint64_t RCON = Op->Src[1].Data.Literal.Value;
auto Res = _VAESKeyGenAssist(Src, RCON);
StoreResult(FPRClass, Op, Res, -1);
}
}
@@ -0,0 +1,810 @@
/*
$info$
tags: frontend|x86-to-ir, opcodes|dispatcher-implementations
desc: Handles x86/64 flag generation
$end_info$
*/
#include "Interface/Core/OpcodeDispatcher.h"
#include <FEXCore/Core/X86Enums.h>
namespace FEXCore::IR {
constexpr std::array<uint32_t, 17> FlagOffsets = {
FEXCore::X86State::RFLAG_CF_LOC,
FEXCore::X86State::RFLAG_PF_LOC,
FEXCore::X86State::RFLAG_AF_LOC,
FEXCore::X86State::RFLAG_ZF_LOC,
FEXCore::X86State::RFLAG_SF_LOC,
FEXCore::X86State::RFLAG_TF_LOC,
FEXCore::X86State::RFLAG_IF_LOC,
FEXCore::X86State::RFLAG_DF_LOC,
FEXCore::X86State::RFLAG_OF_LOC,
FEXCore::X86State::RFLAG_IOPL_LOC,
FEXCore::X86State::RFLAG_NT_LOC,
FEXCore::X86State::RFLAG_RF_LOC,
FEXCore::X86State::RFLAG_VM_LOC,
FEXCore::X86State::RFLAG_AC_LOC,
FEXCore::X86State::RFLAG_VIF_LOC,
FEXCore::X86State::RFLAG_VIP_LOC,
FEXCore::X86State::RFLAG_ID_LOC,
};
void OpDispatchBuilder::SetPackedRFLAG(bool Lower8, OrderedNode *Src) {
uint8_t NumFlags = FlagOffsets.size();
if (Lower8) {
NumFlags = 5;
}
auto OneConst = _Constant(1);
for (int i = 0; i < NumFlags; ++i) {
auto Tmp = _And(_Lshr(Src, _Constant(FlagOffsets[i])), OneConst);
SetRFLAG(Tmp, FlagOffsets[i]);
}
}
OrderedNode *OpDispatchBuilder::GetPackedRFLAG(bool Lower8) {
OrderedNode *Original = _Constant(2);
uint8_t NumFlags = FlagOffsets.size();
if (Lower8) {
NumFlags = 5;
}
for (int i = 0; i < NumFlags; ++i) {
OrderedNode *Flag = _LoadFlag(FlagOffsets[i]);
Flag = _Bfe(4, 32, 0, Flag);
Flag = _Lshl(Flag, _Constant(FlagOffsets[i]));
Original = _Or(Original, Flag);
}
return Original;
}
void OpDispatchBuilder::GenerateFlags_ADC(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2, OrderedNode *CF) {
auto Size = GetSrcSize(Op) * 8;
// AF
{
OrderedNode *AFRes = _Xor(_Xor(Src1, Src2), Res);
AFRes = _Bfe(1, 4, AFRes);
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(AFRes);
}
// SF
{
auto SignBitConst = _Constant(GetSrcSize(Op) * 8 - 1);
auto LshrOp = _Lshr(Res, SignBitConst);
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(LshrOp);
}
// PF
if (!CTX->Config.ABINoPF) {
auto PopCountOp = _Popcount(_And(Res, _Constant(0xFF)));
auto XorOp = _Xor(PopCountOp, _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(SelectOp);
}
// CF
// Unsigned
{
auto SelectOpLT = _Select(FEXCore::IR::COND_ULT, Res, Src2, _Constant(1), _Constant(0));
auto SelectOpLE = _Select(FEXCore::IR::COND_ULE, Res, Src2, _Constant(1), _Constant(0));
auto SelectCF = _Select(FEXCore::IR::COND_EQ, CF, _Constant(1), SelectOpLE, SelectOpLT);
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(SelectCF);
}
// OF
// Signed
{
auto NegOne = _Constant(~0ULL);
auto XorOp1 = _Xor(_Xor(Src1, Src2), NegOne);
auto XorOp2 = _Xor(Res, Src1);
OrderedNode *AndOp1 = _And(XorOp1, XorOp2);
switch (Size) {
case 8:
AndOp1 = _Bfe(1, 7, AndOp1);
break;
case 16:
AndOp1 = _Bfe(1, 15, AndOp1);
break;
case 32:
AndOp1 = _Bfe(1, 31, AndOp1);
break;
case 64:
AndOp1 = _Bfe(1, 63, AndOp1);
break;
default: LOGMAN_MSG_A("Unknown BFESize: %d", Size); break;
}
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(AndOp1);
}
}
void OpDispatchBuilder::GenerateFlags_SBB(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2, OrderedNode *CF) {
// AF
{
OrderedNode *AFRes = _Xor(_Xor(Src1, Src2), Res);
AFRes = _Bfe(1, 4, AFRes);
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(AFRes);
}
// SF
{
auto SignBitConst = _Constant(GetSrcSize(Op) * 8 - 1);
auto LshrOp = _Lshr(Res, SignBitConst);
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(LshrOp);
}
// PF
if (!CTX->Config.ABINoPF) {
auto PopCountOp = _Popcount(_And(Res, _Constant(0xFF)));
auto XorOp = _Xor(PopCountOp, _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(SelectOp);
}
// CF
// Unsigned
{
auto SelectOpLT = _Select(FEXCore::IR::COND_UGT, Res, Src1, _Constant(1), _Constant(0));
auto SelectOpLE = _Select(FEXCore::IR::COND_UGE, Res, Src1, _Constant(1), _Constant(0));
auto SelectCF = _Select(FEXCore::IR::COND_EQ, CF, _Constant(1), SelectOpLE, SelectOpLT);
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(SelectCF);
}
// OF
// Signed
{
auto XorOp1 = _Xor(Src1, Src2);
auto XorOp2 = _Xor(Res, Src1);
OrderedNode *AndOp1 = _And(XorOp1, XorOp2);
switch (GetSrcSize(Op)) {
case 1:
AndOp1 = _Bfe(1, 7, AndOp1);
break;
case 2:
AndOp1 = _Bfe(1, 15, AndOp1);
break;
case 4:
AndOp1 = _Bfe(1, 31, AndOp1);
break;
case 8:
AndOp1 = _Bfe(1, 63, AndOp1);
break;
default: LOGMAN_MSG_A("Unknown BFESize: %d", GetSrcSize(Op)); break;
}
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(AndOp1);
}
}
void OpDispatchBuilder::GenerateFlags_SUB(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2, bool UpdateCF) {
// AF
{
OrderedNode *AFRes = _Xor(_Xor(Src1, Src2), Res);
AFRes = _Bfe(1, 4, AFRes);
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(AFRes);
}
// SF
{
auto SignBitConst = _Constant(GetSrcSize(Op) * 8 - 1);
auto LshrOp = _Lshr(Res, SignBitConst);
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(LshrOp);
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// ZF
{
auto ZeroConst = _Constant(0);
auto OneConst = _Constant(1);
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, ZeroConst, OneConst, ZeroConst);
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(SelectOp);
}
// CF
if (UpdateCF) {
auto ZeroConst = _Constant(0);
auto OneConst = _Constant(1);
auto SelectOp = _Select(FEXCore::IR::COND_ULT,
Src1, Src2, OneConst, ZeroConst);
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(SelectOp);
}
// OF
{
auto XorOp1 = _Xor(Src1, Src2);
auto XorOp2 = _Xor(Res, Src1);
OrderedNode *FinalAnd = _And(XorOp1, XorOp2);
FinalAnd = _Bfe(1, GetSrcSize(Op) * 8 - 1, FinalAnd);
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(FinalAnd);
}
}
void OpDispatchBuilder::GenerateFlags_ADD(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2, bool UpdateCF) {
// AF
{
OrderedNode *AFRes = _Xor(_Xor(Src1, Src2), Res);
AFRes = _Bfe(1, 4, AFRes);
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(AFRes);
}
// SF
{
auto SignBitConst = _Constant(GetSrcSize(Op) * 8 - 1);
auto LshrOp = _Lshr(Res, SignBitConst);
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(LshrOp);
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(SelectOp);
}
// CF
if (UpdateCF) {
auto SelectOp = _Select(FEXCore::IR::COND_ULT, Res, Src2, _Constant(1), _Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(SelectOp);
}
// OF
{
auto NegOne = _Constant(~0ULL);
auto XorOp1 = _Xor(_Xor(Src1, Src2), NegOne);
auto XorOp2 = _Xor(Res, Src1);
OrderedNode *AndOp1 = _And(XorOp1, XorOp2);
switch (GetSrcSize(Op)) {
case 1:
AndOp1 = _Bfe(1, 7, AndOp1);
break;
case 2:
AndOp1 = _Bfe(1, 15, AndOp1);
break;
case 4:
AndOp1 = _Bfe(1, 31, AndOp1);
break;
case 8:
AndOp1 = _Bfe(1, 63, AndOp1);
break;
default: LOGMAN_MSG_A("Unknown BFESize: %d", GetSrcSize(Op)); break;
}
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(AndOp1);
}
}
void OpDispatchBuilder::GenerateFlags_MUL(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *High) {
// PF/AF/ZF/SF
// Undefined
{
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(_Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(_Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(_Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(_Constant(0));
}
// CF/OF
{
// CF and OF are set if the result of the operation can't be fit in to the destination register
// If the value can fit then the top bits will be zero
auto SignBit = _Sbfe(1, GetSrcSize(Op) * 8 - 1, Res);
auto SelectOp = _Select(FEXCore::IR::COND_EQ, High, SignBit, _Constant(0), _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(SelectOp);
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(SelectOp);
}
}
void OpDispatchBuilder::GenerateFlags_UMUL(FEXCore::X86Tables::DecodedOp Op, OrderedNode *High) {
// AF/SF/PF/ZF
// Undefined
{
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(_Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(_Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(_Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(_Constant(0));
}
// CF/OF
{
// CF and OF are set if the result of the operation can't be fit in to the destination register
// The result register will be all zero if it can't fit due to how multiplication behaves
auto SelectOp = _Select(FEXCore::IR::COND_EQ, High, _Constant(0), _Constant(0), _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(SelectOp);
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(SelectOp);
}
}
void OpDispatchBuilder::GenerateFlags_Logical(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2) {
// AF
{
// Undefined
// Set to zero anyway
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(_Constant(0));
}
// SF
{
auto SignBitConst = _Constant(GetSrcSize(Op) * 8 - 1);
auto LshrOp = _Lshr(Res, SignBitConst);
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(LshrOp);
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(SelectOp);
}
// CF/OF
{
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(_Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(_Constant(0));
}
}
#define COND_FLAG_SET(cond, flag, newflag) \
auto oldflag = GetRFLAG(FEXCore::X86State::flag);\
auto newval = _Select(FEXCore::IR::COND_EQ, cond, _Constant(0), oldflag, newflag);\
SetRFLAG<FEXCore::X86State::flag>(newval);
void OpDispatchBuilder::GenerateFlags_ShiftLeft(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2) {
// CF
{
// Extract the last bit shifted in to CF
auto Size = _Constant(GetSrcSize(Op) * 8);
auto ShiftAmt = _Sub(Size, Src2);
auto LastBit = _And(_Lshr(Src1, ShiftAmt), _Constant(1));
COND_FLAG_SET(Src2, RFLAG_CF_LOC, LastBit);
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
COND_FLAG_SET(Src2, RFLAG_PF_LOC, XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// AF
{
// Undefined
// Set to zero anyway
COND_FLAG_SET(Src2, RFLAG_AF_LOC, _Constant(0));
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
COND_FLAG_SET(Src2, RFLAG_ZF_LOC, SelectOp);
}
// SF
{
auto val = _Bfe(1, GetSrcSize(Op) * 8 - 1, Res);
COND_FLAG_SET(Src2, RFLAG_SF_LOC, val);
}
// OF
{
// In the case of left shift. OF is only set from the result of <Top Source Bit> XOR <Top Result Bit>
// When Shift > 1 then OF is undefined
auto val = _Bfe(1, GetSrcSize(Op) * 8 - 1, _Xor(Src1, Res));
COND_FLAG_SET(Src2, RFLAG_OF_LOC, val);
}
}
void OpDispatchBuilder::GenerateFlags_ShiftRight(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2) {
// CF
{
// Extract the last bit shifted in to CF
auto ShiftAmt = _Sub(Src2, _Constant(1));
auto LastBit = _And(_Lshr(Src1, ShiftAmt), _Constant(1));
COND_FLAG_SET(Src2, RFLAG_CF_LOC, LastBit);
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
COND_FLAG_SET(Src2, RFLAG_PF_LOC, XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// AF
{
// Undefined
// Set to zero anyway
COND_FLAG_SET(Src2, RFLAG_AF_LOC, _Constant(0));
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
COND_FLAG_SET(Src2, RFLAG_ZF_LOC, SelectOp);
}
// SF
{
auto val =_Bfe(1, GetSrcSize(Op) * 8 - 1, Res);
COND_FLAG_SET(Src2, RFLAG_SF_LOC, val);
}
// OF
{
// Only defined when Shift is 1 else undefined
// OF flag is set if a sign change occurred
auto val = _Bfe(1, GetSrcSize(Op) * 8 - 1, _Xor(Src1, Res));
COND_FLAG_SET(Src2, RFLAG_OF_LOC, val);
}
}
void OpDispatchBuilder::GenerateFlags_SignShiftRight(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2) {
// CF
{
// Extract the last bit shifted in to CF
auto ShiftAmt = _Sub(Src2, _Constant(1));
auto LastBit = _And(_Lshr(Src1, ShiftAmt), _Constant(1));
COND_FLAG_SET(Src2, RFLAG_CF_LOC, LastBit);
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
COND_FLAG_SET(Src2, RFLAG_PF_LOC, XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// AF
{
// Undefined
// Set to zero anyway
COND_FLAG_SET(Src2, RFLAG_AF_LOC, _Constant(0));
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
COND_FLAG_SET(Src2, RFLAG_ZF_LOC, SelectOp);
}
// SF
{
auto SignBitConst = _Constant(GetSrcSize(Op) * 8 - 1);
auto LshrOp = _Lshr(Res, SignBitConst);
COND_FLAG_SET(Src2, RFLAG_SF_LOC, LshrOp);
}
// OF
{
COND_FLAG_SET(Src2, RFLAG_OF_LOC, _Constant(0));
}
}
void OpDispatchBuilder::GenerateFlags_ShiftLeftImmediate(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, uint64_t Shift) {
// No flags changed if shift is zero
if (Shift == 0) return;
// CF
{
// Extract the last bit shifted in to CF
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(_Bfe(1, GetSrcSize(Op) * 8 - Shift, Src1));
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// AF
{
// Undefined
// Set to zero anyway
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(_Constant(0));
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(SelectOp);
}
// SF
{
auto LshrOp = _Bfe(1, GetSrcSize(Op) * 8 - 1, Res);
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(LshrOp);
// OF
// In the case of left shift. OF is only set from the result of <Top Source Bit> XOR <Top Result Bit>
if (Shift == 1) {
auto SourceBit = _Bfe(1, GetSrcSize(Op) * 8 - 1, Src1);
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(_Xor(SourceBit, LshrOp));
}
}
}
void OpDispatchBuilder::GenerateFlags_SignShiftRightImmediate(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, uint64_t Shift) {
// No flags changed if shift is zero
if (Shift == 0) return;
// CF
{
// Extract the last bit shifted in to CF
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(_Bfe(1, Shift-1, Src1));
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// AF
{
// Undefined
// Set to zero anyway
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(_Constant(0));
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(SelectOp);
}
// SF
{
auto SignBitConst = _Constant(GetSrcSize(Op) * 8 - 1);
auto LshrOp = _Lshr(Res, SignBitConst);
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(LshrOp);
// OF
// Only defined when Shift is 1 else undefined
// Only is set if the top bit was set to 1 when shifted
// So it is set to same value as SF
if (Shift == 1) {
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(_Constant(0));
}
}
}
void OpDispatchBuilder::GenerateFlags_ShiftRightImmediate(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, uint64_t Shift) {
// No flags changed if shift is zero
if (Shift == 0) return;
// CF
{
// Extract the last bit shifted in to CF
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(_Bfe(1, Shift-1, Src1));
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// AF
{
// Undefined
// Set to zero anyway
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(_Constant(0));
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(SelectOp);
}
// SF
{
auto SignBitConst = _Constant(GetSrcSize(Op) * 8 - 1);
auto LshrOp = _Lshr(Res, SignBitConst);
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(LshrOp);
}
// OF
{
// Only defined when Shift is 1 else undefined
// Is set to the MSB of the original value
if (Shift == 1) {
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(_Bfe(1, GetSrcSize(Op) * 8 - 1, Src1));
}
}
}
void OpDispatchBuilder::GenerateFlags_RotateRight(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2) {
auto OpSize = GetSrcSize(Op) * 8;
// Extract the last bit shifted in to CF
auto NewCF = _Bfe(1, OpSize - 1, Res);
// CF
{
auto OldCF = GetRFLAG(FEXCore::X86State::RFLAG_CF_LOC);
auto CF = _Select(FEXCore::IR::COND_EQ, Src2, _Constant(0), OldCF, NewCF);
// Extract the last bit shifted in to CF
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(CF);
}
// OF
{
auto OldOF = GetRFLAG(FEXCore::X86State::RFLAG_OF_LOC);
// OF is set to the XOR of the new CF bit and the most significant bit of the result
auto NewOF = _Xor(_Bfe(1, OpSize - 2, Res), NewCF);
// If shift == 0, don't update flags
auto OF = _Select(FEXCore::IR::COND_EQ, Src2, _Constant(0), OldOF, NewOF);
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(OF);
}
}
void OpDispatchBuilder::GenerateFlags_RotateLeft(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2) {
auto OpSize = GetSrcSize(Op) * 8;
// Extract the last bit shifted in to CF
//auto Size = _Constant(GetSrcSize(Res) * 8);
//auto ShiftAmt = _Sub(Size, Src2);
auto NewCF = _Bfe(1, 0, Res);
// CF
{
auto OldCF = GetRFLAG(FEXCore::X86State::RFLAG_CF_LOC);
auto CF = _Select(FEXCore::IR::COND_EQ, Src2, _Constant(0), OldCF, NewCF);
// Extract the last bit shifted in to CF
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(CF);
}
// OF
{
auto OldOF = GetRFLAG(FEXCore::X86State::RFLAG_OF_LOC);
// OF is set to the XOR of the new CF bit and the most significant bit of the result
auto NewOF = _Xor(_Bfe(1, OpSize - 1, Res), NewCF);
auto OF = _Select(FEXCore::IR::COND_EQ, Src2, _Constant(0), OldOF, NewOF);
// If shift == 0, don't update flags
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(OF);
}
}
void OpDispatchBuilder::GenerateFlags_RotateRightImmediate(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, uint64_t Shift) {
if (Shift == 0) return;
auto OpSize = GetSrcSize(Op) * 8;
auto NewCF = _Bfe(1, OpSize - Shift, Src1);
// CF
{
// Extract the last bit shifted in to CF
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(NewCF);
}
// OF
{
if (Shift == 1) {
// OF is set to the XOR of the new CF bit and the most significant bit of the result
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(_Xor(_Bfe(1, OpSize - 1, Res), NewCF));
}
}
}
void OpDispatchBuilder::GenerateFlags_RotateLeftImmediate(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, uint64_t Shift) {
if (Shift == 0) return;
auto OpSize = GetSrcSize(Op) * 8;
// CF
{
// Extract the last bit shifted in to CF
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(_Bfe(1, Shift, Src1));
}
// OF
{
if (Shift == 1) {
// OF is the top two MSBs XOR'd together
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(_Xor(_Bfe(1, OpSize - 1, Src1), _Bfe(1, OpSize - 2, Src1)));
}
}
}
}
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,285 @@
// This is the vsyscall page for x86_64 guest code
// This was compiled with nasm with the following source then exported to binary
//BITS 64;
//
//align 4096, db 0xcc
// ; __NR_gettimeofday
// mov rax, 96
// syscall
// ret
//
//align 1024, db 0xcc
// ; __NR_time
// mov rax, 201
// syscall
// ret
//
//align 1024, db 0xcc
// ; __NR_getcpu
// mov rax, 309
// syscall
// ret
//
//align 4096, db 0xcc
// We only want one of these pages per FEX process
// One page
const static uint8_t VSyscallData[0x1000] = {
0xB8, 0x60, 0x00, 0x00, 0x00, 0x0F, 0x05, 0xC3, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xB8, 0xC9, 0x00, 0x00, 0x00, 0x0F, 0x05, 0xC3, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xB8, 0x35, 0x01, 0x00, 0x00, 0x0F, 0x05, 0xC3, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC
};
@@ -114,8 +114,6 @@ void InstallDebugInfo() {
GenerateDebugTable(PrimaryInstGroupOps, PrimaryGroupOpTable);
GenerateDebugTable(SecondInstGroupOps, SecondaryExtensionOpTable);
LogMan::Msg::D("Installing debug info");
}
}
#endif
+5 -4
View File
@@ -6,6 +6,7 @@ $end_info$
*/
#include "Interface/Core/X86HelperGen.h"
#include <FEXCore/Utils/Allocator.h>
#include <cstring>
#include <stdlib.h>
@@ -31,7 +32,7 @@ X86GeneratedCode::X86GeneratedCode() {
}
X86GeneratedCode::~X86GeneratedCode() {
munmap(CodePtr, CODE_SIZE);
FEXCore::Allocator::munmap(CodePtr, CODE_SIZE);
}
void* X86GeneratedCode::AllocateGuestCodeSpace(size_t Size) {
@@ -39,7 +40,7 @@ void* X86GeneratedCode::AllocateGuestCodeSpace(size_t Size) {
if (Is64BitMode()) {
// 64bit mode can have its sigret handler anywhere
return mmap(nullptr, Size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
return FEXCore::Allocator::mmap(nullptr, Size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
}
// First 64bit page
@@ -49,14 +50,14 @@ void* X86GeneratedCode::AllocateGuestCodeSpace(size_t Size) {
// We need to have the sigret handler in the lower 32bits of memory space
// Scan top down and try to allocate a location
for (size_t Location = 0xFFFF'E000; Location != 0x0; Location -= 0x1000) {
void *Ptr = mmap(reinterpret_cast<void*>(Location), Size, PROT_READ | PROT_WRITE, MAP_FIXED_NOREPLACE | MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
void *Ptr = ::mmap(reinterpret_cast<void*>(Location), Size, PROT_READ | PROT_WRITE, MAP_FIXED_NOREPLACE | MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
if (Ptr != MAP_FAILED &&
reinterpret_cast<uintptr_t>(Ptr) >= LOCATION_MAX) {
// Failed to map in the lower 32bits
// Try again
// Can happen in the case that host kernel ignores MAP_FIXED_NOREPLACE
munmap(Ptr, Size);
::munmap(Ptr, Size);
continue;
}
-1
View File
@@ -108,7 +108,6 @@ void InitializeInfoTables(Context::OperatingMode Mode) {
#ifndef NDEBUG
X86InstDebugInfo::InstallDebugInfo();
LogMan::Msg::D("X86Tables had %ld total insts, and %ld labeled as understood", Total, NumInsts);
#endif
}
@@ -4,6 +4,8 @@ tags: frontend|x86-tables
$end_info$
*/
#include <FEXCore/Core/Context.h>
#include "Interface/Core/X86Tables/X86Tables.h"
namespace FEXCore::X86Tables {
@@ -41,10 +43,10 @@ void InitializeH0F38Tables() {
{OPD(PF_38_NONE, 0x0B), 1, X86InstInfo{"PMULHRSW", TYPE_INST, GenFlagsSameSize(SIZE_64BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS | FLAGS_SF_MMX, 0, nullptr}},
{OPD(PF_38_66, 0x0B), 1, X86InstInfo{"PMULHRSW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_66, 0x10), 1, X86InstInfo{"PBLENDVB", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(PF_38_66, 0x14), 1, X86InstInfo{"BLENDVPS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(PF_38_66, 0x15), 1, X86InstInfo{"BLENDVPD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(PF_38_66, 0x17), 1, X86InstInfo{"PTEST", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(PF_38_66, 0x10), 1, X86InstInfo{"PBLENDVB", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_66, 0x14), 1, X86InstInfo{"BLENDVPS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_66, 0x15), 1, X86InstInfo{"BLENDVPD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_66, 0x17), 1, X86InstInfo{"PTEST", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_NONE, 0x1C), 1, X86InstInfo{"PABSB", TYPE_INST, GenFlagsSameSize(SIZE_64BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS | FLAGS_SF_MMX, 0, nullptr}},
{OPD(PF_38_66, 0x1C), 1, X86InstInfo{"PABSB", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_NONE, 0x1D), 1, X86InstInfo{"PABSW", TYPE_INST, GenFlagsSameSize(SIZE_64BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS | FLAGS_SF_MMX, 0, nullptr}},
@@ -52,34 +54,34 @@ void InitializeH0F38Tables() {
{OPD(PF_38_NONE, 0x1E), 1, X86InstInfo{"PABSD", TYPE_INST, GenFlagsSameSize(SIZE_64BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS | FLAGS_SF_MMX, 0, nullptr}},
{OPD(PF_38_66, 0x1E), 1, X86InstInfo{"PABSD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_66, 0x20), 1, X86InstInfo{"PMOVSXBW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(PF_38_66, 0x21), 1, X86InstInfo{"PMOVSXBD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(PF_38_66, 0x22), 1, X86InstInfo{"PMOVSXBQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(PF_38_66, 0x23), 1, X86InstInfo{"PMOVSXWD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(PF_38_66, 0x24), 1, X86InstInfo{"PMOVSXWQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(PF_38_66, 0x25), 1, X86InstInfo{"PMOVSXDQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(PF_38_66, 0x28), 1, X86InstInfo{"PMULDQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(PF_38_66, 0x29), 1, X86InstInfo{"PCMPEQQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(PF_38_66, 0x20), 1, X86InstInfo{"PMOVSXBW", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_64BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_66, 0x21), 1, X86InstInfo{"PMOVSXBD", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_32BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_66, 0x22), 1, X86InstInfo{"PMOVSXBQ", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_16BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_66, 0x23), 1, X86InstInfo{"PMOVSXWD", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_64BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_66, 0x24), 1, X86InstInfo{"PMOVSXWQ", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_32BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_66, 0x25), 1, X86InstInfo{"PMOVSXDQ", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_64BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_66, 0x28), 1, X86InstInfo{"PMULDQ", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_66, 0x29), 1, X86InstInfo{"PCMPEQQ", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_66, 0x2A), 1, X86InstInfo{"MOVNTDQA", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_SF_MOD_MEM_ONLY | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_66, 0x2B), 1, X86InstInfo{"PACKUSDW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(PF_38_66, 0x2B), 1, X86InstInfo{"PACKUSDW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_66, 0x30), 1, X86InstInfo{"PMOVZXBW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(PF_38_66, 0x31), 1, X86InstInfo{"PMOVZXBD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(PF_38_66, 0x32), 1, X86InstInfo{"PMOVZXBQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(PF_38_66, 0x33), 1, X86InstInfo{"PMOVZXWD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(PF_38_66, 0x34), 1, X86InstInfo{"PMOVZXWQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(PF_38_66, 0x35), 1, X86InstInfo{"PMOVZXDQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(PF_38_66, 0x38), 1, X86InstInfo{"PMINSB", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(PF_38_66, 0x39), 1, X86InstInfo{"PMINSD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(PF_38_66, 0x3A), 1, X86InstInfo{"PMINUW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(PF_38_66, 0x3B), 1, X86InstInfo{"PMINUD", TYPE_INST, FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_66, 0x3C), 1, X86InstInfo{"PMAXSB", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(PF_38_66, 0x3D), 1, X86InstInfo{"PMAXSD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(PF_38_66, 0x3E), 1, X86InstInfo{"PMAXUW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(PF_38_66, 0x3F), 1, X86InstInfo{"PMAXUD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(PF_38_66, 0x30), 1, X86InstInfo{"PMOVZXBW", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_64BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_66, 0x31), 1, X86InstInfo{"PMOVZXBD", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_32BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_66, 0x32), 1, X86InstInfo{"PMOVZXBQ", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_16BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_66, 0x33), 1, X86InstInfo{"PMOVZXWD", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_64BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_66, 0x34), 1, X86InstInfo{"PMOVZXWQ", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_32BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_66, 0x35), 1, X86InstInfo{"PMOVZXDQ", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_64BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_66, 0x38), 1, X86InstInfo{"PMINSB", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_66, 0x39), 1, X86InstInfo{"PMINSD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_66, 0x3A), 1, X86InstInfo{"PMINUW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_66, 0x3B), 1, X86InstInfo{"PMINUD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_66, 0x3C), 1, X86InstInfo{"PMAXSB", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_66, 0x3D), 1, X86InstInfo{"PMAXSD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_66, 0x3E), 1, X86InstInfo{"PMAXUW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_66, 0x3F), 1, X86InstInfo{"PMAXUD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_66, 0x40), 1, X86InstInfo{"PMULLD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(PF_38_66, 0x41), 1, X86InstInfo{"PHMINPOSUW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(PF_38_66, 0x40), 1, X86InstInfo{"PMULLD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_66, 0x41), 1, X86InstInfo{"PHMINPOSUW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_66, 0xDB), 1, X86InstInfo{"AESIMC", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(PF_38_66, 0xDC), 1, X86InstInfo{"AESENC", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
@@ -16,26 +16,26 @@ void InitializeH0F3ATables(Context::OperatingMode Mode) {
const U16U8InfoStruct H0F3ATable[] = {
{OPD(0, PF_3A_NONE, 0x0F), 1, X86InstInfo{"PALIGNR", TYPE_INST, GenFlagsSameSize(SIZE_64BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS | FLAGS_SF_MMX, 1, nullptr}},
{OPD(0, PF_3A_66, 0x08), 1, X86InstInfo{"ROUNDPS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(0, PF_3A_66, 0x09), 1, X86InstInfo{"ROUNDPD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(0, PF_3A_66, 0x0A), 1, X86InstInfo{"ROUNDSS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(0, PF_3A_66, 0x0B), 1, X86InstInfo{"ROUNDSD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(0, PF_3A_66, 0x0C), 1, X86InstInfo{"BLENDPS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(0, PF_3A_66, 0x0D), 1, X86InstInfo{"BLENDPD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(0, PF_3A_66, 0x0E), 1, X86InstInfo{"PBLENDW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(0, PF_3A_66, 0x08), 1, X86InstInfo{"ROUNDPS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(0, PF_3A_66, 0x09), 1, X86InstInfo{"ROUNDPD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(0, PF_3A_66, 0x0A), 1, X86InstInfo{"ROUNDSS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(0, PF_3A_66, 0x0B), 1, X86InstInfo{"ROUNDSD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(0, PF_3A_66, 0x0C), 1, X86InstInfo{"BLENDPS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(0, PF_3A_66, 0x0D), 1, X86InstInfo{"BLENDPD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(0, PF_3A_66, 0x0E), 1, X86InstInfo{"PBLENDW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(0, PF_3A_66, 0x0F), 1, X86InstInfo{"PALIGNR", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(0, PF_3A_66, 0x14), 1, X86InstInfo{"PEXTRB", TYPE_INST, GenFlagsSizes(SIZE_8BIT, SIZE_128BIT) | FLAGS_MODRM | FLAGS_SF_MOD_DST | FLAGS_SF_DST_GPR | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(0, PF_3A_66, 0x15), 1, X86InstInfo{"PEXTRW", TYPE_INST, GenFlagsSizes(SIZE_16BIT, SIZE_128BIT) | FLAGS_MODRM | FLAGS_SF_MOD_DST | FLAGS_SF_DST_GPR | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(0, PF_3A_66, 0x16), 1, X86InstInfo{"PEXTRD", TYPE_INST, GenFlagsSizes(SIZE_32BIT, SIZE_128BIT) | FLAGS_MODRM | FLAGS_SF_MOD_DST | FLAGS_SF_DST_GPR | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(0, PF_3A_66, 0x17), 1, X86InstInfo{"EXTRACTPS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(0, PF_3A_66, 0x17), 1, X86InstInfo{"EXTRACTPS", TYPE_INST, GenFlagsSizes(SIZE_32BIT, SIZE_128BIT) | FLAGS_MODRM | FLAGS_SF_MOD_DST | FLAGS_SF_DST_GPR | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(0, PF_3A_66, 0x20), 1, X86InstInfo{"PINSRB", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_8BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS | FLAGS_SF_SRC_GPR, 1, nullptr}},
{OPD(0, PF_3A_66, 0x21), 1, X86InstInfo{"INSERTPS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(0, PF_3A_66, 0x20), 1, X86InstInfo{"PINSRB", TYPE_INST, GenFlagsDstSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS | FLAGS_SF_SRC_GPR, 1, nullptr}},
{OPD(0, PF_3A_66, 0x21), 1, X86InstInfo{"INSERTPS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(0, PF_3A_66, 0x22), 1, X86InstInfo{"PINSRD", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_32BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS | FLAGS_SF_SRC_GPR, 1, nullptr}},
{OPD(0, PF_3A_66, 0x40), 1, X86InstInfo{"DPPS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(0, PF_3A_66, 0x41), 1, X86InstInfo{"DPPD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(0, PF_3A_66, 0x42), 1, X86InstInfo{"MPSADBW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(0, PF_3A_66, 0x40), 1, X86InstInfo{"DPPS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(0, PF_3A_66, 0x41), 1, X86InstInfo{"DPPD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(0, PF_3A_66, 0x42), 1, X86InstInfo{"MPSADBW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(0, PF_3A_66, 0x44), 1, X86InstInfo{"PCLMULQDQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(0, PF_3A_66, 0x60), 1, X86InstInfo{"PCMPESTRM", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
@@ -27,7 +27,7 @@ void InitializeSecondaryTables(Context::OperatingMode Mode) {
{0x0B, 1, X86InstInfo{"UD2", TYPE_INST, FLAGS_DEBUG | FLAGS_BLOCK_END | FLAGS_NO_OVERLAY, 0, nullptr}},
{0x0C, 1, X86InstInfo{"", TYPE_INVALID, FLAGS_NO_OVERLAY, 0, nullptr}},
{0x0D, 1, X86InstInfo{"", TYPE_GROUP_P, FLAGS_MODRM | FLAGS_NO_OVERLAY, 0, nullptr}},
{0x0E, 1, X86InstInfo{"FEMMS", TYPE_INST, FLAGS_BLOCK_END | FLAGS_NO_OVERLAY, 0, nullptr}},
{0x0E, 1, X86InstInfo{"FEMMS", TYPE_INST, FLAGS_NO_OVERLAY, 0, nullptr}},
{0x0F, 1, X86InstInfo{"", TYPE_3DNOW_TABLE, FLAGS_NO_OVERLAY, 0, nullptr}},
{0x10, 1, X86InstInfo{"MOVUPS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
@@ -6,6 +6,7 @@ $end_info$
#pragma once
#include <FEXCore/Debug/X86Tables.h>
#include <FEXCore/Core/Context.h>
#include <FEXCore/Utils/LogManager.h>
@@ -33,7 +34,7 @@ static inline void GenerateTable(X86InstInfo *FinalTable, U8U8InfoStruct const *
auto OpNum = Op.first;
X86InstInfo const &Info = Op.Info;
for (uint32_t i = 0; i < Op.second; ++i) {
LogMan::Throw::A(FinalTable[OpNum + i].Type == TYPE_UNKNOWN, "Duplicate Entry %s->%s", FinalTable[OpNum + i].Name, Info.Name);
LOGMAN_THROW_A(FinalTable[OpNum + i].Type == TYPE_UNKNOWN, "Duplicate Entry %s->%s", FinalTable[OpNum + i].Name, Info.Name);
FinalTable[OpNum + i] = Info;
#ifndef NDEBUG
++Total;
@@ -50,7 +51,7 @@ static inline void GenerateTable(X86InstInfo *FinalTable, U16U8InfoStruct const
auto OpNum = Op.first;
X86InstInfo const &Info = Op.Info;
for (uint32_t i = 0; i < Op.second; ++i) {
LogMan::Throw::A(FinalTable[OpNum + i].Type == TYPE_UNKNOWN, "Duplicate Entry %s->%s", FinalTable[OpNum + i].Name, Info.Name);
LOGMAN_THROW_A(FinalTable[OpNum + i].Type == TYPE_UNKNOWN, "Duplicate Entry %s->%s", FinalTable[OpNum + i].Name, Info.Name);
FinalTable[OpNum + i] = Info;
#ifndef NDEBUG
++Total;
@@ -67,7 +68,7 @@ static inline void GenerateTableWithCopy(X86InstInfo *FinalTable, U8U8InfoStruct
auto OpNum = Op.first;
X86InstInfo const &Info = Op.Info;
for (uint32_t i = 0; i < Op.second; ++i) {
LogMan::Throw::A(FinalTable[OpNum + i].Type == TYPE_UNKNOWN, "Duplicate Entry %s->%s", FinalTable[OpNum + i].Name, Info.Name);
LOGMAN_THROW_A(FinalTable[OpNum + i].Type == TYPE_UNKNOWN, "Duplicate Entry %s->%s", FinalTable[OpNum + i].Name, Info.Name);
if (Info.Type == TYPE_COPY_OTHER) {
FinalTable[OpNum + i] = OtherLocal[OpNum + i];
}
@@ -89,7 +90,7 @@ static inline void GenerateX87Table(X86InstInfo *FinalTable, U16U8InfoStruct con
auto OpNum = Op.first;
X86InstInfo const &Info = Op.Info;
for (uint32_t i = 0; i < Op.second; ++i) {
LogMan::Throw::A(FinalTable[OpNum + i].Type == TYPE_UNKNOWN, "Duplicate Entry %s->%s", FinalTable[OpNum + i].Name, Info.Name);
LOGMAN_THROW_A(FinalTable[OpNum + i].Type == TYPE_UNKNOWN, "Duplicate Entry %s->%s", FinalTable[OpNum + i].Name, Info.Name);
if ((OpNum & 0b11'000'000) == 0b11'000'000) {
// If the mod field is 0b11 then it is a regular op
FinalTable[OpNum + i] = Info;
@@ -97,7 +98,7 @@ static inline void GenerateX87Table(X86InstInfo *FinalTable, U16U8InfoStruct con
else {
// If the mod field is !0b11 then this instruction is duplicated through the whole mod [0b00, 0b10] range
// and the modrm.rm space because that is used part of the instruction encoding
LogMan::Throw::A((OpNum & 0b11'000'000) == 0, "Only support mod field of zero in this path");
LOGMAN_THROW_A((OpNum & 0b11'000'000) == 0, "Only support mod field of zero in this path");
for (uint16_t mod = 0b00'000'000; mod < 0b11'000'000; mod += 0b01'000'000) {
for (uint16_t rm = 0b000; rm < 0b1'000; ++rm) {
FinalTable[(OpNum | mod | rm) + i] = Info;
@@ -113,5 +114,7 @@ static inline void GenerateX87Table(X86InstInfo *FinalTable, U16U8InfoStruct con
}
};
void InitializeInfoTables(Context::OperatingMode Mode);
}
+197 -101
View File
@@ -60,6 +60,12 @@
"constexpr static uint8_t ROUND_MODE_TOWARDS_ZERO = 3",
"constexpr static uint8_t ROUND_MODE_FLUSH_TO_ZERO = 1 << 2",
"static constexpr FEXCore::IR::RoundType Round_Nearest {ROUND_MODE_NEAREST}",
"static constexpr FEXCore::IR::RoundType Round_Negative_Infinity {ROUND_MODE_NEGATIVE_INFINITY}",
"static constexpr FEXCore::IR::RoundType Round_Positive_Infinity {ROUND_MODE_POSITIVE_INFINITY}",
"static constexpr FEXCore::IR::RoundType Round_Towards_Zero {ROUND_MODE_TOWARDS_ZERO} /* Truncate */",
"static constexpr FEXCore::IR::RoundType Round_Host {ROUND_MODE_TOWARDS_ZERO + 1}",
"constexpr static FEXCore::IR::MemOffsetType MEM_OFFSET_SXTX {0};",
"constexpr static FEXCore::IR::MemOffsetType MEM_OFFSET_UXTW {1};",
"constexpr static FEXCore::IR::MemOffsetType MEM_OFFSET_SXTW {2};"
@@ -79,7 +85,6 @@
"Blocks"
],
"Args": [
"uint64_t", "Entry",
"uint32_t", "BlockCount"
]
},
@@ -793,6 +798,17 @@
]
},
"CacheLineClear": {
"Desc": ["Does a 64 byte cacheline clear at the address specified"
],
"HasSideEffects": true,
"OpClass": "Memory",
"SSAArgs": "1",
"SSANames": [
"Addr"
]
},
"Add": {
"Desc": [ "Integer Add",
"Will truncate to 64 or 32bits"
@@ -1446,6 +1462,24 @@
]
},
"AtomicFetchNeg": {
"HasSideEffects": true,
"Desc": ["Atomic integer fetch and two's complement negate",
"Dest is the value prior to operating on the value in memory"
],
"OpClass": "Atomic",
"HasDest": true,
"DestClass": "GPR",
"DestSize": "Size",
"SSAArgs": "1",
"SSANames": [
"Addr"
],
"Args": [
"uint8_t", "Size"
]
},
"VExtractToGPR": {
"Desc": ["Extracts an element from a vector and places it in a GPR",
"The element that is extracted from the vector is zero extended to the GPR size"
@@ -1467,24 +1501,6 @@
]
},
"Float_ToGPR_U": {
"Desc": ["Moves the scalar element to a GPR with conversion",
"Converts the 32bit or 64bit float to an unsigned integer",
"Rounding mode determined by host flag's rounding mode"
],
"OpClass": "ALU",
"HasDest": true,
"DestClass": "GPR",
"DestSize": "ElementSize",
"SSAArgs": "1",
"SSANames": [
"Scalar"
],
"Args": [
"uint8_t", "ElementSize"
]
},
"Float_ToGPR_S": {
"Desc": ["Moves the scalar element to a GPR with conversion",
"Converts the 32bit or 64bit float to an signed integer",
@@ -1493,30 +1509,16 @@
"OpClass": "ALU",
"HasDest": true,
"DestClass": "GPR",
"DestSize": "ElementSize",
"DestSize": "DestElementSize",
"SSAArgs": "1",
"SSANames": [
"Scalar"
],
"Args": [
"uint8_t", "ElementSize"
]
},
"Float_ToGPR_ZU": {
"Desc": ["Moves the scalar element to a GPR with conversion",
"Converts the 32bit or 64bit float to an unsigned integer rounding towards zero (Truncating)"
],
"OpClass": "ALU",
"HasDest": true,
"DestClass": "GPR",
"DestSize": "ElementSize",
"SSAArgs": "1",
"SSANames": [
"Scalar"
"HelperArgs": [
"uint8_t", "DestElementSize"
],
"Args": [
"uint8_t", "ElementSize"
"uint8_t", "SrcElementSize"
]
},
@@ -1527,13 +1529,16 @@
"OpClass": "ALU",
"HasDest": true,
"DestClass": "GPR",
"DestSize": "ElementSize",
"DestSize": "DestElementSize",
"SSAArgs": "1",
"SSANames": [
"Scalar"
],
"HelperArgs": [
"uint8_t", "DestElementSize"
],
"Args": [
"uint8_t", "ElementSize"
"uint8_t", "SrcElementSize"
]
},
@@ -1563,7 +1568,6 @@
"Depending on backend, may only support GPR printing"
],
"OpClass": "Misc",
"DestSize": "GetOpSize(ssa0)",
"SSAArgs": "1",
"SSANames": [
"Value"
@@ -1655,6 +1659,23 @@
]
},
"VBic": {
"OpClass": "Vector",
"HasDest": true,
"DestClass": "FPR",
"DestSize": "RegisterSize",
"NumElements": "RegisterSize / ElementSize",
"SSAArgs": "2",
"SSANames": [
"Vector1",
"Vector2"
],
"HelperArgs": [
"uint8_t", "RegisterSize",
"uint8_t", "ElementSize"
]
},
"VOr": {
"OpClass": "Vector",
"HasDest": true,
@@ -1800,8 +1821,8 @@
"NumElements": "RegisterSize / ElementSize",
"SSAArgs": "2",
"SSANames": [
"Vector1",
"Vector2"
"VectorLower",
"VectorUpper"
],
"HelperArgs": [
"uint8_t", "RegisterSize",
@@ -1828,6 +1849,25 @@
]
},
"VUMinV": {
"OpClass": "Vector",
"Desc": ["Does a horizontal vector unsigned minimum of elements across the source vector",
"Result is a zero extended scalar"
],
"HasDest": true,
"DestClass": "FPR",
"DestSize": "RegisterSize",
"NumElements": "RegisterSize / ElementSize",
"SSAArgs": "1",
"SSANames": [
"Vector"
],
"HelperArgs": [
"uint8_t", "RegisterSize",
"uint8_t", "ElementSize"
]
},
"VURAvg": {
"OpClass": "Vector",
"Desc": ["Does an unsigned rounded average", "dst_elem = (src1_elem + src2_elem + 1) >> 1"],
@@ -1864,6 +1904,24 @@
]
},
"VPopcount": {
"OpClass": "Vector",
"Desc": ["Does a popcount for each element of the register"
],
"HasDest": true,
"DestClass": "FPR",
"DestSize": "RegisterSize",
"NumElements": "RegisterSize / ElementSize",
"SSAArgs": "1",
"SSANames": [
"Vector"
],
"HelperArgs": [
"uint8_t", "RegisterSize",
"uint8_t", "ElementSize"
]
},
"VFAdd": {
"OpClass": "Vector",
"HasDest": true,
@@ -1890,8 +1948,8 @@
"NumElements": "RegisterSize / ElementSize",
"SSAArgs": "2",
"SSANames": [
"Vector1",
"Vector2"
"VectorLow",
"VectorHigh"
],
"HelperArgs": [
"uint8_t", "RegisterSize",
@@ -2183,6 +2241,40 @@
]
},
"VUnZip": {
"OpClass": "Vector",
"HasDest": true,
"DestClass": "FPR",
"DestSize": "RegisterSize",
"NumElements": "RegisterSize / ElementSize",
"SSAArgs": "2",
"SSANames": [
"Lower",
"Upper"
],
"HelperArgs": [
"uint8_t", "RegisterSize",
"uint8_t", "ElementSize"
]
},
"VUnZip2": {
"OpClass": "Vector",
"HasDest": true,
"DestClass": "FPR",
"DestSize": "RegisterSize",
"NumElements": "RegisterSize / ElementSize",
"SSAArgs": "2",
"SSANames": [
"Lower",
"Upper"
],
"HelperArgs": [
"uint8_t", "RegisterSize",
"uint8_t", "ElementSize"
]
},
"VBSL": {
"Desc": ["Does a vector bitwise select.",
"If the bit in the field is 1 then the corresponding bit is pulled from VectorTrue",
@@ -2574,6 +2666,26 @@
]
},
"VDupElement": {
"Desc": ["Duplicates one element from the source register across the whole register"],
"OpClass": "Vector",
"HasDest": true,
"DestClass": "FPR",
"DestSize": "RegisterSize",
"NumElements": "RegisterSize / ElementSize",
"SSAArgs": "1",
"SSANames": [
"Vector"
],
"HelperArgs": [
"uint8_t", "RegisterSize",
"uint8_t", "ElementSize"
],
"Args": [
"uint8_t", "Index"
]
},
"VExtr": {
"Desc": ["Concats two vector registers together and extracts a full width register from the element index",
"Index is an element index. So it is offset by ElementSize argument",
@@ -2926,27 +3038,6 @@
]
},
"Float_FromGPR_U": {
"OpClass": "Conv",
"Desc": ["Scalar op: Converts unsigned GPR to Scalar float",
"Zeroes the upper bits of the vector register"
],
"HasDest": true,
"DestClass": "FPR",
"DestSize": "DstElementSize",
"NumElements": "1",
"SSAArgs": "1",
"SSANames": [
"GPR"
],
"HelperArgs": [
"uint8_t", "DstElementSize"
],
"Args": [
"uint8_t", "SrcElementSize"
]
},
"Float_FromGPR_S": {
"OpClass": "Conv",
"Desc": ["Scalar op: Converts signed GPR to Scalar float",
@@ -3023,25 +3114,6 @@
]
},
"Vector_FToU": {
"OpClass": "Conv",
"Desc": ["Vector op: Converts float to unsigned integer",
"Rounding mode determined by host rounding mode"
],
"HasDest": true,
"DestClass": "FPR",
"DestSize": "RegisterSize",
"NumElements": "RegisterSize / ElementSize",
"SSAArgs": "1",
"SSANames": [
"Vector"
],
"HelperArgs": [
"uint8_t", "RegisterSize",
"uint8_t", "ElementSize"
]
},
"Vector_FToS": {
"OpClass": "Conv",
"Desc": ["Vector op: Converts float to signed integer, rounding towards zero",
@@ -3061,23 +3133,6 @@
]
},
"Vector_FToZU": {
"OpClass": "Conv",
"Desc": "Vector op: Converts float to unsigned integer, rounding towards zero",
"HasDest": true,
"DestClass": "FPR",
"DestSize": "RegisterSize",
"NumElements": "RegisterSize / ElementSize",
"SSAArgs": "1",
"SSANames": [
"Vector"
],
"HelperArgs": [
"uint8_t", "RegisterSize",
"uint8_t", "ElementSize"
]
},
"Vector_FToZS": {
"OpClass": "Conv",
"Desc": "Vector op: Converts float to signed integer, rounding towards zero",
@@ -3115,6 +3170,28 @@
]
},
"Vector_FToI": {
"OpClass": "Conv",
"Desc": ["Vector op: Rounds float to integral",
"Rounding mode determined by argument"
],
"HasDest": true,
"DestClass": "FPR",
"DestSize": "RegisterSize",
"NumElements": "RegisterSize / ElementSize",
"SSAArgs": "1",
"SSANames": [
"Vector"
],
"HelperArgs": [
"uint8_t", "RegisterSize",
"uint8_t", "ElementSize"
],
"Args":[
"FEXCore::IR::RoundType", "Round"
]
},
"VUMul": {
"OpClass": "Vector",
"HasDest": true,
@@ -3222,6 +3299,25 @@
]
},
"VUABDL": {
"OpClass": "Vector",
"Desc": ["Unsigned Absolute Difference Long"
],
"HasDest": true,
"DestClass": "FPR",
"DestSize": "RegisterSize",
"NumElements": "RegisterSize / (ElementSize << 1)",
"SSAArgs": "2",
"SSANames": [
"Vector1",
"Vector2"
],
"HelperArgs": [
"uint8_t", "RegisterSize",
"uint8_t", "ElementSize"
]
},
"VTBL1": {
"Desc": ["Does a vector table lookup from one register in to the destination",
"Lookup is byte sized per byte element.",
+13 -3
View File
@@ -37,7 +37,7 @@ static void PrintArg(std::stringstream *out, [[maybe_unused]] IRListView const*
}
static void PrintArg(std::stringstream *out, [[maybe_unused]] IRListView const* IR, CondClassType Arg) {
std::array<std::string, 22> CondNames = {
static constexpr std::array<std::string_view, 22> CondNames = {
"EQ",
"NEQ",
"UGE",
@@ -66,7 +66,7 @@ static void PrintArg(std::stringstream *out, [[maybe_unused]] IRListView const*
}
static void PrintArg(std::stringstream *out, [[maybe_unused]] IRListView const* IR, MemOffsetType Arg) {
std::array<std::string, 3> Names = {
static constexpr std::array<std::string_view, 3> Names = {
"SXTX",
"UXTW",
"SXTW",
@@ -154,6 +154,17 @@ static void PrintArg(std::stringstream *out, [[maybe_unused]] IRListView const*
}
}
static void PrintArg(std::stringstream *out, [[maybe_unused]] IRListView const* IR, FEXCore::IR::RoundType Arg) {
switch (Arg) {
case FEXCore::IR::Round_Nearest: *out << "Nearest"; break;
case FEXCore::IR::Round_Negative_Infinity: *out << "-Inf"; break;
case FEXCore::IR::Round_Positive_Infinity: *out << "+Inf"; break;
case FEXCore::IR::Round_Towards_Zero: *out << "Towards Zero"; break;
case FEXCore::IR::Round_Host: *out << "Host"; break;
default: *out << "<Unknown Round Type>"; break;
}
}
void Dump(std::stringstream *out, IRListView const* IR, IR::RegisterAllocationData *RAData) {
auto HeaderOp = IR->GetHeader();
@@ -167,7 +178,6 @@ void Dump(std::stringstream *out, IRListView const* IR, IR::RegisterAllocationDa
++CurrentIndent;
AddIndent();
*out << "(%ssa0) " << "IRHeader ";
*out << "#0x" << std::hex << HeaderOp->Entry << ", ";
*out << "%ssa" << HeaderOp->Blocks.ID() << ", ";
*out << "#" << std::dec << HeaderOp->BlockCount << std::endl;
+16 -17
View File
@@ -9,18 +9,17 @@ $end_info$
namespace FEXCore::IR {
void IREmitter::ResetWorkingList() {
Data.Reset();
ListData.Reset();
DualListData.Reset();
CodeBlocks.clear();
CurrentWriteCursor = nullptr;
// This is necessary since we do "null" pointer checks
InvalidNode = reinterpret_cast<OrderedNode*>(ListData.Allocate(sizeof(OrderedNode)));
InvalidNode = reinterpret_cast<OrderedNode*>(DualListData.ListAllocate(sizeof(OrderedNode)));
memset(InvalidNode, 0, sizeof(OrderedNode));
CurrentCodeBlock = nullptr;
}
void IREmitter::ReplaceAllUsesWithRange(OrderedNode *Node, OrderedNode *NewNode, AllNodesIterator After, AllNodesIterator End) {
uintptr_t ListBegin = ListData.Begin();
uintptr_t ListBegin = DualListData.ListBegin();
auto NodeId = Node->Wrapped(ListBegin).ID();
while (After != End) {
@@ -45,8 +44,8 @@ void IREmitter::ReplaceAllUsesWithRange(OrderedNode *Node, OrderedNode *NewNode,
}
void IREmitter::ReplaceNodeArgument(OrderedNode *Node, uint8_t Arg, OrderedNode *NewArg) {
uintptr_t ListBegin = ListData.Begin();
uintptr_t DataBegin = Data.Begin();
uintptr_t ListBegin = DualListData.ListBegin();
uintptr_t DataBegin = DualListData.DataBegin();
FEXCore::IR::IROp_Header *IROp = Node->Op(DataBegin);
OrderedNodeWrapper OldArgWrapper = IROp->Args[Arg];
@@ -57,8 +56,8 @@ void IREmitter::ReplaceNodeArgument(OrderedNode *Node, uint8_t Arg, OrderedNode
}
void IREmitter::RemoveArgUses(OrderedNode *Node) {
uintptr_t ListBegin = ListData.Begin();
uintptr_t DataBegin = Data.Begin();
uintptr_t ListBegin = DualListData.ListBegin();
uintptr_t DataBegin = DualListData.DataBegin();
FEXCore::IR::IROp_Header *IROp = Node->Op(DataBegin);
@@ -72,7 +71,7 @@ void IREmitter::RemoveArgUses(OrderedNode *Node) {
void IREmitter::Remove(OrderedNode *Node) {
RemoveArgUses(Node);
Node->Unlink(ListData.Begin());
Node->Unlink(DualListData.ListBegin());
}
IREmitter::IRPair<IROp_CodeBlock> IREmitter::CreateNewCodeBlockAfter(OrderedNode* insertAfter) {
@@ -83,14 +82,14 @@ IREmitter::IRPair<IROp_CodeBlock> IREmitter::CreateNewCodeBlockAfter(OrderedNode
if (insertAfter) {
LinkCodeBlocks(insertAfter, CodeNode);
} else {
LogMan::Throw::A(CurrentCodeBlock != nullptr, "CurrentCodeBlock must not be null here");
LOGMAN_THROW_A_FMT(CurrentCodeBlock != nullptr, "CurrentCodeBlock must not be null here");
// Find last block
auto LastBlock = CurrentCodeBlock;
while (LastBlock->Header.Next.GetNode(ListData.Begin()) != InvalidNode)
LastBlock = LastBlock->Header.Next.GetNode(ListData.Begin());
while (LastBlock->Header.Next.GetNode(DualListData.ListBegin()) != InvalidNode)
LastBlock = LastBlock->Header.Next.GetNode(DualListData.ListBegin());
// Append it after the last block
LinkCodeBlocks(LastBlock, CodeNode);
}
@@ -102,12 +101,12 @@ IREmitter::IRPair<IROp_CodeBlock> IREmitter::CreateNewCodeBlockAfter(OrderedNode
void IREmitter::SetCurrentCodeBlock(OrderedNode *Node) {
CurrentCodeBlock = Node;
LogMan::Throw::A(Node->Op(Data.Begin())->Op == OP_CODEBLOCK, "Node wasn't codeblock. It was '%s'", std::string(IR::GetName(Node->Op(Data.Begin())->Op)).c_str());
SetWriteCursor(Node->Op(Data.Begin())->CW<IROp_CodeBlock>()->Begin.GetNode(ListData.Begin()));
LOGMAN_THROW_A_FMT(Node->Op(DualListData.DataBegin())->Op == OP_CODEBLOCK, "Node wasn't codeblock. It was '{}'", IR::GetName(Node->Op(DualListData.DataBegin())->Op));
SetWriteCursor(Node->Op(DualListData.DataBegin())->CW<IROp_CodeBlock>()->Begin.GetNode(DualListData.ListBegin()));
}
void IREmitter::ReplaceWithConstant(OrderedNode *Node, uint64_t Value) {
auto Header = Node->Op(Data.Begin());
auto Header = Node->Op(DualListData.DataBegin());
if (IRSizes[Header->Op] >= sizeof(IROp_Constant)) {
// Unlink any arguments the node currently has
+113 -117
View File
@@ -66,7 +66,8 @@ std::string DecodeErrorToString(DecodeFailure Failure) {
case DecodeFailure::DECODE_INVALID_CONDFLAG: return "Invalid Conditional name";
case DecodeFailure::DECODE_INVALID_MEMOFFSETTYPE: return "Invalid Memory Offset Type";
case DecodeFailure::DECODE_INVALID_FENCETYPE: return "Invalid Fence Type";
};
}
return "Unknown Error";
}
std::unordered_map<std::string_view, FEXCore::IR::IROps> NameToOpMap;
@@ -74,22 +75,22 @@ std::unordered_map<std::string_view, FEXCore::IR::IROps> NameToOpMap;
class IRParser: public FEXCore::IR::IREmitter {
public:
template<typename Type>
std::pair<DecodeFailure, Type> DecodeValue(std::string &Arg) {
std::pair<DecodeFailure, Type> DecodeValue(const std::string &Arg) {
return {DecodeFailure::DECODE_UNKNOWN_TYPE, {}};
}
template<>
std::pair<DecodeFailure, uint8_t> DecodeValue(std::string &Arg) {
std::pair<DecodeFailure, uint8_t> DecodeValue(const std::string &Arg) {
if (Arg.at(0) != '#') return {DecodeFailure::DECODE_INVALIDCHAR, 0};
uint8_t Result = strtoul(&Arg.at(1), nullptr, 0);
if (errno == ERANGE) return {DecodeFailure::DECODE_INVALIDRANGE, 0};
return {DecodeFailure::DECODE_OKAY, Result};
}
}
template<>
std::pair<DecodeFailure, bool> DecodeValue(std::string &Arg) {
std::pair<DecodeFailure, bool> DecodeValue(const std::string &Arg) {
if (Arg.at(0) != '#') return {DecodeFailure::DECODE_INVALIDCHAR, 0};
uint8_t Result = strtoul(&Arg.at(1), nullptr, 0);
@@ -98,7 +99,7 @@ class IRParser: public FEXCore::IR::IREmitter {
}
template<>
std::pair<DecodeFailure, uint16_t> DecodeValue(std::string &Arg) {
std::pair<DecodeFailure, uint16_t> DecodeValue(const std::string &Arg) {
if (Arg.at(0) != '#') return {DecodeFailure::DECODE_INVALIDCHAR, 0};
uint16_t Result = strtoul(&Arg.at(1), nullptr, 0);
@@ -107,7 +108,7 @@ class IRParser: public FEXCore::IR::IREmitter {
}
template<>
std::pair<DecodeFailure, uint32_t> DecodeValue(std::string &Arg) {
std::pair<DecodeFailure, uint32_t> DecodeValue(const std::string &Arg) {
if (Arg.at(0) != '#') return {DecodeFailure::DECODE_INVALIDCHAR, 0};
uint32_t Result = strtoul(&Arg.at(1), nullptr, 0);
@@ -116,7 +117,7 @@ class IRParser: public FEXCore::IR::IREmitter {
}
template<>
std::pair<DecodeFailure, uint64_t> DecodeValue(std::string &Arg) {
std::pair<DecodeFailure, uint64_t> DecodeValue(const std::string &Arg) {
if (Arg.at(0) != '#') return {DecodeFailure::DECODE_INVALIDCHAR, 0};
uint64_t Result = strtoull(&Arg.at(1), nullptr, 0);
@@ -125,7 +126,7 @@ class IRParser: public FEXCore::IR::IREmitter {
}
template<>
std::pair<DecodeFailure, int64_t> DecodeValue(std::string &Arg) {
std::pair<DecodeFailure, int64_t> DecodeValue(const std::string &Arg) {
if (Arg.at(0) != '#') return {DecodeFailure::DECODE_INVALIDCHAR, 0};
int64_t Result = (int64_t)strtoull(&Arg.at(1), nullptr, 0);
@@ -134,7 +135,7 @@ class IRParser: public FEXCore::IR::IREmitter {
}
template<>
std::pair<DecodeFailure, IR::SHA256Sum> DecodeValue(std::string &Arg) {
std::pair<DecodeFailure, IR::SHA256Sum> DecodeValue(const std::string &Arg) {
IR::SHA256Sum Result;
if (Arg.at(0) != 's' || Arg.at(1) != 'h' || Arg.at(2) != 'a' || Arg.at(3) != '2' || Arg.at(4) != '5' || Arg.at(5) != '6' || Arg.at(6) != ':')
@@ -165,7 +166,7 @@ class IRParser: public FEXCore::IR::IREmitter {
}
template<>
std::pair<DecodeFailure, FEXCore::IR::RegisterClassType> DecodeValue(std::string &Arg) {
std::pair<DecodeFailure, FEXCore::IR::RegisterClassType> DecodeValue(const std::string &Arg) {
if (Arg == "GPR") {
return {DecodeFailure::DECODE_OKAY, FEXCore::IR::GPRClass};
}
@@ -183,7 +184,7 @@ class IRParser: public FEXCore::IR::IREmitter {
}
template<>
std::pair<DecodeFailure, FEXCore::IR::TypeDefinition> DecodeValue(std::string &Arg) {
std::pair<DecodeFailure, FEXCore::IR::TypeDefinition> DecodeValue(const std::string &Arg) {
uint8_t Size{}, Elements{1};
int NumArgs = sscanf(Arg.c_str(), "i%hhdv%hhd", &Size, &Elements);
@@ -195,8 +196,8 @@ class IRParser: public FEXCore::IR::IREmitter {
}
template<>
std::pair<DecodeFailure, FEXCore::IR::CondClassType> DecodeValue(std::string &Arg) {
std::array<std::string, 22> CondNames = {
std::pair<DecodeFailure, FEXCore::IR::CondClassType> DecodeValue(const std::string &Arg) {
static constexpr std::array<std::string_view, 22> CondNames = {
"EQ",
"NEQ",
"UGE",
@@ -230,8 +231,8 @@ class IRParser: public FEXCore::IR::IREmitter {
}
template<>
std::pair<DecodeFailure, FEXCore::IR::MemOffsetType> DecodeValue(std::string &Arg) {
std::array<std::string, 3> Names = {
std::pair<DecodeFailure, FEXCore::IR::MemOffsetType> DecodeValue(const std::string &Arg) {
static constexpr std::array<std::string_view, 3> Names = {
"SXTX",
"UXTW",
"SXTW",
@@ -246,8 +247,8 @@ class IRParser: public FEXCore::IR::IREmitter {
}
template<>
std::pair<DecodeFailure, FEXCore::IR::FenceType> DecodeValue(std::string &Arg) {
std::array<std::string, 3> Names = {
std::pair<DecodeFailure, FEXCore::IR::FenceType> DecodeValue(const std::string &Arg) {
static constexpr std::array<std::string_view, 3> Names = {
"Loads",
"Stores",
"LoadStores",
@@ -262,23 +263,22 @@ class IRParser: public FEXCore::IR::IREmitter {
}
template<>
std::pair<DecodeFailure, OrderedNode*> DecodeValue(std::string &Arg) {
std::pair<DecodeFailure, OrderedNode*> DecodeValue(const std::string &Arg) {
if (Arg.at(0) != '%') return {DecodeFailure::DECODE_INVALIDCHAR, 0};
// Strip off the type qualifier from the ssa value
size_t ArgEnd = std::string::npos;
std::string SSAName = trim(Arg);
ArgEnd = SSAName.find_first_of(" ");
const size_t ArgEnd = SSAName.find_first_of(' ');
if (ArgEnd != std::string::npos) {
SSAName = SSAName.substr(0, ArgEnd);
}
SSAName = SSAName.substr(0, ArgEnd);
}
// Forward declarations may make this not succed
// Forward declarations may make this not succed
auto Op = SSANameMapper.find(SSAName);
if (Op == SSANameMapper.end()) {
if (Op == SSANameMapper.end()) {
return {DecodeFailure::DECODE_UNKNOWN_SSA, nullptr};
}
}
return {DecodeFailure::DECODE_OKAY, Op->second};
}
@@ -302,21 +302,21 @@ class IRParser: public FEXCore::IR::IREmitter {
IRParser(std::istream *text) {
InitializeStaticTables();
std::string TmpLine;
while (!text->eof()) {
std::getline(*text, TmpLine);
if (text->eof()) {
break;
}
if (text->eof()) {
break;
}
if (text->fail()) {
LogMan::Msg::E("Failed to getline on line: %ld", Lines.size());
LogMan::Msg::EFmt("Failed to getline on line: {}", Lines.size());
return;
}
Lines.emplace_back(TmpLine);
}
ResetWorkingList();
ResetWorkingList();
Loaded = Parse();
}
@@ -327,11 +327,11 @@ class IRParser: public FEXCore::IR::IREmitter {
bool Parse() {
auto CheckPrintError = [&](LineDefinition &Def, DecodeFailure Failure) -> bool {
const auto CheckPrintError = [&](const LineDefinition &Def, DecodeFailure Failure) -> bool {
if (Failure != DecodeFailure::DECODE_OKAY) {
LogMan::Msg::E("Error on Line: %d", Def.LineNumber);
LogMan::Msg::E("%s", Lines[Def.LineNumber].c_str());
LogMan::Msg::E("Value Couldn't be decoded due to %s", DecodeErrorToString(Failure).c_str());
LogMan::Msg::EFmt("Error on Line: {}", Def.LineNumber);
LogMan::Msg::EFmt("{}", Lines[Def.LineNumber]);
LogMan::Msg::EFmt("Value Couldn't be decoded due to {}", DecodeErrorToString(Failure));
return false;
}
@@ -339,13 +339,13 @@ class IRParser: public FEXCore::IR::IREmitter {
};
// String parse every line for our definitions
for (size_t i = 0; i < Lines.size(); ++i) {
std::string Line = Lines[i];
for (size_t i = 0; i < Lines.size(); ++i) {
std::string Line = Lines[i];
LineDefinition Def{};
CurrentDef = &Def;
CurrentDef = &Def;
Def.LineNumber = i;
Line = trim(Line);
Line = trim(Line);
// Skip empty lines
if (Line.empty()) {
@@ -359,35 +359,37 @@ class IRParser: public FEXCore::IR::IREmitter {
}
size_t CurrentPos{};
// Let's see if this node is assigning something first
if (Line[0] == '%') {
// Let's see if this node is assigning something first
if (Line[0] == '%') {
size_t DefinitionEnd = std::string::npos;
if ((DefinitionEnd = Line.find_first_of("=", CurrentPos)) != std::string::npos) {
if ((DefinitionEnd = Line.find_first_of('=', CurrentPos)) != std::string::npos) {
Def.Definition = Line.substr(0, DefinitionEnd);
Def.Definition = trim(Def.Definition);
Def.HasDefinition = true;
CurrentPos = DefinitionEnd + 1; // +1 to ensure we go past then assignment
}
else {
LogMan::Msg::E("Error on Line: %d", i);
LogMan::Msg::E("%s", Lines[i].c_str());
LogMan::Msg::E("SSA declaration without assignment");
LogMan::Msg::EFmt("Error on Line: {}", i);
LogMan::Msg::EFmt("{}", Lines[i]);
LogMan::Msg::EFmt("SSA declaration without assignment");
return false;
}
}
}
// Check if we are pulling in some IR from the IR Printer
// Prints (%ssa%d) at the start of lines without a definition
if (Line[0] == '(') {
size_t DefinitionEnd = std::string::npos;
if ((DefinitionEnd = Line.find_first_of(")", CurrentPos)) != std::string::npos) {
if ((DefinitionEnd = Line.find_first_of(')', CurrentPos)) != std::string::npos) {
size_t SSAEnd = std::string::npos;
if ((SSAEnd = Line.find_last_of(" ", DefinitionEnd)) != std::string::npos) {
if ((SSAEnd = Line.find_last_of(' ', DefinitionEnd)) != std::string::npos) {
std::string Type = Line.substr(SSAEnd + 1, DefinitionEnd - SSAEnd - 1);
Type = trim(Type);
auto DefinitionSize = DecodeValue<FEXCore::IR::TypeDefinition>(Type);
if (!CheckPrintError(Def, DefinitionSize.first)) return false;
if (!CheckPrintError(Def, DefinitionSize.first)) {
return false;
}
Def.Size = DefinitionSize.second;
}
@@ -396,9 +398,9 @@ class IRParser: public FEXCore::IR::IREmitter {
CurrentPos = DefinitionEnd + 1;
}
else {
LogMan::Msg::E("Error on Line: %d", i);
LogMan::Msg::E("%s", Lines[i].c_str());
LogMan::Msg::E("SSA value with numbered SSA provided but no closing parentheses");
LogMan::Msg::EFmt("Error on Line: {}", i);
LogMan::Msg::EFmt("{}", Lines[i]);
LogMan::Msg::EFmt("SSA value with numbered SSA provided but no closing parentheses");
return false;
}
}
@@ -406,7 +408,7 @@ class IRParser: public FEXCore::IR::IREmitter {
if (Def.HasDefinition) {
// Let's check if we have a size declared with this variable
size_t NameEnd = std::string::npos;
if ((NameEnd = Def.Definition.find_first_of(" ")) != std::string::npos) {
if ((NameEnd = Def.Definition.find_first_of(' ')) != std::string::npos) {
std::string Type = Def.Definition.substr(NameEnd + 1);
Type = trim(Type);
Def.Definition = trim(Def.Definition.substr(0, NameEnd));
@@ -417,9 +419,9 @@ class IRParser: public FEXCore::IR::IREmitter {
}
if (Def.Definition == "%Invalid") {
LogMan::Msg::E("Error on Line: %d", i);
LogMan::Msg::E("%s", Lines[i].c_str());
LogMan::Msg::E("Definition tried to define reserved %Invalid ssa node");
LogMan::Msg::EFmt("Error on Line: {}", i);
LogMan::Msg::EFmt("{}", Lines[i]);
LogMan::Msg::EFmt("Definition tried to define reserved %Invalid ssa node");
return false;
}
}
@@ -436,9 +438,9 @@ class IRParser: public FEXCore::IR::IREmitter {
}
else {
if (RemainingLine.empty()) {
LogMan::Msg::E("Error on Line: %d", i);
LogMan::Msg::E("%s", Lines[i].c_str());
LogMan::Msg::E("Line without an IROp?");
LogMan::Msg::EFmt("Error on Line: {}", i);
LogMan::Msg::EFmt("{}", Lines[i]);
LogMan::Msg::EFmt("Line without an IROp?");
return false;
}
@@ -455,12 +457,10 @@ class IRParser: public FEXCore::IR::IREmitter {
}
else {
while (!RemainingLine.empty()) {
size_t ArgEnd = std::string::npos;
ArgEnd = RemainingLine.find_first_of(",");
const size_t ArgEnd = RemainingLine.find(',');
std::string Arg = trim(RemainingLine.substr(0, ArgEnd));
std::string Arg = RemainingLine.substr(0, ArgEnd);
Arg = trim(Arg);
Def.Args.emplace_back(Arg);
Def.Args.emplace_back(std::move(Arg));
RemainingLine.erase(0, ArgEnd+1); // +1 to ensure we go past the ','
if (ArgEnd == std::string::npos)
@@ -469,17 +469,17 @@ class IRParser: public FEXCore::IR::IREmitter {
}
}
Defs.emplace_back(Def);
}
CurrentDef = &Defs.emplace_back(std::move(Def));
}
// Ensure all of the ops are real ops
for(size_t i = 0; i < Defs.size(); ++i) {
auto &Def = Defs[i];
auto Op = NameToOpMap.find(Def.IROp);
if (Op == NameToOpMap.end()) {
LogMan::Msg::E("Error on Line: %d", Def.LineNumber);
LogMan::Msg::E("%s", Lines[Def.LineNumber].c_str());
LogMan::Msg::E("IROp '%s' doesn't exist", Def.IROp.c_str());
LogMan::Msg::EFmt("Error on Line: {}", Def.LineNumber);
LogMan::Msg::EFmt("{}", Lines[Def.LineNumber]);
LogMan::Msg::EFmt("IROp '{}' doesn't exist", Def.IROp);
return false;
}
Def.OpEnum = Op->second;
@@ -489,40 +489,38 @@ class IRParser: public FEXCore::IR::IREmitter {
IRPair<IROp_IRHeader> IRHeader;
{
auto &Def = Defs[0];
CurrentDef = &Def;
CurrentDef = &Def;
if (Def.OpEnum != FEXCore::IR::IROps::OP_IRHEADER) {
LogMan::Msg::E("Error on Line: %d", Def.LineNumber);
LogMan::Msg::E("%s", Lines[Def.LineNumber].c_str());
LogMan::Msg::E("First op needs to be IRHeader. Was '%s'", Def.IROp.c_str());
LogMan::Msg::EFmt("Error on Line: {}", Def.LineNumber);
LogMan::Msg::EFmt("{}", Lines[Def.LineNumber]);
LogMan::Msg::EFmt("First op needs to be IRHeader. Was '{}'", Def.IROp);
return false;
}
auto Entry = DecodeValue<uint64_t>(Def.Args[0]);
auto CodeBlockCount = DecodeValue<uint64_t>(Def.Args[2]);
auto CodeBlockCount = DecodeValue<uint64_t>(Def.Args[1]);
if (!CheckPrintError(Def, Entry.first)) return false;
if (!CheckPrintError(Def, CodeBlockCount.first)) return false;
IRHeader = _IRHeader(InvalidNode, Entry.second, CodeBlockCount.second);
IRHeader = _IRHeader(InvalidNode, CodeBlockCount.second);
}
SetWriteCursor(nullptr); // isolate the header from everything following
// Initialize SSANameMapper with Invalid value
SSANameMapper["%Invalid"] = Invalid();
SSANameMapper.insert_or_assign("%Invalid", Invalid());
// Spin through the blocks and generate basic block ops
for(size_t i = 0; i < Defs.size(); ++i) {
auto &Def = Defs[i];
if (Def.OpEnum == FEXCore::IR::IROps::OP_CODEBLOCK) {
auto CodeBlock = _CodeBlock(InvalidNode, InvalidNode);
SSANameMapper[Def.Definition] = CodeBlock.Node;
SSANameMapper.insert_or_assign(Def.Definition, CodeBlock.Node);
Def.Node = CodeBlock.Node;
if (i == 1) {
// First code block is the entry block
// Link the header to the first block
IRHeader.first->Blocks = CodeBlock.Node->Wrapped(ListData.Begin());
IRHeader.first->Blocks = CodeBlock.Node->Wrapped(DualListData.ListBegin());
}
CodeBlocks.emplace_back(CodeBlock.Node);
}
@@ -534,63 +532,64 @@ class IRParser: public FEXCore::IR::IREmitter {
FEXCore::IR::IROp_CodeBlock *CurrentBlockOp{};
for(size_t i = 1; i < Defs.size(); ++i) {
auto &Def = Defs[i];
CurrentDef = &Def;
CurrentDef = &Def;
switch (Def.OpEnum) {
// Special handled
case FEXCore::IR::IROps::OP_IRHEADER:
LogMan::Msg::E("Error on Line: %d", Def.LineNumber);
LogMan::Msg::E("%s", Lines[Def.LineNumber].c_str());
LogMan::Msg::E("IRHEADER used in the middle of the block!");
LogMan::Msg::EFmt("Error on Line: {}", Def.LineNumber);
LogMan::Msg::EFmt("{}", Lines[Def.LineNumber]);
LogMan::Msg::EFmt("IRHEADER used in the middle of the block!");
return false; // only one OP_IRHEADER allowed per block
case FEXCore::IR::IROps::OP_CODEBLOCK: {
SetWriteCursor(nullptr); // isolate from previous block
if (CurrentBlock != nullptr) {
LogMan::Msg::E("Error on Line: %d", Def.LineNumber);
LogMan::Msg::E("%s", Lines[Def.LineNumber].c_str());
LogMan::Msg::E("CodeBlock being used inside of already existing codeblock!");
LogMan::Msg::EFmt("Error on Line: {}", Def.LineNumber);
LogMan::Msg::EFmt("{}", Lines[Def.LineNumber]);
LogMan::Msg::EFmt("CodeBlock being used inside of already existing codeblock!");
return false;
}
CurrentBlock = Def.Node;
CurrentBlockOp = CurrentBlock->Op(Data.Begin())->CW<FEXCore::IR::IROp_CodeBlock>();
CurrentBlockOp = CurrentBlock->Op(DualListData.DataBegin())->CW<FEXCore::IR::IROp_CodeBlock>();
break;
}
case FEXCore::IR::IROps::OP_BEGINBLOCK: {
if (CurrentBlock == nullptr) {
LogMan::Msg::E("Error on Line: %d", Def.LineNumber);
LogMan::Msg::E("%s", Lines[Def.LineNumber].c_str());
LogMan::Msg::E("EndBlock being used outside of a block!");
LogMan::Msg::EFmt("Error on Line: {}", Def.LineNumber);
LogMan::Msg::EFmt("{}", Lines[Def.LineNumber]);
LogMan::Msg::EFmt("EndBlock being used outside of a block!");
return false;
}
auto Adjust = DecodeValue<OrderedNode*>(Def.Args[0]);
if (!CheckPrintError(Def, Adjust.first)) return false;
if (!CheckPrintError(Def, Adjust.first)) {
return false;
}
Def.Node = _BeginBlock(Adjust.second);
CurrentBlockOp->Begin = Def.Node->Wrapped(ListData.Begin());
CurrentBlockOp->Begin = Def.Node->Wrapped(DualListData.ListBegin());
break;
}
case FEXCore::IR::IROps::OP_ENDBLOCK: {
if (CurrentBlock == nullptr) {
LogMan::Msg::E("Error on Line: %d", Def.LineNumber);
LogMan::Msg::E("%s", Lines[Def.LineNumber].c_str());
LogMan::Msg::E("EndBlock being used outside of a block!");
LogMan::Msg::EFmt("Error on Line: {}", Def.LineNumber);
LogMan::Msg::EFmt("{}", Lines[Def.LineNumber]);
LogMan::Msg::EFmt("EndBlock being used outside of a block!");
return false;
}
auto Adjust = DecodeValue<OrderedNode*>(Def.Args[0]);
if (!CheckPrintError(Def, Adjust.first)) return false;
if (!CheckPrintError(Def, Adjust.first)) {
return false;
}
Def.Node = _EndBlock(Adjust.second);
CurrentBlockOp->Last = Def.Node->Wrapped(ListData.Begin());
CurrentBlockOp->Last = Def.Node->Wrapped(DualListData.ListBegin());
CurrentBlock = nullptr;
CurrentBlockOp = nullptr;
@@ -599,25 +598,23 @@ class IRParser: public FEXCore::IR::IREmitter {
}
case FEXCore::IR::IROps::OP_DUMMY: {
LogMan::Msg::E("Error on Line: %d", Def.LineNumber);
LogMan::Msg::E("%s", Lines[Def.LineNumber].c_str());
LogMan::Msg::E("Dummy op must not be used");
LogMan::Msg::EFmt("Error on Line: {}", Def.LineNumber);
LogMan::Msg::EFmt("{}", Lines[Def.LineNumber]);
LogMan::Msg::EFmt("Dummy op must not be used");
break;
}
#define IROP_PARSER_SWITCH_HELPERS
#include <FEXCore/IR/IRDefines.inc>
default: {
LogMan::Msg::E("Error on Line: %d", Def.LineNumber);
LogMan::Msg::E("%s", Lines[Def.LineNumber].c_str());
LogMan::Msg::E("Unhandled Op enum '%s' in parser", Def.IROp.c_str());
LogMan::Msg::EFmt("Error on Line: {}", Def.LineNumber);
LogMan::Msg::EFmt("{}", Lines[Def.LineNumber]);
LogMan::Msg::EFmt("Unhandled Op enum '{}' in parser", Def.IROp);
return false;
break;
}
}
if (Def.HasDefinition) {
auto IROp = Def.Node->Op(Data.Begin());
auto IROp = Def.Node->Op(DualListData.DataBegin());
if (Def.Size.Elements()) {
IROp->Size = Def.Size.Bytes() * Def.Size.Elements();
IROp->ElementSize = Def.Size.Bytes();
@@ -626,7 +623,7 @@ class IRParser: public FEXCore::IR::IREmitter {
IROp->Size = Def.Size.Bytes();
IROp->ElementSize = 0;
}
SSANameMapper[Def.Definition] = Def.Node;
SSANameMapper.insert_or_assign(Def.Definition, Def.Node);
}
}
@@ -634,11 +631,11 @@ class IRParser: public FEXCore::IR::IREmitter {
}
void InitializeStaticTables() {
if (NameToOpMap.size() == 0) {
if (NameToOpMap.empty()) {
for (FEXCore::IR::IROps Op = FEXCore::IR::IROps::OP_DUMMY;
Op <= FEXCore::IR::IROps::OP_LAST;
Op = static_cast<FEXCore::IR::IROps>(static_cast<uint32_t>(Op) + 1)) {
NameToOpMap[FEXCore::IR::GetName(Op)] = Op;
NameToOpMap.insert_or_assign(FEXCore::IR::GetName(Op), Op);
}
}
}
@@ -646,13 +643,12 @@ class IRParser: public FEXCore::IR::IREmitter {
} // anon namespace
IREmitter* Parse(std::istream *in) {
auto parser = new IRParser(in);
std::unique_ptr<IREmitter> Parse(std::istream *in) {
auto parser = std::make_unique<IRParser>(in);
if (parser->Loaded) {
return parser;
} else {
delete parser;
return nullptr;
}
}
+9 -4
View File
@@ -19,6 +19,13 @@ void PassManager::AddDefaultPasses(bool InlineConstants, bool StaticRegisterAllo
if (!DisablePasses()) {
InsertPass(CreateContextLoadStoreElimination());
if (Is64BitMode()) {
// This needs to run after RCLSE
// This only matters for 64-bit code since these instructions don't exist in 32-bit
InsertPass(CreateLongDivideEliminationPass());
}
InsertPass(CreateDeadStoreElimination());
InsertPass(CreatePassDeadCodeElimination());
InsertPass(CreateConstProp(InlineConstants));
@@ -38,10 +45,9 @@ void PassManager::AddDefaultPasses(bool InlineConstants, bool StaticRegisterAllo
InsertPass(CreateStaticRegisterAllocationPass());
}
CompactionPass = CreateIRCompaction();
// If the IR is compacted post-RA then the node indexing gets messed up and the backend isn't able to find the register assigned to a node
// Compact before IR, don't worry about RA generating spills/fills
InsertPass(CompactionPass);
CompactionPass = InsertPass(CreateIRCompaction());
}
void PassManager::AddDefaultValidationPasses() {
@@ -53,8 +59,7 @@ void PassManager::AddDefaultValidationPasses() {
}
void PassManager::InsertRegisterAllocationPass(bool OptimizeSRA) {
RAPass = IR::CreateRegisterAllocationPass(CompactionPass, OptimizeSRA);
InsertPass(RAPass);
RAPass = InsertPass(IR::CreateRegisterAllocationPass(CompactionPass, OptimizeSRA));
}
bool PassManager::Run(IREmitter *IREmit) {
+9 -6
View File
@@ -6,6 +6,7 @@ $end_info$
#pragma once
#include <FEXCore/Config/Config.h>
#include <FEXCore/IR/IntrusiveIRList.h>
#include <FEXCore/IR/IREmitter.h>
@@ -41,9 +42,9 @@ class PassManager final {
public:
void AddDefaultPasses(bool InlineConstants, bool StaticRegisterAllocation);
void AddDefaultValidationPasses();
void InsertPass(Pass *Pass) {
Pass* InsertPass(std::unique_ptr<Pass> Pass) {
Pass->RegisterPassManager(this);
Passes.emplace_back(Pass);
return Passes.emplace_back(std::move(Pass)).get();
}
void InsertRegisterAllocationPass(bool OptimizeSRA);
@@ -51,7 +52,7 @@ public:
bool Run(IREmitter *IREmit);
void RegisterExitHandler(ShouldExitHandler Handler) {
ExitHandler = Handler;
ExitHandler = std::move(Handler);
}
bool HasRAPass() const {
@@ -72,17 +73,19 @@ protected:
private:
Pass *RAPass{};
FEXCore::IR::Pass *CompactionPass{};
Pass *CompactionPass{};
std::vector<std::unique_ptr<Pass>> Passes;
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
std::vector<std::unique_ptr<Pass>> ValidationPasses;
void InsertValidationPass(Pass *Pass) {
void InsertValidationPass(std::unique_ptr<Pass> Pass) {
Pass->RegisterPassManager(this);
ValidationPasses.emplace_back(Pass);
ValidationPasses.emplace_back(std::move(Pass));
}
#endif
FEX_CONFIG_OPT(Is64BitMode, IS64BIT_MODE);
};
}
+15 -12
View File
@@ -1,24 +1,27 @@
#pragma once
#include <memory>
namespace FEXCore::IR {
class Pass;
class RegisterAllocationPass;
class RegisterAllocationData;
FEXCore::IR::Pass* CreateConstProp(bool InlineConstants);
FEXCore::IR::Pass* CreateContextLoadStoreElimination();
FEXCore::IR::Pass* CreateSyscallOptimization();
FEXCore::IR::Pass* CreateDeadFlagCalculationEliminination();
FEXCore::IR::Pass* CreateDeadStoreElimination();
FEXCore::IR::Pass* CreatePassDeadCodeElimination();
FEXCore::IR::Pass* CreateIRCompaction();
FEXCore::IR::RegisterAllocationPass* CreateRegisterAllocationPass(FEXCore::IR::Pass* CompactionPass, bool OptimizeSRA);
FEXCore::IR::Pass* CreateStaticRegisterAllocationPass();
std::unique_ptr<FEXCore::IR::Pass> CreateConstProp(bool InlineConstants);
std::unique_ptr<FEXCore::IR::Pass> CreateContextLoadStoreElimination();
std::unique_ptr<FEXCore::IR::Pass> CreateSyscallOptimization();
std::unique_ptr<FEXCore::IR::Pass> CreateDeadFlagCalculationEliminination();
std::unique_ptr<FEXCore::IR::Pass> CreateDeadStoreElimination();
std::unique_ptr<FEXCore::IR::Pass> CreatePassDeadCodeElimination();
std::unique_ptr<FEXCore::IR::Pass> CreateIRCompaction();
std::unique_ptr<FEXCore::IR::RegisterAllocationPass> CreateRegisterAllocationPass(FEXCore::IR::Pass* CompactionPass, bool OptimizeSRA);
std::unique_ptr<FEXCore::IR::Pass> CreateStaticRegisterAllocationPass();
std::unique_ptr<FEXCore::IR::Pass> CreateLongDivideEliminationPass();
namespace Validation {
FEXCore::IR::Pass* CreateIRValidation();
FEXCore::IR::Pass* CreatePhiValidation();
FEXCore::IR::Pass* CreateValueDominanceValidation();
std::unique_ptr<FEXCore::IR::Pass> CreateIRValidation();
std::unique_ptr<FEXCore::IR::Pass> CreatePhiValidation();
std::unique_ptr<FEXCore::IR::Pass> CreateValueDominanceValidation();
}
}
+375 -318
View File
@@ -19,15 +19,6 @@ $end_info$
namespace FEXCore::IR {
class ConstProp final : public FEXCore::IR::Pass {
std::unordered_map<uint64_t, OrderedNode*> ConstPool;
std::map<OrderedNode*, uint64_t> AddressgenConsts;
public:
bool Run(IREmitter *IREmit) override;
bool InlineConstants;
ConstProp(bool DoInlineConstants) : InlineConstants(DoInlineConstants) { }
};
template<typename T>
uint64_t getMask(T Op) {
uint64_t NumBits = Op->Header.Size * 8;
@@ -69,8 +60,7 @@ static bool IsImmMemory(uint64_t imm, uint8_t AccessSize) {
}
}
std::tuple<MemOffsetType, uint8_t, OrderedNode*, OrderedNode*> MemExtendedAddressing(IREmitter *IREmit, uint8_t AccessSize, IROp_Header* AddressHeader) {
static std::tuple<MemOffsetType, uint8_t, OrderedNode*, OrderedNode*> MemExtendedAddressing(IREmitter *IREmit, uint8_t AccessSize, IROp_Header* AddressHeader) {
auto Src0Header = IREmit->GetOpHeader(AddressHeader->Args[0]);
if (Src0Header->Size == 8) {
//Try to optimize: Base + MUL(Offset, Scale)
@@ -124,7 +114,7 @@ std::tuple<MemOffsetType, uint8_t, OrderedNode*, OrderedNode*> MemExtendedAddres
return { MEM_OFFSET_SXTX, 1, IREmit->UnwrapNode(AddressHeader->Args[0]), IREmit->UnwrapNode(AddressHeader->Args[1]) };
}
OrderedNodeWrapper RemoveUselessMasking(IREmitter *IREmit, OrderedNodeWrapper src, uint64_t mask) {
static OrderedNodeWrapper RemoveUselessMasking(IREmitter *IREmit, OrderedNodeWrapper src, uint64_t mask) {
#if 1 // HOTFIX: We need to clear up the meaning of opsize and dest size. See #594
return src;
#else
@@ -151,7 +141,7 @@ OrderedNodeWrapper RemoveUselessMasking(IREmitter *IREmit, OrderedNodeWrapper sr
#endif
}
bool IsBfeAlreadyDone(IREmitter *IREmit, OrderedNodeWrapper src, uint64_t Width) {
static bool IsBfeAlreadyDone(IREmitter *IREmit, OrderedNodeWrapper src, uint64_t Width) {
auto IROp = IREmit->GetOpHeader(src);
if (IROp->Op == OP_BFE) {
auto Op = IROp->C<IR::IROp_Bfe>();
@@ -162,31 +152,55 @@ bool IsBfeAlreadyDone(IREmitter *IREmit, OrderedNodeWrapper src, uint64_t Width)
return false;
}
bool ConstProp::Run(IREmitter *IREmit) {
class ConstProp final : public FEXCore::IR::Pass {
public:
explicit ConstProp(bool DoInlineConstants) : InlineConstants(DoInlineConstants) { }
bool Run(IREmitter *IREmit) override;
bool InlineConstants;
private:
bool HandleConstantPools(IREmitter *IREmit, const IRListView& CurrentIR);
void CodeMotionAroundSelects(IREmitter *IREmit, const IRListView& CurrentIR);
void FCMPOptimization(IREmitter *IREmit, const IRListView& CurrentIR);
void LoadMemStoreMemImmediatePooling(IREmitter *IREmit, const IRListView& CurrentIR);
bool ZextAndMaskingElimination(IREmitter *IREmit, const IRListView& CurrentIR,
OrderedNode* CodeNode, IROp_Header* IROp);
bool ConstantPropagation(IREmitter *IREmit, const IRListView& CurrentIR,
OrderedNode* CodeNode, IROp_Header* IROp);
bool ConstantInlining(IREmitter *IREmit, const IRListView& CurrentIR);
std::unordered_map<uint64_t, OrderedNode*> ConstPool;
std::map<OrderedNode*, uint64_t> AddressgenConsts;
};
bool ConstProp::HandleConstantPools(IREmitter *IREmit, const IRListView& CurrentIR) {
bool Changed = false;
auto CurrentIR = IREmit->ViewIR();
auto OriginalWriteCursor = IREmit->GetWriteCursor();
{
// constants are pooled per block
for (auto [BlockNode, BlockHeader] : CurrentIR.GetBlocks()) {
for (auto [CodeNode, IROp] : CurrentIR.GetCode(BlockNode)) {
if (IROp->Op == OP_CONSTANT) {
auto Op = IROp->C<IR::IROp_Constant>();
if (ConstPool.count(Op->Constant)) {
IREmit->ReplaceAllUsesWith(CodeNode, ConstPool[Op->Constant]);
Changed = true;
} else {
ConstPool[Op->Constant] = CodeNode;
}
// constants are pooled per block
for (auto [BlockNode, BlockHeader] : CurrentIR.GetBlocks()) {
for (auto [CodeNode, IROp] : CurrentIR.GetCode(BlockNode)) {
if (IROp->Op == OP_CONSTANT) {
auto Op = IROp->C<IR::IROp_Constant>();
if (ConstPool.count(Op->Constant)) {
IREmit->ReplaceAllUsesWith(CodeNode, ConstPool[Op->Constant]);
Changed = true;
} else {
ConstPool[Op->Constant] = CodeNode;
}
}
ConstPool.clear();
}
ConstPool.clear();
}
return Changed;
}
// Code motion around selects
// Moves unary ops that depend on a select before the select, if both inputs are constants
// assumes that unary ops without side effects on constants will be constprop'd
void ConstProp::CodeMotionAroundSelects(IREmitter *IREmit, const IRListView& CurrentIR) {
// Code motion around selects
// Moves unary ops that depend on a select before the select, if both inputs are constants
// assumes that unary ops without side effects on constants will be constprop'd
@@ -243,9 +257,9 @@ bool ConstProp::Run(IREmitter *IREmit) {
}
}
}
}
// FCMP optimization
void ConstProp::FCMPOptimization(IREmitter *IREmit, const IRListView& CurrentIR) {
// Make all FCMPs set no flags
for (auto [CodeNode, IROp] : CurrentIR.GetAllCode()) {
if (IROp->Op == OP_FCMP) {
@@ -260,16 +274,17 @@ bool ConstProp::Run(IREmitter *IREmit) {
auto ghf = IROp->CW<IR::IROp_GetHostFlag>();
auto fcmp = IREmit->GetOpHeader(ghf->GPR)->CW<IR::IROp_FCmp>();
LogMan::Throw::A(fcmp->Header.Op == OP_FCMP || fcmp->Header.Op == OP_F80CMP, "Unexpected OP_GETHOSTFLAG source");
LOGMAN_THROW_A(fcmp->Header.Op == OP_FCMP || fcmp->Header.Op == OP_F80CMP, "Unexpected OP_GETHOSTFLAG source");
if(fcmp->Header.Op == OP_FCMP) {
fcmp->Flags |= 1 << ghf->Flag;
}
}
}
}
// LoadMem / StoreMem imm pooling
// If imms are close by, use address gen to generate the values instead of using a new imm
// LoadMem / StoreMem imm pooling
// If imms are close by, use address gen to generate the values instead of using a new imm
void ConstProp::LoadMemStoreMemImmediatePooling(IREmitter *IREmit, const IRListView& CurrentIR) {
for (auto [BlockNode, BlockIROp] : CurrentIR.GetBlocks()) {
for (auto [CodeNode, IROp] : CurrentIR.GetCode(BlockNode)) {
if (IROp->Op == OP_LOADMEM || IROp->Op == OP_STOREMEM) {
@@ -293,152 +308,165 @@ bool ConstProp::Run(IREmitter *IREmit) {
}
AddressgenConsts.clear();
}
}
for (auto [CodeNode, IROp] : CurrentIR.GetAllCode()) {
// zext / masking elimination
switch (IROp->Op) {
// Generic handling
case OP_OR:
case OP_XOR:
case OP_NOT:
case OP_ADD:
case OP_SUB:
case OP_MUL:
case OP_UMUL:
case OP_DIV:
case OP_UDIV:
case OP_LSHR:
case OP_ASHR:
case OP_LSHL:
case OP_ROR: {
for (int i = 0; i < IROp->NumArgs; i++) {
auto newArg = RemoveUselessMasking(IREmit, IROp->Args[i], getMask(IROp));
if (newArg.ID() != IROp->Args[i].ID()) {
IREmit->ReplaceNodeArgument(CodeNode, i, IREmit->UnwrapNode(newArg));
Changed = true;
}
}
break;
}
bool ConstProp::ZextAndMaskingElimination(IREmitter *IREmit, const IRListView& CurrentIR,
OrderedNode* CodeNode, IROp_Header* IROp) {
bool Changed = false;
case OP_AND: {
// if AND's arguments are imms, they are masking
for (int i = 0; i < IROp->NumArgs; i++) {
auto mask = getMask(IROp);
uint64_t imm = 0;
if (IREmit->IsValueConstant(IROp->Args[i^1], &imm))
mask = imm;
auto newArg = RemoveUselessMasking(IREmit, IROp->Args[i], imm);
if (newArg.ID() != IROp->Args[i].ID()) {
IREmit->ReplaceNodeArgument(CodeNode, i, IREmit->UnwrapNode(newArg));
Changed = true;
}
}
break;
}
case OP_BFE: {
auto Op = IROp->C<IR::IROp_Bfe>();
// Is this value already BFE'd?
if (IsBfeAlreadyDone(IREmit, IROp->Args[0], Op->Width)) {
IREmit->ReplaceAllUsesWith(CodeNode, CurrentIR.GetNode(IROp->Args[0]));
//printf("Removed BFE once \n");
break;
}
// Is this value already ZEXT'd?
if (Op->lsb == 0) {
//LoadMem, LoadMemTSO & LoadContext ZExt
auto source = IROp->Args[0];
auto sourceHeader = IREmit->GetOpHeader(source);
if (Op->Width >= (sourceHeader->Size*8) &&
(sourceHeader->Op == OP_LOADMEM || sourceHeader->Op == OP_LOADMEMTSO || sourceHeader->Op == OP_LOADCONTEXT)
) {
//printf("Eliminated needless zext bfe\n");
// Load mem / load ctx zexts, no need to vmem
IREmit->ReplaceAllUsesWith(CodeNode, CurrentIR.GetNode(source));
break;
}
}
// BFE does implicit masking, remove any masks leading to this, if possible
uint64_t imm = 1ULL << (Op->Width-1);
imm = (imm-1) *2 + 1;
imm <<= Op->lsb;
auto newArg = RemoveUselessMasking(IREmit, IROp->Args[0], imm);
if (newArg.ID() != IROp->Args[0].ID()) {
IREmit->ReplaceNodeArgument(CodeNode, 0, IREmit->UnwrapNode(newArg));
switch (IROp->Op) {
// Generic handling
case OP_OR:
case OP_XOR:
case OP_NOT:
case OP_ADD:
case OP_SUB:
case OP_MUL:
case OP_UMUL:
case OP_DIV:
case OP_UDIV:
case OP_LSHR:
case OP_ASHR:
case OP_LSHL:
case OP_ROR: {
for (int i = 0; i < IROp->NumArgs; i++) {
auto newArg = RemoveUselessMasking(IREmit, IROp->Args[i], getMask(IROp));
if (newArg.ID() != IROp->Args[i].ID()) {
IREmit->ReplaceNodeArgument(CodeNode, i, IREmit->UnwrapNode(newArg));
Changed = true;
}
break;
}
break;
}
case OP_SBFE: {
auto Op = IROp->C<IR::IROp_Sbfe>();
case OP_AND: {
// if AND's arguments are imms, they are masking
for (int i = 0; i < IROp->NumArgs; i++) {
auto mask = getMask(IROp);
uint64_t imm = 0;
if (IREmit->IsValueConstant(IROp->Args[i^1], &imm))
mask = imm;
// BFE does implicit masking
uint64_t imm = 1ULL << (Op->Width-1);
imm = (imm-1) *2 + 1;
imm <<= Op->lsb;
auto newArg = RemoveUselessMasking(IREmit, IROp->Args[i], imm);
auto newArg = RemoveUselessMasking(IREmit, IROp->Args[0], imm);
if (newArg.ID() != IROp->Args[0].ID()) {
IREmit->ReplaceNodeArgument(CodeNode, 0, IREmit->UnwrapNode(newArg));
if (newArg.ID() != IROp->Args[i].ID()) {
IREmit->ReplaceNodeArgument(CodeNode, i, IREmit->UnwrapNode(newArg));
Changed = true;
}
}
break;
}
case OP_BFE: {
auto Op = IROp->C<IR::IROp_Bfe>();
// Is this value already BFE'd?
if (IsBfeAlreadyDone(IREmit, IROp->Args[0], Op->Width)) {
IREmit->ReplaceAllUsesWith(CodeNode, CurrentIR.GetNode(IROp->Args[0]));
//printf("Removed BFE once \n");
break;
}
case OP_VFADD:
case OP_VFSUB:
case OP_VFMUL:
case OP_VFDIV:
case OP_FCMP: {
auto flopSize = IROp->Size;
for (int i = 0; i < IROp->NumArgs; i++) {
auto argHeader = IREmit->GetOpHeader(IROp->Args[i]);
if (argHeader->Op == OP_VMOV) {
auto source = argHeader->Args[0];
auto sourceHeader = IREmit->GetOpHeader(source);
if (sourceHeader->Size >= flopSize) {
IREmit->ReplaceNodeArgument(CodeNode, i, IREmit->UnwrapNode(source));
//printf("VMOV bypassed\n");
}
}
}
break;
}
case OP_VMOV: {
// elim from load mem
// Is this value already ZEXT'd?
if (Op->lsb == 0) {
//LoadMem, LoadMemTSO & LoadContext ZExt
auto source = IROp->Args[0];
auto sourceHeader = IREmit->GetOpHeader(source);
if (IROp->Size >= sourceHeader->Size &&
if (Op->Width >= (sourceHeader->Size*8) &&
(sourceHeader->Op == OP_LOADMEM || sourceHeader->Op == OP_LOADMEMTSO || sourceHeader->Op == OP_LOADCONTEXT)
) {
//printf("Eliminated needless zext VMOV\n");
) {
//printf("Eliminated needless zext bfe\n");
// Load mem / load ctx zexts, no need to vmem
IREmit->ReplaceAllUsesWith(CodeNode, CurrentIR.GetNode(source));
} else if (IROp->Size == sourceHeader->Size) {
// VMOV of same size
//printf("printf vmov of same size?!\n");
IREmit->ReplaceAllUsesWith(CodeNode, CurrentIR.GetNode(source));
break;
}
break;
}
default: break;
// BFE does implicit masking, remove any masks leading to this, if possible
uint64_t imm = 1ULL << (Op->Width-1);
imm = (imm-1) *2 + 1;
imm <<= Op->lsb;
auto newArg = RemoveUselessMasking(IREmit, IROp->Args[0], imm);
if (newArg.ID() != IROp->Args[0].ID()) {
IREmit->ReplaceNodeArgument(CodeNode, 0, IREmit->UnwrapNode(newArg));
Changed = true;
}
break;
}
// constprop + some more per instruction logic
case OP_SBFE: {
auto Op = IROp->C<IR::IROp_Sbfe>();
// BFE does implicit masking
uint64_t imm = 1ULL << (Op->Width-1);
imm = (imm-1) *2 + 1;
imm <<= Op->lsb;
auto newArg = RemoveUselessMasking(IREmit, IROp->Args[0], imm);
if (newArg.ID() != IROp->Args[0].ID()) {
IREmit->ReplaceNodeArgument(CodeNode, 0, IREmit->UnwrapNode(newArg));
Changed = true;
}
break;
}
case OP_VFADD:
case OP_VFSUB:
case OP_VFMUL:
case OP_VFDIV:
case OP_FCMP: {
auto flopSize = IROp->Size;
for (int i = 0; i < IROp->NumArgs; i++) {
auto argHeader = IREmit->GetOpHeader(IROp->Args[i]);
if (argHeader->Op == OP_VMOV) {
auto source = argHeader->Args[0];
auto sourceHeader = IREmit->GetOpHeader(source);
if (sourceHeader->Size >= flopSize) {
IREmit->ReplaceNodeArgument(CodeNode, i, IREmit->UnwrapNode(source));
//printf("VMOV bypassed\n");
}
}
}
break;
}
case OP_VMOV: {
// elim from load mem
auto source = IROp->Args[0];
auto sourceHeader = IREmit->GetOpHeader(source);
if (IROp->Size >= sourceHeader->Size &&
(sourceHeader->Op == OP_LOADMEM || sourceHeader->Op == OP_LOADMEMTSO || sourceHeader->Op == OP_LOADCONTEXT)
) {
//printf("Eliminated needless zext VMOV\n");
// Load mem / load ctx zexts, no need to vmem
IREmit->ReplaceAllUsesWith(CodeNode, CurrentIR.GetNode(source));
} else if (IROp->Size == sourceHeader->Size) {
// VMOV of same size
// XXX: This is unsafe of an optimization since in some cases we can't see through garbage data in the upper bits of a vector
// RCLSE generates VMOV instructions which are being used as a zero extension
//printf("printf vmov of same size?!\n");
//IREmit->ReplaceAllUsesWith(CodeNode, CurrentIR.GetNode(source));
}
break;
}
default:
break;
}
return Changed;
}
// constprop + some more per instruction logic
bool ConstProp::ConstantPropagation(IREmitter *IREmit, const IRListView& CurrentIR,
OrderedNode* CodeNode, IROp_Header* IROp) {
bool Changed = false;
switch (IROp->Op) {
/*
case OP_UMUL:
@@ -463,7 +491,7 @@ bool ConstProp::Run(IREmitter *IREmit) {
if (IREmit->IsValueConstant(IROp->Args[0], &Constant1) &&
IREmit->IsValueConstant(IROp->Args[1], &Constant2)) {
LogMan::Msg::A("Could const prop op: %s", std::string(IR::GetName(IROp->Op)).c_str());
LOGMAN_MSG_A_FMT("Could const prop op: {}", IR::GetName(IROp->Op));
}
break;
}
@@ -479,7 +507,7 @@ bool ConstProp::Run(IREmitter *IREmit) {
uint64_t Constant1;
if (IREmit->IsValueConstant(IROp->Args[0], &Constant1)) {
LogMan::Msg::A("Could const prop op: %s", std::string(IR::GetName(IROp->Op)).c_str());
LOGMAN_MSG_A_FMT("Could const prop op: {}", IR::GetName(IROp->Op));
}
break;
}
@@ -490,7 +518,6 @@ bool ConstProp::Run(IREmitter *IREmit) {
auto AddressHeader = IREmit->GetOpHeader(Op->Header.Args[0]);
if (AddressHeader->Op == OP_ADD && AddressHeader->Size == 8) {
auto [OffsetType, OffsetScale, Arg0, Arg1] = MemExtendedAddressing(IREmit, Op->Size, AddressHeader);
Op->OffsetType = OffsetType;
@@ -530,7 +557,6 @@ bool ConstProp::Run(IREmitter *IREmit) {
uint64_t NewConstant = (Constant1 + Constant2) & getMask(Op) ;
IREmit->ReplaceWithConstant(CodeNode, NewConstant);
Changed = true;
continue;
}
break;
}
@@ -711,9 +737,9 @@ bool ConstProp::Run(IREmitter *IREmit) {
uint64_t NewConstant = (Constant1 * Constant2) & getMask(Op);
IREmit->ReplaceWithConstant(CodeNode, NewConstant);
Changed = true;
} else if (IREmit->IsValueConstant(Op->Header.Args[1], &Constant2) && __builtin_popcountl(Constant2) == 1) {
} else if (IREmit->IsValueConstant(Op->Header.Args[1], &Constant2) && std::popcount(Constant2) == 1) {
if (IROp->Size == 4 || IROp->Size == 8) {
uint64_t amt = __builtin_ctzl(Constant2);
uint64_t amt = std::countr_zero(Constant2);
IREmit->SetWriteCursor(CodeNode);
auto shift = IREmit->_Lshl(CurrentIR.GetNode(Op->Header.Args[0]), IREmit->_Constant(amt));
shift.first->Header.Size = IROp->Size; // force Lshl to be the same size as the original Mul
@@ -753,192 +779,223 @@ bool ConstProp::Run(IREmitter *IREmit) {
default:
break;
}
}
// constant inlining
if (InlineConstants) {
for (auto [CodeNode, IROp] : CurrentIR.GetAllCode()) {
switch(IROp->Op) {
case OP_LSHR:
case OP_ASHR:
case OP_ROR:
case OP_LSHL:
{
auto Op = IROp->C<IR::IROp_Lshr>();
return Changed;
}
uint64_t Constant2{};
if (IREmit->IsValueConstant(Op->Header.Args[1], &Constant2)) {
bool ConstProp::ConstantInlining(IREmitter *IREmit, const IRListView& CurrentIR) {
bool Changed = false;
for (auto [CodeNode, IROp] : CurrentIR.GetAllCode()) {
switch(IROp->Op) {
case OP_LSHR:
case OP_ASHR:
case OP_ROR:
case OP_LSHL:
{
auto Op = IROp->C<IR::IROp_Lshr>();
uint64_t Constant2{};
if (IREmit->IsValueConstant(Op->Header.Args[1], &Constant2)) {
IREmit->SetWriteCursor(CurrentIR.GetNode(Op->Header.Args[1]));
// this shouldn't be here, but rather on the emitter themselves or the constprop transformation?
if (IROp->Size <=4)
Constant2 &= 31;
else
Constant2 &= 63;
IREmit->ReplaceNodeArgument(CodeNode, 1, IREmit->_InlineConstant(Constant2));
Changed = true;
}
break;
}
case OP_ADD:
case OP_SUB:
{
auto Op = IROp->C<IR::IROp_Add>();
uint64_t Constant2{};
if (IREmit->IsValueConstant(Op->Header.Args[1], &Constant2)) {
if (IsImmAddSub(Constant2)) {
IREmit->SetWriteCursor(CurrentIR.GetNode(Op->Header.Args[1]));
// this shouldn't be here, but rather on the emitter themselves or the constprop transformation?
if (IROp->Size <=4)
Constant2 &= 31;
else
Constant2 &= 63;
IREmit->ReplaceNodeArgument(CodeNode, 1, IREmit->_InlineConstant(Constant2));
Changed = true;
}
break;
}
break;
}
case OP_SELECT:
{
auto Op = IROp->C<IR::IROp_Select>();
uint64_t Constant1{};
if (IREmit->IsValueConstant(Op->Header.Args[1], &Constant1)) {
if (IsImmAddSub(Constant1)) {
IREmit->SetWriteCursor(CurrentIR.GetNode(Op->Header.Args[1]));
IREmit->ReplaceNodeArgument(CodeNode, 1, IREmit->_InlineConstant(Constant1));
Changed = true;
}
}
case OP_ADD:
case OP_SUB:
uint64_t Constant2{};
uint64_t Constant3{};
if (IREmit->IsValueConstant(Op->Header.Args[2], &Constant2) &&
IREmit->IsValueConstant(Op->Header.Args[3], &Constant3) &&
Constant2 == 1 &&
Constant3 == 0)
{
auto Op = IROp->C<IR::IROp_Add>();
IREmit->SetWriteCursor(CurrentIR.GetNode(Op->Header.Args[2]));
uint64_t Constant2{};
if (IREmit->IsValueConstant(Op->Header.Args[1], &Constant2)) {
if (IsImmAddSub(Constant2)) {
IREmit->SetWriteCursor(CurrentIR.GetNode(Op->Header.Args[1]));
IREmit->ReplaceNodeArgument(CodeNode, 1, IREmit->_InlineConstant(Constant2));
Changed = true;
}
}
break;
IREmit->ReplaceNodeArgument(CodeNode, 2, IREmit->_InlineConstant(Constant2));
IREmit->ReplaceNodeArgument(CodeNode, 3, IREmit->_InlineConstant(Constant3));
}
case OP_SELECT:
{
auto Op = IROp->C<IR::IROp_Select>();
break;
}
uint64_t Constant1{};
if (IREmit->IsValueConstant(Op->Header.Args[1], &Constant1)) {
if (IsImmAddSub(Constant1)) {
IREmit->SetWriteCursor(CurrentIR.GetNode(Op->Header.Args[1]));
case OP_CONDJUMP:
{
auto Op = IROp->C<IR::IROp_CondJump>();
IREmit->ReplaceNodeArgument(CodeNode, 1, IREmit->_InlineConstant(Constant1));
uint64_t Constant2{};
if (IREmit->IsValueConstant(Op->Header.Args[1], &Constant2)) {
if (IsImmAddSub(Constant2)) {
IREmit->SetWriteCursor(CurrentIR.GetNode(Op->Header.Args[1]));
Changed = true;
}
IREmit->ReplaceNodeArgument(CodeNode, 1, IREmit->_InlineConstant(Constant2));
Changed = true;
}
}
break;
}
uint64_t Constant2{};
uint64_t Constant3{};
if (IREmit->IsValueConstant(Op->Header.Args[2], &Constant2) &&
IREmit->IsValueConstant(Op->Header.Args[3], &Constant3) &&
Constant2 == 1 &&
Constant3 == 0)
{
case OP_EXITFUNCTION:
{
auto Op = IROp->C<IR::IROp_ExitFunction>();
uint64_t Constant{};
if (IREmit->IsValueConstant(Op->NewRIP, &Constant)) {
IREmit->SetWriteCursor(CurrentIR.GetNode(Op->NewRIP));
IREmit->ReplaceNodeArgument(CodeNode, 0, IREmit->_InlineConstant(Constant));
Changed = true;
} else {
auto NewRIP = IREmit->GetOpHeader(Op->NewRIP);
if (NewRIP->Op == OP_ENTRYPOINTOFFSET) {
auto EO = NewRIP->C<IR::IROp_EntrypointOffset>();
IREmit->SetWriteCursor(CurrentIR.GetNode(Op->NewRIP));
IREmit->ReplaceNodeArgument(CodeNode, 0, IREmit->_InlineEntrypointOffset(EO->Offset, EO->Header.Size));
Changed = true;
}
}
break;
}
case OP_OR:
case OP_XOR:
case OP_AND:
{
auto Op = IROp->CW<IR::IROp_Or>();
uint64_t Constant2{};
if (IREmit->IsValueConstant(Op->Header.Args[1], &Constant2)) {
if (IsImmLogical(Constant2, IROp->Size * 8)) {
IREmit->SetWriteCursor(CurrentIR.GetNode(Op->Header.Args[1]));
IREmit->ReplaceNodeArgument(CodeNode, 1, IREmit->_InlineConstant(Constant2));
Changed = true;
}
}
break;
}
case OP_LOADMEM:
{
auto Op = IROp->CW<IR::IROp_LoadMem>();
uint64_t Constant2{};
if (Op->OffsetType == MEM_OFFSET_SXTX && IREmit->IsValueConstant(Op->Header.Args[1], &Constant2)) {
if (IsImmMemory(Constant2, Op->Size)) {
IREmit->SetWriteCursor(CurrentIR.GetNode(Op->Header.Args[1]));
IREmit->ReplaceNodeArgument(CodeNode, 1, IREmit->_InlineConstant(Constant2));
Changed = true;
}
}
break;
}
case OP_STOREMEM:
{
auto Op = IROp->CW<IR::IROp_StoreMem>();
uint64_t Constant2{};
if (Op->OffsetType == MEM_OFFSET_SXTX && IREmit->IsValueConstant(Op->Header.Args[2], &Constant2)) {
if (IsImmMemory(Constant2, Op->Size)) {
IREmit->SetWriteCursor(CurrentIR.GetNode(Op->Header.Args[2]));
IREmit->ReplaceNodeArgument(CodeNode, 2, IREmit->_InlineConstant(Constant2));
IREmit->ReplaceNodeArgument(CodeNode, 3, IREmit->_InlineConstant(Constant3));
}
break;
}
case OP_CONDJUMP:
{
auto Op = IROp->C<IR::IROp_CondJump>();
uint64_t Constant2{};
if (IREmit->IsValueConstant(Op->Header.Args[1], &Constant2)) {
if (IsImmAddSub(Constant2)) {
IREmit->SetWriteCursor(CurrentIR.GetNode(Op->Header.Args[1]));
IREmit->ReplaceNodeArgument(CodeNode, 1, IREmit->_InlineConstant(Constant2));
Changed = true;
}
}
break;
}
case OP_EXITFUNCTION:
{
auto Op = IROp->C<IR::IROp_ExitFunction>();
uint64_t Constant{};
if (IREmit->IsValueConstant(Op->NewRIP, &Constant)) {
IREmit->SetWriteCursor(CurrentIR.GetNode(Op->NewRIP));
IREmit->ReplaceNodeArgument(CodeNode, 0, IREmit->_InlineConstant(Constant));
Changed = true;
} else {
auto NewRIP = IREmit->GetOpHeader(Op->NewRIP);
if (NewRIP->Op == OP_ENTRYPOINTOFFSET) {
auto EO = NewRIP->C<IR::IROp_EntrypointOffset>();
IREmit->SetWriteCursor(CurrentIR.GetNode(Op->NewRIP));
IREmit->ReplaceNodeArgument(CodeNode, 0, IREmit->_InlineEntrypointOffset(EO->Offset, EO->Header.Size));
Changed = true;
}
}
break;
}
case OP_OR:
case OP_XOR:
case OP_AND:
{
auto Op = IROp->CW<IR::IROp_Or>();
uint64_t Constant2{};
if (IREmit->IsValueConstant(Op->Header.Args[1], &Constant2)) {
if (IsImmLogical(Constant2, IROp->Size * 8)) {
IREmit->SetWriteCursor(CurrentIR.GetNode(Op->Header.Args[1]));
IREmit->ReplaceNodeArgument(CodeNode, 1, IREmit->_InlineConstant(Constant2));
Changed = true;
}
}
break;
}
case OP_LOADMEM:
{
auto Op = IROp->CW<IR::IROp_LoadMem>();
uint64_t Constant2{};
if (Op->OffsetType == MEM_OFFSET_SXTX && IREmit->IsValueConstant(Op->Header.Args[1], &Constant2)) {
if (IsImmMemory(Constant2, Op->Size)) {
IREmit->SetWriteCursor(CurrentIR.GetNode(Op->Header.Args[1]));
IREmit->ReplaceNodeArgument(CodeNode, 1, IREmit->_InlineConstant(Constant2));
Changed = true;
}
}
break;
}
case OP_STOREMEM:
{
auto Op = IROp->CW<IR::IROp_StoreMem>();
uint64_t Constant2{};
if (Op->OffsetType == MEM_OFFSET_SXTX && IREmit->IsValueConstant(Op->Header.Args[2], &Constant2)) {
if (IsImmMemory(Constant2, Op->Size)) {
IREmit->SetWriteCursor(CurrentIR.GetNode(Op->Header.Args[2]));
IREmit->ReplaceNodeArgument(CodeNode, 2, IREmit->_InlineConstant(Constant2));
Changed = true;
}
}
break;
}
default: break;
break;
}
default:
break;
}
}
IREmit->SetWriteCursor(OriginalWriteCursor);
return Changed;
}
FEXCore::IR::Pass* CreateConstProp(bool InlineConstants) {
return new ConstProp(InlineConstants);
bool ConstProp::Run(IREmitter *IREmit) {
bool Changed = false;
auto CurrentIR = IREmit->ViewIR();
auto OriginalWriteCursor = IREmit->GetWriteCursor();
if (HandleConstantPools(IREmit, CurrentIR)) {
Changed = true;
}
CodeMotionAroundSelects(IREmit, CurrentIR);
FCMPOptimization(IREmit, CurrentIR);
LoadMemStoreMemImmediatePooling(IREmit, CurrentIR);
for (auto [CodeNode, IROp] : CurrentIR.GetAllCode()) {
if (ZextAndMaskingElimination(IREmit, CurrentIR, CodeNode, IROp)) {
Changed = true;
}
if (ConstantPropagation(IREmit, CurrentIR, CodeNode, IROp)) {
Changed = true;
}
}
if (InlineConstants && ConstantInlining(IREmit, CurrentIR)) {
Changed = true;
}
IREmit->SetWriteCursor(OriginalWriteCursor);
return Changed;
}
std::unique_ptr<FEXCore::IR::Pass> CreateConstProp(bool InlineConstants) {
return std::make_unique<ConstProp>(InlineConstants);
}
}
@@ -59,10 +59,8 @@ void DeadCodeElimination::markUsed(OrderedNodeWrapper *CodeOp, IROp_Header *IROp
}
FEXCore::IR::Pass* CreatePassDeadCodeElimination() {
return new DeadCodeElimination{};
std::unique_ptr<FEXCore::IR::Pass> CreatePassDeadCodeElimination() {
return std::make_unique<DeadCodeElimination>();
}
}
@@ -17,14 +17,36 @@ namespace {
};
enum LastAccessType {
ACCESS_NONE, ///< Was never previously accessed
ACCESS_WRITE, ///< Was fully overwritten
ACCESS_READ, ///< Was fully read
ACCESS_PARTIAL_WRITE, ///< Was partially written
ACCESS_PARTIAL_READ, ///< Was partially read
ACCESS_INVALID, ///< Accessing this is invalid
ACCESS_NONE = (0b000 << 0), ///< Was never previously accessed
ACCESS_WRITE = (0b001 << 0), ///< Was fully overwritten
ACCESS_READ = (0b010 << 0), ///< Was fully read
ACCESS_INVALID = (0b011 << 0), ///< Accessing this is invalid
ACCESS_TYPE_MASK = (0b011 << 0),
ACCESS_PARTIAL = (0b100 << 0),
ACCESS_PARTIAL_WRITE = (ACCESS_PARTIAL | ACCESS_WRITE), ///< Was partially written
ACCESS_PARTIAL_READ = (ACCESS_PARTIAL | ACCESS_READ), ///< Was partially read
};
static bool IsWriteAccess(LastAccessType Type) {
return (Type & ACCESS_TYPE_MASK) == ACCESS_WRITE;
}
static bool IsReadAccess(LastAccessType Type) {
return (Type & ACCESS_TYPE_MASK) == ACCESS_READ;
}
static bool IsInvalidAccess(LastAccessType Type) {
return (Type & ACCESS_TYPE_MASK) == ACCESS_INVALID;
}
static bool IsPartialAccess(LastAccessType Type) {
return (Type & ACCESS_PARTIAL) == ACCESS_PARTIAL;
}
static bool IsFullAccess(LastAccessType Type) {
return (Type & ACCESS_PARTIAL) == 0;
}
struct ContextMemberInfo {
ContextMemberClassification Class;
LastAccessType Accessed;
@@ -40,7 +62,7 @@ namespace {
std::vector<ContextMemberInfo> ClassificationInfo;
};
constexpr static std::array<LastAccessType, 15> DefaultAccess = {
constexpr static std::array<LastAccessType, 16> DefaultAccess = {
ACCESS_NONE,
ACCESS_NONE,
ACCESS_INVALID, // PAD
@@ -56,6 +78,7 @@ namespace {
ACCESS_NONE,
ACCESS_NONE,
ACCESS_NONE,
ACCESS_NONE,
};
static void ClassifyContextStruct(ContextInfo *ContextClassificationInfo) {
@@ -145,7 +168,7 @@ namespace {
DefaultAccess[4],
FEXCore::IR::InvalidClass,
});
ContextClassification->emplace_back(ContextMemberInfo{
ContextMemberClassification {
offsetof(FEXCore::Core::CPUState, fs),
@@ -208,21 +231,32 @@ namespace {
FEXCore::IR::InvalidClass,
});
// FTW
ContextClassification->emplace_back(ContextMemberInfo {
ContextMemberClassification {
offsetof(FEXCore::Core::CPUState, FTW),
sizeof(FEXCore::Core::CPUState::FTW),
},
DefaultAccess[15],
FEXCore::IR::InvalidClass,
});
size_t ClassifiedStructSize{};
ContextClassificationInfo->Lookup.reserve(sizeof(FEXCore::Core::CPUState));
for (auto &it : *ContextClassification) {
LogMan::Throw::A(it.Class.Offset == ContextClassificationInfo->Lookup.size(), "Offset missmatch %d %d", it.Class.Offset == ContextClassificationInfo->Lookup.size());
LOGMAN_THROW_A(it.Class.Offset == ContextClassificationInfo->Lookup.size(), "Offset missmatch %d %d", it.Class.Offset == ContextClassificationInfo->Lookup.size());
for (int i = 0; i < it.Class.Size; i++) {
ContextClassificationInfo->Lookup.push_back(&it);
}
ClassifiedStructSize += it.Class.Size;
}
LogMan::Throw::A(ClassifiedStructSize == sizeof(FEXCore::Core::CPUState),
LOGMAN_THROW_A(ClassifiedStructSize == sizeof(FEXCore::Core::CPUState),
"Classified CPUStruct size doesn't match real CPUState struct size! %ld != %ld",
ClassifiedStructSize, sizeof(FEXCore::Core::CPUState));
LogMan::Throw::A(ContextClassificationInfo->Lookup.size() == sizeof(FEXCore::Core::CPUState),
LOGMAN_THROW_A(ContextClassificationInfo->Lookup.size() == sizeof(FEXCore::Core::CPUState),
"Classified CPUStruct size doesn't match real CPUState struct size! %ld != %ld",
ContextClassificationInfo->Lookup.size(), sizeof(FEXCore::Core::CPUState));
}
@@ -270,6 +304,7 @@ namespace {
}
SetAccess(Offset++, DefaultAccess[14]);
SetAccess(Offset++, DefaultAccess[15]);
}
struct BlockInfo {
@@ -283,7 +318,7 @@ class RCLSE final : public FEXCore::IR::Pass {
public:
RCLSE() {
ClassifyContextStruct(&ClassifiedStruct);
DCE.reset(FEXCore::IR::CreatePassDeadCodeElimination());
DCE = FEXCore::IR::CreatePassDeadCodeElimination();
}
bool Run(FEXCore::IR::IREmitter *IREmit) override;
private:
@@ -306,15 +341,15 @@ ContextMemberInfo *RCLSE::FindMemberInfo(ContextInfo *ContextClassificationInfo,
}
ContextMemberInfo *RCLSE::RecordAccess(ContextMemberInfo *Info, FEXCore::IR::RegisterClassType RegClass, uint32_t Offset, uint8_t Size, LastAccessType AccessType, FEXCore::IR::OrderedNode *Node, FEXCore::IR::OrderedNode *StoreNode) {
LogMan::Throw::A((Offset + Size) <= (Info->Class.Offset + Info->Class.Size), "Access to context item went over member size");
LogMan::Throw::A(Info->Accessed != ACCESS_INVALID, "Tried to access invalid member");
LOGMAN_THROW_A((Offset + Size) <= (Info->Class.Offset + Info->Class.Size), "Access to context item went over member size");
LOGMAN_THROW_A(Info->Accessed != ACCESS_INVALID, "Tried to access invalid member");
// If we aren't fully overwriting the member then it is a partial write that we need to track
if (Size < Info->Class.Size) {
AccessType = AccessType == ACCESS_WRITE ? ACCESS_PARTIAL_WRITE : ACCESS_PARTIAL_READ;
}
if (Size > Info->Class.Size) {
LogMan::Msg::A("Can't handle this");
LOGMAN_MSG_A("Can't handle this");
}
Info->Accessed = AccessType;
@@ -431,6 +466,8 @@ bool RCLSE::RedundantStoreLoadElimination(FEXCore::IR::IREmitter *IREmit) {
ContextInfo &LocalInfo = ClassifiedStruct;
for (auto [BlockNode, BlockHeader] : CurrentIR.GetBlocks()) {
auto BlockOp = BlockHeader->CW<FEXCore::IR::IROp_CodeBlock>();
auto BlockEnd = IREmit->GetIterator(BlockOp->Last);
ResetClassificationAccesses(&LocalInfo);
@@ -443,9 +480,9 @@ bool RCLSE::RedundantStoreLoadElimination(FEXCore::IR::IREmitter *IREmit) {
uint8_t LastSize = Info->AccessSize;
LastAccessType LastAccess = Info->Accessed;
OrderedNode *LastStoreNode = Info->StoreNode;
RecordAccess(Info, Op->Class, Op->Offset, IROp->Size, ACCESS_WRITE, CurrentIR.GetNode(Op->Header.Args[0]), CodeNode);
RecordAccess(Info, Op->Class, Op->Offset, IROp->Size, ACCESS_WRITE, CurrentIR.GetNode(Op->Value), CodeNode);
if ((LastAccess == ACCESS_WRITE || LastAccess == ACCESS_PARTIAL_WRITE) &&
if (IsWriteAccess(LastAccess) &&
LastClass == Op->Class &&
LastOffset == Op->Offset &&
LastSize <= IROp->Size) {
@@ -470,7 +507,7 @@ bool RCLSE::RedundantStoreLoadElimination(FEXCore::IR::IREmitter *IREmit) {
OrderedNode *LastStoreNode = Info->StoreNode;
RecordAccess(Info, Op->Class, Op->Offset, IROp->Size, ACCESS_READ, CodeNode);
if ((LastAccess == ACCESS_WRITE || LastAccess == ACCESS_PARTIAL_WRITE) &&
if (IsWriteAccess(LastAccess) &&
LastClass == Op->Class &&
LastOffset == Op->Offset &&
IROp->Size <= LastSize) {
@@ -491,46 +528,52 @@ bool RCLSE::RedundantStoreLoadElimination(FEXCore::IR::IREmitter *IREmit) {
if (TruncateSize != IREmit->GetOpSize(LastNode)) {
// We need to insert an explict truncation
if (LastClass == FPRClass) {
LastNode = IREmit->_VMov(LastNode, TruncateSize); // Vmov truncates and zexts when register width is smaller than source
}
else if (LastClass == GPRPairClass) {
LastNode = IREmit->_TruncElementPair(LastNode, TruncateSize);
}
else if (LastClass == GPRClass) {
LastNode = IREmit->_Bfe(Info->AccessSize, TruncateSize * 8, 0, LastNode);
} else {
LogMan::Msg::A("Unhandled Register class");
}
LastNode = IREmit->_Bfe(Info->AccessSize, TruncateSize * 8, 0, LastNode);
}
IREmit->ReplaceAllUsesWith(CodeNode, LastNode);
IREmit->ReplaceAllUsesWithRange(CodeNode, LastNode, IREmit->GetIterator(IREmit->WrapNode(CodeNode)), BlockEnd);
RecordAccess(Info, Op->Class, Op->Offset, IROp->Size, ACCESS_READ, LastNode);
Changed = true;
} else {
if (LastClass == FPRClass && LastSize == IROp->Size && LastSize == IREmit->GetOpSize(LastNode)) {
// LoadCtx matches StoreCtx and Node Size
IREmit->ReplaceAllUsesWith(CodeNode, LastNode);
RecordAccess(Info, Op->Class, Op->Offset, IROp->Size, ACCESS_READ, LastNode);
Changed = true;
} else if (LastClass == FPRClass && LastSize >= IROp->Size && IROp->Size == IREmit->GetOpSize(LastNode)) {
} else if (LastClass == FPRClass) {
if (LastSize == IROp->Size && LastSize == IREmit->GetOpSize(LastNode)) {
if (IsFullAccess(Info->Accessed)) {
// LoadCtx matches StoreCtx and Node Size
IREmit->ReplaceAllUsesWithRange(CodeNode, LastNode, IREmit->GetIterator(IREmit->WrapNode(CodeNode)), BlockEnd);
RecordAccess(Info, Op->Class, Op->Offset, IROp->Size, ACCESS_READ, LastNode);
Changed = true;
}
else {
// If this load size is a partial load then it may be expecting a zext of
// the vector element
IREmit->SetWriteCursor(CodeNode);
// zext to size
LastNode = IREmit->_VMov(LastNode, IROp->Size);
IREmit->ReplaceAllUsesWithRange(CodeNode, LastNode, IREmit->GetIterator(IREmit->WrapNode(CodeNode)), BlockEnd);
RecordAccess(Info, Op->Class, Op->Offset, IROp->Size, ACCESS_READ, LastNode);
Changed = true;
}
} else if (LastSize >= IROp->Size &&
IROp->Size == IREmit->GetOpSize(LastNode)) {
// LoadCtx is <= StoreCtx and Node is LoadCtx
IREmit->ReplaceAllUsesWith(CodeNode, LastNode);
IREmit->ReplaceAllUsesWithRange(CodeNode, LastNode, IREmit->GetIterator(IREmit->WrapNode(CodeNode)), BlockEnd);
RecordAccess(Info, Op->Class, Op->Offset, IROp->Size, ACCESS_READ, LastNode);
Changed = true;
} else if (LastClass == FPRClass && LastSize >= IROp->Size && IROp->Size < IREmit->GetOpSize(LastNode)) {
} else if (LastSize >= IROp->Size &&
IROp->Size < IREmit->GetOpSize(LastNode)) {
IREmit->SetWriteCursor(CodeNode);
// trucate to size
LastNode = IREmit->_VMov(LastNode, IROp->Size);
IREmit->ReplaceAllUsesWith(CodeNode, LastNode);
IREmit->ReplaceAllUsesWithRange(CodeNode, LastNode, IREmit->GetIterator(IREmit->WrapNode(CodeNode)), BlockEnd);
RecordAccess(Info, Op->Class, Op->Offset, IROp->Size, ACCESS_READ, LastNode);
Changed = true;
} else if (LastClass == FPRClass && LastSize >= IROp->Size && IROp->Size > IREmit->GetOpSize(LastNode)) {
} else if (LastSize >= IROp->Size &&
IROp->Size > IREmit->GetOpSize(LastNode)) {
IREmit->SetWriteCursor(CodeNode);
// zext to size
LastNode = IREmit->_VMov(LastNode, IROp->Size);
IREmit->ReplaceAllUsesWith(CodeNode, LastNode);
IREmit->ReplaceAllUsesWithRange(CodeNode, LastNode, IREmit->GetIterator(IREmit->WrapNode(CodeNode)), BlockEnd);
RecordAccess(Info, Op->Class, Op->Offset, IROp->Size, ACCESS_READ, LastNode);
Changed = true;
} else {
@@ -538,13 +581,13 @@ bool RCLSE::RedundantStoreLoadElimination(FEXCore::IR::IREmitter *IREmit) {
}
}
}
else if ((LastAccess == ACCESS_READ || LastAccess == ACCESS_PARTIAL_READ) &&
else if (IsReadAccess(LastAccess) &&
IsReadAccess(Info->Accessed) &&
LastClass == Op->Class &&
LastOffset == Op->Offset &&
LastSize == IROp->Size &&
(Info->Accessed == ACCESS_READ || Info->Accessed == ACCESS_PARTIAL_READ)) {
LastSize == IROp->Size) {
// Did we read and then read again?
IREmit->ReplaceAllUsesWith(CodeNode, LastNode);
IREmit->ReplaceAllUsesWithRange(CodeNode, LastNode, IREmit->GetIterator(IREmit->WrapNode(CodeNode)), BlockEnd);
RecordAccess(Info, Op->Class, Op->Offset, IROp->Size, ACCESS_READ, LastNode);
Changed = true;
}
@@ -578,7 +621,7 @@ bool RCLSE::RedundantStoreLoadElimination(FEXCore::IR::IREmitter *IREmit) {
if (LastStoreNode != nullptr)
{
IREmit->SetWriteCursor(CodeNode);
RecordAccess(&LocalInfo, FEXCore::IR::GPRClass, offsetof(FEXCore::Core::CPUState, flags[0]) + F, 1, ACCESS_WRITE, IREmit->_InlineConstant(0), CodeNode);
RecordAccess(&LocalInfo, FEXCore::IR::GPRClass, offsetof(FEXCore::Core::CPUState, flags[0]) + F, 1, ACCESS_WRITE, IREmit->_Constant(0), CodeNode);
IREmit->Remove(LastStoreNode);
Changed = true;
@@ -591,21 +634,21 @@ bool RCLSE::RedundantStoreLoadElimination(FEXCore::IR::IREmitter *IREmit) {
LastAccessType LastAccess = Info->Accessed;
OrderedNode *LastNode = Info->Node;
if (LastAccess == ACCESS_WRITE) { // 1 byte so always a full write
if (IsWriteAccess(LastAccess)) { // 1 byte so always a full write
// If the last store matches this load value then we can replace the loaded value with the previous valid one
IREmit->SetWriteCursor(CodeNode);
IREmit->ReplaceAllUsesWith(CodeNode, LastNode);
RecordAccess(Info, FEXCore::IR::GPRClass, offsetof(FEXCore::Core::CPUState, flags[0]) + Op->Flag, 1, ACCESS_READ, LastNode);
Changed = true;
}
else if (LastAccess == ACCESS_READ) {
else if (IsReadAccess(LastAccess)) {
IREmit->ReplaceAllUsesWith(CodeNode, LastNode);
RecordAccess(Info, FEXCore::IR::GPRClass, offsetof(FEXCore::Core::CPUState, flags[0]) + Op->Flag, 1, ACCESS_READ, LastNode);
Changed = true;
}
}
else if (IROp->Op == OP_STORECONTEXTINDEXED ||
IROp->Op == OP_LOADCONTEXTINDEXED ||
IROp->Op == OP_LOADCONTEXTINDEXED ||
IROp->Op == OP_SYSCALL) {
// We can't track through these
ResetClassificationAccesses(&LocalInfo);
@@ -622,7 +665,7 @@ bool RCLSE::Run(FEXCore::IR::IREmitter *IREmit) {
// XXX: We don't do cross-block optimizations yet
//CalculateControlFlowInfo(IREmit);
bool Changed = false;
// Run up to 5 times
for( int i = 0; i < 5 && RedundantStoreLoadElimination(IREmit); i++) {
Changed = true;
@@ -636,8 +679,8 @@ bool RCLSE::Run(FEXCore::IR::IREmitter *IREmit) {
namespace FEXCore::IR {
FEXCore::IR::Pass* CreateContextLoadStoreElimination() {
return new RCLSE{};
std::unique_ptr<FEXCore::IR::Pass> CreateContextLoadStoreElimination() {
return std::make_unique<RCLSE>();
}
}
@@ -106,7 +106,7 @@ uint64_t FPRBit(uint32_t Offset, uint32_t Size) {
else if (Size == 4)
return 1UL << (bitn);
else
LogMan::Msg::A("Unexpected FPR size %d", Size);
LOGMAN_MSG_A("Unexpected FPR size %d", Size);
return 7UL << (bitn); // Return maximum on failure case
}
@@ -331,8 +331,8 @@ bool DeadStoreElimination::Run(IREmitter *IREmit) {
return Changed;
}
FEXCore::IR::Pass* CreateDeadStoreElimination() {
return new DeadStoreElimination{};
std::unique_ptr<FEXCore::IR::Pass> CreateDeadStoreElimination() {
return std::make_unique<DeadStoreElimination>();
}
}
@@ -66,7 +66,7 @@ bool IRCompaction::Run(IREmitter *IREmit) {
auto HeaderNode = CurrentIR.GetHeaderNode();
auto HeaderOp = CurrentIR.GetHeader();
LogMan::Throw::A(HeaderOp->Header.Op == OP_IRHEADER, "First op wasn't IRHeader");
LOGMAN_THROW_A(HeaderOp->Header.Op == OP_IRHEADER, "First op wasn't IRHeader");
// This compaction pass is something that we need to ensure correct ordering and distances between IROps
// Later on we assume that an IROp's SSA value live range is its Node locations
@@ -84,13 +84,13 @@ bool IRCompaction::Run(IREmitter *IREmit) {
// Zero is always zero(invalid)
OldToNewRemap[0].NodeID = 0;
auto LocalHeaderOp = LocalBuilder._IRHeader(OrderedNodeWrapper::WrapOffset(0).GetNode(ListBegin), HeaderOp->Entry, HeaderOp->BlockCount);
auto LocalHeaderOp = LocalBuilder._IRHeader(OrderedNodeWrapper::WrapOffset(0).GetNode(ListBegin), HeaderOp->BlockCount);
OldToNewRemap[CurrentIR.GetID(HeaderNode)].NodeID = LocalIR.GetID(LocalHeaderOp.Node);
{
// Generate our codeblocks and link them together
for (auto [BlockNode, BlockHeader] : CurrentIR.GetBlocks()) {
LogMan::Throw::A(BlockHeader->Op == OP_CODEBLOCK, "IR type failed to be a code block");
LOGMAN_THROW_A(BlockHeader->Op == OP_CODEBLOCK, "IR type failed to be a code block");
auto LocalBlockIRNode = LocalBuilder._CodeBlock(LocalHeaderOp, LocalHeaderOp); // Use LocalHeaderOp as a dummy arg for now
OldToNewRemap[CurrentIR.GetID(BlockNode)].NodeID = LocalIR.GetID(LocalBlockIRNode.Node);
@@ -153,8 +153,10 @@ bool IRCompaction::Run(IREmitter *IREmit) {
{
// Fixup the arguments of all the IROps
for (auto &Block : GeneratedCodeBlocks) {
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
auto BlockIROp = LocalIR.GetOp<FEXCore::IR::IROp_CodeBlock>(Block.NewNode);
LogMan::Throw::A(BlockIROp->Header.Op == OP_CODEBLOCK, "IR type failed to be a code block");
LOGMAN_THROW_A(BlockIROp->Header.Op == OP_CODEBLOCK, "IR type failed to be a code block");
#endif
for (auto [LocalNode, LocalIROp] : LocalIR.GetCode(Block.NewNode)) {
@@ -165,7 +167,7 @@ bool IRCompaction::Run(IREmitter *IREmit) {
for (uint8_t i = 0; i < NumArgs; ++i) {
uint32_t OldArg = LocalIROp->Args[i].ID();
#ifndef NDEBUG
LogMan::Throw::A(OldToNewRemap[OldArg].NodeID != ~0U, "Tried remapping unfound node %%ssa%d", OldArg);
LOGMAN_THROW_A(OldToNewRemap[OldArg].NodeID != ~0U, "Tried remapping unfound node %%ssa%d", OldArg);
#endif
LocalIROp->Args[i].NodeOffset = OldToNewRemap[OldArg].NodeID * sizeof(OrderedNode);
}
@@ -191,7 +193,7 @@ bool IRCompaction::Run(IREmitter *IREmit) {
// if (NewListSize > OldListSize ||
// NewDataSize > OldDataSize) {
// LogMan::Msg::A("Whoa. Compaction made the IR a different size when it shouldn't have. 0x%lx > 0x%lx or 0x%lx > 0x%lx",NewListSize, OldListSize, NewDataSize, OldDataSize);
// LOGMAN_MSG_A("Whoa. Compaction made the IR a different size when it shouldn't have. 0x%lx > 0x%lx or 0x%lx > 0x%lx",NewListSize, OldListSize, NewDataSize, OldDataSize);
// }
IREmit->CopyData(LocalBuilder);
@@ -199,8 +201,8 @@ bool IRCompaction::Run(IREmitter *IREmit) {
return true;
}
FEXCore::IR::Pass* CreateIRCompaction() {
return new IRCompaction{};
std::unique_ptr<FEXCore::IR::Pass> CreateIRCompaction() {
return std::make_unique<IRCompaction>();
}
}
@@ -11,7 +11,7 @@ $end_info$
#include "Interface/Core/OpcodeDispatcher.h"
#include "Common/BitSet.h"
#include <iostream>
#include <sstream>
namespace {
struct BlockInfo {
@@ -54,8 +54,10 @@ bool IRValidation::Run(IREmitter *IREmit) {
std::vector<uint32_t> Uses(CurrentIR.GetSSACount(), 0);
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
auto HeaderOp = CurrentIR.GetHeader();
LogMan::Throw::A(HeaderOp->Header.Op == OP_IRHEADER, "First op wasn't IRHeader");
LOGMAN_THROW_A_FMT(HeaderOp->Header.Op == OP_IRHEADER, "First op wasn't IRHeader");
#endif
IR::RegisterAllocationData * RAData{};
if (Manager->HasRAPass()) {
@@ -66,7 +68,7 @@ bool IRValidation::Run(IREmitter *IREmit) {
for (auto [BlockNode, BlockHeader] : CurrentIR.GetBlocks()) {
auto BlockIROp = BlockHeader->CW<FEXCore::IR::IROp_CodeBlock>();
LogMan::Throw::A(BlockIROp->Header.Op == OP_CODEBLOCK, "IR type failed to be a code block");
LOGMAN_THROW_A_FMT(BlockIROp->Header.Op == OP_CODEBLOCK, "IR type failed to be a code block");
uint32_t BlockID = CurrentIR.GetID(BlockNode);
@@ -209,7 +211,7 @@ bool IRValidation::Run(IREmitter *IREmit) {
break;
}
default:
// LogMan::Msg::A("Unknown IR Op: %d(%s)", IROp->Op, FEXCore::IR::GetName(IROp->Op).data());
// LOGMAN_MSG_A_FMT("Unknown IR Op: {}({})", IROp->Op, FEXCore::IR::GetName(IROp->Op));
break;
}
}
@@ -265,10 +267,9 @@ bool IRValidation::Run(IREmitter *IREmit) {
}
}
std::stringstream Out;
HadWarning = false;
if (HadError || HadWarning) {
std::stringstream Out;
FEXCore::IR::Dump(&Out, &CurrentIR, RAData);
if (HadError) {
@@ -279,13 +280,13 @@ bool IRValidation::Run(IREmitter *IREmit) {
Out << "Warnings:" << std::endl << Warnings.str() << std::endl;
}
LogMan::Msg::E("%s", Out.str().c_str());
LogMan::Msg::EFmt("{}", Out.str());
}
return false;
}
FEXCore::IR::Pass* CreateIRValidation() {
return new IRValidation{};
std::unique_ptr<FEXCore::IR::Pass> CreateIRValidation() {
return std::make_unique<IRValidation>();
}
}
@@ -0,0 +1,112 @@
/*
$info$
tags: ir|opts
desc: Long divide elimination pass
$end_info$
*/
#include "Interface/IR/PassManager.h"
#include <FEXCore/Utils/LogManager.h>
namespace FEXCore::IR {
class LongDivideEliminationPass final : public FEXCore::IR::Pass {
public:
bool Run(IREmitter *IREmit) override;
private:
bool IsZeroOp(IREmitter *IREmit, OrderedNodeWrapper Arg);
bool IsSextOp(IREmitter *IREmit, OrderedNodeWrapper Lower, OrderedNodeWrapper Upper);
};
bool LongDivideEliminationPass::IsZeroOp(IREmitter *IREmit, OrderedNodeWrapper Arg) {
auto IROp = IREmit->GetOpHeader(Arg);
uint64_t Value;
// XOR based zero
if (IROp->Op == OP_XOR) {
return IROp->Args[0] == IROp->Args[1];
}
else if (IREmit->IsValueConstant(Arg, &Value)) {
// Zero constant based zero op
return Value == 0;
}
return false;
}
bool LongDivideEliminationPass::IsSextOp(IREmitter *IREmit, OrderedNodeWrapper Lower, OrderedNodeWrapper Upper) {
// We need to check if the upper source is a sext of the lower source
auto UpperIROp = IREmit->GetOpHeader(Upper);
if (UpperIROp->Op == OP_SBFE) {
auto Op = UpperIROp->C<IR::IROp_Sbfe>();
if (Op->Width == 1 && Op->lsb == 63) {
// CQO: OrderedNode *Upper = _Sbfe(1, Size * 8 - 1, Src);
// If the lower is the upper in this case then it can be optimized
return Op->Header.Args[0] == Lower;
}
}
return false;
}
bool LongDivideEliminationPass::Run(IREmitter *IREmit) {
bool Changed = false;
auto CurrentIR = IREmit->ViewIR();
auto OriginalWriteCursor = IREmit->GetWriteCursor();
for (auto [BlockNode, BlockHeader] : CurrentIR.GetBlocks()) {
for (auto [CodeNode, IROp] : CurrentIR.GetCode(BlockNode)) {
if (IROp->Size == 8) {
if (IROp->Op == OP_LDIV ||
IROp->Op == OP_LREM) {
auto Op = IROp->C<IR::IROp_LDiv>();
// Check upper Op to see if it came from a CQO
// CQO: OrderedNode *Upper = _Sbfe(1, Size * 8 - 1, Src);
// If it does then it we only need a 64bit SDIV
if (IsSextOp(IREmit, Op->Lower, Op->Upper)) {
IREmit->SetWriteCursor(CodeNode);
OrderedNode *Lower = CurrentIR.GetNode(Op->Lower);
OrderedNode *Divisor = CurrentIR.GetNode(Op->Divisor);
OrderedNode *SDivOp{};
if (IROp->Op == OP_LDIV) {
SDivOp = IREmit->_Div(Lower, Divisor);
}
else {
SDivOp = IREmit->_Rem(Lower, Divisor);
}
IREmit->ReplaceAllUsesWith(CodeNode, SDivOp);
Changed = true;
}
}
else if (IROp->Op == OP_LUDIV ||
IROp->Op == OP_LUREM) {
auto Op = IROp->C<IR::IROp_LUDiv>();
// Check upper Op to see if it came from a xor zeroing op
// XOR: Result = _Xor(Dest, Src);
// If it does then it we only need a 64bit UDIV
if (IsZeroOp(IREmit, Op->Upper)) {
IREmit->SetWriteCursor(CodeNode);
OrderedNode *Lower = CurrentIR.GetNode(Op->Lower);
OrderedNode *Divisor = CurrentIR.GetNode(Op->Divisor);
OrderedNode *UDivOp{};
if (IROp->Op == OP_LUDIV) {
UDivOp = IREmit->_UDiv(Lower, Divisor);
}
else {
UDivOp = IREmit->_URem(Lower, Divisor);
}
IREmit->ReplaceAllUsesWith(CodeNode, UDivOp);
Changed = true;
}
}
}
}
}
IREmit->SetWriteCursor(OriginalWriteCursor);
return Changed;
}
std::unique_ptr<FEXCore::IR::Pass> CreateLongDivideEliminationPass() {
return std::make_unique<LongDivideEliminationPass>();
}
}
@@ -8,7 +8,7 @@ $end_info$
#include "Interface/IR/PassManager.h"
#include "Interface/Core/OpcodeDispatcher.h"
#include <iostream>
#include <sstream>
namespace FEXCore::IR::Validation {
@@ -52,22 +52,18 @@ bool PhiValidation::Run(IREmitter *IREmit) {
}
}
std::stringstream Out;
if (HadError) {
std::stringstream Out;
FEXCore::IR::Dump(&Out, &CurrentIR, nullptr);
Out << "Errors:" << std::endl << Errors.str() << std::endl;
LogMan::Msg::E(Out.str().c_str());
LogMan::Msg::EFmt("{}", Out.str());
}
return false;
}
FEXCore::IR::Pass* CreatePhiValidation() {
return new PhiValidation{};
std::unique_ptr<FEXCore::IR::Pass> CreatePhiValidation() {
return std::make_unique<PhiValidation>();
}
}
@@ -59,8 +59,8 @@ bool DeadFlagCalculationEliminination::Run(IREmitter *IREmit) {
return Changed;
}
FEXCore::IR::Pass* CreateDeadFlagCalculationEliminination() {
return new DeadFlagCalculationEliminination{};
std::unique_ptr<FEXCore::IR::Pass> CreateDeadFlagCalculationEliminination() {
return std::make_unique<DeadFlagCalculationEliminination>();
}
}
@@ -8,9 +8,11 @@ $end_info$
#include "Interface/IR/Passes/RegisterAllocationPass.h"
#include "Interface/IR/Passes.h"
#include "Interface/Core/OpcodeDispatcher.h"
#include <FEXCore/Utils/Allocator.h>
#include <iterator>
#include <unordered_set>
#include <sys/mman.h>
#define SRA_DEBUG(...) // printf(__VA_ARGS__)
@@ -45,7 +47,7 @@ namespace {
for (int i = 1; i < Size; i++)
Items[i] = 0xDEADBEEF;
#endif
Next.release();
Next.reset();
}
BucketList() {
@@ -65,7 +67,7 @@ namespace {
Enumerator(Item);
if (++i == Bucket->Size) {
LogMan::Throw::A(Bucket->Next != nullptr, "Interference bug");
LOGMAN_THROW_A(Bucket->Next != nullptr, "Interference bug");
Bucket = Bucket->Next.get();
i = 0;
}
@@ -86,7 +88,7 @@ namespace {
return true;
if (++i == Bucket->Size) {
LogMan::Throw::A(Bucket->Next != nullptr, "Bucket in bad state");
LOGMAN_THROW_A(Bucket->Next != nullptr, "Bucket in bad state");
Bucket = Bucket->Next.get();
i = 0;
}
@@ -130,7 +132,7 @@ namespace {
}
else if (++i == Size) {
i = 0;
LogMan::Throw::A(that->Next != nullptr, "Bucket::Erase but element not contained");
LOGMAN_THROW_A(that->Next != nullptr, "Bucket::Erase but element not contained");
that = that->Next.get();
}
}
@@ -143,7 +145,7 @@ namespace {
}
else if (++i == Size) {
if (that->Next->Items[0] == 0) {
that->Next.release();
that->Next.reset();
foundThat->Items[foundI] = that->Items[Size-1];
that->Items[Size-1] = 0;
break;
@@ -176,6 +178,7 @@ namespace {
};
static_assert(sizeof(RegisterNode) == 128 * 4);
constexpr size_t REGISTER_NODES_PER_PAGE = FEXCore::Core::PAGE_SIZE / sizeof(RegisterNode);
struct RegisterSet {
std::vector<RegisterClass> Classes;
@@ -203,8 +206,8 @@ namespace {
struct RegisterGraph {
std::unique_ptr<FEXCore::IR::RegisterAllocationData, FEXCore::IR::RegisterAllocationDataDeleter> AllocData;
RegisterSet Set;
std::vector<RegisterNode> Nodes;
uint32_t NodeCount;
std::vector<RegisterNode> Nodes{};
uint32_t NodeCount{};
std::vector<SpillStackUnit> SpillStack;
std::unordered_map<uint32_t, std::unordered_set<uint32_t>> BlockPredecessors;
std::unordered_map<uint32_t, std::unordered_set<uint32_t>> VisitedNodePredecessors;
@@ -257,12 +260,15 @@ namespace {
}
void ResetRegisterGraph(RegisterGraph *Graph, uint64_t NodeCount) {
NodeCount = AlignUp(NodeCount, sizeof(uint64_t));
NodeCount = AlignUp(NodeCount, REGISTER_NODES_PER_PAGE);
// Clear to free the Bucketlists which have unique_ptrs
// Resize to our correct size
Graph->Nodes.clear();
Graph->Nodes.resize(NodeCount);
Graph->VisitedNodePredecessors.clear();
Graph->AllocData.reset();
Graph->AllocData.reset((FEXCore::IR::RegisterAllocationData*)malloc(FEXCore::IR::RegisterAllocationData::Size(NodeCount)));
Graph->AllocData.reset((FEXCore::IR::RegisterAllocationData*)FEXCore::Allocator::malloc(FEXCore::IR::RegisterAllocationData::Size(NodeCount)));
memset(&Graph->AllocData->Map[0], INVALID_REGCLASS.Raw, NodeCount);
Graph->AllocData->MapCount = NodeCount;
Graph->AllocData->IsShared = false; // not shared by default
@@ -445,8 +451,8 @@ namespace FEXCore::IR {
}
void ConstrainedRAPass::AllocateRegisterSet(uint32_t RegisterCount, uint32_t ClassCount) {
LogMan::Throw::A(RegisterCount <= INVALID_REG, "Up to %d regs supported", INVALID_REG);
LogMan::Throw::A(ClassCount <= INVALID_CLASS, "Up to %d classes supported", INVALID_CLASS);
LOGMAN_THROW_A(RegisterCount <= INVALID_REG, "Up to %d regs supported", INVALID_REG);
LOGMAN_THROW_A(ClassCount <= INVALID_CLASS, "Up to %d classes supported", INVALID_CLASS);
Graph = AllocateRegisterGraph(ClassCount);
@@ -459,7 +465,7 @@ namespace FEXCore::IR {
}
void ConstrainedRAPass::AddRegisters(FEXCore::IR::RegisterClassType Class, uint32_t RegisterCount) {
LogMan::Throw::A(RegisterCount <= INVALID_REG, "Up to %d regs supported", INVALID_REG);
LOGMAN_THROW_A(RegisterCount <= INVALID_REG, "Up to %d regs supported", INVALID_REG);
AllocatePhysicalRegisters(Graph, Class, RegisterCount);
}
@@ -486,7 +492,7 @@ namespace FEXCore::IR {
auto Op = IROp->C<IROp_CodeBlock>();
LogMan::Throw::A(Op->Header.Op == OP_CODEBLOCK, "Block not defined by codeblock?");
LOGMAN_THROW_A(Op->Header.Op == OP_CODEBLOCK, "Block not defined by codeblock?");
LiveRange->Begin = std::min(LiveRange->Begin, Op->Begin.ID());
LiveRange->End = std::max(LiveRange->End, Op->Begin.ID());
@@ -514,7 +520,7 @@ namespace FEXCore::IR {
// If the destination hasn't yet been set then set it now
if (IROp->HasDest) {
LogMan::Throw::A(LiveRanges[Node].Begin == ~0U, "Node begin already defined?");
LOGMAN_THROW_A(LiveRanges[Node].Begin == ~0U, "Node begin already defined?");
LiveRanges[Node].Begin = Node;
// Default to ending right where after it starts
LiveRanges[Node].End = Node + 1;
@@ -546,7 +552,7 @@ namespace FEXCore::IR {
if (IR->GetOp<IROp_Header>(IROp->Args[i])->Op == OP_INLINEENTRYPOINTOFFSET) continue;
if (IR->GetOp<IROp_Header>(IROp->Args[i])->Op == OP_IRHEADER) continue;
uint32_t ArgNode = IROp->Args[i].ID();
LogMan::Throw::A(LiveRanges[ArgNode].Begin != ~0U, "%%ssa%d used by %%ssa%d before defined?", ArgNode, Node);
LOGMAN_THROW_A(LiveRanges[ArgNode].Begin != ~0U, "%%ssa%d used by %%ssa%d before defined?", ArgNode, Node);
auto ArgNodeBlockID = Graph->Nodes[ArgNode].Head.BlockID;
if (ArgNodeBlockID == BlockNodeID) {
@@ -600,7 +606,7 @@ namespace FEXCore::IR {
} else if (StaticClass == FPRFixedClass) {
return Size == 16;
} else {
LogMan::Throw::A(false, "Unexpected static class %d", StaticClass);
LOGMAN_THROW_A(false, "Unexpected static class %d", StaticClass);
}
return false; // Unknown
};
@@ -612,7 +618,7 @@ namespace FEXCore::IR {
} else if (StaticClass == FPRFixedClass) {
return (Size == 16 /*|| Size == 8 || Size == 4*/) && ((Offset & 15) == 0); // We need more meta info to support not-size-of-reg
} else {
LogMan::Throw::A(false, "Unexpected static class %d", StaticClass);
LOGMAN_THROW_A(false, "Unexpected static class %d", StaticClass);
}
return false; // Unknown
};
@@ -632,7 +638,7 @@ namespace FEXCore::IR {
auto reg = (Offset - beginFpr) / 16;
return PhysicalRegister(FPRFixedClass, reg);
} else {
LogMan::Throw::A(false, "Unexpected Offset %d", Offset);
LOGMAN_THROW_A(false, "Unexpected Offset %d", Offset);
return INVALID_REGCLASS;
}
};
@@ -656,7 +662,7 @@ namespace FEXCore::IR {
auto reg = (Offset - beginFpr) / 16;
return &StaticMaps[GprSize + reg];
} else {
LogMan::Throw::A(false, "Unexpected offset %d", Offset);
LOGMAN_THROW_A(false, "Unexpected offset %d", Offset);
return (LiveRange**)nullptr;
}
};
@@ -668,7 +674,7 @@ namespace FEXCore::IR {
} else if (PhyReg.Class == FPRFixedClass.Val) {
return &StaticMaps[GprSize + PhyReg.Reg];
} else {
LogMan::Throw::A(false, "Unexpected Class %d", PhyReg.Class);
LOGMAN_THROW_A(false, "Unexpected Class %d", PhyReg.Class);
return (LiveRange**)nullptr;
}
};
@@ -807,7 +813,7 @@ namespace FEXCore::IR {
for (auto [BlockNode, BlockHeader] : IR->GetBlocks()) {
auto BlockIROp = BlockHeader->CW<FEXCore::IR::IROp_CodeBlock>();
LogMan::Throw::A(BlockIROp->Header.Op == IR::OP_CODEBLOCK, "IR type failed to be a code block");
LOGMAN_THROW_A(BlockIROp->Header.Op == IR::OP_CODEBLOCK, "IR type failed to be a code block");
BlockInterferences *BlockInterferenceVector = &LocalBlockInterferences.try_emplace(IR->GetID(BlockNode)).first->second;
BlockInterferenceVector->reserve(BlockIROp->Last.ID() - BlockIROp->Begin.ID());
@@ -910,7 +916,7 @@ namespace FEXCore::IR {
SpanEnd.resize(NodeCount);
for (uint32_t i = 0; i < NodeCount; ++i) {
if (LiveRanges[i].Begin != ~0U) {
LogMan::Throw::A(LiveRanges[i].Begin < LiveRanges[i].End , "Span must Begin before Ending");
LOGMAN_THROW_A(LiveRanges[i].Begin < LiveRanges[i].End , "Span must Begin before Ending");
auto Class = GetClass(Graph->AllocData->Map[i]);
SpanStart[LiveRanges[i].Begin].Append(INFO_MAKE(i, Class));
@@ -938,7 +944,7 @@ namespace FEXCore::IR {
});
}
LogMan::Throw::A(Active.Items[0] == 0, "Interference bug");
LOGMAN_THROW_A(Active.Items[0] == 0, "Interference bug");
SpanStart.clear();
SpanEnd.clear();
}
@@ -957,7 +963,7 @@ namespace FEXCore::IR {
RegisterClass *RAClass = &Graph->Set.Classes[RegClass];
if (CurrentNode->Head.PhiPartner) {
LogMan::Msg::A("Phi nodes not supported");
LOGMAN_MSG_A("Phi nodes not supported");
#if 0
// In the case that we have a list of nodes that need the same register allocated we need to do something special
// We need to gather the data from the forward linked list and make sure they all match the virtual register
@@ -1156,7 +1162,7 @@ namespace FEXCore::IR {
// This would ensure something will spill earlier if its previous use and next use are farther away
auto InterferenceNodeNextUse = FindFirstUse(IREmit, InterferenceOrderedNode, NodeOpBeginIter, InterferenceNodeOpEndIter);
auto InterferenceNodePrevUse = FindLastUseBefore(IREmit, InterferenceOrderedNode, InterferenceNodeOpBeginIter, NodeOpBeginIter);
LogMan::Throw::A(InterferenceNodeNextUse != IR::NodeIterator::Invalid(), "Couldn't find next usage of op");
LOGMAN_THROW_A(InterferenceNodeNextUse != IR::NodeIterator::Invalid(), "Couldn't find next usage of op");
// If there is no use of the interference op prior to our op then it only has initial definition
if (InterferenceNodePrevUse == IR::NodeIterator::Invalid()) InterferenceNodePrevUse = InterferenceNodeOpBeginIter;
@@ -1322,7 +1328,7 @@ namespace FEXCore::IR {
LogMan::Msg::D("\tInt%d: %%ssa%d Remat: %d [%d, %d)", j++, InterferenceNode, InterferenceLiveRange->RematCost, InterferenceLiveRange->Begin, InterferenceLiveRange->End);
});
}
LogMan::Throw::A(InterferenceIdToSpill != 0, "Couldn't find Node to spill");
LOGMAN_THROW_A(InterferenceIdToSpill != 0, "Couldn't find Node to spill");
return InterferenceIdToSpill;
}
@@ -1357,7 +1363,7 @@ namespace FEXCore::IR {
auto LastCursor = IREmit->GetWriteCursor();
auto [CodeNode, IROp] = IR.at(SpillPointId)();
LogMan::Throw::A(IROp->HasDest, "Can't spill with no dest");
LOGMAN_THROW_A(IROp->HasDest, "Can't spill with no dest");
uint32_t Node = IR.GetID(CodeNode);
RegisterNode *CurrentNode = &Graph->Nodes[Node];
@@ -1381,7 +1387,7 @@ namespace FEXCore::IR {
// First op post Spill
auto NextIter = IR.at(CodeNode);
auto FirstUseLocation = FindFirstUse(IREmit, ConstantNode, NextIter, NodeIterator::Invalid());
LogMan::Throw::A(FirstUseLocation != IR::NodeIterator::Invalid(), "At %%ssa%d Spilling Op %%ssa%d but Failure to find op use", Node, InterferenceNode);
LOGMAN_THROW_A(FirstUseLocation != IR::NodeIterator::Invalid(), "At %%ssa%d Spilling Op %%ssa%d but Failure to find op use", Node, InterferenceNode);
if (FirstUseLocation != IR::NodeIterator::Invalid()) {
--FirstUseLocation;
auto [FirstUseOrderedNode, _] = FirstUseLocation();
@@ -1398,11 +1404,13 @@ namespace FEXCore::IR {
if (InterferenceNode != ~0U) {
FEXCore::IR::RegisterClassType InterferenceRegClass = FEXCore::IR::RegisterClassType{Graph->AllocData->Map[InterferenceNode].Class};
uint32_t SpillSlot = FindSpillSlot(InterferenceNode, InterferenceRegClass);
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
RegisterNode *InterferenceRegisterNode = &Graph->Nodes[InterferenceNode];
LogMan::Throw::A(SpillSlot != ~0U, "Interference Node doesn't have a spill slot!");
//LogMan::Throw::A(InterferenceRegisterNode->Head.RegAndClass.Reg != INVALID_REG, "Interference node never assigned a register?");
LogMan::Throw::A(InterferenceRegClass != ~0U, "Interference node never assigned a register class?");
LogMan::Throw::A(InterferenceRegisterNode->Head.PhiPartner == nullptr, "We don't support spilling PHI nodes currently");
LOGMAN_THROW_A(SpillSlot != ~0U, "Interference Node doesn't have a spill slot!");
//LOGMAN_THROW_A(InterferenceRegisterNode->Head.RegAndClass.Reg != INVALID_REG, "Interference node never assigned a register?");
LOGMAN_THROW_A(InterferenceRegClass != ~0U, "Interference node never assigned a register class?");
LOGMAN_THROW_A(InterferenceRegisterNode->Head.PhiPartner == nullptr, "We don't support spilling PHI nodes currently");
#endif
// This is the op that we need to dump
auto [InterferenceOrderedNode, InterferenceIROp] = IR.at(InterferenceNode)();
@@ -1435,7 +1443,7 @@ namespace FEXCore::IR {
++FirstIter;
auto FirstUseLocation = FindFirstUse(IREmit, InterferenceOrderedNode, FirstIter, NodeIterator::Invalid());
LogMan::Throw::A(FirstUseLocation != NodeIterator::Invalid(), "At %%ssa%d Spilling Op %%ssa%d but Failure to find op use", Node, InterferenceNode);
LOGMAN_THROW_A(FirstUseLocation != NodeIterator::Invalid(), "At %%ssa%d Spilling Op %%ssa%d but Failure to find op use", Node, InterferenceNode);
if (FirstUseLocation != IR::NodeIterator::Invalid()) {
// We want to fill just before the first use
--FirstUseLocation;
@@ -1537,7 +1545,7 @@ namespace FEXCore::IR {
return Changed;
}
FEXCore::IR::RegisterAllocationPass* CreateRegisterAllocationPass(FEXCore::IR::Pass* CompactionPass, bool OptimizeSRA) {
return new ConstrainedRAPass{CompactionPass, OptimizeSRA};
std::unique_ptr<FEXCore::IR::RegisterAllocationPass> CreateRegisterAllocationPass(FEXCore::IR::Pass* CompactionPass, bool OptimizeSRA) {
return std::make_unique<ConstrainedRAPass>(CompactionPass, OptimizeSRA);
}
}
@@ -22,7 +22,7 @@ bool IsStaticAllocGpr(uint32_t Offset, RegisterClassType Class) {
if (Offset >= begin && Offset < end) {
auto reg = (Offset - begin) / 8;
LogMan::Throw::A(Class == IR::GPRClass, "unexpected Class %d", Class);
LOGMAN_THROW_A(Class == IR::GPRClass, "unexpected Class %d", Class);
rv = reg < 16; // 0..15 -> 16 in total
}
@@ -37,7 +37,7 @@ bool IsStaticAllocFpr(uint32_t Offset, RegisterClassType Class, bool AllowGpr) {
if (Offset >= begin && Offset < end) {
auto reg = (Offset - begin)/16;
LogMan::Throw::A(Class == IR::FPRClass || (AllowGpr && Class == IR::GPRClass), "unexpected Class %d, AllowGpr %d", Class, AllowGpr);
LOGMAN_THROW_A(Class == IR::FPRClass || (AllowGpr && Class == IR::GPRClass), "unexpected Class %d, AllowGpr %d", Class, AllowGpr);
rv = reg < 16; // 0..15 -> 16 in total
}
@@ -94,8 +94,8 @@ bool StaticRegisterAllocationPass::Run(IREmitter *IREmit) {
return true;
}
FEXCore::IR::Pass* CreateStaticRegisterAllocationPass() {
return new StaticRegisterAllocationPass{};
std::unique_ptr<FEXCore::IR::Pass> CreateStaticRegisterAllocationPass() {
return std::make_unique<StaticRegisterAllocationPass>();
}
}
@@ -44,12 +44,11 @@ bool SyscallOptimization::Run(IREmitter *IREmit) {
}
}
return Changed;
}
FEXCore::IR::Pass* CreateSyscallOptimization() {
return new SyscallOptimization{};
std::unique_ptr<FEXCore::IR::Pass> CreateSyscallOptimization() {
return std::make_unique<SyscallOptimization>();
}
}
Loaded 100 of 353 files, more files were not shown because too many files have changed in this diff. Show more