Compare commits

...
269 Commits
Author SHA1 Message Date
Ryan Houdek cd4586b67d Docs: Update for release FEX-2109 2021-09-05 01:57:02 -07:00
Ryan Houdek 2c02dcac9f Merge pull request #1257 from CallumDev/caspair_fix_armv8
Fix unaligned CASPair on ARMv8.0
2021-09-05 01:35:43 -07:00
CallumDev dec512187e JIT: Remove nops in ARMv8.0 CASPair 2021-09-05 17:49:57 +09:30
Ryan Houdek dd34316562 Merge pull request #1256 from Sonicadvance1/stabilize_fexmountdaemon
FEXMountDaemon: Fixes dangling mounts problem
2021-09-05 01:13:01 -07:00
CallumDev 5f7532c569 Interpreter: Lower 4 byte CASPair to inline assembly 2021-09-05 17:30:18 +09:30
Ryan Houdek 0fa7af15b2 FEXMountDaemon: Fixes dangling mounts problem
The FEXMountDaemon no longer uses the inotify interface for refcounting
instances of FEX.
The inotify interface fails to send close events when an application
crashes. Which is either an API oversight or intentional choice.

Now to use FEXMountDaemon the FEX process must send the daemon a pipe
fd.
The FEXMountDaemon then uses the write end of the pipe to determine if
the read end of the pipe is still open. It does this using the epoll API
and ref counting how many pipes are still active.
This is possible since epoll will tell us if pipe status has changed to
error. Signalling to the write end that the read end has closed for
whatever reason.

Now we only use the "lock" file to remove races and tell the new
instances of FEX where the rootfs is mounted
2021-09-05 00:36:44 -07:00
CallumDev 8654d19f02 JIT: Fix unaligned CASPair on ARMv8.0 2021-09-05 16:54:53 +09:30
Ryan Houdek befd0aa9cf Merge pull request #1255 from Sonicadvance1/syscall_fixes
Syscall fixes
2021-09-04 18:39:33 -07:00
Ryan Houdek 96cef80a25 Linux: Adds missing Namespace handlers
Missed committing this at some point
2021-09-03 17:04:15 -07:00
Ryan Houdek fd3a88389b Linux: Fixes some 32-bit syscalls 2021-09-03 16:53:39 -07:00
Ryan Houdek d8350353d6 Linux: Creates a Types.h header that matches between architectures 2021-09-03 16:48:53 -07:00
Ryan Houdek 083d3a464a StructPackVerifier: Add a couple missing defines 2021-09-03 16:46:16 -07:00
Ryan Houdek 70931bf388 Merge pull request #1250 from Sonicadvance1/fix_signal_nodefer
Linux: Setup signal mask correctly to block signal-in-signal situations
2021-09-03 16:45:39 -07:00
Ryan Houdek 4f93259332 Linux: Setup signal mask correctly to block signal-in-signal situations
Due to how we emulate the guest signal handlers, we do the state setup
in the real host signal handler, then we jump out after state setup.
This was causing a situation where we were setting up the guest signal
handler state with the correct sa_mask.
Then after setting up the guest state we would leave the FEX signal
handler, restoring the signal mask to our original mask.

Instead now as we are setting up the guest state, we save our host
signal mask. Then on signal handler return we modify our host signal
mask to match what the guest wants.

Once we hit our sigreturn emulation we then restore the original signal
mask.

This looks to improve some stability problems regarding how wine uses
signals but it still doesn't fix the gvisor test sadly.
2021-09-02 20:21:10 -07:00
Ryan Houdek ad34cddbf2 Merge pull request #1249 from Sonicadvance1/fix_fexmountdaemon_messages
FEXMountDaemon: Fixes some minor issues
2021-09-02 16:32:41 -07:00
Ryan Houdek c74620f083 Merge pull request #1248 from Sonicadvance1/sigaltstack_ignore_onstack
Linux: sigaltstack ignore SS_ONSTACK
2021-09-02 16:32:24 -07:00
Ryan Houdek eefcde369a FEXMountDaemon: Fixes some minor issues
Now that we deparent FEXMountDaemon we can no longer use
PR_SET_PDEATHSIG.
Instead we rely on the ref counting and checking the pipe status to see
if the original parent has left us.
Further improvements that could be done in the future is that every user
of the mount point talks to the daemon to give it a pipe to check is
still live, since the ref counting sometimes is incorrect.
2021-09-02 15:42:55 -07:00
Ryan Houdek 005818177c unittests: Update posix tests known failures
This test is relying on legacy behaviour which is no longer true.
Keep running it but expect it to fail
2021-09-02 15:23:10 -07:00
Ryan Houdek 2ae47eae48 Linux: sigaltstack ignore SS_ONSTACK
This flag is ignored with sigaltstack.
Fixes an early assert in:
- Splice
- No Time to Explain Remastered
- Ittledew
- Hyperdrive Massacre
- English Country Tune
2021-09-02 15:15:20 -07:00
Ryan Houdek 1c4503e26a Merge pull request #1247 from Sonicadvance1/handle_xmm_state_32bit
Linux: Handle fpstate in the signal delegator correctly
2021-09-02 12:36:02 -07:00
Ryan Houdek 425ee98f81 Merge pull request #1246 from Sonicadvance1/itimer_32_fixes
Linux: Fixes 32-bit interval timers
2021-09-02 12:35:56 -07:00
Ryan Houdek 9945542375 SignalDelegator: Minor fix with SA_NODEFER
if a test application set NODEFER then changed the signal handler to
have one without it then we weren't correctly removing it
2021-09-02 03:31:58 -07:00
Ryan Houdek 0468bb4496 Linux: Handle fpstate in the signal delegator correctly
We were using the glibc context structure layout which doesn't match
what the kernel is doing.

Switches over to allocating fpstate independentally of the ucontext_t,
then pointing to it from uc_mcontext how we're supposed to.

glibc uses the __fpregs_mem region for other purposes.

This also fixes xmm state being stored on the 32-bit side, and also
fixes SIGALRM and SIGVTALRM siginfo overwriting structure data.
On 32-bit we can't just memcpy the siginfo_t over because the sizes
don't match. Throw a message instead.
2021-09-02 03:29:24 -07:00
Ryan Houdek e55e3a58b1 Linux: Fixes 32-bit interval timers
getitimer and setitimer use an itimerval struct with 32-bit members in
it.

Handles this case and now the timers work
2021-09-02 03:26:50 -07:00
Ryan Houdek 25e9585564 Merge pull request #1244 from CallumDev/fix_cas_armv8
Properly implement single CAS on ARMv8.0
2021-09-01 17:37:47 -07:00
CallumDev 9ab294b2f8 Interpreter: Templated AtomicCompareAndSwap 2021-09-01 21:02:07 +09:30
CallumDev d7f4fe7564 Interpreter: Fix x86 build 2021-09-01 19:24:58 +09:30
CallumDev e09219e5e5 Properly implement single CAS on ARMv8.0 2021-09-01 18:13:18 +09:30
Ryan Houdek c645d8683a Merge pull request #1243 from phire/24bit_assert
RA: Add max NoteCount assert
2021-08-31 06:43:13 -07:00
Scott Mansell eb9d3b11f2 RA: Add max NoteCount assert
The chance of hitting this is near zero, but still safer to have
an assert.
2021-09-01 01:32:41 +12:00
Ryan Houdek 7795078f7e Merge pull request #1242 from Sonicadvance1/deparent_fexmountdaemon
FEXMountDaemon: Early fork to deparent child
2021-08-30 23:07:19 -07:00
Ryan Houdek 0b564652d2 FEXMountDaemon: Early fork to deparent child
This allows FEXMountDaemon to remove FEXInterpreter as its parent.
Instead becoming the parent of whatever the current reaper process is.

Do it as early as possible this way FEXInterpreter won't get an
erroneous SIGCHLD.
2021-08-30 22:57:45 -07:00
Ryan Houdek 4f66d3e9dc Merge pull request #1240 from phire/extract_bucketlist
Move BucketList into it's own file
2021-08-30 19:21:12 -07:00
Scott Mansell bd7822bbe9 Move BucketList into it's own file 2021-08-31 14:06:07 +12:00
Ryan Houdek 0c484ac49c Merge pull request #1239 from Sonicadvance1/fix_ioctl_definition
Linux: Fix V3d and VC4 ioctl definitions
2021-08-30 19:02:46 -07:00
Ryan Houdek 5d21a1e6d6 Linux: Fix V3d and VC4 ioctl definitions
Oops, used the wrong definition

Also fix two struct definitions
2021-08-30 18:54:26 -07:00
Ryan Houdek d600b34b8e Merge pull request #1237 from Sonicadvance1/gvisor_fixes
Gvisor fixes
2021-08-30 18:08:47 -07:00
Ryan Houdek 4003ede7ee unittests: gvisor: Remove stale tmp files
Sometimes a test leaves a tmp file in the root of the rootfs.
This should be fixed but to ensure it stops happening, make sure to
delete it
2021-08-30 18:00:10 -07:00
Ryan Houdek 127d7c1a9e Merge pull request #1238 from Sonicadvance1/vc4_v3d_ioctl
Linux: x86: Initial V3D and VC4 ioctl emulation
2021-08-30 17:45:37 -07:00
Ryan Houdek f45de45568 unittests: Updates gvisor lists on changed behaviour 2021-08-30 17:43:12 -07:00
Ryan Houdek 5fdb66249b Linux: x86: Initial V3D and VC4 ioctl emulation
Untested but with how the structs are laid out, it likely just works
2021-08-30 17:35:58 -07:00
Ryan Houdek bb525e2291 unittests: Updates posix tests with expected failures
These are broken due to how sa_mask currently works. Will need to
resolve this.
2021-08-30 03:00:12 -07:00
Ryan Houdek de29c65585 Linux: Be more verbose about VFORK in code
With a comment claiming we don't support it yet since it causes problems
2021-08-30 02:27:54 -07:00
Ryan Houdek ef0f2246ac Linux: Fixes tkill syscall
There is no glibc wrapper for tkill and tgkill requires a tgid.
Kernel will reject us if we tried using -1 or 0 even though that is what
it does internally.
2021-08-30 02:26:46 -07:00
Ryan Houdek 76877e08bc Linux: Add some comments claiming shmctl and msqctl is incorrect on x86
These aren't correctly handled, so they will get garbage data for now
2021-08-30 02:25:48 -07:00
Ryan Houdek d2636f63f2 Linux: Return -EPERM on non-canonical TLS
If it lives outside of the canonical range then immediately reject with
-EPERM. Just like the official kernel.
2021-08-30 02:24:51 -07:00
Ryan Houdek c241c2f5e7 Linux: Report that we don't support RSEQ
This way the gvisor tests skip rather than just breaking
2021-08-30 02:24:19 -07:00
Ryan Houdek 99e24c284c Linux: Let an application correctly reset its signal handlers
If it is going back to SIG_IGN or SIG_DFL then let them unregister.
This is useful for when they are wanting to ignore a signal after a
while.
Or only capture a fault during a time, then afterwards want the
application to crash on error.

Additionally clears up some other minor logic which wasn't being used
anymore
2021-08-30 02:22:52 -07:00
Ryan Houdek bb3bd3faa1 Linux: Register guest signal delegators for all signals including SIGRT32
Had accidentally missed the last one.
2021-08-30 02:21:41 -07:00
Ryan Houdek 6b3a5470c4 Linux: Fixes tracking of /proc/self with dirfd
Previously we would miss this. Causing things like lscpu to leak state.
2021-08-30 02:20:39 -07:00
Ryan Houdek d241c925c0 Linux: Fixes /proc/self/cmdline arguments
With some changes in the frontend this had gotten out of sync.
2021-08-30 02:19:50 -07:00
Ryan Houdek 378dfcf164 Linux: Fixes semctl syscall on AArch64
Turns out the semid_ds struct is a different layout on x86-64 versus
AArch64.
This was causing some minor failures
2021-08-30 02:19:07 -07:00
Ryan Houdek 9fd558e173 Linux: Fixes signalfd
Was using the wrong signal mask for this.
We need to use the one passed in to the syscall, not the current active
mask
2021-08-30 02:16:26 -07:00
Ryan Houdek 3fbc3c347a Linux: Fixes the pread/pwrite family of syscalls
These syscall arguments are not laid out in a sane way.
Looks like they wanted to keep the interface the same for both x86-64
and 32-bit x86 so they split the offset argument in to two values.

Weirdly enough, even though these are 32-bit offsets on 32-bit x86; On
x86-64 these are still 64-bit. Which means the kernel weirdly allows you
to overlap the two 64-bit halves.

eg: `uint64_t Offset = (pos_high << 32) | pos_low;`
So you can have a 64-bit value that is the full range, but if you have
data in the lower 32-bits of pos_high then you corrupt the offset.
Additionally this allows you to interleave low and high if you want to
be obtuse.
2021-08-30 02:12:35 -07:00
Ryan Houdek fb0b03808f Linux: Fixes waitid syscall
The raw syscall has an rusage argument that can give you child process
rusage similar to wait4
2021-08-30 02:11:07 -07:00
Ryan Houdek a1aeb0d7ee Linux: Fixes execve with missing arguments
Most of the arguments to execve can be nullptr.
Fixes this from crashing
2021-08-30 02:09:51 -07:00
Ryan Houdek 4b6b7495df Signals: Fix incorrect SIGINFO check
Initially this was a small hack to work around sigqueueinfo sending over
siginfo_t. Now this is unnecessary and would crash if an application
used sigqueueinfo to a signal without the SIGINFO flag.
2021-08-30 02:06:55 -07:00
Ryan Houdek 29e30773ee Signals: On jumping to signal frame FPU state is reset
All registers are set to zero and setup to be able to do float
operations.
2021-08-30 02:05:56 -07:00
Ryan Houdek 3e05544e78 Merge pull request #1236 from Sonicadvance1/fexconfig_advanced
FEXConfig: Load application config and advanced tab
2021-08-29 13:05:19 -07:00
Ryan Houdek 61260c2a62 FEXConfig: Load application config and advanced tab
Loading a preexisting application config was impossible through the gui.
Adds a way to do so.

Also adds an advanced tab that just displays all the items.
Allows pruning of application config options to be fairly efficient
2021-08-29 12:30:39 -07:00
Ryan Houdek 109c42a629 Merge pull request #1235 from Sonicadvance1/fix_emulated_openat
EmulatedFiles: Fixes openat for emulated files not using FDCWD
2021-08-28 18:09:42 -07:00
Ryan Houdek 9140ba28b7 Merge pull request #1234 from Sonicadvance1/FEXBash_init
FEXBash: Allow creating a bash instance easily
2021-08-28 17:55:33 -07:00
Ryan Houdek 1c3be542f7 Merge pull request #1233 from Sonicadvance1/fix_execve_escape
Linux: Fixes accidental execve escape
2021-08-28 17:55:22 -07:00
Ryan Houdek 8a331202ee EmulatedFiles: Fixes openat for emulated files not using FDCWD
Fixes lscpu and other applications that open the directory first and
then files inside of that directory.
2021-08-28 17:54:05 -07:00
Ryan Houdek f72ecacd86 FEXBash: Allow creating a bash instance easily
If not passing in any arguments then just immediately create a bash
instance. Incredibly nice little helper
2021-08-28 16:54:25 -07:00
Ryan Houdek 057c1de69d Linux: Fixes accidental execve escape
If the binfmt_misc interpreter was installed then we were running execve
directly.
This allowed shebang programs to escape and see the host architecture
when we weren't planning on it.

Fixes `FEXBash steam` from complaining about missing packages.
2021-08-28 16:37:29 -07:00
Ryan Houdek 10793e89a9 Merge pull request #1232 from Sonicadvance1/iwyu_fixes
Massive amount of IWYU cleanup
2021-08-28 09:54:45 -07:00
Ryan Houdek f161e3bfb0 Massive amount of IWYU cleanup
This isn't quite a 100% clean sweep of IWYU.
There are some false positives where clang fails.
Additionally there are still a few missed in the frontend side of things
that I didn't get to
2021-08-28 00:32:15 -07:00
Ryan Houdek c206942b59 Merge pull request #1231 from Sonicadvance1/Arm64Emitter_iwyu
Arm64Emitter: Resolves some IWYU warnings
2021-08-27 17:18:10 -07:00
Ryan Houdek 10922293c9 Merge pull request #1230 from Sonicadvance1/fix_stdio_log
FEXLoader: Fixes potential bug in log output to stdout/stderr
2021-08-27 17:09:04 -07:00
Ryan Houdek 38ce5876f8 Arm64Emitter: Resolves some IWYU warnings
Should help some build errors
2021-08-27 17:08:29 -07:00
Ryan Houdek de64db5852 FEXLoader: Fixes potential bug in log output to stdout/stderr
Comment in the file for why this can be a bug.

Noticed a game opening a file and our logs were ending up in their
files.
2021-08-27 15:23:17 -07:00
Ryan Houdek 097b3ad881 Merge pull request #1229 from Sonicadvance1/more_signal_splitting
SignalDelegator: More splitting and cleanup
2021-08-27 13:57:07 -07:00
Ryan Houdek c25ae9b2c3 Dispatcher: Changes SRA assert in to an error message instead
In some instances this is safe but we can't currently distinguish safe
or not.
2021-08-27 12:58:40 -07:00
Ryan Houdek b625437071 SignalDelegator: More splitting and cleanup
This is working towards getting the stack frame for guest signals being
pushed over to the Frontend.
Still some more work to do but this is the first step that can be split
up.
2021-08-27 12:46:41 -07:00
Ryan Houdek b0c8710b2c X86Enums: Adds some more defines around signals 2021-08-27 12:42:26 -07:00
Ryan Houdek dfce1dc476 SignalDispatcher: Minor fix for SRA
In the case of SRA plus a signal not using siginfo then we were not
spilling SRA registers.
In this case we would then shift the signal frame and fill the guest
context with garbage register data.
This would potentially cause some issues but haven't noticed anything
outside outside of my test applications.
2021-08-27 12:38:19 -07:00
Ryan Houdek 77db25fc24 Merge pull request #1227 from Sonicadvance1/initial_hangover
Hangover: Initial support for the syscall handling.
2021-08-26 01:49:59 -07:00
Ryan Houdek fa224a3557 Merge pull request #1226 from Sonicadvance1/fix_callback
Fixes Callback interface to take a thread argument
2021-08-26 01:41:16 -07:00
Ryan Houdek b373d0fcfe Merge pull request #1225 from Sonicadvance1/fix_libs
Fixes jemalloc library ordering
2021-08-26 01:40:08 -07:00
Ryan Houdek a3490aad5b Hangover: Initial support for the syscall handling.
Hangover currently abuses the syscall op for thunking purposes.
Very likely this will be changed in the future but for now make sure to
support that use case.
2021-08-26 01:34:13 -07:00
Ryan Houdek 304db72d6b Fixes Callback interface to take a thread argument
This was using the implicit thread TLS object. All users of this
have access to the thread object directly.

Use that instead. Fixes a subtle bug were the frontend could be trying
to do a callback and TLS sections weren't correctly set.
2021-08-26 01:23:17 -07:00
Ryan Houdek b15e0c5f6c Fixes jemalloc library ordering
FEXCore relies on jemalloc symbols if compiled with it.
Have FEXCore link to jemalloc instead of the frontend.

Fixes a missing symbol if someone loads libFEXCore

Additionally, stop trying to compile JEMalloc if not enabled
2021-08-26 01:21:56 -07:00
Ryan Houdek 97f413cfec Merge pull request #1224 from Sonicadvance1/expose_parent_thread
FEXCore: Return the ParentThread with InitCore
2021-08-25 22:47:37 -07:00
Ryan Houdek 6da3330646 Merge pull request #1223 from Sonicadvance1/move_config_to_fexcore
Config: Moves non-OS specific configuration loading to FEXCore
2021-08-25 22:47:30 -07:00
Ryan Houdek 5debdf8d57 Merge pull request #1222 from Sonicadvance1/minor_visibility_fixes
FEXCore: Minor symbol visibility fixes
2021-08-25 22:47:23 -07:00
Ryan Houdek 6483740553 FEXCore: Return the ParentThread with InitCore
This always returns success and for easier state management, just return
our parent thread.

Frontend needs full visibility of this state anyway for thread
management.
2021-08-24 23:20:44 -07:00
Ryan Houdek f131f07612 Config: Moves non-OS specific configuration loading to FEXCore
Puts the visibility of the main layer, application layers, and
environment in to FEXCore instead of FEX.
These layers aren't specific to FEX/FEXLoader and should live in
FEXCore.

Only the EmptyMapper remains in FEX, which should eventually move over
to FEXConfig since that is the only user.
2021-08-24 23:17:32 -07:00
Ryan Houdek f268a28caf FEXCore: Minor symbol visibility fixes
Noticed these missing
2021-08-24 22:35:38 -07:00
Ryan Houdek 6afc3ca13a Merge pull request #1219 from Sonicadvance1/fix_large_offset_syscalls
Linux: Fixes 32-bit syscalls that use 64-bit values
2021-08-23 22:42:08 -07:00
Ryan Houdek 35c664295d Merge pull request #1215 from Sonicadvance1/fix_arm64_signal_handling
Arm64: Fixes SRA spilling on signal
2021-08-23 00:47:10 -07:00
Ryan Houdek b7af5c641c Linux: Fixes 32-bit syscalls that use 64-bit values
A bunch of these were defined incorrectly. I tested a few of these
locally to ensure they were correct after the fact.

Shows that most 32-bit applications that we've encountered aren't
dealing with files larger than 4GB.
2021-08-23 00:44:04 -07:00
Ryan Houdek 6b87839437 Merge pull request #1216 from Sonicadvance1/fix_32bit_sigsegv
x86: Fixes siginfo_t si_addr for SIGBUS/SIGSEGV
2021-08-22 22:26:27 -07:00
Ryan Houdek 9563b5aa3a Merge pull request #1218 from Sonicadvance1/hotfix_nasm_fix
unittests: Hotfix for older nasm
2021-08-22 22:26:07 -07:00
Ryan Houdek 28b3bc3508 unittests: Hotfix for older nasm
Newer nasm takes the size specifier for LEA, older ones do not
2021-08-22 21:59:57 -07:00
Ryan Houdek 8099dfc830 Merge pull request #1210 from Sonicadvance1/proton_6.3_fixes
Proton 6.3 32-bit fixes
2021-08-22 21:32:26 -07:00
Ryan Houdek 926ddabbd1 Merge pull request #1211 from Sonicadvance1/implement_repne_strings
OpcodeDispatcher: Implements undocumented repne on string ops
2021-08-22 21:32:17 -07:00
Ryan Houdek af3af9d048 x86: Fixes siginfo_t si_addr for SIGBUS/SIGSEGV
si_addr is set to the address that is trying to be accessed, not the RIP
that is trying to access it.
We just need to copy our host value over for this.
SIGFPE and SIGILL we still don't have a good answer for.
2021-08-22 21:10:51 -07:00
Ryan Houdek 3b8f24d74f Arm64: Fixes SRA spilling on signal
We were checking the PC after we set the new location in the data
structure.
Didn't matter for x86-64 since it doesn't use SRA but it does matter for
ARM.

Now on signal while in JIT code it will spill correctly
2021-08-22 19:14:04 -07:00
Ryan Houdek 8931ddc382 unittests: Duplicates rep string unit tests with repne
These are exactly the same except using the other prefix.
Hardware tests confirm that this behave the same
2021-08-21 20:19:38 -07:00
Ryan Houdek dd8225be8a OpcodeDispatcher: Implements undocumented repne on string ops
MOVS, LODS, and STOS ops are only documented to support the REP prefix.
These instructions actually repeat correctly with the REPNE prefix as
well.
Behaviour is the same.

Fixes POD Gold
2021-08-21 20:16:25 -07:00
Ryan Houdek 02c10b9671 unittests: Adds tests for FXSave/FXRStor 2021-08-21 17:45:52 -07:00
Ryan Houdek 4d7455989c OpcodeDispatcher: Ensure FXSave/FXRStor doesn't store too many XMM registers in 32-bit
32-bit doesn't store XMM registers 8-15 since they don't exist there.
This technically falls under the "reserved" slot so applications can't
rely on them to not be written.
Saves us a bit of CPU overhead at the very least
2021-08-21 17:44:06 -07:00
Ryan Houdek 23fb4baf46 unittests: Adds unit tests for storing segment register sizes 2021-08-21 17:43:39 -07:00
Ryan Houdek 6e5fc5cdde Fixes segment register storing to memory
These were storing to memory as 32-bits but when storing to memory these
are only ever stored as 16-bit
2021-08-21 17:41:57 -07:00
Ryan Houdek 6a08587d2e Merge pull request #1209 from Sonicadvance1/wine_fixes2
32-bit wine fixes
2021-08-21 17:41:16 -07:00
Ryan Houdek 7bfa1c4838 Linux: Fixes 32-bit sigaltstack
Due to an incorrect pointer check, we were never setting the sigaltstack
on 32-bit applications.
Additionally the stack_t type didn't have the members in the correct
order.

This fixes wine 32-bit applications where wine sends an application
SIGUSR1 and expects the altstack to be used. This is because the
altstack has the thread's TEB region at the start of the stack.

Without this when the application was getting sent a SIGUSR1, it would
remain in the application stack, thus getting an invalid TEB and loading
an FS register with zero.
2021-08-20 23:55:56 -07:00
Ryan Houdek 84f42f6155 Dispatcher: Fixes alt stack check and redzone offset
There are two checks to the alt stack that the signal handler needs to
check.
First it needs to check if the signal handler was registered with teh
flag SA_ONSTACK.

Then it needs to check if the alt stack is actually enabled by not
having flag SS_DISABLE.

Additionally, 32-bit x86 doesn't have a redzone so stop offsetting by
128
2021-08-20 23:54:09 -07:00
Ryan Houdek e4a230c3ec OpcodeDispatcher: Fixes FTW saving/storing in FXSave/FXRStor
Missed this when implementing FTW
2021-08-20 23:53:13 -07:00
Ryan Houdek 09296fe73e unittests: Adds IRET unit tests 2021-08-20 23:52:42 -07:00
Ryan Houdek d2130e1df3 OpcodeDispatcher: Fixes 32-bit IRET
We were failing to store the updated ESP on 32-bit.
On 64-bit or CPL change (which we don't support) the stack pointer is
pulled from the frame.

This resolves an issue in wine's 32-bit loader where it was getting zero
for one of the context pointers since the stack pointer wasn't adjusted
2021-08-20 23:50:04 -07:00
Ryan Houdek 01c49dbb9c FEXLoader: Updates RanAsInterpreter check for FD exec
Only the interpreter can run when executed as FD.
This happens when executed with binfmt_misc and can resolve an issue if
someone sets up the hardlink incorrectly.
2021-08-20 23:48:42 -07:00
Ryan Houdek 6d60689ad4 FileManagement: Stop calling getpid for every file access
We can cache the pid result, we know every case in which the pid is
changed.
This makes watching strace significantly less annoying.
2021-08-20 23:47:48 -07:00
Ryan Houdek 63af80fce3 Merge pull request #1205 from Sonicadvance1/fix_jemalloc_missing_alias
Updates jemalloc to fix missing alias posix_memalign
2021-08-15 03:19:31 -07:00
Ryan Houdek 43052a5707 Merge pull request #1204 from Sonicadvance1/pressure_vessel_option_program
Adds new FEXGetConfig program
2021-08-15 03:19:24 -07:00
Ryan Houdek db5a26991f Updates jemalloc to fix missing alias posix_memalign
This was causing us to fail compiling in debug.
xbyak uses this.
2021-08-14 02:59:40 -07:00
Ryan Houdek c07b5e480b Adds new FEXGetConfig program
This is a simple program to get a few configuration options that are necessary
to expose for pressure-vessel
2021-08-13 20:53:30 -07:00
Ryan Houdek 7aae9b7e44 Merge pull request #1201 from Sonicadvance1/pressure-vessel_fixes
Linux: Implements support for clone with namespaces
2021-08-12 12:57:48 -07:00
Ryan Houdek e90892a20b unittests: Disable gvisor test that doesn't pass when namespaces are disabled
It's expecting EPERM (Which is what we used to return) but it is also
valid for the kernel to return EINVAL when not compiled with namespaces enabled
2021-08-10 19:13:32 -07:00
Ryan Houdek a46773a9ff unittests: Disables gvisor test that uses unsupported clone flags
It uses CLONE_VM which breaks our cloning
2021-08-10 18:39:40 -07:00
Ryan Houdek 740270c05f Linux: Implements support for clone with namespaces
This is very tricky to handle and it has a bunch of rough edges.
One of the major problems that we can't workaround is that if we receive a
clone flag that pthreads can't support with THREAD, then we are required to fall down
the pthreads code path.
This is because threads going down the clone path will break TLS and we don't have
a way to work around it currently.

So this adds a clone path, a clone3 path, and keeps the legacy path as well.
Which makes this fairly convoluted but it gets pressure-vessel working on x86-64 host.
It's a bit tricky to setup but it does work.

Still some work necessary to get pressure-vessel working on AArch64 host, but I'm working on that.
2021-08-10 18:31:04 -07:00
Ryan Houdek 83d20d8f34 Allocator: Leak the allocator object until we fix static initializer allocations
When we are running a 32-bit process we end up mixing VMA Region allocators which
can cause crashes on shutdown.
This is because if a statically initialized object allocates memory, and then frees that
memory in the atexit handler. There is a chance that if it had to reallocate memory during the
VMA allocator switch, that the atexit handler will try freeing memory from the FEX VMA region allocator
AFTER it has already been deallocated.

This can't be safely worked around with atexit handlers.
So until we resolve this issue, we HAVE to leak the allocator so it can safely clean up and then let the kernel
clean up after us
2021-08-10 18:23:39 -07:00
Ryan Houdek e388729403 Config: Changes string config default values to be string_view
These were causing static initializer construction all over the codebase.
Scope of these variables has also been changed over to the module instead of the header as well
2021-08-10 18:20:15 -07:00
Ryan Houdek e3fda9f232 Telemetry: Changes telemetry names map to use string_view
These don't need to be std::string and was causing allocations to occur in the static
initializer
2021-08-10 18:19:03 -07:00
Ryan Houdek f5940df822 RAPass: Removes static initialization of INVALID_REGCLASS
We were just using this as a reference and it was causing a static initializer
2021-08-10 18:18:07 -07:00
Ryan Houdek e6c4f9aad1 IRParser: Removes static initialization of map
This didn't need to be global
2021-08-10 18:16:49 -07:00
Ryan Houdek 8eb0df96f9 Merge pull request #1199 from MerryMage/GetCursorAddress
FEXCore: Use GetCursorAddress when able
2021-08-08 13:26:09 -07:00
Merry 55d981fcb0 FEXCore: Use GetCursorAddress when able 2021-08-08 21:06:06 +01:00
Ryan Houdek b120a8ea84 Merge pull request #1196 from Sonicadvance1/offline_telemetry
Implements support for offline *only* telemetry
2021-08-06 23:23:38 -07:00
Ryan Houdek 5d73ac3234 Merge pull request #1198 from Sonicadvance1/fix_binfmt_misc_arch
Arm64: Reimplements support for binfmt_misc without update-binfmts
2021-08-06 23:23:25 -07:00
Ryan Houdek b05adaeba3 Arm64: Reimplements support for binfmt_misc without update-binfmts
Arch doesn't have update-binfmts. Fall back to the classic approach
without it.
2021-08-06 23:14:25 -07:00
Ryan Houdek 0be16baebf Merge pull request #1197 from Sonicadvance1/rebase_skmp/no-sra
Rebase skmp/no sra
2021-08-06 23:09:13 -07:00
Ryan Houdek 8dd41e7fcd Merge pull request #1193 from Sonicadvance1/fix_readlinkat_self
Linux: Fixes readlinkat for self
2021-08-06 22:59:47 -07:00
Ryan Houdek 83cdf0f377 Fix no-sra crash 2021-08-06 22:57:33 -07:00
Stefanos Kornilios Mitsis Poiitidis b010ab42c7 JIT: Add an option to disable SRA 2021-08-06 22:52:58 -07:00
Ryan Houdek 1c1f40e5af Linux: Fixes readlinkat for self
Similar code to readlinkat. Needs to be correct for self otherwise we
return EINVAL which is unexpected since self should always be a symlink

Fixes a bug in running bwrap
2021-08-06 22:51:31 -07:00
Ryan Houdek c6c94570b4 Implements support for offline *only* telemetry
This information is only ever going to be offline. Will be useful for multiple reasons.

1) Searching for split lock usage in applications, which can be a programming bug.
  a) This isn't visible on AMD systems and on Intel is a fairly new linux feature
2) Having more information about when an application breaks.
3) Useful for some minor profiling for devs looking for statistical data
2021-08-06 22:40:19 -07:00
Ryan Houdek cce3f365cc Merge pull request #1194 from Sonicadvance1/implement_pivot_root
Linux: Implements pivot_root syscall
2021-08-06 22:36:30 -07:00
Ryan Houdek 4641e44276 Linux: Implements pivot_root syscall
Somehow missed this one. Easy enough and matches between architectures.
Used by bubblewrap
2021-08-03 23:30:38 -07:00
Ryan Houdek 1f65bf9e47 Docs: Update for release FEX-2108 2021-08-02 23:21:04 -07:00
Ryan Houdek 4c431518b0 Merge pull request #1192 from Sonicadvance1/workaround_static_pie
AArch64: Workaround static-pie crashing
2021-08-02 23:19:45 -07:00
Ryan Houdek b586592643 Make sure our std::filesystem users use the std::error_code versions
Otherwise these crash out.
2021-08-02 23:10:23 -07:00
Ryan Houdek 83bdbc8392 AArch64: Check if we launched with FD argument
This will give us another way to check if the interpreter is installed rather than file checks.
The binfmt_misc files won't exist inside of a rootfs
2021-08-02 22:47:49 -07:00
Ryan Houdek c49fa5a64f Safely check if we can create paths rather than crashing on failure. 2021-08-02 22:47:42 -07:00
Ryan Houdek d052e87d47 AArch64: Workaround static-pie crashing
Static-pie can only work with GNU ld instead of lld on AArch64 hosts.
This is a known problem on the llvm side: https://bugs.llvm.org/show_bug.cgi?id=49672
2021-08-02 22:02:12 -07:00
Ryan Houdek ff98a43f70 Merge pull request #1191 from Sonicadvance1/improve_fexmountdaemon_stability
FEXMountDaemon: Make squashfs mounting more robust
2021-07-30 16:49:20 -07:00
Ryan Houdek a6c4ad889b FEXMountDaemon: Make squashfs mounting more robust
Instead of watching to ensure our parent process is still alive. Mount the
squashfs once and use a combination of file leases and inotify to
ref count how many processes are using the rootfs.

This makes it so in the common case, the FEXMountDaemon only ever executes once
and runs until all FEX processes stop running.
In the rare edge case there is a race condition where multiple FEXMountDaemon
applications will start, but only one will end up mounting the squashfs.
In this case, one application wins and the one that failed to grab the lease
will wait until the other one completes.

With this change, squashfs should be reasonable to use now.
2021-07-29 21:45:30 -07:00
Ryan Houdek 7e8577a50a Merge pull request #1180 from Sonicadvance1/support_setxid
Linux: Implements support for signals 32 and 33
2021-07-28 00:22:30 -07:00
Ryan Houdek a21cead9dc Fixes memcpy of siginfo_t data
_sifields are offset by a uint32_t between a 32-bit and 64-bit arch.

Fixes thread cancelling on 32-bit processes
2021-07-28 00:10:15 -07:00
Ryan Houdek 555cdb558b Linux: Implements support for signals 32 and 33
When a guest tries to use the setxid syscalls, the guest glibc has
a mechanism in place to ensure that the process wide setxid is handled.
The mechanism is fairly complex but it uses signal 33 and sends the signal to all
active threads to ensure every thread sets the correct state here.

We need to intercept this and pass the context information correctly to the guest instead.
Otherwise FEX just crashes when the glibc HOST handler tries handling the guest applications signal.

Fixes the game SOMA https://store.steampowered.com/app/282140/SOMA/
2021-07-28 00:10:08 -07:00
Ryan Houdek ceaaac3b7a Linux: Pass setxid syscalls directly to the kernel
We don't want these to be captured by the host glibc handlers around these
2021-07-27 18:55:30 -07:00
Ryan Houdek de0337db47 Merge pull request #1188 from Sonicadvance1/wine_fixes
Wine fixes
2021-07-27 18:07:53 -07:00
Ryan Houdek a48e41e4a9 Merge pull request #1187 from Sonicadvance1/cleanup_compile_service_on_fork
Cleanup compile service on fork
2021-07-27 18:07:42 -07:00
Ryan Houdek 896aa52851 Merge pull request #1186 from Sonicadvance1/fix_partial_rclse
RCLSE: Fixes an assumption in RCLSE
2021-07-27 18:07:32 -07:00
Ryan Houdek 0315964848 Merge pull request #1185 from Sonicadvance1/allow_stall_config
Adds an option to stall processes on launch
2021-07-27 18:07:19 -07:00
Ryan Houdek dec822192a Merge pull request #1182 from Sonicadvance1/disable_jemalloc_option
Adds an option to disable jemalloc from cmake
2021-07-27 18:07:09 -07:00
Ryan Houdek a5b142cf02 Merge pull request #1178 from Sonicadvance1/implement_vsyscall
Linux: Implements support for vsyscall
2021-07-27 18:06:54 -07:00
Ryan Houdek 007b1cbe9d Merge pull request #1174 from Sonicadvance1/binfmt_support_preserve_and_fd
binfmt_misc: Support loading ELFs from FD and support preserve
2021-07-27 18:06:39 -07:00
Ryan Houdek b0a28e595f Merge pull request #1181 from Sonicadvance1/fix_rapass
Fixes bug in RAPass
2021-07-27 18:06:27 -07:00
Ryan Houdek b8c5cf2736 Adds FXAM unit tests for ABI conformance
Wine was doing a fun x87 stack clean by checking tag bits for empty.
We hadn't previously supported anything but valid. Which cause wine to overflow the stack
trying to save all of the x87 values.

We now support valid and empty well enough that this doesn't break and we can unit test it
2021-07-27 08:25:40 -07:00
Ryan Houdek 2803113e35 Improves 32-bit signal handling
A decent amount of this was broken.
To the point that the ABI was incorrect even, causing 32-bit signals to completely crash in most instances.
This is enough to get my 32-bit signal handler test application to work and basic SIGUSR behaviour
2021-07-27 08:23:25 -07:00
Ryan Houdek 7a84ab8cfc Implements partial support for x87 FTW
This only implements the Empty and Valid tags.
Anything that is not empty is considered valid currently.
2021-07-27 08:22:11 -07:00
Ryan Houdek 77a7ecbae0 Add x87 FTW to CoreState
This will be needed for x87 FTW emulation
2021-07-27 07:57:41 -07:00
Ryan Houdek 2310ea83bb Cleanup compile service on fork
On fork this thread will be gone. Make sure to delete the shared_ptr.
2021-07-27 07:53:02 -07:00
Ryan Houdek 109df27bba Amd64: Very minor optimization in Push/Pop Regs
Just generates a bit nicer code to look at. Doesn't really change much.
2021-07-24 08:45:38 -07:00
Ryan Houdek 2afa14d1f3 ConstProp: Removes VMOV assumption
ConstProp has an assumption that a VMOV of the same size arguments is a no-op.
This isn't quite 100% true as this could be behaving as a zext and the pass can't see through the op correctly.

Something to be said that the vector ops should be looked over to see which op is leaving dirty bits in the upper bits
of the register. But that's a battle for a different time
2021-07-24 08:43:35 -07:00
Ryan Houdek 744b778013 RCLSE: Fixes an assumption in RCLSE
There is an assumption in the RCLSE pass that if a StoreContext + LoadContext pair of the same size
with a vector register that it is safe to remove the LoadContext.
This isn't quite safe since it can't entirely see through the IR to determine if the store op
had already zext the upper bits of the register.

This was causing 8byte loadcontext operations to "load" with garbage data in the upper
bits when it was expecting a zext.

Which granted it is likely a bug that not all vector ops zero their upper bits but that's a battle for a different day.

Additionally:
- Removes a couple dead optimizations in the GPR path that was never hit.
- Cleans up how access types are checked
- Uses ReplaceAllUsesWithRange to reinforce that this is purely a per block optimization still
2021-07-24 08:37:03 -07:00
Ryan Houdek b3965ab9b4 Adds an option to stall processes on launch
For misbehaving applications before we can fully diagnose, allow
a config that hangs a process on load.
2021-07-23 15:18:52 -07:00
Ryan Houdek ec6cc5cb4f Merge pull request #1184 from lioncash/vec-alloc
64BitAllocator: Convert std::vector table into std::array
2021-07-23 14:29:03 -07:00
Ryan Houdek 7931dc2c09 Merge pull request #1183 from lioncash/size
FlexBitSet: Use non-template type parameter for index parameters
2021-07-23 14:17:13 -07:00
Lioncash 22590806a0 64BitAllocator: Convert std::vector table into std::array
This is small enough that it can be converted over to a std::array so
the heap use isn't necessary
2021-07-23 17:12:37 -04:00
Lioncash f9313f4000 FlexBitSet: Use non-template type parameter for index parameters
These parameters are used as indexes into the tracked memory, so sizing
the index variable relative to the type being stored is kind of sketchy.

e.g. If the tracked types were uint8_t for example, we'd still want the
indexing parameters to be regularly sized so that we aren't implicitly
truncating values all the time when passing values to a uint8_t
parameter (and while all usages are currently using uint64_t as type
T, we may as well address this).

While we're at it, we can make both Get() and operator[] const member
functions, since they don't directly modify any underlying data, they
only read it.
2021-07-23 17:07:41 -04:00
Ryan Houdek ddcc18277b Check for valgrind for memory region check
If running under valgrind then ignore memory region check.
Otherwise you'd need to modify source to get past this check
2021-07-22 18:17:16 -07:00
Ryan Houdek 19de33f9ca Adds an option to disable jemalloc from cmake
While not recommended. It is necessary to allow disabling jemalloc if you want to run asan or tsan
2021-07-22 18:16:24 -07:00
Ryan Houdek a9ee4123ef Fixes bug in RAPass
RegisterNode isn't a POD so we can't memset it.
This was overwriting some unique_ptrs which were memory leaking and causing crashes in other PRs
2021-07-22 18:13:44 -07:00
Ryan Houdek 138e88bc80 Linux: Implements support for vsyscall
vsyscall is a legacy interface to expose three syscalls in a faster fashion.
Most applications should have moved to vDSO but this isn't entirely the case.

golang specifically still used vsyscall in the case that vdso isn't available.
Which that isn't completely safe since vsyscall can be completely disabled.
This is fixed in a newer version of their runtime but we must support legacy behaviour.

This makes it so FEX checks if the incoming RIP is a "special" region and adjusts where it
loads instructions from accordingly. Lying to the guest application about the instruction stream
while maintaining the illusion that its RIP is in the correct location.
This was fairly straightforward since we already decoupled the instruction stream from the RIP location.

This fixes golang execution as long as you also disable their preempt signaling.
environment variable to do so is `GODEBUG=asyncpreemptoff=1`
2021-07-22 16:58:17 -07:00
Ryan Houdek 62775161fe Merge pull request #1176 from Sonicadvance1/fix_static_pie_crashing
Works around static-pie crashing
2021-07-20 19:37:56 -07:00
Ryan Houdek 310cfcbcc4 Works around static-pie crashing
static-pie is crashing early due to an issue with pthread symbols being exposed as weak.
For some reason the weak symbols never get resolved and jump to zero or ELF base.

When linking static-pie, force search for the pthread symbols using an undefined glob.

With this, static-pie works as long as you link with a new enough version of lld (version 13.0)
2021-07-20 05:24:57 -07:00
Ryan Houdek 2fd41de5e0 Merge pull request #1153 from Sonicadvance1/split_opdispatcher
OpcodeDispatcher: Split the opcode handling to multiple files
2021-07-20 04:43:53 -07:00
Ryan Houdek c2a97d4fc1 Minor header shuffling to improve compilation time
Shaves a few seconds off compile time
2021-07-20 04:27:28 -07:00
Ryan Houdek ebb21e86b0 OpcodeDispatcher: Split the opcode handling to multiple files
These are some fairly large separate files still.
Don't want these to be too terribly small but helps significantly with compile time when working
in the OpcodeDispatcher
2021-07-20 04:27:28 -07:00
Ryan Houdek d8da4ce2a5 binfmt_misc: Support loading ELFs from FD and support preserve
This lets us support the final two flags in binfmt_misc that we needed.
1) Support open_binary
2) Support preserve

1)
We already supported the credentials flag, which implied open_binary, but we weren't handling half of it.
With the open_binary flag, the Linux kernel passes us the executable in an FD instead of as a pathname.
This can be found inside of auxv on startup, inside of AT_EXECFD.
If AT_EXECFD is available then we prioritize using that instead of the pathname passed in.
This fixes a potential permissions issue where an executable is executed without read permissions.

2)
The preserve flag has the Linux kernel preserve the original argv[0] that was passed to the application.
Prior to supporting this flag, the kernel would provide us with a resolved program path.
This can happen in the instance where something like `blah` resolves to `/usr/loca/bin/blah` which
isn't what the user originally typed.
This works around this problem by handing the interpreter both the resolved path and the original typed path.

Alongside open_binary, we can just use the FD passed in instead of the resolved path, this means
we can just drop the argv[0] for the guest (which is the kernel resolved path) and pass through arguments
unmangled.
We do have to make a minor assumption here that if we are using EXECFD that we assume preserve.
It isn't until kernel v5.12 that we can actually check AT_FLAGS to see if that was true.
2021-07-20 01:35:23 -07:00
Ryan Houdek aa767b3922 Merge pull request #1166 from Sonicadvance1/static_pie_support
Adds cmake option ENABLE_STATIC_PIE
2021-07-19 23:33:46 -07:00
Ryan Houdek 14480f3ee0 Merge pull request #1167 from Sonicadvance1/tsan_problems
Fixes some issues I found when running under tsan and asan
2021-07-19 23:30:25 -07:00
Ryan Houdek 6ac23003e1 Merge pull request #1168 from Sonicadvance1/fix_spill_slot
Arm64: Fixes fill and spill slot offset calculation
2021-07-19 23:29:01 -07:00
Ryan Houdek 04aacea58b Merge pull request #1173 from Sonicadvance1/fix_binfmt_misc_not_usr
Fixes binfmt_misc install when not installed to /usr
2021-07-19 23:26:33 -07:00
Ryan Houdek 3d46c40aaf Fixes binfmt_misc install when not installed to /usr
We had hardcoded paths in our binfmt_misc files. Change it to a generated
file instead.

Additionally for the binfmt_misc install targets, since we aren't installing to the
global binfmt_misc folder, we need to pass in the import directory

Fixes #1169
2021-07-18 13:05:12 -07:00
Ryan Houdek 7052d022c1 Merge pull request #1172 from lioncash/jits
Jits: Migrate logs over to fmt where applicable
2021-07-16 11:10:27 -07:00
Lioncash 9c7ebca235 JIT/x86_64: Migrate logs over to fmt 2021-07-16 13:57:15 -04:00
Lioncash 22656aedb2 JIT/Arm64: Migrate logs over to fmt 2021-07-16 13:57:12 -04:00
Ryan Houdek 80caf9038f Merge pull request #1171 from lioncash/string
General: Remove redundant string constructions in log calls related to IR op name retrieval
2021-07-16 10:09:43 -07:00
Lioncash 7fddac7b22 General: Remove redundant string constructions in log calls
Prior to the introduction of fmt, there wasn't a nice and easy way to
format std::string_view, but now that we have it in place, we can modify
relevant log calls to make use of it and get rid of the need to
construct a string around the view.

While we're in the same area, we can make the rest of the modified file
use fmt where appropriate.
2021-07-16 09:53:13 -04:00
Ryan Houdek 7f5b352453 Merge pull request #1170 from lioncash/dead
Validation Passes: Remove unused variables
2021-07-16 06:13:35 -07:00
Lioncash f8968b8296 IRValidation: Move stringstream int HadError conditonal body
Same behavior, but now we only construct it when we need it.
2021-07-16 08:54:09 -04:00
Lioncash 914861ba42 PhiValidation: Move stringstream int HadError conditonal body
Same behavior, but now we don't construct it unless we need to.
2021-07-16 08:52:51 -04:00
Lioncash 5d4efa8caa ValueDominanceValidation: Remove unused variables
No warnings are ever appended to the warning string stream, and
HadWarning is never set to anything other than false, so we can remove
it to bring it in line with the PhiValidation pass.
2021-07-16 08:50:33 -04:00
Ryan Houdek 0075e4ed19 Arm64: Fixes fill and spill slot offset calculation
+16 doesn't matter anymore. This can cause stack corruption
2021-07-15 06:47:28 -07:00
Scott Mansell 3a0da68b94 Merge pull request #1165 from Sonicadvance1/fix_lock_flags_ops
Fixes flag setting for 8bit and 16bit LOCK ALU ops
2021-07-15 14:00:33 +12:00
Ryan Houdek d0edd04e58 Amd64: Fixes X86Dispatcher allocator
The entire object didn't need to be an Allocator.
Additionally there was an overwrite happening on the Size variable.
This was causing the shutdown to free a size of zero.
2021-07-14 13:25:29 -07:00
Ryan Houdek 4332c22797 Linux: Fixes BRK deallocation
The SyscallHandler takes over ownership of BRK from the ELFLoader.
We were failing to deallocate the full range.
Additionally if the guest changed BRK size to something smaller than initial load then the size would underflow.
2021-07-14 13:23:58 -07:00
Ryan Houdek 5e12c38298 SignalDelegator: Switch Required flag to atomic bool
This is shared between multiple threads but it isn't never changed beyond first set.
Change it to atomic bool with relaxed loads to quiet tsan
2021-07-14 13:22:38 -07:00
Ryan Houdek bf8e28f935 Fixes TLS for softfloat
softfloat uses TLS for setting the rounding mode per thread.
We had never set the `THREAD_LOCAL` environment variable, so this
rounding mode was being set globally rather than per thread.
We are expecting per thread state here.

Noticed this while running through tsan
2021-07-14 13:21:02 -07:00
Ryan Houdek 6b90663be3 Adds cmake option ENABLE_STATIC_PIE
This option does cmake checks to determine if your system can handle static-pie.
With upstream projects static-pie only works if you use the binutils linker.
Using lld doesn't currently work because it defines __rela_iplt_{start,end} symbols.

Our cmake file will now compile a test application and check for these symbols.
Either the symbols will not exist at all or they will exist but be a null address

Once your system passes the checks then it will allow you to enable static-pie
2021-07-14 09:56:47 -07:00
Ryan Houdek c362bd883d Update jemalloc for more symbols 2021-07-14 08:21:29 -07:00
Ryan Houdek 0ade57614a Fixes flag setting for 8bit and 16bit LOCK ALU ops
MUL is handled in a different function, so this isn't needed there anymore.
This was causing a problem where if you force enabled LOCK on all capable ops it was causing a crash.

Issue #1164 is set to track getting unit tests for these.
2021-07-13 01:37:17 -07:00
Ryan Houdek f4e044a53c Merge pull request #1163 from Sonicadvance1/add_named_thunk_configs
Adds a convenience of named thunk configs
2021-07-13 00:26:36 -07:00
Ryan Houdek 108081e2cd Merge pull request #1162 from Sonicadvance1/update_man_page_with_json_key
Update man page to show JSON key for config options
2021-07-13 00:26:31 -07:00
Ryan Houdek 6ff9b9664d Merge pull request #1161 from Sonicadvance1/fix_thunks_finding_gl
Fixes Host thunks lib not being able to GL
2021-07-13 00:26:26 -07:00
Ryan Houdek 0a0d8f8885 Adds a convenience of named thunk configs
If the thunk config is not a path then search for the filename in $XDG_DATA_DIR/.fex-emu/ThunkConfigs/
for the file.

Just makes it easier to use rather than having full paths
2021-07-12 19:30:03 -07:00
Ryan Houdek bc09d73b9d Update man page to show JSON key for config options
Allows users to reference the man page for the json key instead of code diving
2021-07-12 19:22:57 -07:00
Ryan Houdek ed891f09f1 Fixes Host thunks lib not being able to GL 2021-07-12 19:21:58 -07:00
Ryan Houdek a24d63b5f9 Merge pull request #1160 from Sonicadvance1/allocator_fixes
Allocator fixes
2021-07-12 16:38:22 -07:00
Ryan Houdek 9434a59902 Remove some logs that aren't worth seeing anymore 2021-07-12 00:25:09 -07:00
Ryan Houdek 786a341652 BRKHandler: Don't rely on 64Bit Allocator 32-bit hack.
Use the actual 32-bit allocator if we are running a 32-bit process.
This hack in the 64bit allocator is likely to go away in the future
2021-07-12 00:22:59 -07:00
Ryan Houdek 74bff6c3a5 64BitAllocator: Add a backward scan path
First thing, do a backwards scan to try and fill any holes in the allocator
This reduces the number of holes we have in our VA space
2021-07-12 00:22:44 -07:00
Ryan Houdek 8cfda9c86d RAPass: Mostly revert the PR switching this over to mmap.
1) We needed to switch it to the FEXCore allocator so it didn't live in 32-bit space
2) This generates holes in the VA space due to how the allocations line up.
This makes us almost immediate hit the 65k VMA region.
Switching over to jemalloc allocations means we don't hit that.

Not worth creating an mmap based allocator that can work around that.
2021-07-12 00:16:20 -07:00
Ryan Houdek f79c832f88 X86HelperGen: Use System mmap for 32-bit fixed location
This needs to end up 32-bit space which means we shouldn't use the FEXCore Allocator.
FEXCore allocator has a hack in it for fixed offset in 32-bit space, but we shouldn't rely on it
2021-07-12 00:13:32 -07:00
Ryan Houdek 1fa51f7f74 Frontend: Use mmap helper for DecodeBuffer
This ensures that on 32-bit applications we don't consume 416KB per thread inside of 32-bit space
2021-07-12 00:12:30 -07:00
Ryan Houdek 53417e5322 Merge pull request #1158 from Sonicadvance1/atomic_improvements
Atomic improvements
2021-07-11 18:17:16 -07:00
Ryan Houdek 94e38bb556 Arm64: Fixes duplicated symbols problem in the interpreter
Uses local symbol numbers so the compiler doesn't get upset if this is inlined.
2021-07-10 23:51:02 -07:00
Ryan Houdek 59a4481e5b unittests: Adds explicit 16bit, 32bit, 64bit unaligned unit tests 2021-07-10 23:40:18 -07:00
Ryan Houdek 31a672b1d5 unittests: Adds explicit 128bit vector unaligned load test 2021-07-10 23:39:04 -07:00
Ryan Houdek e03ece0ee6 unittests: Adds LOCK NEG unit tests 2021-07-10 23:38:19 -07:00
Ryan Houdek b1151428e8 Arm64: Stop backpatching loadstores in paranoid TSO mode
When paroid tso is enabled, stop backpatching almost entirely.
Only code still backpatching is 128bit loadstores
2021-07-10 23:37:10 -07:00
Ryan Houdek 818808c3e1 Arm64: Implements unaligned atomic loads store handler
In the case that paranoid TSO is enabled, we can force all unaligned loadstores
down this path.

Instead of backpatching, just do the unaligned loadstore in the handler
2021-07-10 23:32:18 -07:00
Ryan Houdek ee40300eb5 OpcodeDispatcher: Implements support for LOCK NEG
Previously this was unimplemented since it isn't really used on x86.
Fixes #844
2021-07-10 23:30:42 -07:00
Ryan Houdek 484d756a26 Arm64: Implements AtomicFetchNeg IR op 2021-07-10 23:29:49 -07:00
Ryan Houdek d0bcdeda6b amd64: Implements AtomicFetchNeg IR op 2021-07-10 23:29:04 -07:00
Ryan Houdek ea218f8758 Interpreter: Implements AtomicFetchNeg
Does a two's complement negation of what is in memory atomically

Requires a loadstore exclusive loop to implement correctly
2021-07-10 23:27:45 -07:00
Ryan Houdek 821634e300 Arm64: Make sure ARMv8.0 Atomic Swap follows our pattern
The most minor of optimizations as well
2021-07-10 23:26:26 -07:00
Ryan Houdek 5df8354339 Arm64: Allow capturing loadstore exclusive pairing for atomic memory ops
This allows us to capture LDAXR* instructions which appear in ARMv8.0 memory ops.
These scan the memory space to see what the loop is doing and then make an atomic decision to handle
the loadstore exclusive loop with our signal handler based CAS atomics.

This will be necessary to support `LOCK NEG` on ARMv8.1+
2021-07-10 23:23:16 -07:00
Ryan Houdek fe10db9396 Adds an assert if we tried restoring a signal without known frames
Noticed this happen while tinkering. Best to get a debug only message in that case.
2021-07-10 23:19:46 -07:00
Ryan Houdek 3b4119dd31 Arm64: Add some helpers to unaligned atomic load
If we want to do atomic memory loads that are unaligned then we need some helpers to do these.

LoadAcquire128 needs to specifically avoid the std::atomic helper because it can try storing to read only memory
It does this to clear the exclusive lock from its ldaxp but we can just use clrex.
2021-07-10 23:18:46 -07:00
Ryan Houdek 60da70b07e ARMv8.0: Adds some NOPs around CASPair loadstore exclusive pair
The unaligned signal handler will backpatch these with DMBs.
Previously generating corrupted code. This will fix that corruption but not the atomicity problem
2021-07-10 23:15:49 -07:00
Ryan Houdek 35a4f52d7e Merge pull request #1156 from Sonicadvance1/actually_fix_jemalloc
Actually fix jemalloc for FEXConfig
2021-07-09 04:58:51 -07:00
Ryan Houdek 78a9dcf505 Actually fix jemalloc for FEXConfig 2021-07-09 03:48:29 -07:00
Ryan Houdek 1eee2dd004 Merge pull request #1155 from Sonicadvance1/improve_startup_time
Switch some memory allocations over to mmap
2021-07-09 03:47:05 -07:00
Ryan Houdek 25866a00ed Disable rtsignal gvisor test
This test is getting frustrating and is becoming MORE flakey as threads become faster
2021-07-09 01:57:48 -07:00
Ryan Houdek a27c05f577 Switch some memory allocations over to mmap
This cuts start up time from 8.962ms to 3.949ms and lets us use less physical memory.
We pay a small amortized cost from the faulting to populate pages which will be fairly low
2021-07-09 01:57:47 -07:00
Ryan Houdek 0d9ab9ca42 Merge pull request #1154 from Sonicadvance1/fix_jemalloc_override
Update jemalloc to fix glibc override.
2021-07-08 14:48:43 -07:00
Ryan Houdek 9c4c9ea566 Update jemalloc to fix glibc override.
For some reason jemalloc was missing an override
2021-07-08 14:33:35 -07:00
Stefanos Kornilios Mitsis Poiitidis a9bb33b8c2 Merge pull request #1152 from Sonicadvance1/fix_jemalloc_malloc_replace
Fix jemalloc malloc replace
2021-07-08 18:13:20 +03:00
Ryan Houdek faf09fec5f Removes glibc symbol replacement inside FEX
FEX_jemalloc now handles this entirely by itself.
It now replaces glibc during startup rather than when we call for it.
2021-07-06 21:52:57 -07:00
Ryan Houdek 4cd2b1a959 Update Externals jemalloc 2021-07-06 21:20:17 -07:00
Stefanos Kornilios Mitsis Poiitidis 8f09bd9215 Merge pull request #1150 from Sonicadvance1/fix_old_kernel_defines
Fixes old kernel defines for sockios
2021-07-06 15:39:54 +03:00
Stefanos Kornilios Mitsis Poiitidis 84dee03a16 Merge pull request #1151 from Sonicadvance1/error_on_immediate_failure
FEXLoader: stderr on immediate failure
2021-07-06 14:36:11 +03:00
Stefanos Kornilios Mitsis Poiitidis 0a8ee43081 Merge pull request #1149 from Sonicadvance1/enable_local_xxhash
Use externals xxhash if not found installed
2021-07-06 14:35:18 +03:00
Ryan Houdek 9fbeba3b79 FEXLoader: stderr on immediate failure
In the case that there is an immediate configuration failure. Use stderr specifically for outputting.
These errors won't be output typically because silent logging is enabled by default.
In the case of executable missing or rootfs configuration failure, print directly to stderr.
Previously it looked like FEX just exited for no reason.
We had multiple users encounter this and be confused
2021-07-05 18:52:06 -07:00
Ryan Houdek b3bab814de Fixes old kernel defines for sockios
These defines were changed at in kernel v5.2 but the debian defines didn't seem to change?
Redefine them if missing
2021-07-05 18:26:44 -07:00
Ryan Houdek 728e9fc187 Use externals xxhash if not found installed
Also in the case that you don't have v0.8.0 minimum installed
2021-07-05 17:42:35 -07:00
Ryan Houdek e01e6d9293 Adds xxhash external submodule 2021-07-05 17:39:19 -07:00
Stefanos Kornilios Mitsis Poiitidis 1a64c908e5 Merge pull request #1146 from Sonicadvance1/implement_sigqueueinfo
Linux: Implements rt_{tg,}sigqueueinfo
2021-07-05 16:55:34 +03:00
Stefanos Kornilios Mitsis Poiitidis 7dd700394e Merge pull request #1147 from Sonicadvance1/fix_imul_flags
OpDispatcher: Fixes imul flags calculations
2021-07-05 16:55:16 +03:00
Ryan Houdek b7b9d4089f unittests: Adds more imul unit tests for flags 2021-07-03 21:48:18 -07:00
Ryan Houdek 5a7dd844b0 OpDispatcher: Fixes imul flags calculations
We were calculating the high bits incorrectly in a couple variants
2021-07-03 21:45:15 -07:00
Ryan Houdek 80da08b15b Fixes Signal handler check for queue signals
Any signals sent with sigqueue or kill/sigsend are safe on the siginfo path.
2021-07-03 17:12:42 -07:00
Ryan Houdek 20a519a250 unittests: Update posix tests that have been fixed 2021-07-02 23:00:48 -07:00
Ryan Houdek b9efbd6159 Linux: Implements rt_{tg,}sigqueueinfo
We already support receiving user signals in our signal handlers for this.
Just need to push through syscalls.

This fixes Dead Island Definitive Edition so it runs.
2021-07-02 22:53:27 -07:00
Ryan Houdek 749f1eba51 Merge pull request #1145 from lioncash/stub
GdbStub: Fix memory leak in GdbServerLoop()
2021-07-02 03:21:00 -07:00
Lioncash bc52a16fde GdbStub: Fix memory leak in GdbServerLoop()
Previously the opened socket stream would be leaked.
2021-07-02 05:53:59 -04:00
Stefanos Kornilios Mitsis Poiitidis 2db44cf2ed Merge pull request #1142 from Sonicadvance1/implement_base_signalfd
Linux: Implements a base implementation of signalfd{4,}
2021-07-02 10:04:26 +03:00
Ryan Houdek 336eb9a8b4 Merge pull request #1143 from Sonicadvance1/remove_fork_log
Linux: Remove logs about parent or child stack usage
2021-07-01 12:46:18 -07:00
Ryan Houdek b923a82c39 unittests: fallocate gvisor test no longer fails 2021-07-01 07:11:00 -07:00
Ryan Houdek 1304279eb5 Linux: Remove logs about parent or child stack usage
We don't need to care about what the guest stacks are doing now
2021-07-01 07:06:14 -07:00
Ryan Houdek 1f6926a245 Linux: Implements a base implementation of signalfd{4,}
This is a base implementation of signalfd.
Signalfd allows the application to receive siginfo_t information through an FD.
The FD is either provided by the application or created by the kernel depending.
This specifically doesn't pick up *true* synchronous signals. tgkill of the number
should theoretically go through this interface.

This very specifically skips our internal required signals for now.
This means it won't pick up SIGILL, SIGBUS, or SIG63.

This is enough to capture an application that just wants to poll for SIGCHLD.
Anything more complex has the same problems of the guest handling a siginfo_t.
2021-07-01 06:51:39 -07:00
Stefanos Kornilios Mitsis Poiitidis 2d11df03cb Merge pull request #1141 from Sonicadvance1/modify_cpack_name
CPack: Update package name to remove conflict
2021-07-01 16:44:44 +03:00
Stefanos Kornilios Mitsis Poiitidis d142d7c363 Merge pull request #1140 from Sonicadvance1/cpuid_tm
CPUID: Update TM and TM2 CPUID bits
2021-07-01 16:44:09 +03:00
Ryan Houdek b546869d9d CPack: Update package name to remove conflict
'fex' is already taken by another projects. Use 'fex-emu' instead; Similar to other emulator package names.
2021-07-01 05:08:57 -07:00
Ryan Houdek 2a49ad941f CPUID: Update TM and TM2 CPUID bits
These are in place to let applications know that we do our own thermal management

TM was introduced with Pentium M, it would dispatch idle uops to counter thermals.
TM2 was later and is what allows the CPU to dynamically change its multiplier and frequency.
All in an effort to not cook themselves.

Our ARM CPUs also do all this. Claim it as well.
2021-07-01 04:26:39 -07:00
288 changed files with 16441 additions and 7227 deletions

No files matched your search

+3
View File
@@ -39,3 +39,6 @@
[submodule "External/drm-headers"]
path = External/drm-headers
url = https://github.com/FEX-Emu/drm-headers.git
[submodule "External/xxhash"]
path = External/xxhash
url = https://github.com/FEX-Emu/xxHash.git
+186 -36
View File
@@ -14,6 +14,9 @@ option(ENABLE_ASSERTIONS "Enables assertions in build" FALSE)
option(ENABLE_VISUAL_DEBUGGER "Enables the visual debugger for compiling" FALSE)
option(ENABLE_STRICT_WERROR "Enables stricter -Werror for CI" FALSE)
option(ENABLE_WERROR "Enables -Werror" FALSE)
option(ENABLE_STATIC_PIE "Enables static-pie build" FALSE)
option(ENABLE_JEMALLOC "Enables jemalloc allocator" TRUE)
option(ENABLE_OFFLINE_TELEMETRY "Enables FEX offline telemetry" TRUE)
set (X86_C_COMPILER "x86_64-linux-gnu-gcc" CACHE STRING "c compiler for compiling x86 guest libs")
set (X86_CXX_COMPILER "x86_64-linux-gnu-g++" CACHE STRING "c++ compiler for compiling x86 guest libs")
@@ -44,38 +47,6 @@ else()
set(CMAKE_INTERPROCEDURAL_OPTIMIZATION FALSE)
endif()
find_program(CCACHE_PROGRAM ccache)
if(CCACHE_PROGRAM)
message(STATUS "CCache enabled")
set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE "${CCACHE_PROGRAM}")
endif()
if (ENABLE_XRAY)
add_compile_options(-fxray-instrument)
link_libraries(-fxray-instrument)
endif()
if (ENABLE_LLD)
link_libraries(-fuse-ld=lld)
endif()
if (ENABLE_ASAN)
add_definitions(-DENABLE_ASAN=1)
add_compile_options(-fno-omit-frame-pointer -fsanitize=address -fsanitize-address-use-after-scope)
link_libraries(-fno-omit-frame-pointer -fsanitize=address -fsanitize-address-use-after-scope)
endif()
if (ENABLE_TSAN)
add_compile_options(-fno-omit-frame-pointer -fsanitize=thread)
link_libraries(-fno-omit-frame-pointer -fsanitize=thread)
endif()
set (CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} -fno-omit-frame-pointer")
set (CMAKE_LINKER_FLAGS_RELWITHDEBINFO "${CMAKE_LINKER_FLAGS_RELWITHDEBINFO} -fno-omit-frame-pointer")
set (CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -fomit-frame-pointer")
set (CMAKE_LINKER_FLAGS_RELEASE "${CMAKE_LINKER_FLAGS_RELEASE} -fomit-frame-pointer")
if (CMAKE_SYSTEM_PROCESSOR MATCHES "x86_64")
option(ENABLE_X86_HOST_DEBUG "Enables compiling on x86_64 host" FALSE)
if (NOT ENABLE_X86_HOST_DEBUG)
@@ -95,6 +66,181 @@ if (CMAKE_SYSTEM_PROCESSOR MATCHES "aarch64")
add_definitions(-D_M_ARM_64=1)
endif()
find_program(CCACHE_PROGRAM ccache)
if(CCACHE_PROGRAM)
message(STATUS "CCache enabled")
set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE "${CCACHE_PROGRAM}")
endif()
if (ENABLE_XRAY)
add_compile_options(-fxray-instrument)
link_libraries(-fxray-instrument)
endif()
set (PTHREAD_LIB pthread)
if (ENABLE_LLD)
set (LD_OVERRIDE "-fuse-ld=lld")
link_libraries(${LD_OVERRIDE})
endif()
if (NOT ENABLE_OFFLINE_TELEMETRY)
# Disable FEX offline telemetry entirely if asked
add_definitions(-DFEX_DISABLE_TELEMETRY=1)
endif()
if (ENABLE_STATIC_PIE)
if (_M_ARM_64 AND ENABLE_LLD)
message (FATAL_ERROR "Static linking does not currently work with AArch64+LLD. Use GNU ld for now.")
endif()
file(WRITE ${PROJECT_BINARY_DIR}/CMakeFiles/CMakeTmp/Determine_iplt.c
"int main(int argc, char* argv[])
{
return 0;
}")
# Compile the test application with our LD_OVERRIDE and static-pie options
try_compile(
COMPILE_RESULT
${PROJECT_BINARY_DIR}/CMakeFiles/CMakeTmp
${PROJECT_BINARY_DIR}/CMakeFiles/CMakeTmp/Determine_iplt.c
COMPILE_DEFINITIONS "-fPIE ${LD_OVERRIDE}"
LINK_LIBRARIES "-static-pie ${LD_OVERRIDE}"
COPY_FILE ${PROJECT_BINARY_DIR}/CMakeFiles/CMakeTmp/Determine_iplt
)
if (${COMPILE_RESULT})
# Read the symbols from the elf
execute_process(COMMAND
readelf -s ${PROJECT_BINARY_DIR}/CMakeFiles/CMakeTmp/Determine_iplt
OUTPUT_FILE ${PROJECT_BINARY_DIR}/CMakeFiles/CMakeTmp/plt_out.txt
OUTPUT_VARIABLE PLT_SYMBOLS)
# Pull out the __rela_iplt_{start,end} symbols if they exist
execute_process(COMMAND
"grep" "__rela_iplt" ${PROJECT_BINARY_DIR}/CMakeFiles/CMakeTmp/plt_out.txt
OUTPUT_VARIABLE PLT_SYMBOLS)
set (SYMBOLS_FINE TRUE)
set (HAS_IPLT -1)
# Check if we have any symbols in our grep output
# The symbols must either not exist at all OR the symbols are zero
if (PLT_SYMBOLS)
string(FIND ${PLT_SYMBOLS} "__rela_iplt_start" HAS_IPLT)
endif()
if (NOT HAS_IPLT EQUAL -1)
# We have some symbols from readelf. Let's parse the results to check if they are zero
# Format: '35: 0000000000000000 0 NOTYPE LOCAL HIDDEN UND __rela_iplt_start'
string(REPLACE "\n" ";" SYMBOL_LIST ${PLT_SYMBOLS})
foreach (SYMBOL ${SYMBOL_LIST})
# strip any leading and trailing whitespace
string (STRIP ${SYMBOL} SYMBOL)
# Convert string to a list
string(REPLACE " " ";" SYMBOL_VALUES ${SYMBOL}})
# Pull out the address argument
list(GET SYMBOL_VALUES 1 OFFSET)
# Check against integer zero
if (NOT ${OFFSET} EQUAL 0)
# Symbol wasn't zero, this now fails
set (SYMBOLS_FINE FALSE)
endif()
endforeach()
endif()
if (SYMBOLS_FINE)
# We can now exnable static-pie
set (STATIC_PIE_OPTIONS "-static-pie")
# Pthreads has an issue with exposing symbols
# We need to make some concessions to the pthread gods
if (ENABLE_LLD)
set (PTHREAD_LIB
-Wl,--undefined-glob=pthread_*
-Wl,--undefined=__cxa_finalize
-Wl,--undefined=_pthread_cleanup_push_defer
-Wl,--undefined=_pthread_cleanup_pop_restore
-Wl,--undefined=__pthread_cleanup_upto
pthread)
else()
set (PTHREAD_LIB
-Wl,--undefined=pthread_join
-Wl,--undefined=pthread_attr_getdetachstate
-Wl,--undefined=pthread_sigmask
-Wl,--undefined=pthread_mutex_lock
-Wl,--undefined=pthread_cond_init
-Wl,--undefined=pthread_attr_init
-Wl,--undefined=pthread_mutex_unlock
-Wl,--undefined=pthread_mutexattr_destroy
-Wl,--undefined=pthread_detach
-Wl,--undefined=pthread_mutex_init
-Wl,--undefined=pthread_getattr_np
-Wl,--undefined=pthread_cond_timedwait
-Wl,--undefined=pthread_attr_destroy
-Wl,--undefined=pthread_mutexattr_settype
-Wl,--undefined=pthread_rwlock_unlock
-Wl,--undefined=pthread_rwlock_wrlock
-Wl,--undefined=pthread_setspecific
-Wl,--undefined=pthread_create
-Wl,--undefined=pthread_cond_clockwait
-Wl,--undefined=pthread_key_create
-Wl,--undefined=pthread_rwlock_rdlock
-Wl,--undefined=pthread_setname_np
-Wl,--undefined=pthread_cond_signal
-Wl,--undefined=pthread_mutexattr_init
-Wl,--undefined=pthread_attr_setstack
-Wl,--undefined=pthread_self
-Wl,--undefined=pthread_getaffinity_np
-Wl,--undefined=pthread_cond_wait
-Wl,--undefined=pthread_mutex_trylock
-Wl,--undefined=pthread_cond_broadcast
-Wl,--undefined=pthread_cond_destroy
-Wl,--undefined=pthread_getspecific
-Wl,--undefined=pthread_key_delete
-Wl,--undefined=pthread_once
-Wl,--undefined=__cxa_finalize
-Wl,--undefined=_pthread_cleanup_push_defer
-Wl,--undefined=_pthread_cleanup_pop_restore
-Wl,--undefined=__pthread_cleanup_upto
pthread)
endif()
else()
message (FATAL_ERROR "Application has __rela_iplt_{start,end} symbols. Which means static-pie can't be enabled")
endif()
else()
message (FATAL_ERROR "Couldn't compile static-pie test. Static-pie can't be enabled!")
endif()
endif()
if (ENABLE_ASAN)
add_definitions(-DENABLE_ASAN=1)
add_compile_options(-fno-omit-frame-pointer -fsanitize=address -fsanitize-address-use-after-scope)
link_libraries(-fno-omit-frame-pointer -fsanitize=address -fsanitize-address-use-after-scope)
endif()
if (ENABLE_TSAN)
add_compile_options(-fno-omit-frame-pointer -fsanitize=thread)
link_libraries(-fno-omit-frame-pointer -fsanitize=thread)
endif()
if (ENABLE_JEMALLOC)
add_definitions(-DENABLE_JEMALLOC=1)
add_subdirectory(External/jemalloc/)
include_directories(External/jemalloc/pregen/include/)
else()
message (STATUS
" jemalloc disabled!\n"
" This is not a recommended configuration!\n"
" This will very explicitly break 32-bit application execution!\n"
" Use at your own risk!")
endif()
set (CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} -fno-omit-frame-pointer")
set (CMAKE_LINKER_FLAGS_RELWITHDEBINFO "${CMAKE_LINKER_FLAGS_RELWITHDEBINFO} -fno-omit-frame-pointer")
set (CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -fomit-frame-pointer")
set (CMAKE_LINKER_FLAGS_RELEASE "${CMAKE_LINKER_FLAGS_RELEASE} -fomit-frame-pointer")
add_subdirectory(External/vixl/)
include_directories(External/vixl/src/)
@@ -105,14 +251,17 @@ endif()
find_package(PkgConfig REQUIRED)
find_package(Python 3.0 REQUIRED COMPONENTS Interpreter)
pkg_check_modules(XXHASH libxxhash REQUIRED)
pkg_check_modules(XXHASH libxxhash>=0.8.0 QUIET)
if (NOT XXHASH_FOUND)
message(STATUS "xxHash not found. Using Externals")
add_subdirectory(External/xxhash/)
include_directories(External/xxhash/)
endif()
add_definitions(-Wno-trigraphs)
add_definitions(-DGLOBAL_DATA_DIRECTORY="${DATA_DIRECTORY}/")
add_subdirectory(External/jemalloc/)
include_directories(External/jemalloc/pregen/include/)
add_subdirectory(External/cpp-optparse/)
include_directories(External/cpp-optparse/)
@@ -356,6 +505,7 @@ endif()
# Package creation
set (CPACK_GENERATOR "DEB")
set (CPACK_PACKAGE_NAME fex-emu)
set (CPACK_PACKAGE_CONTACT "team@fex-emu.org")
set (CPACK_PACKAGE_VERSION_MAJOR "${FEX_VERSION_MAJOR}")
set (CPACK_PACKAGE_VERSION_MINOR "${FEX_VERSION_MINOR}")
+5
View File
@@ -0,0 +1,5 @@
{
"Config": {
"StallProcess": "1"
}
}
+17 -4
View File
@@ -1,4 +1,17 @@
install(FILES FEX-x86
DESTINATION ${CMAKE_INSTALL_PREFIX}/share/binfmts/)
install(FILES FEX-x86_64
DESTINATION ${CMAKE_INSTALL_PREFIX}/share/binfmts/)
function(GenBinFmt Name)
# Get the filename only component
get_filename_component(FMT_NAME ${Name} NAME_WE)
# Configure it
configure_file(
${Name}
${CMAKE_BINARY_DIR}/Data/binfmts/${FMT_NAME})
# Then install the configured binfmt
install(
FILES ${CMAKE_BINARY_DIR}/Data/binfmts/${FMT_NAME}
DESTINATION ${CMAKE_INSTALL_PREFIX}/share/binfmts/)
endfunction()
GenBinFmt(FEX-x86.in)
GenBinFmt(FEX-x86_64.in)
@@ -1,9 +1,8 @@
package fex
interpreter /usr/bin/FEXInterpreter
interpreter @CMAKE_INSTALL_PREFIX@/bin/FEXInterpreter
magic \x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x03\x00
offset 0
mask \xff\xff\xff\xff\xff\xfe\xfe\x00\x00\x00\x00\xff\xff\xff\xff\xff\xfe\xff\xff\xff
credentials yes
fix_binary yes
preserve no
preserve yes
@@ -1,8 +1,8 @@
package fex
interpreter /usr/bin/FEXInterpreter
interpreter @CMAKE_INSTALL_PREFIX@/bin/FEXInterpreter
magic \x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x3e\x00
offset 0
mask \xff\xff\xff\xff\xff\xfe\xfe\x00\x00\x00\x00\xff\xff\xff\xff\xff\xfe\xff\xff\xff
credentials yes
fix_binary yes
preserve no
preserve yes
+13 -7
View File
@@ -98,14 +98,19 @@ def print_man_option(short, long, desc, default):
output_man.write("\\fBdefault:\\fR {0}\n".format(default))
output_man.write(".Pp\n\n")
def print_man_env_option(name, desc, default):
output_man.write("\\fBFEX_{0}\\fR\n".format(name))
def print_man_env_option(name, desc, default, no_json_key):
output_man.write("\\fBFEX_{0}\\fR\n".format(name.upper()))
# Print description
for line in desc:
output_man.write(".Pp\n")
output_man.write("{0}\n".format(line))
if (not no_json_key):
output_man.write(".Pp\n")
output_man.write("\\fBJSON key:\\fR '{0}'\n".format(name))
output_man.write(".Pp\n\n")
output_man.write(".Pp\n")
output_man.write("\\fBdefault:\\fR {0}\n".format(default))
output_man.write(".Pp\n\n")
@@ -154,9 +159,10 @@ def print_man_environment(options):
# Wrap the string argument in quotes
default = "'" + default + "'"
print_man_env_option(
op_key.upper(),
op_key,
op_vals["Desc"],
default
default,
False
)
print_man_environment_tail()
@@ -172,7 +178,7 @@ def print_man_environment_tail():
"By default FEX will look in {$HOME, $XDG_CONFIG_HOME}/.fex-emu/",
"This will override the full path",
],
"''")
"''", True)
print_man_env_option(
"FEX_APP_CONFIG",
@@ -183,7 +189,7 @@ def print_man_environment_tail():
"One must be careful with this option as it will override any applications that load with execve as well"
"If you need to support applications that execve then use FEX_APP_CONFIG_LOCATION instead"
],
"''")
"''", True)
print_man_env_option(
"FEX_APP_DATA_LOCATION",
@@ -193,7 +199,7 @@ def print_man_environment_tail():
"This will override the full path",
"This is the folder where FEX stores generated files like IR cache"
],
"''")
"''", True)
def print_man_header():
header ='''.Dd {0}
+14 -3
View File
@@ -81,7 +81,12 @@ set (SRCS
Interface/Core/Frontend.cpp
Interface/Core/GdbServer.cpp
Interface/Core/HostFeatures.cpp
Interface/Core/OpcodeDispatcher/Crypto.cpp
Interface/Core/OpcodeDispatcher/Flags.cpp
Interface/Core/OpcodeDispatcher/Vector.cpp
Interface/Core/OpcodeDispatcher/X87.cpp
Interface/Core/OpcodeDispatcher.cpp
Interface/Core/SignalDelegator.cpp
Interface/Core/X86Tables.cpp
Interface/Core/X86DebugInfo.cpp
Interface/Core/X86HelperGen.cpp
@@ -125,6 +130,7 @@ set (SRCS
Utils/Allocator.cpp
Utils/Allocator/64BitAllocator.cpp
Utils/LogManager.cpp
Utils/Telemetry.cpp
Utils/Threads.cpp
)
@@ -133,7 +139,7 @@ if(_M_ARM_64)
Interface/Core/ArchHelpers/Arm64.cpp)
endif()
set(DEFINES )
set(DEFINES -DTHREAD_LOCAL=_Thread_local)
if (_M_X86_64)
list(APPEND DEFINES -D_M_X86_64=1)
@@ -179,6 +185,11 @@ if (ENABLE_JITSYMBOLS)
list(APPEND DEFINES -DENABLE_JITSYMBOLS=1)
endif()
set (LIBS vixl dl fmt::fmt xxhash tiny-json)
if (ENABLE_JEMALLOC)
list (APPEND LIBS FEX_jemalloc)
endif()
# Generate IR include file
set(OUTPUT_IR_FOLDER "${CMAKE_BINARY_DIR}/include/FEXCore/IR")
set(OUTPUT_NAME "${OUTPUT_IR_FOLDER}/IRDefines.inc")
@@ -265,7 +276,7 @@ function(AddObject Name Type)
add_dependencies(${Name} IR_INC)
add_dependencies(${Name} CONFIG_INC)
target_link_libraries(${Name} pthread vixl dl fmt::fmt xxhash FEX_jemalloc)
target_link_libraries(${Name} ${LIBS})
set_target_properties(${Name} PROPERTIES OUTPUT_NAME FEXCore)
set_target_properties(${Name} PROPERTIES C_VISIBILITY_PRESET hidden)
set_target_properties(${Name} PROPERTIES CXX_VISIBILITY_PRESET hidden)
@@ -306,7 +317,7 @@ endfunction()
function(AddLibrary Name Type)
add_library(${Name} ${Type} $<TARGET_OBJECTS:${PROJECT_NAME}_object>)
target_link_libraries(${Name} pthread vixl dl fmt::fmt xxhash FEX_jemalloc)
target_link_libraries(${Name} ${LIBS})
set_target_properties(${Name} PROPERTIES OUTPUT_NAME FEXCore)
set_target_properties(${Name} PROPERTIES C_VISIBILITY_PRESET hidden)
set_target_properties(${Name} PROPERTIES CXX_VISIBILITY_PRESET hidden)
+1 -5
View File
@@ -1,11 +1,7 @@
#include "NetStream.h"
#include <cstring>
#include <sys/types.h>
#include <sys/socket.h>
#include <stdio.h>
#include <unistd.h>
int NetStream::NetBuf::flushBuffer(const char *buffer, size_t size) {
@@ -29,7 +25,7 @@ std::streamsize NetStream::NetBuf::xsputn(const char* buffer, std::streamsize si
// Check if the string fits neatly in our buffer
if (size <= buf_remaining) {
std::memcpy(pptr(), buffer, size);
::memcpy(pptr(), buffer, size);
pbump(size);
return size;
}
+1
View File
@@ -2,6 +2,7 @@
#include <array>
#include <iostream>
#include <iterator>
#include <string.h>
class NetStream : public std::iostream {
+36 -3
View File
@@ -3,12 +3,44 @@
#include <cstdlib>
#include <filesystem>
#include <sys/stat.h>
#include <memory>
#include <pwd.h>
#include <system_error>
#include <unistd.h>
namespace FEXCore::Paths {
std::unique_ptr<std::string> CachePath;
std::unique_ptr<std::string> EntryCache;
char const* FindUserHomeThroughUID() {
auto passwd = getpwuid(geteuid());
if (passwd) {
return passwd->pw_dir;
}
return nullptr;
}
const char *GetHomeDirectory() {
char const *HomeDir = getenv("HOME");
// Try to get home directory from uid
if (!HomeDir) {
HomeDir = FindUserHomeThroughUID();
}
// try the PWD
if (!HomeDir) {
HomeDir = getenv("PWD");
}
// Still doesn't exit? You get local
if (!HomeDir) {
HomeDir = ".";
}
return HomeDir;
}
void InitializePaths() {
CachePath = std::make_unique<std::string>();
EntryCache = std::make_unique<std::string>();
@@ -36,9 +68,10 @@ namespace FEXCore::Paths {
*CachePath += "/.fex-emu/";
*EntryCache = *CachePath + "/EntryCache/";
std::error_code ec{};
// Ensure the folder structure is created for our Data
if (!std::filesystem::exists(*EntryCache) &&
!std::filesystem::create_directories(*EntryCache)) {
if (!std::filesystem::exists(*EntryCache, ec) &&
!std::filesystem::create_directories(*EntryCache, ec)) {
LogMan::Msg::D("Couldn't create EntryCache directory: '%s'", EntryCache->c_str());
}
}
+3
View File
@@ -4,6 +4,9 @@
namespace FEXCore::Paths {
void InitializePaths();
void ShutdownPaths();
const char *GetHomeDirectory();
std::string GetCachePath();
std::string GetEntryCachePath();
}
+319 -51
View File
@@ -1,43 +1,164 @@
#include "Common/StringConv.h"
#include <FEXCore/Utils/LogManager.h>
#include "Interface/Context/Context.h"
#include "Common/Paths.h"
#include <FEXCore/Config/Config.h>
#include <FEXCore/Utils/LogManager.h>
#include <array>
#include <assert.h>
#include <cstdlib>
#include <filesystem>
#include <pwd.h>
#include <fstream>
#include <functional>
#include <map>
#include <memory>
#include <list>
#include <optional>
#include <stddef.h>
#include <stdint.h>
#include <string>
#include <string_view>
#include <sys/sysinfo.h>
#include <unistd.h>
#include <system_error>
#include <type_traits>
#include <unordered_map>
#include <utility>
#include <vector>
#include <tiny-json.h>
namespace FEXCore::Context {
struct Context;
}
namespace FEXCore::Config {
char const* FindUserHomeThroughUID() {
auto passwd = getpwuid(geteuid());
if (passwd) {
return passwd->pw_dir;
namespace DefaultValues {
#define P(x) x
#define OPT_BASE(type, group, enum, json, default) const P(type) P(enum) = P(default);
#define OPT_STR(group, enum, json, default) const std::string_view P(enum) = P(default);
#define OPT_STRARRAY(group, enum, json, default) OPT_STR(group, enum, json, default)
#include <FEXCore/Config/ConfigValues.inl>
}
namespace JSON {
static bool LoadConfigFile(std::vector<char> &Data, const std::string &Config) {
std::fstream ConfigFile;
ConfigFile.open(Config, std::ios::in);
if (!ConfigFile.is_open()) {
return false;
}
return nullptr;
if (!ConfigFile.seekg(0, std::fstream::end)) {
LogMan::Msg::D("Couldn't load configuration file: Seek end");
return false;
}
auto FileSize = ConfigFile.tellg();
if (ConfigFile.fail()) {
LogMan::Msg::D("Couldn't load configuration file: tellg");
return false;
}
if (!ConfigFile.seekg(0, std::fstream::beg)) {
LogMan::Msg::D("Couldn't load configuration file: Seek beginning");
return false;
}
if (FileSize > 0) {
Data.resize(FileSize);
if (!ConfigFile.read(&Data.at(0), FileSize)) {
// Probably means permissions aren't set. Just early exit
return false;
}
ConfigFile.close();
}
else {
return false;
}
return true;
}
const char *GetHomeDirectory() {
char const *HomeDir = getenv("HOME");
struct JsonAllocator {
jsonPool_t PoolObject;
std::unique_ptr<std::list<json_t>> json_objects;
};
static_assert(offsetof(JsonAllocator, PoolObject) == 0, "This needs to be at offset zero");
// Try to get home directory from uid
if (!HomeDir) {
HomeDir = FindUserHomeThroughUID();
json_t* PoolInit(jsonPool_t* Pool) {
JsonAllocator* alloc = reinterpret_cast<JsonAllocator*>(Pool);
alloc->json_objects = std::make_unique<std::list<json_t>>();
return &*alloc->json_objects->emplace(alloc->json_objects->end());
}
json_t* PoolAlloc(jsonPool_t* Pool) {
JsonAllocator* alloc = reinterpret_cast<JsonAllocator*>(Pool);
return &*alloc->json_objects->emplace(alloc->json_objects->end());
}
static void LoadJSonConfig(const std::string &Config, std::function<void(const char *Name, const char *ConfigSring)> Func) {
std::vector<char> Data;
if (!LoadConfigFile(Data, Config)) {
return;
}
// try the PWD
if (!HomeDir) {
HomeDir = getenv("PWD");
JsonAllocator Pool {
.PoolObject = {
.init = PoolInit,
.alloc = PoolAlloc,
},
};
json_t const *json = json_createWithPool(&Data.at(0), &Pool.PoolObject);
if (!json) {
LogMan::Msg::E("Couldn't create json");
return;
}
// Still doesn't exit? You get local
if (!HomeDir) {
HomeDir = ".";
json_t const* ConfigList = json_getProperty(json, "Config");
if (!ConfigList) {
LogMan::Msg::E("Couldn't get config list");
return;
}
return HomeDir;
for (json_t const* ConfigItem = json_getChild(ConfigList);
ConfigItem != nullptr;
ConfigItem = json_getSibling(ConfigItem)) {
const char* ConfigName = json_getName(ConfigItem);
const char* ConfigString = json_getValue(ConfigItem);
if (!ConfigName) {
LogMan::Msg::E("Couldn't get config name");
return;
}
if (!ConfigString) {
LogMan::Msg::E("Couldn't get ConfigString for '%s'", ConfigName);
return;
}
Func(ConfigName, ConfigString);
}
}
}
std::string GetDataDirectory() {
std::string DataDir{};
char const *HomeDir = Paths::GetHomeDirectory();
char const *DataXDG = getenv("XDG_DATA_HOME");
char const *DataOverride = getenv("FEX_APP_DATA_LOCATION");
if (DataOverride) {
// Data override will override the complete directory
DataDir = DataOverride;
}
else {
DataDir = DataXDG ?: HomeDir;
DataDir += "/.fex-emu/";
}
return DataDir;
}
std::string GetConfigDirectory(bool Global) {
@@ -46,7 +167,7 @@ namespace FEXCore::Config {
ConfigDir = GLOBAL_DATA_DIRECTORY;
}
else {
char const *HomeDir = GetHomeDirectory();
char const *HomeDir = Paths::GetHomeDirectory();
char const *ConfigXDG = getenv("XDG_CONFIG_HOME");
char const *ConfigOverride = getenv("FEX_APP_CONFIG_LOCATION");
if (ConfigOverride) {
@@ -59,9 +180,9 @@ namespace FEXCore::Config {
}
// Ensure the folder structure is created for our configuration
if (!std::filesystem::exists(ConfigDir) &&
!std::filesystem::create_directories(ConfigDir)) {
LogMan::Msg::D("Couldn't create config directory: '%s'", ConfigDir.c_str());
std::error_code ec{};
if (!std::filesystem::exists(ConfigDir, ec) &&
!std::filesystem::create_directories(ConfigDir, ec)) {
// Let's go local in this case
return "./";
}
@@ -85,9 +206,11 @@ namespace FEXCore::Config {
std::string GetApplicationConfig(const std::string &Filename, bool Global) {
std::string ConfigFile = GetConfigDirectory(Global);
std::error_code ec{};
if (!Global &&
!std::filesystem::exists(ConfigFile) &&
!std::filesystem::create_directories(ConfigFile)) {
!std::filesystem::exists(ConfigFile, ec) &&
!std::filesystem::create_directories(ConfigFile, ec)) {
LogMan::Msg::D("Couldn't create config directory: '%s'", ConfigFile.c_str());
// Let's go local in this case
return "./" + Filename + ".json";
@@ -97,9 +220,8 @@ namespace FEXCore::Config {
// Attempt to create the local folder if it doesn't exist
if (!Global &&
!std::filesystem::exists(ConfigFile) &&
!std::filesystem::create_directories(ConfigFile)) {
LogMan::Msg::D("Couldn't create AppConfig directory: '%s'", ConfigFile.c_str());
!std::filesystem::exists(ConfigFile, ec) &&
!std::filesystem::create_directories(ConfigFile, ec)) {
// Let's go local in this case
return "./" + Filename + ".json";
}
@@ -108,23 +230,6 @@ namespace FEXCore::Config {
return ConfigFile;
}
std::string GetDataDirectory() {
std::string DataDir{};
char const *HomeDir = GetHomeDirectory();
char const *DataXDG = getenv("XDG_DATA_HOME");
char const *DataOverride = getenv("FEX_APP_DATA_LOCATION");
if (DataOverride) {
// Data override will override the complete directory
DataDir = DataOverride;
}
else {
DataDir = DataXDG ?: HomeDir;
DataDir += "/.fex-emu/";
}
return DataDir;
}
void SetConfig(FEXCore::Context::Context *CTX, ConfigOption Option, uint64_t Config) {
}
@@ -273,7 +378,8 @@ namespace FEXCore::Config {
Path = std::filesystem::absolute(Path);
// Only return if it exists
if (std::filesystem::exists(Path)) {
std::error_code ec{};
if (std::filesystem::exists(Path, ec)) {
return Path;
}
}
@@ -309,7 +415,8 @@ namespace FEXCore::Config {
else if (!PathName().empty()) {
// If the filesystem doesn't exist then let's see if it exists in the fex-emu folder
std::string NamedRootFS = GetDataDirectory() + "RootFS/" + PathName();
if (std::filesystem::exists(NamedRootFS)) {
std::error_code ec{};
if (std::filesystem::exists(NamedRootFS, ec)) {
FEXCore::Config::EraseSet(FEXCore::Config::CONFIG_ROOTFS, NamedRootFS);
}
}
@@ -324,7 +431,19 @@ namespace FEXCore::Config {
}
if (FEXCore::Config::Exists(FEXCore::Config::CONFIG_THUNKCONFIG)) {
FEX_CONFIG_OPT(PathName, THUNKCONFIG);
ExpandPathIfExists(FEXCore::Config::CONFIG_THUNKCONFIG, PathName());
auto ExpandedString = ExpandPath(PathName());
if (!ExpandedString.empty()) {
// Adjust the path if it ended up being relative
FEXCore::Config::EraseSet(FEXCore::Config::CONFIG_THUNKCONFIG, ExpandedString);
}
else if (!PathName().empty()) {
// If the filesystem doesn't exist then let's see if it exists in the fex-emu folder
std::string NamedConfig = GetDataDirectory() + "ThunkConfigs/" + PathName();
std::error_code ec{};
if (std::filesystem::exists(NamedConfig, ec)) {
FEXCore::Config::EraseSet(FEXCore::Config::CONFIG_THUNKCONFIG, NamedConfig);
}
}
}
if (FEXCore::Config::Exists(FEXCore::Config::CONFIG_OUTPUTLOG)) {
FEX_CONFIG_OPT(PathName, OUTPUTLOG);
@@ -402,6 +521,17 @@ namespace FEXCore::Config {
}
}
template<>
std::string Value<std::string>::GetIfExists(FEXCore::Config::ConfigOption Option, std::string_view Default) {
auto Value = FEXCore::Config::Get(Option);
if (Value) {
return **Value;
}
else {
return std::string(Default);
}
}
template bool Value<bool>::GetIfExists(FEXCore::Config::ConfigOption Option, bool Default);
template int8_t Value<int8_t>::GetIfExists(FEXCore::Config::ConfigOption Option, int8_t Default);
template uint8_t Value<uint8_t>::GetIfExists(FEXCore::Config::ConfigOption Option, uint8_t Default);
@@ -427,5 +557,143 @@ namespace FEXCore::Config {
}
}
template void Value<std::string>::GetListIfExists(FEXCore::Config::ConfigOption Option, std::list<std::string> *List);
// Application loaders
class MainLoader final : public FEXCore::Config::OptionMapper {
public:
explicit MainLoader();
explicit MainLoader(std::string ConfigFile);
void Load() override;
private:
std::string Config;
};
class AppLoader final : public FEXCore::Config::OptionMapper {
public:
explicit AppLoader(const std::string& Filename, bool Global);
void Load();
private:
std::string Config;
};
class EnvLoader final : public FEXCore::Config::Layer {
public:
explicit EnvLoader(char *const _envp[]);
void Load() override;
private:
char *const *envp;
};
static const std::map<std::string, FEXCore::Config::ConfigOption, std::less<>> ConfigLookup = {{
#define OPT_BASE(type, group, enum, json, default) {#json, FEXCore::Config::ConfigOption::CONFIG_##enum},
#include <FEXCore/Config/ConfigValues.inl>
}};
static const std::vector<std::pair<const char*, FEXCore::Config::ConfigOption>> EnvConfigLookup = {{
#define OPT_BASE(type, group, enum, json, default) {"FEX_" #enum, FEXCore::Config::ConfigOption::CONFIG_##enum},
#include <FEXCore/Config/ConfigValues.inl>
}};
OptionMapper::OptionMapper(FEXCore::Config::LayerType Layer)
: FEXCore::Config::Layer(Layer) {
}
void OptionMapper::MapNameToOption(const char *ConfigName, const char *ConfigString) {
auto it = ConfigLookup.find(ConfigName);
if (it != ConfigLookup.end()) {
Set(it->second, ConfigString);
}
}
MainLoader::MainLoader()
: FEXCore::Config::OptionMapper(FEXCore::Config::LayerType::LAYER_MAIN)
, Config{FEXCore::Config::GetConfigFileLocation()} {
}
MainLoader::MainLoader(std::string ConfigFile)
: FEXCore::Config::OptionMapper(FEXCore::Config::LayerType::LAYER_MAIN)
, Config{std::move(ConfigFile)} {
}
void MainLoader::Load() {
JSON::LoadJSonConfig(Config, [this](const char *Name, const char *ConfigString) {
MapNameToOption(Name, ConfigString);
});
}
AppLoader::AppLoader(const std::string& Filename, bool Global)
: FEXCore::Config::OptionMapper(Global ? FEXCore::Config::LayerType::LAYER_GLOBAL_APP : FEXCore::Config::LayerType::LAYER_LOCAL_APP) {
Config = FEXCore::Config::GetApplicationConfig(Filename, Global);
// Immediately load so we can reload the meta layer
Load();
}
void AppLoader::Load() {
JSON::LoadJSonConfig(Config, [this](const char *Name, const char *ConfigString) {
MapNameToOption(Name, ConfigString);
});
}
EnvLoader::EnvLoader(char *const _envp[])
: FEXCore::Config::Layer(FEXCore::Config::LayerType::LAYER_ENVIRONMENT)
, envp {_envp} {
}
void EnvLoader::Load() {
std::unordered_map<std::string_view, std::string_view> EnvMap;
for(const char *const *pvar=envp; pvar && *pvar; pvar++) {
std::string_view Var(*pvar);
size_t pos = Var.rfind('=');
if (std::string::npos == pos)
continue;
std::string_view Ident = Var.substr(0,pos);
std::string_view Value = Var.substr(pos+1);
EnvMap[Ident]=Value;
}
std::function GetVar = [=](const std::string_view id) -> std::optional<std::string_view> {
if (EnvMap.find(id) != EnvMap.end())
return EnvMap.at(id);
// If envp[] was empty, search using std::getenv()
const char* vs = std::getenv(id.data());
if (vs) {
return vs;
}
else {
return std::nullopt;
}
};
std::optional<std::string_view> Value;
for (auto &it : EnvConfigLookup) {
if ((Value = GetVar(it.first)).has_value()) {
Set(it.second, std::string(*Value));
}
}
}
std::unique_ptr<FEXCore::Config::Layer> CreateMainLayer(std::string const *File) {
if (File) {
return std::make_unique<FEXCore::Config::MainLoader>(*File);
}
else {
return std::make_unique<FEXCore::Config::MainLoader>();
}
}
std::unique_ptr<FEXCore::Config::Layer> CreateAppLayer(const std::string& Filename, bool Global) {
return std::make_unique<FEXCore::Config::AppLoader>(Filename, Global);
}
std::unique_ptr<FEXCore::Config::Layer> CreateEnvironmentLayer(char *const _envp[]) {
return std::make_unique<FEXCore::Config::EnvLoader>(_envp);
}
}
+23 -1
View File
@@ -77,7 +77,14 @@
"Default": "",
"ShortArg": "k",
"Desc": [
"A json file specifying where to overlay the thunks."
"A json file specifying where to overlay the thunks.",
"This can be a filesystem path",
"\teg: ~/MyThunkConfig.json",
"Or this can be a named of a Thunk config file",
"If the named config file exists in the FEX data folder folder the it will use that one",
"\teg: $HOME/.fex-emu/ThunkConfigs/<ThunkConfig name>",
"Or if you have XDG_DATA_HOME the config will search in that directory",
"\teg: $XDG_DATA_HOME/.fex-emu/ThunkConfigs/<ThunkConfig name>"
]
},
"Env": {
@@ -130,6 +137,13 @@
"Disables optimizations passes for debugging."
]
},
"SRA": {
"Type": "bool",
"Default": "true",
"Desc": [
"Set to false to disable Static Register Allocation"
]
},
"Force32BitAllocator": {
"Type": "bool",
"Default": "false",
@@ -207,6 +221,14 @@
"Makes TSO operations even more strict.",
"Forces vector loadstores to also become atomic."
]
},
"StallProcess": {
"Type": "bool",
"Default": "false",
"Desc": [
"Forces a process to stall out on initialization",
"Useful for a process that keeps restarting and doesn't work"
]
}
},
"Misc": {
+19 -5
View File
@@ -2,10 +2,20 @@
#include "Interface/Context/Context.h"
#include "Interface/Core/Core.h"
#include "Interface/Core/OpcodeDispatcher.h"
#include "Interface/Core/X86Tables/X86Tables.h"
#include <FEXCore/Config/Config.h>
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/Debug/X86Tables.h>
#include <FEXCore/Core/Context.h>
#include <FEXCore/Core/CPUID.h>
#include <FEXCore/Core/SignalDelegator.h>
#include "FEXCore/Debug/InternalThreadState.h"
#include <string.h>
#include <utility>
namespace FEXCore::HLE {
class SyscallVisitor;
}
namespace FEXCore::Context {
void InitializeStaticTables(OperatingMode Mode) {
@@ -33,7 +43,7 @@ namespace FEXCore::Context {
delete CTX;
}
bool InitCore(FEXCore::Context::Context *CTX, FEXCore::CodeLoader *Loader) {
FEXCore::Core::InternalThreadState* InitCore(FEXCore::Context::Context *CTX, FEXCore::CodeLoader *Loader) {
return CTX->InitCore(Loader);
}
@@ -100,8 +110,8 @@ namespace FEXCore::Context {
void RegisterExternalSyscallVisitor(FEXCore::Context::Context *CTX, [[maybe_unused]] uint64_t Syscall, [[maybe_unused]] FEXCore::HLE::SyscallVisitor *Visitor) {
}
void HandleCallback(FEXCore::Context::Context *CTX, uint64_t RIP) {
CTX->HandleCallback(RIP);
void HandleCallback(FEXCore::Context::Context *CTX, FEXCore::Core::InternalThreadState *Thread, uint64_t RIP) {
CTX->HandleCallback(Thread, RIP);
}
void RegisterHostSignalHandler(FEXCore::Context::Context *CTX, int Signal, HostSignalDelegatorFunction Func, bool Required) {
@@ -116,6 +126,10 @@ namespace FEXCore::Context {
return CTX->CreateThread(NewThreadState, ParentTID);
}
void ExecutionThread(FEXCore::Context::Context *CTX, FEXCore::Core::InternalThreadState *Thread) {
return CTX->ExecutionThread(Thread);
}
void InitializeThread(FEXCore::Context::Context *CTX, FEXCore::Core::InternalThreadState *Thread) {
return CTX->InitializeThread(Thread);
}
+14 -14
View File
@@ -1,48 +1,47 @@
#pragma once
#include "Common/JitSymbols.h"
#include "Interface/Core/CPUID.h"
#include "Interface/Core/Frontend.h"
#include "Interface/Core/HostFeatures.h"
#include "Interface/Core/InternalThreadState.h"
#include "Interface/Core/X86HelperGen.h"
#include "Interface/IR/PassManager.h"
#include "Interface/IR/Passes/RegisterAllocationPass.h"
#include <FEXCore/Config/Config.h>
#include <FEXCore/Core/CPUBackend.h>
#include <FEXCore/Core/Context.h>
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/Core/SignalDelegator.h>
#include <FEXCore/Debug/InternalThreadState.h>
#include <FEXCore/Utils/CompilerDefs.h>
#include <FEXCore/Utils/Event.h>
#include <stdint.h>
#include <atomic>
#include <condition_variable>
#include <functional>
#include <istream>
#include <map>
#include <memory>
#include <mutex>
#include <optional>
#include <ostream>
#include <set>
#include <shared_mutex>
#include <stddef.h>
#include <string>
#include <unordered_map>
#include <queue>
#include <vector>
namespace FEXCore {
class CodeLoader;
class ThunkHandler;
class BlockSamplingData;
class GdbServer;
class SiganlDelegator;
namespace CPU {
class Arm64JITCore;
class X86JITCore;
}
namespace HLE {
struct SyscallArguments;
class SyscallHandler;
}
}
namespace FEXCore::IR {
class RegisterAllocationPass;
class RegisterAllocationData;
class IRListView;
namespace Validation {
@@ -122,6 +121,7 @@ namespace FEXCore::Context {
FEX_CONFIG_OPT(RootFSPath, ROOTFS);
FEX_CONFIG_OPT(ThunkHostLibsPath, THUNKHOSTLIBS);
FEX_CONFIG_OPT(DumpIR, DUMPIR);
FEX_CONFIG_OPT(StaticRegisterAllocation, SRA);
} Config;
using IntCallbackReturn = FEX_NAKED void(*)(FEXCore::Core::InternalThreadState *Thread, volatile void *Host_RSP);
@@ -183,7 +183,7 @@ namespace FEXCore::Context {
Context();
~Context();
bool InitCore(FEXCore::CodeLoader *Loader);
FEXCore::Core::InternalThreadState* InitCore(FEXCore::CodeLoader *Loader);
FEXCore::Context::ExitReason RunUntilExit();
int GetProgramStatus() const;
bool IsPaused() const { return !Running; }
@@ -199,7 +199,7 @@ namespace FEXCore::Context {
bool GetGdbServerStatus() const { return DebugServer != nullptr; }
void StartGdbServer();
void StopGdbServer();
void HandleCallback(uint64_t RIP);
void HandleCallback(FEXCore::Core::InternalThreadState *Thread, uint64_t RIP);
void RegisterHostSignalHandler(int Signal, HostSignalDelegatorFunction Func, bool Required);
void RegisterFrontendHostSignalHandler(int Signal, HostSignalDelegatorFunction Func, bool Required);
+872 -27
View File
@@ -1,6 +1,8 @@
#include "Interface/Core/ArchHelpers/Arm64.h"
#include "Interface/Core/ArchHelpers/MContext.h"
#include <FEXCore/Utils/LogManager.h>
#include <FEXCore/Utils/Telemetry.h>
#include <atomic>
#include <stdint.h>
@@ -8,6 +10,30 @@
#include <signal.h>
namespace FEXCore::ArchHelpers::Arm64 {
FEXCORE_TELEMETRY_STATIC_INIT(SplitLock, TYPE_HAS_SPLIT_LOCKS);
FEXCORE_TELEMETRY_STATIC_INIT(SplitLock16B, TYPE_16BYTE_SPLIT);
static __uint128_t LoadAcquire128(uint64_t Addr) {
__uint128_t Result{};
uint64_t Lower;
uint64_t Upper;
// This specifically avoids using std::atomic<__uint128_t>
// std::atomic helper does a ldaxp + stxp pair that crashes when the page is only mapped readable
__asm volatile(
R"(
ldaxp %[ResultLower], %[ResultUpper], [%[Addr]];
clrex;
)"
: [ResultLower] "=r" (Lower)
, [ResultUpper] "=r" (Upper)
: [Addr] "r" (Addr)
: "memory");
Result = Upper;
Result <<= 64;
Result |= Lower;
return Result;
}
static uint64_t LoadAcquire64(uint64_t Addr) {
std::atomic<uint64_t> *Atom = reinterpret_cast<std::atomic<uint64_t>*>(Addr);
return Atom->load(std::memory_order_acquire);
@@ -38,22 +64,12 @@ static bool StoreCAS8(uint8_t &Expected, uint8_t Val, uint64_t Addr) {
return Atom->compare_exchange_strong(Expected, Val);
}
bool HandleCASPAL(void *_ucontext, void *_info, uint32_t Instr) {
static bool RunCASPAL(void *_ucontext, void *_info, uint32_t Size, uint32_t DesiredReg1, uint32_t DesiredReg2, uint32_t ExpectedReg1, uint32_t ExpectedReg2, uint32_t AddressReg) {
mcontext_t* mcontext = &reinterpret_cast<ucontext_t*>(_ucontext)->uc_mcontext;
siginfo_t* info = reinterpret_cast<siginfo_t*>(_info);
if (info->si_code != BUS_ADRALN) {
// This only handles alignment problems
return false;
}
uint32_t Size = (Instr >> 30) & 1;
uint32_t DesiredReg1 = Instr & 0b11111;
uint32_t DesiredReg2 = DesiredReg1 + 1;
uint32_t ExpectedReg1 = (Instr >> 16) & 0b11111;
uint32_t ExpectedReg2 = ExpectedReg1 + 1;
uint32_t AddressReg = (Instr >> 5) & 0b11111;
//Bus_ADRALN check happens in HandleCASPAL and HandleCASPAL_ARMv8
if (Size == 0) {
// 32bit
@@ -72,8 +88,15 @@ bool HandleCASPAL(void *_ucontext, void *_info, uint32_t Instr) {
// Both cross-cacheline and cross 16byte both need dual CAS loops that can tear
// ARMv8.4 LSE2 solves all atomic issues except cross-cacheline
// Check for Split lock across a cacheline
if ((Addr & 63) > 56) {
FEXCORE_TELEMETRY_SET(SplitLock, 1);
}
uint64_t AlignmentMask = 0b1111;
if ((Addr & AlignmentMask) > 8) {
FEXCORE_TELEMETRY_SET(SplitLock16B, 1);
uint64_t Alignment = Addr & 0b111;
Addr &= ~0b111ULL;
uint64_t AddrUpper = Addr + 8;
@@ -226,6 +249,256 @@ bool HandleCASPAL(void *_ucontext, void *_info, uint32_t Instr) {
return false;
}
bool HandleCASPAL(void *_ucontext, void *_info, uint32_t Instr) {
siginfo_t* info = reinterpret_cast<siginfo_t*>(_info);
if (info->si_code != BUS_ADRALN) {
// This only handles alignment problems
return false;
}
uint32_t Size = (Instr >> 30) & 1;
uint32_t DesiredReg1 = Instr & 0b11111;
uint32_t DesiredReg2 = DesiredReg1 + 1;
uint32_t ExpectedReg1 = (Instr >> 16) & 0b11111;
uint32_t ExpectedReg2 = ExpectedReg1 + 1;
uint32_t AddressReg = (Instr >> 5) & 0b11111;
return RunCASPAL(_ucontext, _info, Size, DesiredReg1, DesiredReg2, ExpectedReg1, ExpectedReg2, AddressReg);
}
uint64_t HandleCASPAL_ARMv8(void *_ucontext, void *_info, uint32_t Instr) {
mcontext_t* mcontext = &reinterpret_cast<ucontext_t*>(_ucontext)->uc_mcontext;
siginfo_t* info = reinterpret_cast<siginfo_t*>(_info);
if (info->si_code != BUS_ADRALN) {
// This only handles alignment problems
return 0;
}
// caspair
// [1] ldaxp(TMP2.W(), TMP3.W(), MemOperand(MemSrc)); <-- DataReg & AddrReg
// [2] cmp(TMP2.W(), Expected.first.W()); <-- ExpectedReg1
// [3] ccmp(TMP3.W(), Expected.second.W(), NoFlag, Condition::eq); <-- ExpectedREg2
// [4] b(&LoopNotExpected, Condition::ne);
// [5] stlxp(TMP2.W(), Desired.first.W(), Desired.second.W(), MemOperand(MemSrc)); <-- DesiredReg
// [6] cbnz(TMP2.W(), &LoopTop);
// [7] mov(Dst.first.W(), Expected.first.W());
// [8] mov(Dst.second.W(), Expected.second.W());
// [9] b(&LoopExpected);
// [10] mov(Dst.first.W(), TMP2.W());
// [11] mov(Dst.second.W(), TMP3.W());
// [12] clrex();
uint32_t *PC = (uint32_t*)ArchHelpers::Context::GetPc(_ucontext);
uint32_t Size = (Instr >> 30) & 1;
uint32_t AddrReg = (Instr >> 5) & 0x1F;
uint32_t DataReg = Instr & 0x1F;
uint32_t DataReg2 = (Instr >> 10) & 0x1F;
uint32_t ExpectedReg1{};
uint32_t ExpectedReg2{};
uint32_t DesiredReg1{};
uint32_t DesiredReg2{};
if(Size != 0) { //Only 32-bit pairs
return 0;
}
for(int i = 1; i < 10; i++) {
uint32_t NextInstr = PC[i];
if ((NextInstr & FEXCore::ArchHelpers::Arm64::ALU_OP_MASK) == FEXCore::ArchHelpers::Arm64::CMP_INST) {
ExpectedReg1 = GetRmReg(NextInstr);
} else if ((NextInstr & FEXCore::ArchHelpers::Arm64::CCMP_MASK) == FEXCore::ArchHelpers::Arm64::CCMP_INST) {
ExpectedReg2 = GetRmReg(NextInstr);
} else if ((NextInstr & FEXCore::ArchHelpers::Arm64::STLXP_MASK) == FEXCore::ArchHelpers::Arm64::STLXP_INST) {
DesiredReg1 = (NextInstr & 0x1F);
DesiredReg2 = (NextInstr >> 10) & 0x1F;
}
}
//mov expected into the temp registers used by JIT
mcontext->regs[DataReg] = mcontext->regs[ExpectedReg1];
mcontext->regs[DataReg2] = mcontext->regs[ExpectedReg2];
if(RunCASPAL(_ucontext, _info, Size, DesiredReg1, DesiredReg2, DataReg, DataReg2, AddrReg)) {
return 9 * sizeof(uint32_t); // skip to mov + clrex
} else {
return 0;
}
}
uint16_t DoLoad16(uint64_t Addr) {
uint64_t AlignmentMask = 0b1111;
if ((Addr & AlignmentMask) == 15) {
// Address crosses over 16byte or 64byte threshold
// Needs two loads
uint64_t AddrUpper = Addr + 1;
uint8_t ActualUpper{};
uint8_t ActualLower{};
// Careful ordering here
ActualUpper = LoadAcquire8(AddrUpper);
ActualLower = LoadAcquire8(Addr);
uint16_t Result = ActualUpper;
Result <<= 8;
Result |= ActualLower;
return Result;
}
else {
AlignmentMask = 0b111;
if ((Addr & AlignmentMask) == 7) {
// Crosses 8byte boundary
// Needs 128bit load
// Fits within a 16byte region
uint64_t Alignment = Addr & 0b1111;
Addr &= ~0b1111ULL;
__uint128_t TmpResult = LoadAcquire128(Addr);
// Zexts the result
uint16_t Result = TmpResult >> (Alignment * 8);
return Result;
}
else {
AlignmentMask = 0b11;
if ((Addr & AlignmentMask) == 3) {
// Crosses 4byte boundary
// Needs 64bit Load
uint64_t Alignment = Addr & AlignmentMask;
Addr &= ~AlignmentMask;
std::atomic<uint64_t> *Atomic = reinterpret_cast<std::atomic<uint64_t>*>(Addr);
uint64_t TmpResult = Atomic->load();
// Zexts the result
uint16_t Result = TmpResult >> (Alignment * 8);
return Result;
}
else {
// Fits within 4byte boundary
// Only needs 32bit Load
// Only alignment offset will be 1 here
uint64_t Alignment = Addr & AlignmentMask;
Addr &= ~AlignmentMask;
std::atomic<uint32_t> *Atomic = reinterpret_cast<std::atomic<uint32_t>*>(Addr);
uint32_t TmpResult = Atomic->load();
// Zexts the result
uint16_t Result = TmpResult >> (Alignment * 8);
return Result;
}
}
}
}
uint32_t DoLoad32(uint64_t Addr) {
uint64_t AlignmentMask = 0b1111;
if ((Addr & AlignmentMask) > 12) {
// Address crosses over 16byte threshold
// Needs dual 32bit load
uint64_t Alignment = Addr & 0b11;
Addr &= ~0b11ULL;
uint64_t AddrUpper = Addr + 4;
// Careful ordering here
uint32_t ActualUpper = LoadAcquire32(AddrUpper);
uint32_t ActualLower = LoadAcquire32(Addr);
uint64_t Result = ActualUpper;
Result <<= 32;
Result |= ActualLower;
return Result >> (Alignment * 8);
}
else {
AlignmentMask = 0b111;
if ((Addr & AlignmentMask) >= 5) {
// Crosses 8byte boundary
// Needs 128bit load
// Fits within a 16byte region
uint64_t Alignment = Addr & 0b1111;
Addr &= ~0b1111ULL;
__uint128_t TmpResult = LoadAcquire128(Addr);
return TmpResult >> (Alignment * 8);
}
else {
// Fits within 8byte boundary
// Only needs 64bit CAS
// Alignments can be [1,5)
uint64_t Alignment = Addr & AlignmentMask;
Addr &= ~AlignmentMask;
std::atomic<uint64_t> *Atomic = reinterpret_cast<std::atomic<uint64_t>*>(Addr);
uint64_t TmpResult = Atomic->load();
return TmpResult >> (Alignment * 8);
}
}
}
uint64_t DoLoad64(uint64_t Addr) {
uint64_t AlignmentMask = 0b1111;
if ((Addr & AlignmentMask) > 8) {
uint64_t Alignment = Addr & 0b111;
Addr &= ~0b111ULL;
uint64_t AddrUpper = Addr + 8;
// Crosses a 16byte boundary
// Needs two 8 byte loads
uint64_t ActualUpper{};
uint64_t ActualLower{};
// Careful ordering here
ActualUpper = LoadAcquire64(AddrUpper);
ActualLower = LoadAcquire64(Addr);
__uint128_t Result = ActualUpper;
Result <<= 64;
Result |= ActualLower;
return Result >> (Alignment * 8);
}
else {
// Fits within a 16byte region
uint64_t Alignment = Addr & AlignmentMask;
Addr &= ~AlignmentMask;
__uint128_t TmpResult = LoadAcquire128(Addr);
uint64_t Result = TmpResult >> (Alignment * 8);
return Result;
}
}
std::pair<uint64_t, uint64_t> DoLoad128(uint64_t Addr) {
// Any misalignment here means we cross a 16byte boundary
// So we need two 128bit loads
uint64_t Alignment = Addr & 0b1111;
Addr &= ~0b1111ULL;
uint64_t AddrUpper = Addr + 16;
union AlignedData {
struct {
__uint128_t Lower;
__uint128_t Upper;
} Large;
struct {
uint8_t Data[32];
} Bytes;
};
AlignedData *Data = reinterpret_cast<AlignedData*>(alloca(sizeof(AlignedData)));
Data->Large.Upper = LoadAcquire128(AddrUpper);
Data->Large.Lower = LoadAcquire128(Addr);
uint64_t ResultLower{}, ResultUpper{};
memcpy(&ResultLower, &Data->Bytes.Data[Alignment], sizeof(uint64_t));
memcpy(&ResultUpper, &Data->Bytes.Data[Alignment + sizeof(uint64_t)], sizeof(uint64_t));
return {ResultLower, ResultUpper};
}
template <typename T>
using CASExpectedFn = T (*)(T Src, T Expected);
template <typename T>
@@ -239,9 +512,16 @@ uint16_t DoCAS16(
uint64_t Addr,
CASExpectedFn<uint16_t> ExpectedFunction,
CASDesiredFn<uint16_t> DesiredFunction) {
if ((Addr & 63) == 63) {
FEXCORE_TELEMETRY_SET(SplitLock, 1);
}
// 16 bit
uint64_t AlignmentMask = 0b1111;
if ((Addr & AlignmentMask) == 15) {
FEXCORE_TELEMETRY_SET(SplitLock16B, 1);
// Address crosses over 16byte or 64byte threshold
// Need a dual 8bit CAS loop
uint64_t AddrUpper = Addr + 1;
@@ -515,9 +795,16 @@ uint32_t DoCAS32(
uint64_t Addr,
CASExpectedFn<uint32_t> ExpectedFunction,
CASDesiredFn<uint32_t> DesiredFunction) {
if ((Addr & 63) > 60) {
FEXCORE_TELEMETRY_SET(SplitLock, 1);
}
// 32 bit
uint64_t AlignmentMask = 0b1111;
if ((Addr & AlignmentMask) > 12) {
FEXCORE_TELEMETRY_SET(SplitLock16B, 1);
// Address crosses over 16byte threshold
// Needs dual 4 byte CAS loop
uint64_t Alignment = Addr & 0b11;
@@ -745,9 +1032,16 @@ uint64_t DoCAS64(
uint64_t Addr,
CASExpectedFn<uint64_t> ExpectedFunction,
CASDesiredFn<uint64_t> DesiredFunction) {
if ((Addr & 63) > 56) {
FEXCORE_TELEMETRY_SET(SplitLock, 1);
}
// 64bit
uint64_t AlignmentMask = 0b1111;
if ((Addr & AlignmentMask) > 8) {
FEXCORE_TELEMETRY_SET(SplitLock16B, 1);
uint64_t Alignment = Addr & 0b111;
Addr &= ~0b111ULL;
uint64_t AddrUpper = Addr + 8;
@@ -900,21 +1194,10 @@ uint64_t DoCAS64(
}
}
bool HandleCASAL(void *_ucontext, void *_info, uint32_t Instr) {
static bool RunCASAL(void *_ucontext, void *_info, uint32_t Size, uint32_t DesiredReg, uint32_t ExpectedReg, uint32_t AddressReg) {
mcontext_t* mcontext = &reinterpret_cast<ucontext_t*>(_ucontext)->uc_mcontext;
siginfo_t* info = reinterpret_cast<siginfo_t*>(_info);
if (info->si_code != BUS_ADRALN) {
// This only handles alignment problems
return false;
}
uint32_t Size = 1 << (Instr >> 30);
uint32_t DesiredReg = Instr & 0b11111;
uint32_t ExpectedReg = (Instr >> 16) & 0b11111;
uint32_t AddressReg = (Instr >> 5) & 0b11111;
uint64_t Addr = mcontext->regs[AddressReg];
// Cross-cacheline CAS doesn't work on ARM
@@ -994,6 +1277,23 @@ bool HandleCASAL(void *_ucontext, void *_info, uint32_t Instr) {
return false;
}
bool HandleCASAL(void *_ucontext, void *_info, uint32_t Instr) {
mcontext_t* mcontext = &reinterpret_cast<ucontext_t*>(_ucontext)->uc_mcontext;
siginfo_t* info = reinterpret_cast<siginfo_t*>(_info);
if (info->si_code != BUS_ADRALN) {
// This only handles alignment problems
return false;
}
uint32_t Size = 1 << (Instr >> 30);
uint32_t DesiredReg = Instr & 0b11111;
uint32_t ExpectedReg = (Instr >> 16) & 0b11111;
uint32_t AddressReg = (Instr >> 5) & 0b11111;
return RunCASAL(_ucontext, _info, Size, DesiredReg, ExpectedReg, AddressReg);
}
bool HandleAtomicMemOp(void *_ucontext, void *_info, uint32_t Instr) {
mcontext_t* mcontext = &reinterpret_cast<ucontext_t*>(_ucontext)->uc_mcontext;
siginfo_t* info = reinterpret_cast<siginfo_t*>(_info);
@@ -1202,4 +1502,549 @@ bool HandleAtomicMemOp(void *_ucontext, void *_info, uint32_t Instr) {
return false;
}
bool HandleAtomicLoad(void *_ucontext, void *_info, uint32_t Instr) {
mcontext_t* mcontext = &reinterpret_cast<ucontext_t*>(_ucontext)->uc_mcontext;
siginfo_t* info = reinterpret_cast<siginfo_t*>(_info);
if (info->si_code != BUS_ADRALN) {
// This only handles alignment problems
return false;
}
uint32_t Size = 1 << (Instr >> 30);
uint32_t ResultReg = Instr & 0b11111;
uint32_t AddressReg = (Instr >> 5) & 0b11111;
uint64_t Addr = mcontext->regs[AddressReg];
if (Size == 2) {
auto Res = DoLoad16(Addr);
// We set the result register if it isn't a zero register
if (ResultReg != 31) {
mcontext->regs[ResultReg] = Res;
}
return true;
}
else if (Size == 4) {
auto Res = DoLoad32(Addr);
// We set the result register if it isn't a zero register
if (ResultReg != 31) {
mcontext->regs[ResultReg] = Res;
}
return true;
}
else if (Size == 8) {
auto Res = DoLoad64(Addr);
// We set the result register if it isn't a zero register
if (ResultReg != 31) {
mcontext->regs[ResultReg] = Res;
}
return true;
}
return false;
}
bool HandleAtomicStore(void *_ucontext, void *_info, uint32_t Instr) {
mcontext_t* mcontext = &reinterpret_cast<ucontext_t*>(_ucontext)->uc_mcontext;
siginfo_t* info = reinterpret_cast<siginfo_t*>(_info);
if (info->si_code != BUS_ADRALN) {
// This only handles alignment problems
return false;
}
uint32_t Size = 1 << (Instr >> 30);
uint32_t DataReg = Instr & 0x1F;
uint32_t AddressReg = (Instr >> 5) & 0b11111;
uint64_t Addr = mcontext->regs[AddressReg];
constexpr bool DoRetry = false;
if (Size == 2) {
DoCAS16<DoRetry>(
mcontext->regs[DataReg],
0, // Unused
Addr,
[](uint16_t SrcVal, uint16_t) -> uint16_t {
// Expected is just src
return SrcVal;
},
[](uint16_t, uint16_t Desired) -> uint16_t {
// Desired is just Desired
return Desired;
});
return true;
}
else if (Size == 4) {
DoCAS32<DoRetry>(
mcontext->regs[DataReg],
0, // Unused
Addr,
[](uint32_t SrcVal, uint32_t) -> uint32_t {
// Expected is just src
return SrcVal;
},
[](uint32_t, uint32_t Desired) -> uint32_t {
// Desired is just Desired
return Desired;
});
return true;
}
else if (Size == 8) {
DoCAS64<DoRetry>(
mcontext->regs[DataReg],
0, // Unused
Addr,
[](uint64_t SrcVal, uint64_t) -> uint64_t {
// Expected is just src
return SrcVal;
},
[](uint64_t, uint64_t Desired) -> uint64_t {
// Desired is just Desired
return Desired;
});
return true;
}
return false;
}
bool HandleAtomicLoad128(void *_ucontext, void *_info, uint32_t Instr) {
mcontext_t* mcontext = &reinterpret_cast<ucontext_t*>(_ucontext)->uc_mcontext;
siginfo_t* info = reinterpret_cast<siginfo_t*>(_info);
if (info->si_code != BUS_ADRALN) {
// This only handles alignment problems
return false;
}
uint32_t ResultReg = Instr & 0b11111;
uint32_t ResultReg2 = (Instr >> 10) & 0x1F;
uint32_t AddressReg = (Instr >> 5) & 0b11111;
uint64_t Addr = mcontext->regs[AddressReg];
auto Res = DoLoad128(Addr);
// We set the result register if it isn't a zero register
if (ResultReg != 31) {
mcontext->regs[ResultReg] = std::get<0>(Res);
}
if (ResultReg2 != 31) {
mcontext->regs[ResultReg2] = std::get<1>(Res);
}
return true;
}
static uint64_t HandleCAS_NoAtomics(void *_ucontext, void *_info)
{
mcontext_t* mcontext = &reinterpret_cast<ucontext_t*>(_ucontext)->uc_mcontext;
siginfo_t* info = reinterpret_cast<siginfo_t*>(_info);
// ARMv8.0 CAS
// [1] ldaxrb(TMP2.W(), MemOperand(MemSrc))
// [2] cmp (TMP2.W(), Expected.W())
// [3] b
// [4] stlxrb(TMP3.W(), Desired.W(), MemOperand(MemSrc)
// [5] cbnz
// [6] mov
// [7] b
// [8] mov (.., TMP2.W());
// [9] clrex
uint32_t *PC = (uint32_t*)ArchHelpers::Context::GetPc(_ucontext);
uint32_t Instr = PC[0];
uint32_t Size = 1 << (Instr >> 30);
uint32_t AddressReg = GetRnReg(Instr);
uint32_t ResultReg = GetRdReg(Instr); //TMP2
uint32_t DesiredReg = 0;
uint32_t ExpectedReg = 0;
for (size_t i = 1; i < 6; ++i) {
uint32_t NextInstr = PC[i];
if ((NextInstr & FEXCore::ArchHelpers::Arm64::STLXR_MASK) == FEXCore::ArchHelpers::Arm64::STLXR_INST) {
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
// Just double check that the memory destination matches
uint32_t StoreAddressReg = GetRnReg(NextInstr);
LOGMAN_THROW_A(StoreAddressReg == AddressReg, "StoreExclusive memory register didn't match the store exclusive register");
#endif
DesiredReg = GetRdReg(NextInstr);
}
else if ((NextInstr & FEXCore::ArchHelpers::Arm64::ALU_OP_MASK) == FEXCore::ArchHelpers::Arm64::CMP_INST) {
ExpectedReg = GetRmReg(NextInstr);
}
}
//set up CASAL by doing mov(TMP2, Expected)
mcontext->regs[ResultReg] = mcontext->regs[ExpectedReg];
if(RunCASAL(_ucontext, _info, Size, DesiredReg, ResultReg, AddressReg)) {
return 7 * sizeof(uint32_t); //jump to mov to allocated register
} else {
return 0;
}
}
uint64_t HandleAtomicLoadstoreExclusive(void *_ucontext, void *_info) {
mcontext_t* mcontext = &reinterpret_cast<ucontext_t*>(_ucontext)->uc_mcontext;
siginfo_t* info = reinterpret_cast<siginfo_t*>(_info);
if (info->si_code != BUS_ADRALN) {
// This only handles alignment problems
return 0;
}
uint32_t *PC = (uint32_t*)ArchHelpers::Context::GetPc(_ucontext);
uint32_t Instr = PC[0];
// Atomic Add
// [1] ldaxrb(TMP2.W(), MemOperand(MemSrc));
// [2] add(TMP2.W(), TMP2.W(), GetReg<RA_32>(Op->Header.Args[1].ID()));
// [3] stlxrb(TMP2.W(), TMP2.W(), MemOperand(MemSrc));
// [4] cbnz(TMP2.W(), &LoopTop);
//
// Atomic Fetch Add
// [1] ldaxrb(TMP2.W(), MemOperand(MemSrc));
// [2] add(TMP3.W(), TMP2.W(), GetReg<RA_32>(Op->Header.Args[1].ID()));
// [3] stlxrb(TMP4.W(), TMP3.W(), MemOperand(MemSrc));
// [4] cbnz(TMP4.W(), &LoopTop);
// [5] mov(GetReg<RA_32>(Node), TMP2.W());
//
// Atomic Swap
//
// [1] ldaxrb(TMP2.W(), MemOperand(MemSrc));
// [2] stlxrb(TMP4.W(), GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc));
// [3] cbnz(TMP4.W(), &LoopTop);
// [4] uxtb(GetReg<RA_64>(Node), TMP2.W());
//
// ASSUMPTIONS:
// - Both cases:
// - The [2]ALU op: (Non NEG case)
// - First source is from [1]ldaxr
// - Second source is incoming value
// - The [2]ALU op: (NEG case)
// - First source is zero register
// - The second source is the from [1]ldaxr
// - No ALU op: (SWAP case)
// - No DataSourceRegister
//
// - In Atomic case (non-fetch)
// - The [3]stlxr instruction status + memory register are the SAME register
//
// - In Atomic FETCH case
// - The [3]stlxr instruction's status + memory register are never the same register
// - The [5]mov instruction source is always the destination register from [1] ldaxr*
uint32_t ResultReg = GetRdReg(Instr);
uint32_t AddressReg = GetRnReg(Instr);
uint64_t Addr = mcontext->regs[AddressReg];
size_t NumInstructionsToSkip = 0;
// Are we an Atomic op or AtomicFetch?
bool AtomicFetch = false;
// This is the register that is the incoming source to the ALU operation
// <DataResultReg> = <Load Exclusive Value> <Op> <DataSourceReg>
// NEG case is special
// <DataResultReg> = Zero <Sub> <Load Exclusive Value>
// DataSourceRegister must always be the Rm register
uint32_t DataSourceReg {};
ExclusiveAtomicPairType AtomicOp {ExclusiveAtomicPairType::TYPE_SWAP};
// Scan forward at most five instructions to find our instructions
for (size_t i = 1; i < 6; ++i) {
uint32_t NextInstr = PC[i];
if ((NextInstr & FEXCore::ArchHelpers::Arm64::ALU_OP_MASK) == FEXCore::ArchHelpers::Arm64::ADD_INST) {
AtomicOp = ExclusiveAtomicPairType::TYPE_ADD;
DataSourceReg = GetRmReg(NextInstr);
}
else if ((NextInstr & FEXCore::ArchHelpers::Arm64::ALU_OP_MASK) == FEXCore::ArchHelpers::Arm64::SUB_INST) {
uint32_t RnReg = GetRnReg(NextInstr);
if (RnReg == REGISTER_MASK) {
// Zero reg means neg
AtomicOp = ExclusiveAtomicPairType::TYPE_NEG;
}
else {
AtomicOp = ExclusiveAtomicPairType::TYPE_SUB;
}
DataSourceReg = GetRmReg(NextInstr);
}
else if ((NextInstr & FEXCore::ArchHelpers::Arm64::ALU_OP_MASK) == FEXCore::ArchHelpers::Arm64::CMP_INST) {
return HandleCAS_NoAtomics(_ucontext, _info); //ARMv8.0 CAS
}
else if ((NextInstr & FEXCore::ArchHelpers::Arm64::ALU_OP_MASK) == FEXCore::ArchHelpers::Arm64::AND_INST) {
AtomicOp = ExclusiveAtomicPairType::TYPE_AND;
DataSourceReg = GetRmReg(NextInstr);
}
else if ((NextInstr & FEXCore::ArchHelpers::Arm64::ALU_OP_MASK) == FEXCore::ArchHelpers::Arm64::OR_INST) {
AtomicOp = ExclusiveAtomicPairType::TYPE_OR;
DataSourceReg = GetRmReg(NextInstr);
}
else if ((NextInstr & FEXCore::ArchHelpers::Arm64::ALU_OP_MASK) == FEXCore::ArchHelpers::Arm64::EOR_INST) {
AtomicOp = ExclusiveAtomicPairType::TYPE_EOR;
DataSourceReg = GetRmReg(NextInstr);
}
else if ((NextInstr & FEXCore::ArchHelpers::Arm64::STLXR_MASK) == FEXCore::ArchHelpers::Arm64::STLXR_INST) {
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
// Just double check that the memory destination matches
uint32_t StoreAddressReg = GetRnReg(NextInstr);
LOGMAN_THROW_A(StoreAddressReg == AddressReg, "StoreExclusive memory register didn't match the store exclusive register");
#endif
uint32_t StatusReg = GetRmReg(NextInstr);
uint32_t StoreResultReg = GetRdReg(NextInstr);
// We are an atomic fetch instruction if the data register isn't the status register
AtomicFetch = !(StatusReg == StoreResultReg);
if (AtomicOp == ExclusiveAtomicPairType::TYPE_SWAP) {
// In the case of swap we don't have an ALU op inbetween
// Source is directly in STLXR
DataSourceReg = StoreResultReg;
}
}
else if ((NextInstr & FEXCore::ArchHelpers::Arm64::CBNZ_MASK) == FEXCore::ArchHelpers::Arm64::CBNZ_INST) {
// Found the CBNZ, we want to skip to just after this instruction when done
NumInstructionsToSkip = i + 1;
// This is the last instruction we care about. Leave now
break;
}
else {
LogMan::Msg::A("Unknown instruction 0x%08x", NextInstr);
}
}
uint32_t Size = 1 << (Instr >> 30);
constexpr bool DoRetry = true;
if (Size == 2) {
using AtomicType = uint16_t;
auto NOPExpected = [](AtomicType SrcVal, AtomicType) -> AtomicType {
return SrcVal;
};
auto ADDDesired = [](AtomicType SrcVal, AtomicType Desired) -> AtomicType {
return SrcVal + Desired;
};
auto SUBDesired = [](AtomicType SrcVal, AtomicType Desired) -> AtomicType {
return SrcVal - Desired;
};
auto ANDDesired = [](AtomicType SrcVal, AtomicType Desired) -> AtomicType {
return SrcVal & Desired;
};
auto ORDesired = [](AtomicType SrcVal, AtomicType Desired) -> AtomicType {
return SrcVal | Desired;
};
auto EORDesired = [](AtomicType SrcVal, AtomicType Desired) -> AtomicType {
return SrcVal ^ Desired;
};
auto NEGDesired = [](AtomicType SrcVal, AtomicType Desired) -> AtomicType {
return -SrcVal;
};
auto SWAPDesired = [](AtomicType SrcVal, AtomicType Desired) -> AtomicType {
return Desired;
};
CASDesiredFn<AtomicType> DesiredFunction{};
switch (AtomicOp) {
case ExclusiveAtomicPairType::TYPE_SWAP:
DesiredFunction = SWAPDesired;
break;
case ExclusiveAtomicPairType::TYPE_ADD:
DesiredFunction = ADDDesired;
break;
case ExclusiveAtomicPairType::TYPE_SUB:
DesiredFunction = SUBDesired;
break;
case ExclusiveAtomicPairType::TYPE_AND:
DesiredFunction = ANDDesired;
break;
case ExclusiveAtomicPairType::TYPE_OR:
DesiredFunction = ORDesired;
break;
case ExclusiveAtomicPairType::TYPE_EOR:
DesiredFunction = EORDesired;
break;
case ExclusiveAtomicPairType::TYPE_NEG:
DesiredFunction = NEGDesired;
break;
default:
LogMan::Msg::E("Unhandled JIT SIGBUS Atomic mem op 0x%02x", AtomicOp);
return false;
break;
}
auto Res = DoCAS16<DoRetry>(
mcontext->regs[DataSourceReg],
0, // Unused
Addr,
NOPExpected,
DesiredFunction);
if (AtomicFetch && ResultReg != 31) {
// On atomic fetch then we store the resulting value back in to the loadacquire destination register
// We want the memory value BEFORE the ALU op
mcontext->regs[ResultReg] = Res;
}
}
else if (Size == 4) {
using AtomicType = uint32_t;
auto NOPExpected = [](AtomicType SrcVal, AtomicType) -> AtomicType {
return SrcVal;
};
auto ADDDesired = [](AtomicType SrcVal, AtomicType Desired) -> AtomicType {
return SrcVal + Desired;
};
auto SUBDesired = [](AtomicType SrcVal, AtomicType Desired) -> AtomicType {
return SrcVal - Desired;
};
auto ANDDesired = [](AtomicType SrcVal, AtomicType Desired) -> AtomicType {
return SrcVal & Desired;
};
auto ORDesired = [](AtomicType SrcVal, AtomicType Desired) -> AtomicType {
return SrcVal | Desired;
};
auto EORDesired = [](AtomicType SrcVal, AtomicType Desired) -> AtomicType {
return SrcVal ^ Desired;
};
auto NEGDesired = [](AtomicType SrcVal, AtomicType Desired) -> AtomicType {
return -SrcVal;
};
auto SWAPDesired = [](AtomicType SrcVal, AtomicType Desired) -> AtomicType {
return Desired;
};
CASDesiredFn<AtomicType> DesiredFunction{};
switch (AtomicOp) {
case ExclusiveAtomicPairType::TYPE_SWAP:
DesiredFunction = SWAPDesired;
break;
case ExclusiveAtomicPairType::TYPE_ADD:
DesiredFunction = ADDDesired;
break;
case ExclusiveAtomicPairType::TYPE_SUB:
DesiredFunction = SUBDesired;
break;
case ExclusiveAtomicPairType::TYPE_AND:
DesiredFunction = ANDDesired;
break;
case ExclusiveAtomicPairType::TYPE_OR:
DesiredFunction = ORDesired;
break;
case ExclusiveAtomicPairType::TYPE_EOR:
DesiredFunction = EORDesired;
break;
case ExclusiveAtomicPairType::TYPE_NEG:
DesiredFunction = NEGDesired;
break;
default:
LogMan::Msg::E("Unhandled JIT SIGBUS Atomic mem op 0x%02x", AtomicOp);
return false;
break;
}
auto Res = DoCAS32<DoRetry>(
mcontext->regs[DataSourceReg],
0, // Unused
Addr,
NOPExpected,
DesiredFunction);
if (AtomicFetch && ResultReg != 31) {
// On atomic fetch then we store the resulting value back in to the loadacquire destination register
// We want the memory value BEFORE the ALU op
mcontext->regs[ResultReg] = Res;
}
}
else if (Size == 8) {
using AtomicType = uint64_t;
auto NOPExpected = [](AtomicType SrcVal, AtomicType) -> AtomicType {
return SrcVal;
};
auto ADDDesired = [](AtomicType SrcVal, AtomicType Desired) -> AtomicType {
return SrcVal + Desired;
};
auto SUBDesired = [](AtomicType SrcVal, AtomicType Desired) -> AtomicType {
return SrcVal - Desired;
};
auto ANDDesired = [](AtomicType SrcVal, AtomicType Desired) -> AtomicType {
return SrcVal & Desired;
};
auto ORDesired = [](AtomicType SrcVal, AtomicType Desired) -> AtomicType {
return SrcVal | Desired;
};
auto EORDesired = [](AtomicType SrcVal, AtomicType Desired) -> AtomicType {
return SrcVal ^ Desired;
};
auto NEGDesired = [](AtomicType SrcVal, AtomicType Desired) -> AtomicType {
return -SrcVal;
};
auto SWAPDesired = [](AtomicType SrcVal, AtomicType Desired) -> AtomicType {
return Desired;
};
CASDesiredFn<AtomicType> DesiredFunction{};
switch (AtomicOp) {
case ExclusiveAtomicPairType::TYPE_SWAP:
DesiredFunction = SWAPDesired;
break;
case ExclusiveAtomicPairType::TYPE_ADD:
DesiredFunction = ADDDesired;
break;
case ExclusiveAtomicPairType::TYPE_SUB:
DesiredFunction = SUBDesired;
break;
case ExclusiveAtomicPairType::TYPE_AND:
DesiredFunction = ANDDesired;
break;
case ExclusiveAtomicPairType::TYPE_OR:
DesiredFunction = ORDesired;
break;
case ExclusiveAtomicPairType::TYPE_EOR:
DesiredFunction = EORDesired;
break;
case ExclusiveAtomicPairType::TYPE_NEG:
DesiredFunction = NEGDesired;
break;
default:
LogMan::Msg::E("Unhandled JIT SIGBUS Atomic mem op 0x%02x", AtomicOp);
return false;
break;
}
auto Res = DoCAS64<DoRetry>(
mcontext->regs[DataSourceReg],
0, // Unused
Addr,
NOPExpected,
DesiredFunction);
if (AtomicFetch && ResultReg != 31) {
// On atomic fetch then we store the resulting value back in to the loadacquire destination register
// We want the memory value BEFORE the ALU op
mcontext->regs[ResultReg] = Res;
}
}
// Multiply by 4 for number of bytes to skip
return NumInstructionsToSkip * 4;
}
}
@@ -18,6 +18,36 @@ namespace FEXCore::ArchHelpers::Arm64 {
constexpr uint32_t STLXP_MASK = 0xBF'E0'80'00;
constexpr uint32_t STLXP_INST = 0x88'20'80'00;
constexpr uint32_t LDAXR_MASK = 0x3F'FF'FC'00;
constexpr uint32_t LDAXR_INST = 0x08'5F'FC'00;
constexpr uint32_t STLXR_MASK = 0x3F'E0'FC'00;
constexpr uint32_t STLXR_INST = 0x08'00'FC'00;
constexpr uint32_t CBNZ_MASK = 0x7F'00'00'00;
constexpr uint32_t CBNZ_INST = 0x35'00'00'00;
constexpr uint32_t ALU_OP_MASK = 0x7F'00'00'00;
constexpr uint32_t ADD_INST = 0x0B'00'00'00;
constexpr uint32_t SUB_INST = 0x4B'00'00'00;
constexpr uint32_t CMP_INST = 0x6B'00'00'00;
constexpr uint32_t AND_INST = 0x0A'00'00'00;
constexpr uint32_t OR_INST = 0x2A'00'00'00;
constexpr uint32_t EOR_INST = 0x4A'00'00'00;
constexpr uint32_t CCMP_MASK = 0x7F'E0'0C'10;
constexpr uint32_t CCMP_INST = 0x7A'40'00'00;
enum ExclusiveAtomicPairType {
TYPE_SWAP,
TYPE_ADD,
TYPE_SUB,
TYPE_AND,
TYPE_OR,
TYPE_EOR,
TYPE_NEG, // This is just a sub with zero. Need to know the differences
};
// Load ops are 4 bits
// Acquire and release bits are independent on the instruction
constexpr uint32_t ATOMIC_ADD_OP = 0b0000;
@@ -30,7 +60,29 @@ namespace FEXCore::ArchHelpers::Arm64 {
constexpr uint32_t ATOMIC_UMIN_OP = 0b0111;
constexpr uint32_t ATOMIC_SWAP_OP = 0b1000;
constexpr uint32_t REGISTER_MASK = 0b11111;
constexpr uint32_t RD_OFFSET = 0;
constexpr uint32_t RN_OFFSET = 5;
constexpr uint32_t RM_OFFSET = 16;
inline uint32_t GetRdReg(uint32_t Instr) {
return (Instr >> RD_OFFSET) & REGISTER_MASK;
}
inline uint32_t GetRnReg(uint32_t Instr) {
return (Instr >> RN_OFFSET) & REGISTER_MASK;
}
inline uint32_t GetRmReg(uint32_t Instr) {
return (Instr >> RM_OFFSET) & REGISTER_MASK;
}
bool HandleAtomicLoad(void *_ucontext, void *_info, uint32_t Instr);
bool HandleAtomicStore(void *_ucontext, void *_info, uint32_t Instr);
bool HandleAtomicLoad128(void *_ucontext, void *_info, uint32_t Instr);
uint64_t HandleAtomicLoadstoreExclusive(void *_ucontext, void *_info);
bool HandleCASPAL(void *_ucontext, void *_info, uint32_t Instr);
uint64_t HandleCASPAL_ARMv8(void *_ucontext, void *_info, uint32_t Instr);
bool HandleCASAL(void *_ucontext, void *_info, uint32_t Instr);
bool HandleAtomicMemOp(void *_ucontext, void *_info, uint32_t Instr);
}
@@ -4,6 +4,11 @@
#include <FEXCore/Core/CoreState.h>
#include "aarch64/cpu-aarch64.h"
#include "cpu-features.h"
#include "aarch64/instructions-aarch64.h"
#include "utils-vixl.h"
#include <tuple>
namespace FEXCore::CPU {
#define STATE x28
@@ -143,22 +148,26 @@ void Arm64Emitter::PopCalleeSavedRegisters() {
void Arm64Emitter::SpillStaticRegs() {
for (size_t i = 0; i < SRA64.size(); i+=2) {
stp(SRA64[i], SRA64[i+1], MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[i])));
}
if (StaticRegisterAllocation()) {
for (size_t i = 0; i < SRA64.size(); i+=2) {
stp(SRA64[i], SRA64[i+1], MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[i])));
}
for (size_t i = 0; i < SRAFPR.size(); i+=2) {
stp(SRAFPR[i].Q(), SRAFPR[i+1].Q(), MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.xmm[i][0])));
for (size_t i = 0; i < SRAFPR.size(); i+=2) {
stp(SRAFPR[i].Q(), SRAFPR[i+1].Q(), MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.xmm[i][0])));
}
}
}
void Arm64Emitter::FillStaticRegs() {
for (size_t i = 0; i < SRA64.size(); i+=2) {
ldp(SRA64[i], SRA64[i+1], MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[i])));
}
if (StaticRegisterAllocation()) {
for (size_t i = 0; i < SRA64.size(); i+=2) {
ldp(SRA64[i], SRA64[i+1], MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[i])));
}
for (size_t i = 0; i < SRAFPR.size(); i+=2) {
ldp(SRAFPR[i].Q(), SRAFPR[i+1].Q(), MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.xmm[i][0])));
for (size_t i = 0; i < SRAFPR.size(); i+=2) {
ldp(SRAFPR[i].Q(), SRAFPR[i+1].Q(), MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.xmm[i][0])));
}
}
}
@@ -222,7 +231,7 @@ void Arm64Emitter::ResetStack() {
}
void Arm64Emitter::Align16B() {
uint64_t CurrentOffset = GetBuffer()->GetOffsetAddress<uint64_t>(GetCursorOffset());
uint64_t CurrentOffset = GetCursorAddress<uint64_t>();
for (uint64_t i = (16 - (CurrentOffset & 0xF)); i != 0; i -= 4) {
nop();
}
@@ -1,7 +1,16 @@
#pragma once
#include "aarch64/assembler-aarch64.h"
#include "aarch64/constants-aarch64.h"
#include "aarch64/cpu-aarch64.h"
#include "aarch64/operands-aarch64.h"
#include "platform-vixl.h"
#include "FEXCore/Config/Config.h"
#include <array>
#include <stddef.h>
#include <stdint.h>
#include <utility>
namespace FEXCore::CPU {
using namespace vixl;
@@ -72,6 +81,8 @@ protected:
uint32_t DCacheLineSize{};
uint32_t ICacheLineSize{};
FEX_CONFIG_OPT(StaticRegisterAllocation, SRA);
};
}
@@ -1,6 +1,7 @@
#include "Interface/Core/ArchHelpers/Arm64.h"
#include <FEXCore/Utils/LogManager.h>
#include <stdint.h>
namespace FEXCore::ArchHelpers::Arm64 {
@@ -23,4 +24,4 @@ bool HandleAtomicMemOp(void *_ucontext, void *_info, uint32_t Instr) {
}
#endif
}
}
@@ -18,6 +18,7 @@ struct X86ContextBackup {
// RIP and RSP is stored in GPRs here
uint64_t GPRs[23];
FEXCore::x86_64::_libc_fpstate FPRState;
uint64_t sa_mask;
// Guest state
int Signal;
@@ -35,6 +36,7 @@ struct ArmContextBackup {
uint32_t FPSR;
uint32_t FPCR;
__uint128_t FPRs[32];
uint64_t sa_mask;
// Guest state
int Signal;
@@ -44,6 +46,11 @@ struct ArmContextBackup {
static constexpr int RedZoneSize = 0;
};
static inline ucontext_t* GetUContext(void* ucontext) {
ucontext_t* _context = (ucontext_t*)ucontext;
return _context;
}
static inline mcontext_t* GetMContext(void* ucontext) {
ucontext_t* _context = (ucontext_t*)ucontext;
return &_context->uc_mcontext;
@@ -102,6 +109,7 @@ using ContextBackup = ArmContextBackup;
template <typename T>
static inline void BackupContext(void* ucontext, T *Backup) {
if constexpr (std::is_same<T, ArmContextBackup>::value) {
auto _ucontext = GetUContext(ucontext);
auto _mcontext = GetMContext(ucontext);
memcpy(&Backup->GPRs[0], &_mcontext->regs[0], 31 * sizeof(uint64_t));
@@ -115,6 +123,9 @@ static inline void BackupContext(void* ucontext, T *Backup) {
Backup->FPSR = HostState->FPSR;
Backup->FPCR = HostState->FPCR;
memcpy(&Backup->FPRs[0], &HostState->FPRs[0], 32 * sizeof(__uint128_t));
// Save the signal mask so we can restore it
memcpy(&Backup->sa_mask, &_ucontext->uc_sigmask, sizeof(uint64_t));
} else {
ERROR_AND_DIE("Wrong context type"); // This must be a runtime error
}
@@ -123,6 +134,7 @@ static inline void BackupContext(void* ucontext, T *Backup) {
template <typename T>
static inline void RestoreContext(void* ucontext, T *Backup) {
if constexpr (std::is_same<T, ArmContextBackup>::value) {
auto _ucontext = GetUContext(ucontext);
auto _mcontext = GetMContext(ucontext);
HostFPRState *HostState = reinterpret_cast<HostFPRState*>(&_mcontext->__reserved[0]);
@@ -136,6 +148,9 @@ static inline void RestoreContext(void* ucontext, T *Backup) {
ArchHelpers::Context::SetPc(ucontext, Backup->PrevPC);
ArchHelpers::Context::SetSp(ucontext, Backup->PrevSP);
memcpy(&_mcontext->regs[0], &Backup->GPRs[0], 31 * sizeof(uint64_t));
// Restore the signal mask now
memcpy(&_ucontext->uc_sigmask, &Backup->sa_mask, sizeof(uint64_t));
} else {
ERROR_AND_DIE("Wrong context type"); // This must be a runtime error
}
@@ -181,6 +196,7 @@ using ContextBackup = X86ContextBackup;
template <typename T>
static inline void BackupContext(void* ucontext, T *Backup) {
if constexpr (std::is_same<T, X86ContextBackup>::value) {
auto _ucontext = GetUContext(ucontext);
auto _mcontext = GetMContext(ucontext);
// Copy the GPRs
@@ -188,6 +204,9 @@ static inline void BackupContext(void* ucontext, T *Backup) {
// Copy the FPRState
memcpy(&Backup->FPRState, _mcontext->fpregs, sizeof(X86ContextBackup::FPRState));
// XXX: Save 256bit and 512bit AVX register state
// Save the signal mask so we can restore it
memcpy(&Backup->sa_mask, &_ucontext->uc_sigmask, sizeof(uint64_t));
} else {
ERROR_AND_DIE("Wrong context type"); // This must be a runtime error
}
@@ -196,12 +215,16 @@ static inline void BackupContext(void* ucontext, T *Backup) {
template <typename T>
static inline void RestoreContext(void* ucontext, T *Backup) {
if constexpr (std::is_same<T, X86ContextBackup>::value) {
auto _ucontext = GetUContext(ucontext);
auto _mcontext = GetMContext(ucontext);
// Copy the GPRs
memcpy(&_mcontext->gregs[0], &Backup->GPRs[0], sizeof(X86ContextBackup::GPRs));
// Copy the FPRState
memcpy(_mcontext->fpregs, &Backup->FPRState, sizeof(X86ContextBackup::FPRState));
// Restore the signal mask now
memcpy(&_ucontext->uc_sigmask, &Backup->sa_mask, sizeof(uint64_t));
} else {
ERROR_AND_DIE("Wrong context type"); // This must be a runtime error
}
@@ -209,4 +232,4 @@ static inline void RestoreContext(void* ucontext, T *Backup) {
#endif
} // namespace FEXCore::ArchHelpers::Context
} // namespace FEXCore::ArchHelpers::Context
@@ -2,6 +2,7 @@
#include <FEXCore/Utils/LogManager.h>
#include <cstring>
#include <fstream>
#include <utility>
namespace FEXCore {
void BlockSamplingData::DumpBlockData() {
+2 -1
View File
@@ -1,6 +1,7 @@
#pragma once
#include <cstdint>
#include <unordered_map>
#include <stdint.h>
namespace FEXCore {
class BlockSamplingData {
+5 -2
View File
@@ -5,8 +5,11 @@ desc: Handles presented capability bits for guest cpu
$end_info$
*/
#include <FEXCore/Config/Config.h>
#include <FEXCore/Core/CPUID.h>
#include "Interface/Context/Context.h"
#include "Interface/Core/CPUID.h"
#include "Interface/Core/HostFeatures.h"
#include "git_version.h"
#include <cstring>
@@ -96,7 +99,7 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_01h(uint32_t Leaf) {
(0 << 5) | // VMX
(0 << 6) | // SMX
(0 << 7) | // Intel SpeedStep
(0 << 8) | // Thermal Monitor 2
(1 << 8) | // Thermal Monitor 2
(1 << 9) | // SSSE3
(0 << 10) | // L1 context ID
(0 << 11) | // Silicon debug
@@ -151,7 +154,7 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_01h(uint32_t Leaf) {
(1 << 26) | // SSE2
(0 << 27) | // Self Snoop
(1 << 28) | // Max APIC IDs reserved field is valid
(0 << 29) | // Thermal monitor
(1 << 29) | // Thermal monitor
(0 << 30) | // Reserved
(0 << 31); // Pending break enable
return Res;
+3 -4
View File
@@ -4,7 +4,9 @@
#include <FEXCore/Core/CPUID.h>
#include <FEXCore/Config/Config.h>
#include <FEXCore/Utils/LogManager.h>
#include <cstdint>
#include <utility>
namespace FEXCore {
namespace Context {
@@ -28,9 +30,6 @@ public:
auto Handler = FunctionHandlers.find(Function);
if (Handler == FunctionHandlers.end()) {
#ifndef NDEBUG
LogMan::Msg::E("Unhandled CPU ID function, 0x%x-0x%x", Function, Leaf);
#endif
return Function_Reserved(Leaf);
}
+14 -1
View File
@@ -1,8 +1,21 @@
#include "Interface/Context/Context.h"
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/CompileService.h"
#include "Interface/Core/InternalThreadState.h"
#include "Interface/Core/OpcodeDispatcher.h"
#include "FEXCore/Debug/InternalThreadState.h"
#include "FEXCore/HLE/Linux/ThreadManagement.h"
#include "Interface/IR/PassManager.h"
#include <FEXCore/Core/CPUBackend.h>
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/Core/SignalDelegator.h>
#include <FEXCore/Utils/Event.h>
#include <FEXCore/Utils/LogManager.h>
#include <FEXCore/Utils/Threads.h>
#include <memory>
#include <pthread.h>
#include <stdio.h>
namespace FEXCore {
static void* ThreadHandler(void *Arg) {
+6 -7
View File
@@ -1,23 +1,22 @@
#pragma once
#include <FEXCore/Core/CPUBackend.h>
#include <FEXCore/Debug/InternalThreadState.h>
#include <FEXCore/Utils/Event.h>
#include <FEXCore/Utils/Threads.h>
#include <atomic>
#include <memory>
#include <thread>
#include <unordered_map>
#include <mutex>
#include <queue>
#include <stdint.h>
#include <vector>
namespace FEXCore {
namespace Context {
struct Context;
}
namespace Core {
struct InternalThreadState;
}
namespace IR {
class IRListView;
class RegisterAllocationData;
};
class CompileService final {
+59 -24
View File
@@ -7,42 +7,70 @@ desc: Glues Frontend, OpDispatcher and IR Opts & Compilation, LookupCache, Dispa
$end_info$
*/
#include "Common/MathUtils.h"
#include "Common/Paths.h"
#include "Interface/Context/Context.h"
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/BlockSamplingData.h"
#include "Interface/Core/CompileService.h"
#include "Interface/Core/Core.h"
#include "Interface/Core/DebugData.h"
#include "Interface/Core/CPUID.h"
#include "Interface/Core/Frontend.h"
#include "Interface/Core/GdbServer.h"
#include "Interface/Core/OpcodeDispatcher.h"
#include "Interface/Core/Interpreter/InterpreterCore.h"
#include "Interface/Core/JIT/JITCore.h"
#include "Interface/HLE/Thunks/Thunks.h"
#include "Interface/IR/Passes/RegisterAllocationPass.h"
#include "Interface/IR/Passes.h"
#include "Interface/IR/PassManager.h"
#include <FEXCore/Config/Config.h>
#include <FEXCore/Core/CodeLoader.h>
#include <FEXCore/Core/Context.h>
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/Core/CPUBackend.h>
#include <FEXCore/Core/SignalDelegator.h>
#include <FEXCore/Core/X86Enums.h>
#include <FEXCore/Debug/InternalThreadState.h>
#include <FEXCore/Debug/X86Tables.h>
#include <FEXCore/HLE/SyscallHandler.h>
#include <FEXCore/HLE/Linux/ThreadManagement.h>
#include <FEXCore/IR/IR.h>
#include <FEXCore/IR/IREmitter.h>
#include <FEXCore/IR/IntrusiveIRList.h>
#include <FEXCore/IR/RegisterAllocationData.h>
#include <FEXCore/Utils/Allocator.h>
#include <FEXCore/Utils/Event.h>
#include <FEXCore/Utils/LogManager.h>
#include <FEXCore/Utils/Threads.h>
#include "Interface/HLE/Thunks/Thunks.h"
#include "FEXCore/Utils/Allocator.h"
#include <xxh3.h>
#include <fstream>
#include <unistd.h>
#include <filesystem>
#include <algorithm>
#include <array>
#include <atomic>
#include <chrono>
#include <condition_variable>
#include <cstdint>
#include <filesystem>
#include <functional>
#include <map>
#include <memory>
#include <mutex>
#include <queue>
#include <set>
#include <shared_mutex>
#include <signal.h>
#include <stdio.h>
#include <string.h>
#include <string>
#include <string_view>
#include <sstream>
#include <sys/mman.h>
#include <unistd.h>
#include <sys/stat.h>
#include "Interface/Core/GdbServer.h"
#include <sys/syscall.h>
#include <type_traits>
#include <unistd.h>
#include <unordered_map>
#include <utility>
#include <vector>
#include <xxhash.h>
namespace FEXCore::CPU {
bool CreateCPUCore(FEXCore::Context::Context *CTX) {
@@ -195,7 +223,7 @@ namespace FEXCore::Context {
}
}
bool Context::InitCore(FEXCore::CodeLoader *Loader) {
FEXCore::Core::InternalThreadState* Context::InitCore(FEXCore::CodeLoader *Loader) {
ThunkHandler.reset(FEXCore::ThunkHandler::Create());
LocalLoader = Loader;
@@ -216,6 +244,7 @@ namespace FEXCore::Context {
NewThreadState.flags[1] = 1;
NewThreadState.flags[9] = 1;
NewThreadState.FCW = 0x37F;
NewThreadState.FTW = 0xFFFF;
FEXCore::Core::InternalThreadState *Thread = CreateThread(&NewThreadState, 0);
@@ -227,8 +256,7 @@ namespace FEXCore::Context {
Thread->CurrentFrame->State.rip = StartingRIP = Loader->DefaultRIP();
InitializeThreadData(Thread);
return true;
return Thread;
}
void Context::StartGdbServer() {
@@ -242,8 +270,7 @@ namespace FEXCore::Context {
DebugServer.reset();
}
void Context::HandleCallback(uint64_t RIP) {
auto Thread = Core::ThreadData.Thread;
void Context::HandleCallback(FEXCore::Core::InternalThreadState *Thread, uint64_t RIP) {
Thread->CPUBackend->CallbackPtr(Thread->CurrentFrame, RIP);
}
@@ -470,8 +497,10 @@ namespace FEXCore::Context {
Stop(false /* Ignore current thread */);
});
State->CTX = this;
#if _M_ARM_64
bool DoSRA = true;
bool DoSRA = State->CTX->Config.StaticRegisterAllocation;
#else
bool DoSRA = false;
#endif
@@ -481,8 +510,6 @@ namespace FEXCore::Context {
State->PassManager->RegisterSyscallHandler(SyscallHandler);
State->CTX = this;
// Create CPU backend
switch (Config.Core) {
case FEXCore::Config::CONFIG_INTERPRETER:
@@ -586,6 +613,12 @@ namespace FEXCore::Context {
// Clean up dead stacks
FEXCore::Threads::Thread::CleanupAfterFork();
if (LiveThread->CompileService) {
// If this live thread had a compile service then it no longer exists
// Erase the shared_ptr
LiveThread->CompileService.reset();
}
}
void Context::AddBlockMapping(FEXCore::Core::InternalThreadState *Thread, uint64_t Address, void *Ptr, uint64_t Start, uint64_t Length) {
@@ -1084,7 +1117,9 @@ namespace FEXCore::Context {
if (NewBlock == 0) {
LogMan::Msg::E("CompileBlockJit: Failed to compile code %lX - aborting process", GuestRIP);
abort();
// Return similar behaviour of SIGILL abort
Frame->Thread->StatusCode = 128 + SIGILL;
Stop(false /* Ignore current thread */);
}
}
@@ -1,16 +1,29 @@
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/ArchHelpers/MContext.h"
#include "Interface/Core/Dispatcher/Arm64Dispatcher.h"
#include "Interface/Core/Interpreter/InterpreterClass.h"
#include "Interface/Context/Context.h"
#include "Interface/Context/Context.h"
#include "Interface/Core/X86HelperGen.h"
#include <FEXCore/Core/CPUBackend.h>
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/Core/X86Enums.h>
#include <FEXCore/Debug/InternalThreadState.h>
#include <array>
#include <bit>
#include <cmath>
#include <cstdint>
#include <memory>
#include <stddef.h>
#include "aarch64/assembler-aarch64.h"
#include "aarch64/constants-aarch64.h"
#include "aarch64/operands-aarch64.h"
#include "aarch64/cpu-aarch64.h"
#include "aarch64/disasm-aarch64.h"
#include "code-buffer-vixl.h"
#include "platform-vixl.h"
namespace FEXCore::CPU {
@@ -25,8 +38,7 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::
SRAEnabled = config.StaticRegisterAssignment;
SetAllowAssembler(true);
auto Buffer = GetBuffer();
DispatchPtr = Buffer->GetOffsetAddress<CPUBackend::AsmDispatch>(GetCursorOffset());
DispatchPtr = GetCursorAddress<CPUBackend::AsmDispatch>();
// while (true) {
// Ptr = FindBlock(RIP)
@@ -59,7 +71,7 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::
add(x0, sp, 0);
str(x0, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, ReturningStackLocation)));
AbsoluteLoopTopAddressFillSRA = Buffer->GetOffsetAddress<uint64_t>(GetCursorOffset());
AbsoluteLoopTopAddressFillSRA = GetCursorAddress<uint64_t>();
if (SRAEnabled) {
FillStaticRegs();
@@ -178,11 +190,11 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::
{
bind(&ExitSpillSRA);
ThreadStopHandlerAddressSpillSRA = Buffer->GetOffsetAddress<uint64_t>(GetCursorOffset());
ThreadStopHandlerAddressSpillSRA = GetCursorAddress<uint64_t>();
if (SRAEnabled)
SpillStaticRegs();
ThreadStopHandlerAddress = Buffer->GetOffsetAddress<uint64_t>(GetCursorOffset());
ThreadStopHandlerAddress = GetCursorAddress<uint64_t>();
PopCalleeSavedRegisters();
@@ -192,7 +204,7 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::
}
{
ExitFunctionLinkerAddress = Buffer->GetOffsetAddress<uint64_t>(GetCursorOffset());
ExitFunctionLinkerAddress = GetCursorAddress<uint64_t>();
if (SRAEnabled)
SpillStaticRegs();
@@ -229,7 +241,7 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::
}
{
SignalHandlerReturnAddress = Buffer->GetOffsetAddress<uint64_t>(GetCursorOffset());
SignalHandlerReturnAddress = GetCursorAddress<uint64_t>();
// Now to get back to our old location we need to do a fault dance
// We can't use SIGTRAP here since gdb catches it and never gives it to the application!
@@ -237,12 +249,12 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::
}
{
ThreadPauseHandlerAddressSpillSRA = Buffer->GetOffsetAddress<uint64_t>(GetCursorOffset());
ThreadPauseHandlerAddressSpillSRA = GetCursorAddress<uint64_t>();
if (SRAEnabled)
SpillStaticRegs();
bind(&ThreadPauseHandler);
ThreadPauseHandlerAddress = Buffer->GetOffsetAddress<uint64_t>(GetCursorOffset());
ThreadPauseHandlerAddress = GetCursorAddress<uint64_t>();
// We are pausing, this means the frontend should be waiting for this thread to idle
// We will have faulted and jumped to this location at this point
@@ -252,7 +264,7 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::
ldr(x2, &l_Sleep);
blr(x2);
PauseReturnInstruction = Buffer->GetOffsetAddress<uint64_t>(GetCursorOffset());
PauseReturnInstruction = GetCursorAddress<uint64_t>();
// Fault to start running again
hlt(0);
}
@@ -273,7 +285,7 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::
// On return to the thunk, the thunk can get whatever its return value is from the thread context depending on ABI handling on its end
// When the thunk itself returns, it'll do its regular return logic there
// void ReentrantCallback(FEXCore::Core::InternalThreadState *Thread, uint64_t RIP);
CallbackPtr = Buffer->GetOffsetAddress<CPUBackend::JITCallback>(GetCursorOffset());
CallbackPtr = GetCursorAddress<CPUBackend::JITCallback>();
// We expect the thunk to have previously pushed the registers it was using
PushCalleeSavedRegisters();
@@ -322,7 +334,7 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::
FinalizeCode();
Start = reinterpret_cast<uint64_t>(DispatchPtr);
End = Buffer->GetOffsetAddress<uint64_t>(GetCursorOffset());
End = GetCursorAddress<uint64_t>();
vixl::aarch64::CPU::EnsureIAndDCacheCoherency(reinterpret_cast<void*>(DispatchPtr), End - reinterpret_cast<uint64_t>(DispatchPtr));
GetBuffer()->SetExecutable();
@@ -3,7 +3,13 @@
#include "Interface/Core/ArchHelpers/Arm64Emitter.h"
#include "Interface/Core/Dispatcher/Dispatcher.h"
#include "aarch64/assembler-aarch64.h"
namespace FEXCore::Context {
struct Context;
}
namespace FEXCore::Core {
struct InternalThreadState;
}
namespace FEXCore::CPU {
@@ -15,4 +21,4 @@ class Arm64Dispatcher final : public Dispatcher, public Arm64Emitter {
void SpillSRA(void *ucontext) override;
};
}
}
@@ -1,8 +1,23 @@
#include "Interface/Core/Dispatcher/Dispatcher.h"
#include "Common/MathUtils.h"
#include "Interface/Core/ArchHelpers/MContext.h"
#include "Interface/Core/Dispatcher/Dispatcher.h"
#include "Interface/Core/X86HelperGen.h"
#include <FEXCore/Config/Config.h>
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/Core/SignalDelegator.h>
#include <FEXCore/Core/UContext.h>
#include <FEXCore/Core/X86Enums.h>
#include <FEXCore/Debug/InternalThreadState.h>
#include <FEXCore/Utils/Event.h>
#include <FEXCore/Utils/LogManager.h>
#include <atomic>
#include <condition_variable>
#include <bits/types/siginfo_t.h>
#include <signal.h>
#include <string.h>
namespace FEXCore::CPU {
@@ -54,6 +69,7 @@ void Dispatcher::StoreThreadState(int Signal, void *ucontext) {
}
void Dispatcher::RestoreThreadState(void *ucontext) {
LOGMAN_THROW_A(!SignalFrames.empty(), "Trying to restore a signal frame when we don't have any");
uint64_t OldSP = SignalFrames.top();
SignalFrames.pop();
uintptr_t NewSP = OldSP;
@@ -64,10 +80,38 @@ void Dispatcher::RestoreThreadState(void *ucontext) {
// Now restore host state
ArchHelpers::Context::RestoreContext(ucontext, Context);
}
// Restore the previous signal state
// This allows recursive signals to properly handle signal masking as we are walking back up the list of signals
CTX->SignalDelegation->SetCurrentSignal(Context->Signal);
static uint32_t ConvertSignalToTrapNo(int Signal, siginfo_t *HostSigInfo) {
switch (Signal) {
case SIGSEGV:
if (HostSigInfo->si_code == SEGV_MAPERR ||
HostSigInfo->si_code == SEGV_ACCERR) {
// Protection fault
return X86State::X86_TRAPNO_PF;
}
break;
}
// Unknown mapping, fall back to old behaviour and just pass signal
return Signal;
}
static uint32_t ConvertSignalToError(int Signal, siginfo_t *HostSigInfo) {
switch (Signal) {
case SIGSEGV:
if (HostSigInfo->si_code == SEGV_MAPERR ||
HostSigInfo->si_code == SEGV_ACCERR) {
// Protection fault
// Always a user fault for us
// XXX: PF_PROT and PF_WRITE
return X86State::X86_PF_USER;
}
break;
}
// Not a page fault issue
return 0;
}
bool Dispatcher::HandleGuestSignal(int Signal, void *info, void *ucontext, GuestSigAction *GuestAction, stack_t *GuestStack) {
@@ -78,60 +122,97 @@ bool Dispatcher::HandleGuestSignal(int Signal, void *info, void *ucontext, Guest
// We use this to track if it is safe to clear cache
++SignalHandlerRefCounter;
uint64_t OldPC = ArchHelpers::Context::GetPc(ucontext);
// Set the new PC
ArchHelpers::Context::SetPc(ucontext, AbsoluteLoopTopAddressFillSRA);
// Set our state register to point to our guest thread data
ArchHelpers::Context::SetState(ucontext, reinterpret_cast<uint64_t>(Frame));
uint64_t OldGuestSP = Frame->State.gregs[X86State::REG_RSP];
uint64_t NewGuestSP = OldGuestSP;
if (!(GuestStack->ss_flags & SS_DISABLE)) {
// If our guest is already inside of the alternative stack
// Then that means we are hitting recursive signals and we need to walk back the stack correctly
uint64_t AltStackBase = reinterpret_cast<uint64_t>(GuestStack->ss_sp);
uint64_t AltStackEnd = AltStackBase + GuestStack->ss_size;
if (OldGuestSP >= AltStackBase &&
OldGuestSP <= AltStackEnd) {
// We are already in the alt stack, the rest of the code will handle adjusting this
}
else {
NewGuestSP = AltStackEnd;
// Pulling from context here
bool Is64BitMode = CTX->Config.Is64BitMode;
uint64_t SignalReturn = CTX->X86CodeGen.SignalReturn;
// Spill the SRA regardless of signal handler type
// We are going to be returning to the top of the dispatcher which will fill again
// Otherwise we might load garbage
if (SRAEnabled) {
if (IsAddressInJITCode(OldPC, false)) {
// We are in jit, SRA must be spilled
SpillSRA(ucontext);
} else {
if (!IsAddressInJITCode(OldPC, true)) {
// This is likely to cause issues but in some cases it isn't fatal
// This can also happen if we have put a signal on hold, then we just reenabled the signal
// So we are in the syscall handler
// Only throw a log message in this case
LogMan::Msg::E("Signals in dispatcher have unsynchronized context");
}
}
}
// Back up past the redzone, which is 128bytes
// Don't need this offset if we aren't going to be putting siginfo in to it
NewGuestSP -= 128;
if (GuestAction->sa_flags & SA_SIGINFO) {
if (SRAEnabled) {
if (!IsAddressInJITCode(ArchHelpers::Context::GetPc(ucontext), false)) {
LOGMAN_THROW_A(!IsAddressInJITCode(ArchHelpers::Context::GetPc(ucontext), true), "Signals in dispatcher have unsynchronized context");
} else {
// We are in jit, SRA must be spilled
SpillSRA(ucontext);
// altstack is only used if the signal handler was setup with SA_ONSTACK
if (GuestAction->sa_flags & SA_ONSTACK) {
// Additionally the altstack is only used if the enabled (SS_DISABLE flag is not set)
if (!(GuestStack->ss_flags & SS_DISABLE)) {
// If our guest is already inside of the alternative stack
// Then that means we are hitting recursive signals and we need to walk back the stack correctly
uint64_t AltStackBase = reinterpret_cast<uint64_t>(GuestStack->ss_sp);
uint64_t AltStackEnd = AltStackBase + GuestStack->ss_size;
if (OldGuestSP >= AltStackBase &&
OldGuestSP <= AltStackEnd) {
// We are already in the alt stack, the rest of the code will handle adjusting this
}
else {
NewGuestSP = AltStackEnd;
}
}
}
if (Is64BitMode) {
// Back up past the redzone, which is 128bytes
// 32-bit doesn't have a redzone
NewGuestSP -= 128;
}
// siginfo_t
siginfo_t *HostSigInfo = reinterpret_cast<siginfo_t*>(info);
if (GuestAction->sa_flags & SA_SIGINFO) {
// Setup ucontext a bit
if (CTX->Config.Is64BitMode) {
if (Is64BitMode) {
NewGuestSP -= sizeof(FEXCore::x86_64::_libc_fpstate);
NewGuestSP = AlignDown(NewGuestSP, alignof(FEXCore::x86_64::_libc_fpstate));
uint64_t FPStateLocation = NewGuestSP;
NewGuestSP -= sizeof(FEXCore::x86_64::ucontext_t);
NewGuestSP = AlignDown(NewGuestSP, alignof(FEXCore::x86_64::ucontext_t));
uint64_t UContextLocation = NewGuestSP;
NewGuestSP -= sizeof(siginfo_t);
NewGuestSP = AlignDown(NewGuestSP, alignof(siginfo_t));
uint64_t SigInfoLocation = NewGuestSP;
FEXCore::x86_64::ucontext_t *guest_uctx = reinterpret_cast<FEXCore::x86_64::ucontext_t*>(UContextLocation);
siginfo_t *guest_siginfo = reinterpret_cast<siginfo_t*>(SigInfoLocation);
// We have extended float information
guest_uctx->uc_flags |= FEXCore::x86_64::UC_FP_XSTATE;
guest_uctx->uc_flags = FEXCore::x86_64::UC_FP_XSTATE;
// Pointer to where the fpreg memory is
guest_uctx->uc_mcontext.fpregs = &guest_uctx->__fpregs_mem;
guest_uctx->uc_mcontext.fpregs = reinterpret_cast<FEXCore::x86_64::_libc_fpstate*>(FPStateLocation);
FEXCore::x86_64::_libc_fpstate *fpstate = reinterpret_cast<FEXCore::x86_64::_libc_fpstate*>(FPStateLocation);
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_RIP] = Frame->State.rip;
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_EFL] = 0;
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_CSGSFS] = 0;
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_ERR] = ConvertSignalToError(Signal, HostSigInfo);
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_TRAPNO] = ConvertSignalToTrapNo(Signal, HostSigInfo);
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_OLDMASK] = 0;
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_CR2] = 0;
#define COPY_REG(x) \
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_##x] = Frame->State.gregs[X86State::REG_##x];
@@ -154,14 +235,15 @@ bool Dispatcher::HandleGuestSignal(int Signal, void *info, void *ucontext, Guest
#undef COPY_REG
// Copy float registers
memcpy(guest_uctx->__fpregs_mem._st, Frame->State.mm, sizeof(Frame->State.mm));
memcpy(guest_uctx->__fpregs_mem._xmm, Frame->State.xmm, sizeof(Frame->State.xmm));
memcpy(fpstate->_st, Frame->State.mm, sizeof(Frame->State.mm));
memcpy(fpstate->_xmm, Frame->State.xmm, sizeof(Frame->State.xmm));
// FCW store default
guest_uctx->__fpregs_mem.fcw = Frame->State.FCW;
fpstate->fcw = Frame->State.FCW;
fpstate->ftw = Frame->State.FTW;
// Reconstruct FSW
guest_uctx->__fpregs_mem.fsw =
fpstate->fsw =
(Frame->State.flags[FEXCore::X86State::X87FLAG_TOP_LOC] << 11) |
(Frame->State.flags[FEXCore::X86State::X87FLAG_C0_LOC] << 8) |
(Frame->State.flags[FEXCore::X86State::X87FLAG_C1_LOC] << 9) |
@@ -173,8 +255,6 @@ bool Dispatcher::HandleGuestSignal(int Signal, void *info, void *ucontext, Guest
guest_uctx->uc_stack.ss_sp = GuestStack->ss_sp;
guest_uctx->uc_stack.ss_size = GuestStack->ss_size;
// siginfo_t
siginfo_t *HostSigInfo = reinterpret_cast<siginfo_t*>(info);
// aarch64 and x86_64 siginfo_t matches. We can just copy this over
// SI_USER could also potentially have random data in it, needs to be bit perfect
// For guest faults we don't have a real way to reconstruct state to a real guest RIP
@@ -184,14 +264,77 @@ bool Dispatcher::HandleGuestSignal(int Signal, void *info, void *ucontext, Guest
Frame->State.gregs[X86State::REG_RDX] = UContextLocation;
}
else {
// XXX: 32bit Support
NewGuestSP -= sizeof(FEXCore::x86::_libc_fpstate);
NewGuestSP = AlignDown(NewGuestSP, alignof(FEXCore::x86::_libc_fpstate));
uint64_t FPStateLocation = NewGuestSP;
NewGuestSP -= sizeof(FEXCore::x86::ucontext_t);
uint64_t UContextLocation = 0; // NewGuestSP;
NewGuestSP = AlignDown(NewGuestSP, alignof(FEXCore::x86::ucontext_t));
uint64_t UContextLocation = NewGuestSP;
NewGuestSP -= sizeof(FEXCore::x86::siginfo_t);
NewGuestSP = AlignDown(NewGuestSP, alignof(FEXCore::x86::siginfo_t));
uint64_t SigInfoLocation = NewGuestSP;
FEXCore::x86::ucontext_t *guest_uctx = reinterpret_cast<FEXCore::x86::ucontext_t*>(UContextLocation);
FEXCore::x86::siginfo_t *guest_siginfo = reinterpret_cast<FEXCore::x86::siginfo_t*>(SigInfoLocation);
siginfo_t *HostSigInfo = reinterpret_cast<siginfo_t*>(info);
// We have extended float information
guest_uctx->uc_flags = FEXCore::x86::UC_FP_XSTATE;
// Pointer to where the fpreg memory is
guest_uctx->uc_mcontext.fpregs = static_cast<uint32_t>(FPStateLocation);
FEXCore::x86::_libc_fpstate *fpstate = reinterpret_cast<FEXCore::x86::_libc_fpstate*>(FPStateLocation);
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_GS] = Frame->State.gs;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_FS] = Frame->State.fs;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_ES] = Frame->State.es;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_DS] = Frame->State.ds;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_TRAPNO] = ConvertSignalToTrapNo(Signal, HostSigInfo);
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_ERR] = ConvertSignalToError(Signal, HostSigInfo);
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_EIP] = Frame->State.rip;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_CS] = Frame->State.cs;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_EFL] = 0;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_UESP] = 0;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_SS] = Frame->State.ss;
#define COPY_REG(x) \
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_##x] = Frame->State.gregs[X86State::REG_##x];
COPY_REG(RDI);
COPY_REG(RSI);
COPY_REG(RBP);
COPY_REG(RBX);
COPY_REG(RDX);
COPY_REG(RAX);
COPY_REG(RCX);
COPY_REG(RSP);
#undef COPY_REG
// Copy float registers
for (size_t i = 0; i < 8; ++i) {
// 32-bit st register size is only 10 bytes. Not padded to 16byte like x86-64
memcpy(&fpstate->_st[i], &Frame->State.mm[i], 10);
}
// Extended XMM state
fpstate->status = FEXCore::x86::fpstate_magic::MAGIC_XFPSTATE;
memcpy(fpstate->_xmm, Frame->State.xmm, sizeof(Frame->State.xmm));
// FCW store default
fpstate->fcw = Frame->State.FCW;
fpstate->ftw = Frame->State.FTW;
// Reconstruct FSW
fpstate->fsw =
(Frame->State.flags[FEXCore::X86State::X87FLAG_TOP_LOC] << 11) |
(Frame->State.flags[FEXCore::X86State::X87FLAG_C0_LOC] << 8) |
(Frame->State.flags[FEXCore::X86State::X87FLAG_C1_LOC] << 9) |
(Frame->State.flags[FEXCore::X86State::X87FLAG_C2_LOC] << 10) |
(Frame->State.flags[FEXCore::X86State::X87FLAG_C3_LOC] << 14);
// Copy over signal stack information
guest_uctx->uc_stack.ss_flags = GuestStack->ss_flags;
guest_uctx->uc_stack.ss_sp = static_cast<uint32_t>(reinterpret_cast<uint64_t>(GuestStack->ss_sp));
guest_uctx->uc_stack.ss_size = GuestStack->ss_size;
// These three elements are in every siginfo
guest_siginfo->si_signo = HostSigInfo->si_signo;
@@ -201,6 +344,12 @@ bool Dispatcher::HandleGuestSignal(int Signal, void *info, void *ucontext, Guest
switch (Signal) {
case SIGSEGV:
case SIGBUS:
// Macro expansion to get the si_addr
// This is the address trying to be accessed, not the RIP
guest_siginfo->_sifields._sigfault.addr = static_cast<uint32_t>(reinterpret_cast<uintptr_t>(HostSigInfo->si_addr));
break;
case SIGFPE:
case SIGILL:
// Macro expansion to get the si_addr
// Can't really give a real result here. Pull from the context for now
guest_siginfo->_sifields._sigfault.addr = Frame->State.rip;
@@ -212,40 +361,59 @@ bool Dispatcher::HandleGuestSignal(int Signal, void *info, void *ucontext, Guest
guest_siginfo->_sifields._sigchld.utime = HostSigInfo->si_utime;
guest_siginfo->_sifields._sigchld.stime = HostSigInfo->si_stime;
break;
default:
LogMan::Msg::D("Unhandled siginfo_t signal: %d", Signal);
// Hope for the best, most things just copy over
memcpy(guest_siginfo, info, sizeof(siginfo_t));
break;
case SIGALRM:
case SIGVTALRM:
guest_siginfo->_sifields._timer.tid = HostSigInfo->si_timerid;
guest_siginfo->_sifields._timer.overrun = HostSigInfo->si_overrun;
guest_siginfo->_sifields._timer.sigval.sival_int = HostSigInfo->si_int;
break;
default:
LogMan::Msg::E("Unhandled siginfo_t for signal: %d\n", Signal);
break;
}
NewGuestSP -= 4;
*(uint32_t*)NewGuestSP = UContextLocation;
NewGuestSP -= 4;
*(uint32_t*)NewGuestSP = SigInfoLocation;
NewGuestSP -= 4;
*(uint32_t*)NewGuestSP = Signal;
}
Frame->State.rip = reinterpret_cast<uint64_t>(GuestAction->sigaction_handler.sigaction);
}
else {
if (!Is64BitMode) {
NewGuestSP -= 4;
*(uint32_t*)NewGuestSP = Signal;
}
Frame->State.rip = reinterpret_cast<uint64_t>(GuestAction->sigaction_handler.handler);
}
if (CTX->Config.Is64BitMode) {
Frame->State.gregs[X86State::REG_RDI] = Signal;
if (Is64BitMode) {
Frame->State.gregs[FEXCore::X86State::REG_RDI] = Signal;
// Set up the new SP for stack handling
NewGuestSP -= 8;
*(uint64_t*)NewGuestSP = CTX->X86CodeGen.SignalReturn;
Frame->State.gregs[X86State::REG_RSP] = NewGuestSP;
*(uint64_t*)NewGuestSP = SignalReturn;
Frame->State.gregs[FEXCore::X86State::REG_RSP] = NewGuestSP;
}
else {
NewGuestSP -= 4;
*(uint32_t*)NewGuestSP = CTX->X86CodeGen.SignalReturn;
LOGMAN_THROW_A(CTX->X86CodeGen.SignalReturn < 0x1'0000'0000ULL, "This needs to be below 4GB");
Frame->State.gregs[X86State::REG_RSP] = NewGuestSP;
*(uint32_t*)NewGuestSP = SignalReturn;
LOGMAN_THROW_A(SignalReturn < 0x1'0000'0000ULL, "This needs to be below 4GB");
Frame->State.gregs[FEXCore::X86State::REG_RSP] = NewGuestSP;
}
// The guest starts its signal frame with a zero initialized FPU
// Set that up now. Little bit costly but it's a requirement
// This state will be restored on rt_sigreturn
memset(Frame->State.xmm, 0, sizeof(Frame->State.xmm));
memset(Frame->State.mm, 0, sizeof(Frame->State.mm));
Frame->State.FCW = 0x37F;
Frame->State.FTW = 0xFFFF;
return true;
}
@@ -291,9 +459,6 @@ bool Dispatcher::HandleSignalPause(int Signal, void *info, void *ucontext) {
ArchHelpers::Context::SetPc(ucontext, ThreadPauseHandlerAddress);
}
// Set the new PC
ArchHelpers::Context::SetPc(ucontext, ThreadPauseHandlerAddress);
// Set our state register to point to our guest thread data
ArchHelpers::Context::SetState(ucontext, reinterpret_cast<uint64_t>(Frame));
@@ -1,11 +1,24 @@
#pragma once
#include <FEXCore/Core/CPUBackend.h>
#include <FEXCore/Core/SignalDelegator.h>
#include "Interface/Context/Context.h"
#include <bits/types/stack_t.h>
#include <cstdint>
#include <stddef.h>
#include <stack>
#include <tuple>
#include <vector>
namespace FEXCore {
struct GuestSigAction;
}
namespace FEXCore::Core {
struct CpuStateFrame;
struct InternalThreadState;
}
namespace FEXCore::CPU {
@@ -1,10 +1,23 @@
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/Dispatcher/X86Dispatcher.h"
#include "Interface/Core/Interpreter/InterpreterClass.h"
#include "Interface/Core/X86HelperGen.h"
#include "Interface/Context/Context.h"
#include <FEXCore/Core/X86Enums.h>
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/Core/CPUBackend.h>
#include <FEXCore/Debug/InternalThreadState.h>
#include <FEXCore/Utils/Allocator.h>
#include <cmath>
#include <memory>
#include <stddef.h>
#include <stdint.h>
#include <sys/mman.h>
#include "xbyak/xbyak.h"
namespace FEXCore::CPU {
static constexpr size_t MAX_DISPATCHER_CODE_SIZE = 4096;
@@ -12,7 +25,9 @@ static constexpr size_t MAX_DISPATCHER_CODE_SIZE = 4096;
X86Dispatcher::X86Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::InternalThreadState *Thread, DispatcherConfig &config)
: Dispatcher(ctx, Thread)
, Xbyak::CodeGenerator(MAX_DISPATCHER_CODE_SIZE, nullptr, this) {
, Xbyak::CodeGenerator(MAX_DISPATCHER_CODE_SIZE,
FEXCore::Allocator::mmap(nullptr, MAX_DISPATCHER_CODE_SIZE, PROT_READ | PROT_WRITE | PROT_EXEC, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0),
nullptr) {
using namespace Xbyak;
using namespace Xbyak::util;
@@ -298,7 +313,7 @@ X86Dispatcher::X86Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::Inte
}
X86Dispatcher::~X86Dispatcher() {
FEXCore::Allocator::munmap(top_, MAX_DISPATCHER_CODE_SIZE);
}
#ifdef _M_X86_64
@@ -2,26 +2,24 @@
#include "Interface/Core/Dispatcher/Dispatcher.h"
#include <FEXCore/Utils/Allocator.h>
#define XBYAK64
#include <xbyak/xbyak.h>
namespace FEXCore::Context {
struct Context;
}
namespace FEXCore::Core {
struct InternalThreadState;
}
namespace FEXCore::CPU {
class X86Dispatcher final : public Dispatcher, public Xbyak::CodeGenerator, public Xbyak::Allocator {
class X86Dispatcher final : public Dispatcher, public Xbyak::CodeGenerator {
public:
X86Dispatcher(FEXCore::Context::Context *ctx, FEXCore::Core::InternalThreadState *Thread, DispatcherConfig &config);
virtual ~X86Dispatcher() override;
// Xbyak::Allocator
Xbyak::uint8 *alloc(size_t size) override { Size = size; return reinterpret_cast<uint8_t*>(FEXCore::Allocator::mmap(nullptr, size, PROT_READ | PROT_WRITE | PROT_EXEC, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0)); }
void free(Xbyak::uint8 *p) override { FEXCore::Allocator::munmap(p, Size); }
bool useProtect() const override { return false; }
private:
size_t Size{};
};
}
+59 -19
View File
@@ -7,17 +7,24 @@ $end_info$
#include "Interface/Context/Context.h"
#include "Interface/Core/Frontend.h"
#include "Interface/Core/InternalThreadState.h"
#include <array>
#include <assert.h>
#include <algorithm>
#include <cstring>
#include <FEXCore/Config/Config.h>
#include <FEXCore/Core/X86Enums.h>
#include <FEXCore/Debug/X86Tables.h>
#include <FEXCore/HLE/SyscallHandler.h>
#include <FEXCore/Utils/Allocator.h>
#include <FEXCore/Utils/LogManager.h>
#include <FEXCore/Utils/Telemetry.h>
#include <set>
#include <sys/mman.h>
namespace FEXCore::Frontend {
#include "Interface/Core/VSyscall/VSyscall.inc"
using namespace FEXCore::X86Tables;
static uint32_t MapModRMToReg(uint8_t REX, uint8_t bits, bool HighBits, bool HasREX, bool HasXMM, bool HasMM, uint8_t InvalidOffset = 16) {
@@ -121,8 +128,18 @@ static uint32_t MapModRMToReg(uint8_t REX, uint8_t bits, bool HighBits, bool Has
}
Decoder::Decoder(FEXCore::Context::Context *ctx)
: CTX {ctx} {
DecodedBuffer.resize(DefaultDecodedBufferSize);
: CTX {ctx}
, OSABI { ctx->SyscallHandler ? ctx->SyscallHandler->GetOSABI() : FEXCore::HLE::SyscallOSABI::OS_UNKNOWN } {
// Using mmap is a start-up time optimization
// Take advantage of page faulting to reduce startup time for minimal runtime cost
DecodedBuffer =
reinterpret_cast<FEXCore::X86Tables::DecodedInst *>(
FEXCore::Allocator::mmap(0, sizeof(FEXCore::X86Tables::DecodedInst) * DefaultDecodedBufferSize,
PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0));
}
Decoder::~Decoder() {
FEXCore::Allocator::munmap(DecodedBuffer, sizeof(FEXCore::X86Tables::DecodedInst) * DefaultDecodedBufferSize);
}
uint8_t Decoder::ReadByte() {
@@ -214,28 +231,28 @@ void Decoder::DecodeModRM_16(X86Tables::DecodedOperand *Operand, X86Tables::ModR
{FEXCore::X86State::REG_RBX, FEXCore::X86State::REG_RDI},
{FEXCore::X86State::REG_RBP, FEXCore::X86State::REG_RSI},
{FEXCore::X86State::REG_RBP, FEXCore::X86State::REG_RDI},
{FEXCore::X86State::REG_RSI, 255},
{FEXCore::X86State::REG_RDI, 255},
{255, 255},
{FEXCore::X86State::REG_RBX, 255},
{FEXCore::X86State::REG_RSI, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_RDI, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_INVALID, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_RBX, FEXCore::X86State::REG_INVALID},
// Mod = 0b01
{FEXCore::X86State::REG_RBX, FEXCore::X86State::REG_RSI},
{FEXCore::X86State::REG_RBX, FEXCore::X86State::REG_RDI},
{FEXCore::X86State::REG_RBP, FEXCore::X86State::REG_RSI},
{FEXCore::X86State::REG_RBP, FEXCore::X86State::REG_RDI},
{FEXCore::X86State::REG_RSI, 255},
{FEXCore::X86State::REG_RDI, 255},
{FEXCore::X86State::REG_RBP, 255},
{FEXCore::X86State::REG_RBX, 255},
{FEXCore::X86State::REG_RSI, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_RDI, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_RBP, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_RBX, FEXCore::X86State::REG_INVALID},
// Mod = 0b10
{FEXCore::X86State::REG_RBX, FEXCore::X86State::REG_RSI},
{FEXCore::X86State::REG_RBX, FEXCore::X86State::REG_RDI},
{FEXCore::X86State::REG_RBP, FEXCore::X86State::REG_RSI},
{FEXCore::X86State::REG_RBP, FEXCore::X86State::REG_RDI},
{FEXCore::X86State::REG_RSI, 255},
{FEXCore::X86State::REG_RDI, 255},
{FEXCore::X86State::REG_RBP, 255},
{FEXCore::X86State::REG_RBX, 255},
{FEXCore::X86State::REG_RSI, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_RDI, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_RBP, FEXCore::X86State::REG_INVALID},
{FEXCore::X86State::REG_RBX, FEXCore::X86State::REG_INVALID},
}};
uint8_t LookupIndex = ModRM.mod << 3 | ModRM.rm;
@@ -683,6 +700,7 @@ bool Decoder::NormalOpHeader(FEXCore::X86Tables::X86InstInfo const *Info, uint16
return NormalOp(&X87Ops[X87Op], X87Op);
}
else if (Info->Type == FEXCore::X86Tables::TYPE_VEX_TABLE_PREFIX) {
FEXCORE_TELEMETRY_SET(VEXOpTelem, 1);
uint16_t map_select = 1;
uint16_t pp = 0;
@@ -710,6 +728,7 @@ bool Decoder::NormalOpHeader(FEXCore::X86Tables::X86InstInfo const *Info, uint16
if (LocalInfo->Type >= FEXCore::X86Tables::TYPE_VEX_GROUP_12 &&
LocalInfo->Type <= FEXCore::X86Tables::TYPE_VEX_GROUP_17) {
FEXCORE_TELEMETRY_SET(VEXOpTelem, 1);
// We have ModRM
uint8_t ModRMByte = ReadByte();
DecodeInst->ModRM = ModRMByte;
@@ -727,6 +746,8 @@ bool Decoder::NormalOpHeader(FEXCore::X86Tables::X86InstInfo const *Info, uint16
return NormalOp(LocalInfo, Op);
}
else if (Info->Type == FEXCore::X86Tables::TYPE_GROUP_EVEX) {
FEXCORE_TELEMETRY_SET(EVEXOpTelem, 1);
/* uint8_t P1 = */ ReadByte();
/* uint8_t P2 = */ ReadByte();
/* uint8_t P3 = */ ReadByte();
@@ -929,7 +950,7 @@ bool Decoder::DecodeInstruction(uint64_t PC) {
}
if (DecodeInst->Dest.IsGPR()) {
assert(DecodeInst->Dest.Data.GPR.GPR != 255);
assert(DecodeInst->Dest.Data.GPR.GPR != FEXCore::X86State::REG_INVALID);
}
return true;
@@ -1004,6 +1025,25 @@ void Decoder::BranchTargetInMultiblockRange() {
}
}
const uint8_t *Decoder::AdjustAddrForSpecialRegion(uint8_t const* _InstStream, uint64_t EntryPoint, uint64_t RIP) {
constexpr uint64_t VSyscall_Base = 0xFFFF'FFFF'FF60'0000ULL;
constexpr uint64_t VSyscall_End = VSyscall_Base + 0x1000;
if (OSABI == FEXCore::HLE::SyscallOSABI::OS_LINUX64 &&
RIP >= VSyscall_Base &&
RIP < VSyscall_End) {
// VSyscall
// This doesn't exist on AArch64 and on x86_64 hosts this is emulated with faults to a region mapped with --xp permissions
// Offset 0: vgettimeofday
// Offset 0x400: vtime
// Offset 0x800: vgetcpu
uint64_t Offset = RIP - VSyscall_Base;
return VSyscallData + Offset;
}
return _InstStream - EntryPoint + RIP;
}
bool Decoder::DecodeInstructionsAtEntry(uint8_t const* _InstStream, uint64_t PC) {
Blocks.clear();
BlocksToDecode.clear();
@@ -1047,7 +1087,7 @@ bool Decoder::DecodeInstructionsAtEntry(uint8_t const* _InstStream, uint64_t PC)
uint64_t BlockStartOffset = DecodedSize;
// Do a bit of pointer math to figure out where we are in code
InstStream = _InstStream - EntryPoint + RIPToDecode;
InstStream = AdjustAddrForSpecialRegion(_InstStream, EntryPoint, RIPToDecode);
while (1) {
ErrorDuringDecoding = !DecodeInstruction(RIPToDecode + PCOffset);
@@ -1091,7 +1131,7 @@ bool Decoder::DecodeInstructionsAtEntry(uint8_t const* _InstStream, uint64_t PC)
}
if (DecodedSize >= CTX->Config.MaxInstPerBlock ||
DecodedSize >= DecodedBuffer.size()) {
DecodedSize >= DefaultDecodedBufferSize) {
break;
}
@@ -1108,7 +1148,7 @@ bool Decoder::DecodeInstructionsAtEntry(uint8_t const* _InstStream, uint64_t PC)
// Copy over only the number of instructions we decoded
CurrentBlockDecoding.NumInstructions = BlockNumberOfInstructions;
CurrentBlockDecoding.DecodedInstructions = &DecodedBuffer.at(BlockStartOffset);
CurrentBlockDecoding.DecodedInstructions = &DecodedBuffer[BlockStartOffset];
}
+12 -3
View File
@@ -1,11 +1,13 @@
#pragma once
#include <FEXCore/Debug/X86Tables.h>
#include <FEXCore/HLE/SyscallHandler.h>
#include <FEXCore/Utils/Telemetry.h>
#include <array>
#include <cstdint>
#include <utility>
#include <set>
#include <stack>
#include <stddef.h>
#include <vector>
namespace FEXCore::Context {
@@ -24,6 +26,7 @@ public:
};
Decoder(FEXCore::Context::Context *ctx);
~Decoder();
bool DecodeInstructionsAtEntry(uint8_t const* InstStream, uint64_t PC);
std::vector<DecodedBlocks> const *GetDecodedBlocks() const {
@@ -37,6 +40,7 @@ public:
void SetExternalBranches(std::set<uint64_t> *v) { ExternalBranches = v; }
private:
FEXCore::Context::Context *CTX;
const FEXCore::HLE::SyscallOSABI OSABI{};
bool DecodeInstruction(uint64_t PC);
@@ -50,7 +54,7 @@ private:
bool NormalOpHeader(FEXCore::X86Tables::X86InstInfo const *Info, uint16_t Op);
static constexpr size_t DefaultDecodedBufferSize = 0x10000;
std::vector<FEXCore::X86Tables::DecodedInst> DecodedBuffer;
FEXCore::X86Tables::DecodedInst *DecodedBuffer{};
size_t DecodedSize {};
uint8_t const *InstStream;
@@ -83,5 +87,10 @@ private:
&FEXCore::Frontend::Decoder::DecodeModRM_64,
&FEXCore::Frontend::Decoder::DecodeModRM_16,
};
const uint8_t *AdjustAddrForSpecialRegion(uint8_t const* _InstStream, uint64_t EntryPoint, uint64_t RIP);
FEXCORE_TELEMETRY_INIT(VEXOpTelem, TYPE_USES_VEX_OPS);
FEXCORE_TELEMETRY_INIT(EVEXOpTelem, TYPE_USES_EVEX_OPS);
};
}
+18 -7
View File
@@ -8,30 +8,41 @@ $end_info$
#include <cstdlib>
#include <cstdio>
#include <iomanip>
#include <iostream>
#include <sstream>
#include <string>
#include <memory>
#include <optional>
#include "Common/NetStream.h"
#include "Common/SoftFloat.h"
#include "Interface/Context/Context.h"
#include <FEXCore/Config/Config.h>
#include <FEXCore/Core/Context.h>
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/Core/SignalDelegator.h>
#include <FEXCore/Core/X86Enums.h>
#include <FEXCore/Debug/InternalThreadState.h>
#include <FEXCore/HLE/Linux/ThreadManagement.h>
#include <FEXCore/Utils/CompilerDefs.h>
#include <FEXCore/Utils/LogManager.h>
#include <FEXCore/Utils/Threads.h>
#include <atomic>
#include <cstring>
#include <errno.h>
#include <fcntl.h>
#include <fmt/format.h>
#include <fstream>
#include <fmt/format.h>
#include <netdb.h>
#include <signal.h>
#include <stddef.h>
#include <string_view>
#include <sys/socket.h>
#include <sys/types.h>
#include <unistd.h>
#include <utility>
#include <vector>
#include "GdbServer.h"
#include <FEXCore/Core/CodeLoader.h>
#include <FEXCore/Core/X86Enums.h>
namespace FEXCore
{
@@ -939,7 +950,7 @@ void GdbServer::GdbServerLoop() {
{
std::lock_guard lk(sendMutex);
CommsStream.release();
CommsStream.reset();
}
}
}
+9 -6
View File
@@ -5,18 +5,21 @@ $end_info$
*/
#pragma once
#include <mutex>
#include <thread>
#include "Interface/Context/Context.h"
#include "Common/NetStream.h"
#include <FEXCore/Config/Config.h>
#include <FEXCore/Utils/Threads.h>
#include <istream>
#include <memory>
#include <mutex>
#include <stdint.h>
#include <string>
namespace FEXCore {
namespace Context {
struct Context;
}
class GdbServer {
public:
GdbServer(FEXCore::Context::Context *ctx);
@@ -1,6 +1,5 @@
#pragma once
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/InternalThreadState.h"
#include "Interface/Core/Dispatcher/Dispatcher.h"
@@ -1,30 +1,31 @@
#include "Common/MathUtils.h"
#include "Common/SoftFloat.h"
#include "Interface/Context/Context.h"
#include "Interface/Core/ArchHelpers/Arm64.h"
#include "Interface/Core/ArchHelpers/MContext.h"
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/DebugData.h"
#include "Interface/Core/InternalThreadState.h"
#include "Interface/Core/Dispatcher/Dispatcher.h"
#include "Interface/Core/Interpreter/InterpreterClass.h"
#include <FEXCore/Config/Config.h>
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/Core/SignalDelegator.h>
#include <FEXCore/Debug/InternalThreadState.h>
#include <FEXCore/Utils/LogManager.h>
#include <FEXCore/Core/CPUBackend.h>
#include <FEXCore/HLE/SyscallHandler.h>
#include <FEXCore/IR/IR.h>
#include <FEXCore/IR/IntrusiveIRList.h>
#include "Interface/HLE/Thunks/Thunks.h"
#include <atomic>
#include <cmath>
#include <limits>
#include <vector>
#include <memory>
#include <bits/types/stack_t.h>
#include <signal.h>
#include <stdint.h>
#include <unordered_map>
#include <utility>
#include "InterpreterOps.h"
namespace FEXCore::IR {
class IRListView;
class RegisterAllocationData;
}
namespace FEXCore::CPU {
class CPUBackend;
static void InterpreterExecution(FEXCore::Core::CpuStateFrame *Frame) {
auto Thread = Frame->Thread;
@@ -78,6 +79,18 @@ bool InterpreterCore::HandleSIGBUS(int Signal, void *info, void *ucontext) {
return false;
}
}
else if ((Instr & FEXCore::ArchHelpers::Arm64::LDAXR_MASK) == FEXCore::ArchHelpers::Arm64::LDAXR_INST) { // LDAXR*
uint64_t BytesToSkip = FEXCore::ArchHelpers::Arm64::HandleAtomicLoadstoreExclusive(ucontext, info);
if (BytesToSkip) {
// Skip this instruction now
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) + BytesToSkip);
return true;
}
else {
LogMan::Msg::E("Unhandled JIT SIGBUS LDAXR: PC: %p Instruction: 0x%08x\n", PC, PC[0]);
return false;
}
}
}
return false;
}
@@ -105,7 +118,7 @@ InterpreterCore::InterpreterCore(FEXCore::Context::Context *ctx, FEXCore::Core::
return Core->Dispatcher->HandleGuestSignal(Signal, info, ucontext, GuestAction, GuestStack);
};
for (uint32_t Signal = 0; Signal < SignalDelegator::MAX_SIGNALS; ++Signal) {
for (uint32_t Signal = 0; Signal <= SignalDelegator::MAX_SIGNALS; ++Signal) {
CTX->SignalDelegation->RegisterHostSignalHandlerForGuest(Signal, GuestSignalHandler);
}
}
File diff suppressed because it is too large. Load diff
@@ -1,9 +1,13 @@
#pragma once
#include <stdint.h>
namespace FEXCore::Core {
struct InternalThreadState;
}
namespace FEXCore::IR {
class IRListView;
struct IROp_Header;
}
namespace FEXCore::Core{
@@ -32,11 +36,11 @@ namespace FEXCore::CPU {
FallbackABI ABI;
void *fn;
};
class InterpreterOps {
public:
static void InterpretIR(FEXCore::Core::InternalThreadState *Thread, uint64_t Entry, FEXCore::IR::IRListView *CurrentIR, FEXCore::Core::DebugData *DebugData);
static bool GetFallbackHandler(IR::IROp_Header *IROp, FallbackInfo *Info);
};
};
};
+36 -36
View File
@@ -46,7 +46,7 @@ DEF_OP(TruncElementPair) {
mov(Dst.second, Src.second);
break;
}
default: LOGMAN_MSG_A("Unhandled Truncation size: %d", Op->Size); break;
default: LOGMAN_MSG_A_FMT("Unhandled Truncation size: {}", Op->Size); break;
}
}
@@ -95,7 +95,7 @@ DEF_OP(Add) {
case 8:
add(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()), Const);
break;
default: LOGMAN_MSG_A("Unsupported Add size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported Add size: {}", OpSize);
}
} else {
switch (OpSize) {
@@ -105,7 +105,7 @@ DEF_OP(Add) {
case 8:
add(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()), GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unsupported Add size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported Add size: {}", OpSize);
}
}
}
@@ -121,7 +121,7 @@ DEF_OP(Sub) {
case 8:
sub(GRS(Node), GRS(Op->Header.Args[0].ID()), Const);
break;
default: LOGMAN_MSG_A("Unsupported Sub size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported Sub size: {}", OpSize);
}
} else {
switch (OpSize) {
@@ -131,7 +131,7 @@ DEF_OP(Sub) {
case 8:
sub(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()), GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unsupported Sub size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported Sub size: {}", OpSize);
}
}
@@ -147,7 +147,7 @@ DEF_OP(Neg) {
case 8:
neg(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unsupported Not size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported Neg size: {}", OpSize);
}
}
@@ -163,7 +163,7 @@ DEF_OP(Mul) {
case 8:
mul(Dst, GetReg<RA_64>(Op->Header.Args[0].ID()), GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unknown Sext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown Mul size: %d", OpSize);
}
}
@@ -179,7 +179,7 @@ DEF_OP(UMul) {
case 8:
mul(Dst, GetReg<RA_64>(Op->Header.Args[0].ID()), GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unknown Sext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown UMul size: {}", OpSize);
}
}
@@ -216,7 +216,7 @@ DEF_OP(Div) {
sdiv(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()), GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown DIV Size: %d", Size); break;
default: LOGMAN_MSG_A_FMT("Unknown DIV Size: {}", Size); break;
}
}
@@ -243,7 +243,7 @@ DEF_OP(UDiv) {
udiv(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()), GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown UDIV Size: %d", Size); break;
default: LOGMAN_MSG_A_FMT("Unknown UDIV Size: {}", Size); break;
}
}
@@ -290,7 +290,7 @@ DEF_OP(Rem) {
msub(GetReg<RA_64>(Node), TMP1, Divisor, Dividend);
break;
}
default: LOGMAN_MSG_A("Unknown REM Size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown REM Size: {}", OpSize); break;
}
}
@@ -332,7 +332,7 @@ DEF_OP(URem) {
msub(GetReg<RA_64>(Node), TMP1, Divisor, Dividend);
break;
}
default: LOGMAN_MSG_A("Unknown UREM Size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown UREM Size: {}", OpSize); break;
}
}
@@ -349,7 +349,7 @@ DEF_OP(MulH) {
case 8:
smulh(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()), GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unknown Sext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown Sext size: {}", OpSize);
}
}
@@ -366,7 +366,7 @@ DEF_OP(UMulH) {
case 8:
umulh(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()), GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unknown Sext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown Sext size: {}", OpSize);
}
}
@@ -462,7 +462,7 @@ DEF_OP(Ror) {
break;
}
default: LOGMAN_MSG_A("Unhandled ROR size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled ROR size: {}", OpSize);
}
} else {
switch (OpSize) {
@@ -475,7 +475,7 @@ DEF_OP(Ror) {
break;
}
default: LOGMAN_MSG_A("Unhandled ROR size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled ROR size: {}", OpSize);
}
}
}
@@ -494,7 +494,7 @@ DEF_OP(Extr) {
break;
}
default: LOGMAN_MSG_A("Unhandled EXTR size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled EXTR size: {}", OpSize);
}
}
@@ -539,7 +539,7 @@ DEF_OP(LDiv) {
mov(GetReg<RA_64>(Node), x0);
break;
}
default: LOGMAN_MSG_A("Unknown LDIV Size: %d", Size); break;
default: LOGMAN_MSG_A_FMT("Unknown LDIV Size: {}", Size); break;
}
}
@@ -582,7 +582,7 @@ DEF_OP(LUDiv) {
mov(GetReg<RA_64>(Node), x0);
break;
}
default: LOGMAN_MSG_A("Unknown LUDIV Size: %d", Size); break;
default: LOGMAN_MSG_A_FMT("Unknown LUDIV Size: {}", Size); break;
}
}
@@ -635,7 +635,7 @@ DEF_OP(LRem) {
mov(GetReg<RA_64>(Node), x0);
break;
}
default: LOGMAN_MSG_A("Unknown LREM Size: %d", Size); break;
default: LOGMAN_MSG_A_FMT("Unknown LREM Size: {}", Size); break;
}
}
@@ -685,7 +685,7 @@ DEF_OP(LURem) {
mov(GetReg<RA_64>(Node), x0);
break;
}
default: LOGMAN_MSG_A("Unknown LUREM Size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown LUREM Size: {}", OpSize); break;
}
}
@@ -699,7 +699,7 @@ DEF_OP(Not) {
case 8:
mvn(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unsupported Not size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported Not size: {}", OpSize);
}
}
@@ -730,7 +730,7 @@ DEF_OP(Popcount) {
// fmov has zero extended, unused bytes are zero
addv(VTMP1.B(), VTMP1.V8B());
break;
default: LOGMAN_MSG_A("Unsupported Popcount size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported Popcount size: {}", OpSize);
}
auto Dst = GetReg<RA_32>(Node);
@@ -779,7 +779,7 @@ DEF_OP(FindMSB) {
clz(Dst, GetReg<RA_64>(Op->Header.Args[0].ID()));
sub(Dst, TMP1, Dst);
break;
default: LOGMAN_MSG_A("Unknown REV size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown FindMSB size: {}", OpSize); break;
}
}
@@ -800,7 +800,7 @@ DEF_OP(FindTrailingZeros) {
rbit(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()));
clz(GetReg<RA_64>(Node), GetReg<RA_64>(Node));
break;
default: LOGMAN_MSG_A("Unknown size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown FindTrailingZeros size: {}", OpSize); break;
}
}
@@ -819,7 +819,7 @@ DEF_OP(CountLeadingZeroes) {
case 8:
clz(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unknown size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown CountLeadingZeroes size: {}", OpSize); break;
}
}
@@ -837,7 +837,7 @@ DEF_OP(Rev) {
case 8:
rev(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unknown REV size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown REV size: {}", OpSize); break;
}
}
@@ -859,14 +859,14 @@ DEF_OP(Bfi) {
bfi(TMP1, GetReg<RA_64>(Op->Header.Args[1].ID()), Op->lsb, Op->Width);
mov(GetReg<RA_64>(Node), TMP1);
break;
default: LOGMAN_MSG_A("Unknown BFI size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown BFI size: {}", OpSize); break;
}
}
DEF_OP(Bfe) {
auto Op = IROp->C<IR::IROp_Bfe>();
LOGMAN_THROW_A(IROp->Size <= 8, "OpSize is too large for BFE: %d", IROp->Size);
LOGMAN_THROW_A(Op->Width != 0, "Invalid BFE width of 0");
LOGMAN_THROW_A_FMT(IROp->Size <= 8, "OpSize is too large for BFE: {}", IROp->Size);
LOGMAN_THROW_A_FMT(Op->Width != 0, "Invalid BFE width of 0");
auto Dst = GetReg<RA_64>(Node);
ubfx(Dst, GetReg<RA_64>(Op->Header.Args[0].ID()), Op->lsb, Op->Width);
@@ -880,7 +880,7 @@ DEF_OP(Sbfe) {
if (OpSize == 8) {
sbfx(Dst, GetReg<RA_64>(Op->Header.Args[0].ID()), Op->lsb, Op->Width);
} else {
LogMan::Msg::D("Unimplemented Sbfe size");
LogMan::Msg::DFmt("Unimplemented Sbfe size");
}
}
@@ -911,7 +911,7 @@ Condition MapSelectCC(IR::CondClassType Cond) {
case FEXCore::IR::COND_MI:
case FEXCore::IR::COND_PL:
default:
LOGMAN_MSG_A("Unsupported compare type");
LOGMAN_MSG_A_FMT("Unsupported compare type");
return Condition::nv;
}
}
@@ -929,7 +929,7 @@ DEF_OP(Select) {
} else if (IsFPR(Op->Cmp1.ID())) {
fcmp(GRFCMP(Op->Cmp1.ID()), GRFCMP(Op->Cmp2.ID()));
} else {
LOGMAN_MSG_A("Select: Expected GPR or FPR");
LOGMAN_MSG_A_FMT("Select: Expected GPR or FPR");
}
auto cc = MapSelectCC(Op->Cond);
@@ -940,7 +940,7 @@ DEF_OP(Select) {
if (is_const_true || is_const_false) {
if (is_const_false != true || is_const_true != true || const_true != 1 || const_false != 0) {
LOGMAN_MSG_A("Select: Unsupported compare inline parameters");
LOGMAN_MSG_A_FMT("Select: Unsupported compare inline parameters");
}
cset(GRS(Node), cc);
} else {
@@ -964,7 +964,7 @@ DEF_OP(VExtractToGPR) {
case 8:
umov(GetReg<RA_64>(Node), GetSrc(Op->Header.Args[0].ID()).V2D(), Op->Idx);
break;
default: LOGMAN_MSG_A("Unhandled ExtractElementSize: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled ExtractElementSize: {}", OpSize);
}
}
@@ -1027,7 +1027,7 @@ DEF_OP(FCmp) {
bool set = false;
if (Op->Flags & (1 << IR::FCMP_FLAG_EQ)) {
LOGMAN_THROW_A(IR::FCMP_FLAG_EQ == 0, "IR::FCMP_FLAG_EQ must equal 0");
LOGMAN_THROW_A_FMT(IR::FCMP_FLAG_EQ == 0, "IR::FCMP_FLAG_EQ must equal 0");
// EQ or unordered
cset(Dst, Condition::eq); // Z = 1
csinc(Dst, Dst, xzr, Condition::vc); // IF !V ? Z : 1
@@ -34,7 +34,7 @@ DEF_OP(CASPair) {
mov(Dst.first, TMP3);
mov(Dst.second, TMP4);
break;
default: LOGMAN_MSG_A("Unsupported: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported: {}", OpSize);
}
}
else {
@@ -44,6 +44,7 @@ DEF_OP(CASPair) {
aarch64::Label LoopNotExpected;
aarch64::Label LoopExpected;
bind(&LoopTop);
ldaxp(TMP2.W(), TMP3.W(), MemOperand(MemSrc));
cmp(TMP2.W(), Expected.first.W());
ccmp(TMP3.W(), Expected.second.W(), NoFlag, Condition::eq);
@@ -69,6 +70,7 @@ DEF_OP(CASPair) {
aarch64::Label LoopNotExpected;
aarch64::Label LoopExpected;
bind(&LoopTop);
ldaxp(TMP2.X(), TMP3.X(), MemOperand(MemSrc));
cmp(TMP2.X(), Expected.first.X());
ccmp(TMP3.X(), Expected.second.X(), NoFlag, Condition::eq);
@@ -89,7 +91,7 @@ DEF_OP(CASPair) {
bind(&LoopExpected);
break;
}
default: LOGMAN_MSG_A("Unsupported: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported: {}", OpSize);
}
}
}
@@ -115,7 +117,7 @@ DEF_OP(CAS) {
case 2: casalh(TMP2.W(), Desired.W(), MemOperand(MemSrc)); break;
case 4: casal(TMP2.W(), Desired.W(), MemOperand(MemSrc)); break;
case 8: casal(TMP2.X(), Desired.X(), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unsupported: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported: {}", OpSize);
}
mov(GetReg<RA_64>(Node), TMP2);
}
@@ -206,7 +208,7 @@ DEF_OP(CAS) {
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", OpSize);
}
}
}
@@ -222,7 +224,7 @@ DEF_OP(AtomicAdd) {
case 2: staddlh(GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
case 4: staddl(GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
case 8: staddl(GetReg<RA_64>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
else {
@@ -264,7 +266,7 @@ DEF_OP(AtomicAdd) {
cbnz(TMP2, &LoopTop);
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
}
@@ -281,7 +283,7 @@ DEF_OP(AtomicSub) {
case 2: staddlh(TMP2.W(), MemOperand(MemSrc)); break;
case 4: staddl(TMP2.W(), MemOperand(MemSrc)); break;
case 8: staddl(TMP2.X(), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
else {
@@ -323,7 +325,7 @@ DEF_OP(AtomicSub) {
cbnz(TMP2, &LoopTop);
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
}
@@ -340,7 +342,7 @@ DEF_OP(AtomicAnd) {
case 2: stclrlh(TMP2.W(), MemOperand(MemSrc)); break;
case 4: stclrl(TMP2.W(), MemOperand(MemSrc)); break;
case 8: stclrl(TMP2.X(), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
else {
@@ -382,7 +384,7 @@ DEF_OP(AtomicAnd) {
cbnz(TMP2, &LoopTop);
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
}
@@ -398,7 +400,7 @@ DEF_OP(AtomicOr) {
case 2: stsetlh(GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
case 4: stsetl(GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
case 8: stsetl(GetReg<RA_64>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
else {
@@ -440,7 +442,7 @@ DEF_OP(AtomicOr) {
cbnz(TMP2, &LoopTop);
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
}
@@ -456,7 +458,7 @@ DEF_OP(AtomicXor) {
case 2: steorlh(GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
case 4: steorl(GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
case 8: steorl(GetReg<RA_64>(Op->Header.Args[1].ID()), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
else {
@@ -498,7 +500,7 @@ DEF_OP(AtomicXor) {
cbnz(TMP2, &LoopTop);
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
}
@@ -515,36 +517,35 @@ DEF_OP(AtomicSwap) {
case 2: swplh(TMP2.W(), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 4: swpl(TMP2.W(), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 8: swpl(TMP2.X(), GetReg<RA_64>(Node), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
else {
// TMP2-TMP3
mov(TMP3, GetReg<RA_64>(Op->Header.Args[1].ID()));
switch (Op->Size) {
case 1: {
aarch64::Label LoopTop;
bind(&LoopTop);
ldaxrb(TMP2.W(), MemOperand(MemSrc));
stlxrb(TMP4.W(), TMP3.W(), MemOperand(MemSrc));
stlxrb(TMP4.W(), GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc));
cbnz(TMP4.W(), &LoopTop);
uxtb(GetReg<RA_64>(Node), TMP2.W());
uxtb(GetReg<RA_32>(Node), TMP2.W());
break;
}
case 2: {
aarch64::Label LoopTop;
bind(&LoopTop);
ldaxrh(TMP2.W(), MemOperand(MemSrc));
stlxrh(TMP4.W(), TMP3.W(), MemOperand(MemSrc));
stlxrh(TMP4.W(), GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc));
cbnz(TMP4.W(), &LoopTop);
uxtw(GetReg<RA_64>(Node), TMP2.W());
uxtw(GetReg<RA_32>(Node), TMP2.W());
break;
}
case 4: {
aarch64::Label LoopTop;
bind(&LoopTop);
ldaxr(TMP2.W(), MemOperand(MemSrc));
stlxr(TMP4.W(), TMP3.W(), MemOperand(MemSrc));
stlxr(TMP4.W(), GetReg<RA_32>(Op->Header.Args[1].ID()), MemOperand(MemSrc));
cbnz(TMP4.W(), &LoopTop);
mov(GetReg<RA_32>(Node), TMP2.W());
break;
@@ -553,12 +554,12 @@ DEF_OP(AtomicSwap) {
aarch64::Label LoopTop;
bind(&LoopTop);
ldaxr(TMP2, MemOperand(MemSrc));
stlxr(TMP4, TMP3.X(), MemOperand(MemSrc));
stlxr(TMP4, GetReg<RA_64>(Op->Header.Args[1].ID()), MemOperand(MemSrc));
cbnz(TMP4, &LoopTop);
mov(GetReg<RA_64>(Node), TMP2.X());
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
}
@@ -573,7 +574,7 @@ DEF_OP(AtomicFetchAdd) {
case 2: ldaddalh(GetReg<RA_32>(Op->Header.Args[1].ID()), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 4: ldaddal(GetReg<RA_32>(Op->Header.Args[1].ID()), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 8: ldaddal(GetReg<RA_64>(Op->Header.Args[1].ID()), GetReg<RA_64>(Node), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
else {
@@ -619,7 +620,7 @@ DEF_OP(AtomicFetchAdd) {
mov(GetReg<RA_64>(Node), TMP2);
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
}
@@ -635,7 +636,7 @@ DEF_OP(AtomicFetchSub) {
case 2: ldaddalh(TMP2.W(), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 4: ldaddal(TMP2.W(), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 8: ldaddal(TMP2.X(), GetReg<RA_64>(Node), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
else {
@@ -681,7 +682,7 @@ DEF_OP(AtomicFetchSub) {
mov(GetReg<RA_64>(Node), TMP2);
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
}
@@ -697,7 +698,7 @@ DEF_OP(AtomicFetchAnd) {
case 2: ldclralh(TMP2.W(), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 4: ldclral(TMP2.W(), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 8: ldclral(TMP2.X(), GetReg<RA_64>(Node), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
else {
@@ -743,7 +744,7 @@ DEF_OP(AtomicFetchAnd) {
mov(GetReg<RA_64>(Node), TMP2);
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
}
@@ -758,7 +759,7 @@ DEF_OP(AtomicFetchOr) {
case 2: ldsetalh(GetReg<RA_32>(Op->Header.Args[1].ID()), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 4: ldsetal(GetReg<RA_32>(Op->Header.Args[1].ID()), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 8: ldsetal(GetReg<RA_64>(Op->Header.Args[1].ID()), GetReg<RA_64>(Node), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
else {
@@ -804,7 +805,7 @@ DEF_OP(AtomicFetchOr) {
mov(GetReg<RA_64>(Node), TMP2);
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
}
@@ -819,7 +820,7 @@ DEF_OP(AtomicFetchXor) {
case 2: ldeoralh(GetReg<RA_32>(Op->Header.Args[1].ID()), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 4: ldeoral(GetReg<RA_32>(Op->Header.Args[1].ID()), GetReg<RA_32>(Node), MemOperand(MemSrc)); break;
case 8: ldeoral(GetReg<RA_64>(Op->Header.Args[1].ID()), GetReg<RA_64>(Node), MemOperand(MemSrc)); break;
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
else {
@@ -865,11 +866,61 @@ DEF_OP(AtomicFetchXor) {
mov(GetReg<RA_64>(Node), TMP2);
break;
}
default: LOGMAN_MSG_A("Unhandled Atomic size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
}
DEF_OP(AtomicFetchNeg) {
auto Op = IROp->C<IR::IROp_AtomicFetchNeg>();
auto MemSrc = GetReg<RA_64>(Op->Header.Args[0].ID());
// TMP2-TMP3
switch (Op->Size) {
case 1: {
aarch64::Label LoopTop;
bind(&LoopTop);
ldaxrb(TMP2.W(), MemOperand(MemSrc));
neg(TMP3.W(), TMP2.W());
stlxrb(TMP4.W(), TMP3.W(), MemOperand(MemSrc));
cbnz(TMP4.W(), &LoopTop);
mov(GetReg<RA_32>(Node), TMP2.W());
break;
}
case 2: {
aarch64::Label LoopTop;
bind(&LoopTop);
ldaxrh(TMP2.W(), MemOperand(MemSrc));
neg(TMP3.W(), TMP2.W());
stlxrh(TMP4.W(), TMP3.W(), MemOperand(MemSrc));
cbnz(TMP4.W(), &LoopTop);
mov(GetReg<RA_32>(Node), TMP2.W());
break;
}
case 4: {
aarch64::Label LoopTop;
bind(&LoopTop);
ldaxr(TMP2.W(), MemOperand(MemSrc));
neg(TMP3.W(), TMP2.W());
stlxr(TMP4.W(), TMP3.W(), MemOperand(MemSrc));
cbnz(TMP4.W(), &LoopTop);
mov(GetReg<RA_32>(Node), TMP2.W());
break;
}
case 8: {
aarch64::Label LoopTop;
bind(&LoopTop);
ldaxr(TMP2, MemOperand(MemSrc));
neg(TMP3, TMP2);
stlxr(TMP4, TMP3, MemOperand(MemSrc));
cbnz(TMP4, &LoopTop);
mov(GetReg<RA_64>(Node), TMP2);
break;
}
default: LOGMAN_MSG_A_FMT("Unhandled Atomic size: {}", Op->Size);
}
}
#undef DEF_OP
void Arm64JITCore::RegisterAtomicHandlers() {
#define REGISTER_OP(op, x) OpHandlers[FEXCore::IR::IROps::OP_##op] = &Arm64JITCore::Op_##x
@@ -886,6 +937,7 @@ void Arm64JITCore::RegisterAtomicHandlers() {
REGISTER_OP(ATOMICFETCHAND, AtomicFetchAnd);
REGISTER_OP(ATOMICFETCHOR, AtomicFetchOr);
REGISTER_OP(ATOMICFETCHXOR, AtomicFetchXor);
REGISTER_OP(ATOMICFETCHNEG, AtomicFetchNeg);
#undef REGISTER_OP
}
}
@@ -4,6 +4,8 @@ tags: backend|arm64
$end_info$
*/
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/JIT/Arm64/JITClass.h"
#include "Interface/Core/InternalThreadState.h"
@@ -16,15 +18,15 @@ using namespace vixl;
using namespace vixl::aarch64;
#define DEF_OP(x) void Arm64JITCore::Op_##x(FEXCore::IR::IROp_Header *IROp, uint32_t Node)
DEF_OP(GuestCallDirect) {
LogMan::Msg::D("Unimplemented");
LogMan::Msg::DFmt("Unimplemented");
}
DEF_OP(GuestCallIndirect) {
LogMan::Msg::D("Unimplemented");
LogMan::Msg::DFmt("Unimplemented");
}
DEF_OP(GuestReturn) {
LogMan::Msg::D("Unimplemented");
LogMan::Msg::DFmt("Unimplemented");
}
DEF_OP(SignalReturn) {
@@ -142,7 +144,7 @@ Condition MapBranchCC(IR::CondClassType Cond) {
case FEXCore::IR::COND_MI:
case FEXCore::IR::COND_PL:
default:
LOGMAN_MSG_A("Unsupported compare type");
LOGMAN_MSG_A_FMT("Unsupported compare type");
return Condition::nv;
}
}
@@ -169,10 +171,10 @@ DEF_OP(CondJump) {
bool isConst = IsInlineConstant(Op->Cmp2, &Const);
if (isConst && Const == 0 && Op->Cond.Val == FEXCore::IR::COND_EQ) {
LOGMAN_THROW_A(IsGPR(Op->Cmp1.ID()), "CondJump: Expected GPR");
LOGMAN_THROW_A_FMT(IsGPR(Op->Cmp1.ID()), "CondJump: Expected GPR");
cbz(GRCMP(Op->Cmp1.ID()), TrueTargetLabel);
} else if (isConst && Const == 0 && Op->Cond.Val == FEXCore::IR::COND_NEQ) {
LOGMAN_THROW_A(IsGPR(Op->Cmp1.ID()), "CondJump: Expected GPR");
LOGMAN_THROW_A_FMT(IsGPR(Op->Cmp1.ID()), "CondJump: Expected GPR");
cbnz(GRCMP(Op->Cmp1.ID()), TrueTargetLabel);
} else {
if (IsGPR(Op->Cmp1.ID())) {
@@ -183,7 +185,7 @@ DEF_OP(CondJump) {
} else if (IsFPR(Op->Cmp1.ID())) {
fcmp(GRFCMP(Op->Cmp1.ID()), GRFCMP(Op->Cmp2.ID()));
} else {
LOGMAN_MSG_A("CondJump: Expected GPR or FPR");
LOGMAN_MSG_A_FMT("CondJump: Expected GPR or FPR");
}
b(TrueTargetLabel, MapBranchCC(Op->Cond));
@@ -31,7 +31,7 @@ DEF_OP(VInsGPR) {
ins(GetDst(Node).V2D(), Op->Index, GetReg<RA_64>(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -52,7 +52,7 @@ DEF_OP(VCastFromGPR) {
case 8:
fmov(GetDst(Node).D(), GetReg<RA_64>(Op->Header.Args[0].ID()).X());
break;
default: LOGMAN_MSG_A("Unknown castGPR element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown castGPR element size: {}", Op->Header.ElementSize);
}
}
@@ -91,7 +91,7 @@ DEF_OP(Float_FToF) {
fcvt(GetDst(Node).S(), GetSrc(Op->Header.Args[0].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown FCVT sizes: 0x%x", Conv);
default: LOGMAN_MSG_A_FMT("Unknown FCVT sizes: 0x{:x}", Conv);
}
}
@@ -104,7 +104,7 @@ DEF_OP(Vector_SToF) {
case 8:
scvtf(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
default: LOGMAN_MSG_A("Unknown castGPR element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Vector_SToF element size: {}", Op->Header.ElementSize);
}
}
@@ -117,7 +117,7 @@ DEF_OP(Vector_FToZS) {
case 8:
fcvtzs(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
default: LOGMAN_MSG_A("Unknown castGPR element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Vector_FToZS element size: {}", Op->Header.ElementSize);
}
}
@@ -132,7 +132,7 @@ DEF_OP(Vector_FToS) {
frinti(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
fcvtzs(GetDst(Node).V2D(), GetDst(Node).V2D());
break;
default: LOGMAN_MSG_A("Unknown castGPR element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Vector_FToS element size: {}", Op->Header.ElementSize);
}
}
@@ -149,7 +149,7 @@ DEF_OP(Vector_FToF) {
fcvtn(GetDst(Node).V2S(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Conversion Type : 0%04x", Conv); break;
default: LOGMAN_MSG_A_FMT("Unknown Vector_FToF Type : 0x{:04x}", Conv); break;
}
}
+91 -65
View File
@@ -11,6 +11,7 @@ $end_info$
*/
#include "Interface/Context/Context.h"
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/ArchHelpers/Arm64.h"
#include "Interface/Core/ArchHelpers/MContext.h"
@@ -45,8 +46,7 @@ void Arm64JITCore::Op_Unhandled(FEXCore::IR::IROp_Header *IROp, uint32_t Node) {
FallbackInfo Info;
if (!InterpreterOps::GetFallbackHandler(IROp, &Info)) {
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
auto Name = FEXCore::IR::GetName(IROp->Op);
LOGMAN_MSG_A("Unhandled IR Op: %s", std::string(Name).c_str());
LOGMAN_MSG_A_FMT("Unhandled IR Op: {}", FEXCore::IR::GetName(IROp->Op));
#endif
} else {
switch(Info.ABI) {
@@ -295,8 +295,7 @@ void Arm64JITCore::Op_Unhandled(FEXCore::IR::IROp_Header *IROp, uint32_t Node) {
case FABI_UNKNOWN:
default:
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
auto Name = FEXCore::IR::GetName(IROp->Op);
LOGMAN_MSG_A("Unhandled IR Fallback abi: %s %d", std::string(Name).c_str(), Info.ABI);
LOGMAN_MSG_A_FMT("Unhandled IR Fallback ABI: {} {}", FEXCore::IR::GetName(IROp->Op), Info.ABI);
#endif
break;
}
@@ -315,7 +314,7 @@ Arm64JITCore::CodeBuffer Arm64JITCore::AllocateNewCodeBuffer(size_t Size) {
PROT_READ | PROT_WRITE | PROT_EXEC,
MAP_PRIVATE | MAP_ANONYMOUS,
-1, 0));
LOGMAN_THROW_A(!!Buffer.Ptr, "Couldn't allocate code buffer");
LOGMAN_THROW_A_FMT(!!Buffer.Ptr, "Couldn't allocate code buffer");
Dispatcher->RegisterCodeBuffer(Buffer.Ptr, Buffer.Size);
return Buffer;
}
@@ -345,54 +344,70 @@ bool Arm64JITCore::HandleSIGBUS(int Signal, void *info, void *ucontext) {
0b1011'0000'0000; // Inner shareable all
if ((Instr & 0x3F'FF'FC'00) == 0x08'DF'FC'00 || // LDAR*
(Instr & 0x3F'FF'FC'00) == 0x38'BF'C0'00) { // LDAPR*
uint32_t LDR = 0b0011'1000'0111'1111'0110'1000'0000'0000;
LDR |= Size << 30;
LDR |= AddrReg << 5;
LDR |= DataReg;
PC[-1] = DMB;
PC[0] = LDR;
PC[1] = DMB;
// Back up one instruction and have another go
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) - 4);
if (ParanoidTSO()) {
if (FEXCore::ArchHelpers::Arm64::HandleAtomicLoad(ucontext, info, Instr)) {
// Skip this instruction now
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) + 4);
return true;
}
else {
LogMan::Msg::EFmt("Unhandled JIT SIGBUS LDAR*: PC: {} Instruction: 0x{:08x}\n", fmt::ptr(PC), PC[0]);
return false;
}
}
else {
uint32_t LDR = 0b0011'1000'0111'1111'0110'1000'0000'0000;
LDR |= Size << 30;
LDR |= AddrReg << 5;
LDR |= DataReg;
PC[-1] = DMB;
PC[0] = LDR;
PC[1] = DMB;
// Back up one instruction and have another go
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) - 4);
}
}
else if ( (Instr & 0x3F'FF'FC'00) == 0x08'9F'FC'00) { // STLR*
uint32_t STR = 0b0011'1000'0011'1111'0110'1000'0000'0000;
STR |= Size << 30;
STR |= AddrReg << 5;
STR |= DataReg;
PC[-1] = DMB;
PC[0] = STR;
PC[1] = DMB;
// Back up one instruction and have another go
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) - 4);
if (ParanoidTSO()) {
if (FEXCore::ArchHelpers::Arm64::HandleAtomicStore(ucontext, info, Instr)) {
// Skip this instruction now
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) + 4);
return true;
}
else {
LogMan::Msg::EFmt("Unhandled JIT SIGBUS STLR*: PC: {} Instruction: 0x{:08x}\n", fmt::ptr(PC), PC[0]);
return false;
}
}
else {
uint32_t STR = 0b0011'1000'0011'1111'0110'1000'0000'0000;
STR |= Size << 30;
STR |= AddrReg << 5;
STR |= DataReg;
PC[-1] = DMB;
PC[0] = STR;
PC[1] = DMB;
// Back up one instruction and have another go
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) - 4);
}
}
else if ((Instr & FEXCore::ArchHelpers::Arm64::LDAXP_MASK) == FEXCore::ArchHelpers::Arm64::LDAXP_INST) { // LDAXP
uint32_t DataReg2 = (Instr >> 10) & 0x1F;
// Convert to LDP
uint32_t LDP = 0b0010'1001'0100'0000'0000'0000'0000'0000;
LDP |= Size << 31;
LDP |= DataReg2 << 10;
LDP |= AddrReg << 5;
LDP |= DataReg;
PC[-1] = DMB;
PC[0] = LDP;
PC[1] = DMB;
// Back up one instruction and have another go
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) - 4);
//Should be compare and swap pair only. LDAXP not used elsewhere
uint64_t BytesToSkip = FEXCore::ArchHelpers::Arm64::HandleCASPAL_ARMv8(ucontext, info, Instr);
if (BytesToSkip) {
// Skip this instruction now
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) + BytesToSkip);
return true;
}
else {
LogMan::Msg::EFmt("Unhandled JIT SIGBUS LDAXP: PC: {} Instruction: 0x{:08x}\n", fmt::ptr(PC), PC[0]);
return false;
}
}
else if ((Instr & FEXCore::ArchHelpers::Arm64::STLXP_MASK) == FEXCore::ArchHelpers::Arm64::STLXP_INST) { // STLXP
uint32_t DataReg2 = (Instr >> 10) & 0x1F;
// Convert to STP
uint32_t STP = 0b0010'1001'0000'0000'0000'0000'0000'0000;
STP |= Size << 31;
STP |= DataReg2 << 10;
STP |= AddrReg << 5;
STP |= DataReg;
PC[-1] = DMB;
PC[0] = STP;
PC[1] = DMB;
// Back up one instruction and have another go
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) - 4);
//Should not trigger - middle of an LDAXP/STAXP pair.
LogMan::Msg::EFmt("Unhandled JIT SIGBUS STLXP: PC: {} Instruction: 0x{:08x}\n", fmt::ptr(PC), PC[0]);
return false;
}
else if ((Instr & FEXCore::ArchHelpers::Arm64::CASPAL_MASK) == FEXCore::ArchHelpers::Arm64::CASPAL_INST) { // CASPAL
if (FEXCore::ArchHelpers::Arm64::HandleCASPAL(ucontext, info, Instr)) {
@@ -401,7 +416,7 @@ bool Arm64JITCore::HandleSIGBUS(int Signal, void *info, void *ucontext) {
return true;
}
else {
LogMan::Msg::E("Unhandled JIT SIGBUS CASPAL: PC: %p Instruction: 0x%08x\n", PC, PC[0]);
LogMan::Msg::EFmt("Unhandled JIT SIGBUS CASPAL: PC: {} Instruction: 0x{:08x}\n", fmt::ptr(PC), PC[0]);
return false;
}
}
@@ -412,7 +427,7 @@ bool Arm64JITCore::HandleSIGBUS(int Signal, void *info, void *ucontext) {
return true;
}
else {
LogMan::Msg::E("Unhandled JIT SIGBUS CASAL: PC: %p Instruction: 0x%08x\n", PC, PC[0]);
LogMan::Msg::EFmt("Unhandled JIT SIGBUS CASAL: PC: {} Instruction: 0x{:08x}\n", fmt::ptr(PC), PC[0]);
return false;
}
}
@@ -424,12 +439,24 @@ bool Arm64JITCore::HandleSIGBUS(int Signal, void *info, void *ucontext) {
}
else {
uint8_t Op = (PC[0] >> 12) & 0xF;
LogMan::Msg::E("Unhandled JIT SIGBUS Atomic mem op 0x%02x: PC: %p Instruction: 0x%08x\n", Op, PC, PC[0]);
LogMan::Msg::EFmt("Unhandled JIT SIGBUS Atomic mem op 0x{:02x}: PC: {} Instruction: 0x{:08x}\n", Op, fmt::ptr(PC), PC[0]);
return false;
}
}
else if ((Instr & FEXCore::ArchHelpers::Arm64::LDAXR_MASK) == FEXCore::ArchHelpers::Arm64::LDAXR_INST) { // LDAXR*
uint64_t BytesToSkip = FEXCore::ArchHelpers::Arm64::HandleAtomicLoadstoreExclusive(ucontext, info);
if (BytesToSkip) {
// Skip this instruction now
ArchHelpers::Context::SetPc(ucontext, ArchHelpers::Context::GetPc(ucontext) + BytesToSkip);
return true;
}
else {
LogMan::Msg::EFmt("Unhandled JIT SIGBUS LDAXR: PC: {} Instruction: 0x{:08x}\n", fmt::ptr(PC), PC[0]);
return false;
}
}
else {
LogMan::Msg::E("Unhandled JIT SIGBUS: PC: %p Instruction: 0x%08x\n", PC, PC[0]);
LogMan::Msg::EFmt("Unhandled JIT SIGBUS: PC: {} Instruction: 0x{:08x}\n", fmt::ptr(PC), PC[0]);
return false;
}
@@ -445,7 +472,7 @@ Arm64JITCore::Arm64JITCore(FEXCore::Context::Context *ctx, FEXCore::Core::Intern
DispatcherConfig config;
config.ExitFunctionLink = reinterpret_cast<uintptr_t>(&ExitFunctionLink);
config.ExitFunctionLinkThis = reinterpret_cast<uintptr_t>(this);
config.StaticRegisterAssignment = true;
config.StaticRegisterAssignment = ctx->Config.StaticRegisterAllocation;
Dispatcher = std::make_unique<Arm64Dispatcher>(CTX, ThreadState, config);
DispatchPtr = Dispatcher->DispatchPtr;
@@ -524,7 +551,7 @@ Arm64JITCore::Arm64JITCore(FEXCore::Context::Context *ctx, FEXCore::Core::Intern
return Core->Dispatcher->HandleGuestSignal(Signal, info, ucontext, GuestAction, GuestStack);
};
for (uint32_t Signal = 0; Signal < SignalDelegator::MAX_SIGNALS; ++Signal) {
for (uint32_t Signal = 0; Signal <= SignalDelegator::MAX_SIGNALS; ++Signal) {
CTX->SignalDelegation->RegisterHostSignalHandlerForGuest(Signal, GuestSignalHandler);
}
}
@@ -584,7 +611,7 @@ Arm64JITCore::~Arm64JITCore() {
IR::PhysicalRegister Arm64JITCore::GetPhys(uint32_t Node) const {
auto PhyReg = RAData->GetNodeRegister(Node);
LOGMAN_THROW_A(!PhyReg.IsInvalid(), "Couldn't Allocate register for node: ssa%d. Class: %d", Node, PhyReg.Class);
LOGMAN_THROW_A_FMT(!PhyReg.IsInvalid(), "Couldn't Allocate register for node: ssa{}. Class: {}", Node, PhyReg.Class);
return PhyReg;
}
@@ -598,7 +625,7 @@ aarch64::Register Arm64JITCore::GetReg<Arm64JITCore::RA_32>(uint32_t Node) const
} else if (Reg.Class == IR::GPRClass.Val) {
return RA64[Reg.Reg].W();
} else {
LOGMAN_THROW_A(false, "Unexpected Class: %d", Reg.Class);
LOGMAN_THROW_A_FMT(false, "Unexpected Class: {}", Reg.Class);
}
FEX_UNREACHABLE;
@@ -613,7 +640,7 @@ aarch64::Register Arm64JITCore::GetReg<Arm64JITCore::RA_64>(uint32_t Node) const
} else if (Reg.Class == IR::GPRClass.Val) {
return RA64[Reg.Reg];
} else {
LOGMAN_THROW_A(false, "Unexpected Class: %d", Reg.Class);
LOGMAN_THROW_A_FMT(false, "Unexpected Class: {}", Reg.Class);
}
FEX_UNREACHABLE;
@@ -639,7 +666,7 @@ aarch64::VRegister Arm64JITCore::GetSrc(uint32_t Node) const {
} else if (Reg.Class == IR::FPRClass.Val) {
return RAFPR[Reg.Reg];
} else {
LOGMAN_THROW_A(false, "Unexpected Class: %d", Reg.Class);
LOGMAN_THROW_A_FMT(false, "Unexpected Class: {}", Reg.Class);
}
FEX_UNREACHABLE;
@@ -653,7 +680,7 @@ aarch64::VRegister Arm64JITCore::GetDst(uint32_t Node) const {
} else if (Reg.Class == IR::FPRClass.Val) {
return RAFPR[Reg.Reg];
} else {
LOGMAN_THROW_A(false, "Unexpected Class: %d", Reg.Class);
LOGMAN_THROW_A_FMT(false, "Unexpected Class: {}", Reg.Class);
}
FEX_UNREACHABLE;
@@ -744,8 +771,7 @@ void *Arm64JITCore::CompileCode(uint64_t Entry, [[maybe_unused]] FEXCore::IR::IR
// X1-X3 = Temp
// X4-r18 = RA
auto Buffer = GetBuffer();
auto GuestEntry = Buffer->GetOffsetAddress<uint64_t>(GetCursorOffset());
auto GuestEntry = GetCursorAddress<uint64_t>();
if (CTX->GetGdbServerStatus()) {
aarch64::Label RunBlock;
@@ -791,7 +817,7 @@ void *Arm64JITCore::CompileCode(uint64_t Entry, [[maybe_unused]] FEXCore::IR::IR
using namespace FEXCore::IR;
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
auto BlockIROp = BlockHeader->CW<FEXCore::IR::IROp_CodeBlock>();
LOGMAN_THROW_A(BlockIROp->Header.Op == IR::OP_CODEBLOCK, "IR type failed to be a code block");
LOGMAN_THROW_A_FMT(BlockIROp->Header.Op == IR::OP_CODEBLOCK, "IR type failed to be a code block");
#endif
{
@@ -812,7 +838,7 @@ void *Arm64JITCore::CompileCode(uint64_t Entry, [[maybe_unused]] FEXCore::IR::IR
}
if (DebugData) {
DebugData->Subblocks.push_back({Buffer->GetOffsetAddress<uintptr_t>(GetCursorOffset()), 0, IR->GetID(BlockNode)});
DebugData->Subblocks.push_back({GetCursorAddress<uintptr_t>(), 0, IR->GetID(BlockNode)});
}
for (auto [CodeNode, IROp] : IR->GetCode(BlockNode)) {
@@ -824,7 +850,7 @@ void *Arm64JITCore::CompileCode(uint64_t Entry, [[maybe_unused]] FEXCore::IR::IR
}
if (DebugData) {
DebugData->Subblocks.back().HostCodeSize = Buffer->GetOffsetAddress<uintptr_t>(GetCursorOffset()) - DebugData->Subblocks.back().HostCodeStart;
DebugData->Subblocks.back().HostCodeSize = GetCursorAddress<uintptr_t>() - DebugData->Subblocks.back().HostCodeStart;
}
}
@@ -837,7 +863,7 @@ void *Arm64JITCore::CompileCode(uint64_t Entry, [[maybe_unused]] FEXCore::IR::IR
FinalizeCode();
auto CodeEnd = Buffer->GetOffsetAddress<uint64_t>(GetCursorOffset());
auto CodeEnd = GetCursorAddress<uint64_t>();
CPU.EnsureIAndDCacheCoherency(reinterpret_cast<void*>(GuestEntry), CodeEnd - reinterpret_cast<uint64_t>(GuestEntry));
if (DebugData) {
@@ -856,7 +882,7 @@ uint64_t Arm64JITCore::ExitFunctionLink(Arm64JITCore *core, FEXCore::Core::CpuSt
auto HostCode = Thread->LookupCache->FindBlock(GuestRip);
if (!HostCode) {
//printf("ExitFunctionLink: Aborting, %lX not in cache\n", GuestRip);
//fmt::print("ExitFunctionLink: Aborting, {:X} not in cache\n", GuestRip);
Frame->State.rip = GuestRip;
return core->ThreadSharedData.Dispatcher->AbsoluteLoopTopAddress;
}
@@ -6,7 +6,6 @@ $end_info$
#pragma once
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/ArchHelpers/Arm64Emitter.h"
#include "Interface/Core/Dispatcher/Dispatcher.h"
@@ -257,6 +256,7 @@ private:
DEF_OP(AtomicFetchAnd);
DEF_OP(AtomicFetchOr);
DEF_OP(AtomicFetchXor);
DEF_OP(AtomicFetchNeg);
///< Branch ops
DEF_OP(GuestCallDirect);
@@ -30,7 +30,7 @@ DEF_OP(LoadContext) {
case 8:
ldr(GetReg<RA_64>(Node), MemOperand(STATE, Op->Offset));
break;
default: LOGMAN_MSG_A("Unhandled LoadContext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled LoadContext size: {}", OpSize);
}
}
else {
@@ -51,7 +51,7 @@ DEF_OP(LoadContext) {
case 16:
ldr(Dst, MemOperand(STATE, Op->Offset));
break;
default: LOGMAN_MSG_A("Unhandled LoadContext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled LoadContext size: {}", OpSize);
}
}
}
@@ -73,7 +73,7 @@ DEF_OP(StoreContext) {
case 8:
str(GetReg<RA_64>(Op->Header.Args[0].ID()), MemOperand(STATE, Op->Offset));
break;
default: LOGMAN_MSG_A("Unhandled StoreContext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled StoreContext size: {}", OpSize);
}
}
else {
@@ -94,7 +94,7 @@ DEF_OP(StoreContext) {
case 16:
str(Src, MemOperand(STATE, Op->Offset));
break;
default: LOGMAN_MSG_A("Unhandled LoadContext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled StoreContext size: {}", OpSize);
}
}
}
@@ -107,29 +107,29 @@ DEF_OP(LoadRegister) {
auto regId = (Op->Offset - offsetof(FEXCore::Core::CpuStateFrame, State.gregs[0])) / 8;
auto regOffs = Op->Offset & 7;
LOGMAN_THROW_A(regId < SRA64.size(), "out of range regId");
LOGMAN_THROW_A_FMT(regId < SRA64.size(), "out of range regId");
auto reg = SRA64[regId];
switch(Op->Header.Size) {
case 1:
LOGMAN_THROW_A(regOffs == 0 || regOffs == 1, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0 || regOffs == 1, "unexpected regOffs");
ubfx(GetReg<RA_64>(Node), reg, regOffs * 8, 8);
break;
case 2:
LOGMAN_THROW_A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
ubfx(GetReg<RA_64>(Node), reg, 0, 16);
break;
case 4:
LOGMAN_THROW_A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
if (GetReg<RA_64>(Node).GetCode() != reg.GetCode())
mov(GetReg<RA_32>(Node), reg.W());
break;
case 8:
LOGMAN_THROW_A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
if (GetReg<RA_64>(Node).GetCode() != reg.GetCode())
mov(GetReg<RA_64>(Node), reg);
break;
@@ -138,24 +138,24 @@ DEF_OP(LoadRegister) {
auto regId = (Op->Offset - offsetof(FEXCore::Core::CpuStateFrame, State.xmm[0][0])) / 16;
auto regOffs = Op->Offset & 15;
LOGMAN_THROW_A(regId < SRAFPR.size(), "out of range regId");
LOGMAN_THROW_A_FMT(regId < SRAFPR.size(), "out of range regId");
auto guest = SRAFPR[regId];
auto host = GetSrc(Node);
switch(Op->Header.Size) {
case 1:
LOGMAN_THROW_A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
mov(host.B(), guest.B());
break;
case 2:
LOGMAN_THROW_A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
fmov(host.H(), guest.H());
break;
case 4:
LOGMAN_THROW_A((regOffs & 3) == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT((regOffs & 3) == 0, "unexpected regOffs");
if (regOffs == 0) {
if (host.GetCode() != guest.GetCode())
fmov(host.S(), guest.S());
@@ -165,7 +165,7 @@ DEF_OP(LoadRegister) {
break;
case 8:
LOGMAN_THROW_A((regOffs & 7) == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT((regOffs & 7) == 0, "unexpected regOffs");
if (regOffs == 0) {
if (host.GetCode() != guest.GetCode())
mov(host.D(), guest.D());
@@ -175,13 +175,13 @@ DEF_OP(LoadRegister) {
break;
case 16:
LOGMAN_THROW_A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
if (host.GetCode() != guest.GetCode())
mov(host.Q(), guest.Q());
break;
}
} else {
LOGMAN_THROW_A(false, "Unhandled Op->Class %d", Op->Class);
LOGMAN_THROW_A_FMT(false, "Unhandled Op->Class {}", Op->Class);
}
}
@@ -192,28 +192,28 @@ DEF_OP(StoreRegister) {
auto regId = Op->Offset / 8 - 1;
auto regOffs = Op->Offset & 7;
LOGMAN_THROW_A(regId < SRA64.size(), "out of range regId");
LOGMAN_THROW_A_FMT(regId < SRA64.size(), "out of range regId");
auto reg = SRA64[regId];
switch(Op->Header.Size) {
case 1:
LOGMAN_THROW_A(regOffs == 0 || regOffs == 1, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0 || regOffs == 1, "unexpected regOffs");
bfi(reg, GetReg<RA_64>(Op->Value.ID()), regOffs * 8, 8);
break;
case 2:
LOGMAN_THROW_A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
bfi(reg, GetReg<RA_64>(Op->Value.ID()), 0, 16);
break;
case 4:
LOGMAN_THROW_A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
bfi(reg, GetReg<RA_64>(Op->Value.ID()), 0, 32);
break;
case 8:
LOGMAN_THROW_A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
if (GetReg<RA_64>(Op->Value.ID()).GetCode() != reg.GetCode())
mov(reg, GetReg<RA_64>(Op->Value.ID()));
break;
@@ -222,7 +222,7 @@ DEF_OP(StoreRegister) {
auto regId = (Op->Offset - offsetof(FEXCore::Core::CpuStateFrame, State.xmm[0][0])) / 16;
auto regOffs = Op->Offset & 15;
LOGMAN_THROW_A(regId < SRAFPR.size(), "regId out of range");
LOGMAN_THROW_A_FMT(regId < SRAFPR.size(), "regId out of range");
auto guest = SRAFPR[regId];
auto host = GetSrc(Op->Value.ID());
@@ -233,28 +233,28 @@ DEF_OP(StoreRegister) {
break;
case 2:
LOGMAN_THROW_A((regOffs & 1) == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT((regOffs & 1) == 0, "unexpected regOffs");
ins(guest.V8H(), regOffs/2, host.V8H(), 0);
break;
case 4:
LOGMAN_THROW_A((regOffs & 3) == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT((regOffs & 3) == 0, "unexpected regOffs");
ins(guest.V4S(), regOffs/4, host.V4S(), 0);
break;
case 8:
LOGMAN_THROW_A((regOffs & 7) == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT((regOffs & 7) == 0, "unexpected regOffs");
ins(guest.V2D(), regOffs / 8, host.V2D(), 0);
break;
case 16:
LOGMAN_THROW_A(regOffs == 0, "unexpected regOffs");
LOGMAN_THROW_A_FMT(regOffs == 0, "unexpected regOffs");
if (guest.GetCode() != host.GetCode())
mov(guest.Q(), host.Q());
break;
}
} else {
LOGMAN_THROW_A(false, "Unhandled Op->Class %d", Op->Class);
LOGMAN_THROW_A_FMT(false, "Unhandled Op->Class {}", Op->Class);
}
}
@@ -288,15 +288,17 @@ DEF_OP(LoadContextIndexed) {
ldr(GetReg<RA_64>(Node), MemOperand(TMP1, Op->BaseOffset));
break;
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed size: %d", Op->Size);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed size: {}", Op->Size);
break;
}
break;
}
case 16:
LOGMAN_MSG_A("Invalid Class load of size 16");
LOGMAN_MSG_A_FMT("Invalid Class load of size 16");
break;
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed stride: %d", Op->Stride);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed stride: {}", Op->Stride);
break;
}
}
else {
@@ -333,12 +335,14 @@ DEF_OP(LoadContextIndexed) {
}
break;
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed size: %d", Op->Size);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed size: {}", Op->Size);
break;
}
break;
}
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed stride: %d", Op->Stride);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed stride: {}", Op->Stride);
break;
}
}
}
@@ -374,15 +378,17 @@ DEF_OP(StoreContextIndexed) {
str(value, MemOperand(TMP1, Op->BaseOffset));
break;
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed size: %d", Op->Size);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed size: {}", Op->Size);
break;
}
break;
}
case 16:
LOGMAN_MSG_A("Invalid Class load of size 16");
LOGMAN_MSG_A_FMT("Invalid Class store of size 16");
break;
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed stride: %d", Op->Stride);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed stride: {}", Op->Stride);
break;
}
}
else {
@@ -421,12 +427,14 @@ DEF_OP(StoreContextIndexed) {
}
break;
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed size: %d", Op->Size);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed size: {}", Op->Size);
break;
}
break;
}
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed stride: %d", Op->Stride);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed stride: {}", Op->Stride);
break;
}
}
}
@@ -434,7 +442,7 @@ DEF_OP(StoreContextIndexed) {
DEF_OP(SpillRegister) {
auto Op = IROp->C<IR::IROp_SpillRegister>();
uint8_t OpSize = IROp->Size;
uint32_t SlotOffset = Op->Slot * 16 + 16;
uint32_t SlotOffset = Op->Slot * 16;
if (Op->Class == FEXCore::IR::GPRClass) {
switch (OpSize) {
@@ -454,7 +462,7 @@ DEF_OP(SpillRegister) {
str(GetReg<RA_64>(Op->Header.Args[0].ID()), MemOperand(sp, SlotOffset));
break;
}
default: LOGMAN_MSG_A("Unhandled SpillRegister size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled SpillRegister size: {}", OpSize);
}
} else if (Op->Class == FEXCore::IR::FPRClass) {
switch (OpSize) {
@@ -470,17 +478,17 @@ DEF_OP(SpillRegister) {
str(GetSrc(Op->Header.Args[0].ID()), MemOperand(sp, SlotOffset));
break;
}
default: LOGMAN_MSG_A("Unhandled SpillRegister size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled SpillRegister size: {}", OpSize);
}
} else {
LOGMAN_MSG_A("Unhandled SpillRegister class: %d", Op->Class.Val);
LOGMAN_MSG_A_FMT("Unhandled SpillRegister class: {}", Op->Class.Val);
}
}
DEF_OP(FillRegister) {
auto Op = IROp->C<IR::IROp_FillRegister>();
uint8_t OpSize = IROp->Size;
uint32_t SlotOffset = Op->Slot * 16 + 16;
uint32_t SlotOffset = Op->Slot * 16;
if (Op->Class == FEXCore::IR::GPRClass) {
switch (OpSize) {
@@ -500,7 +508,7 @@ DEF_OP(FillRegister) {
ldr(GetReg<RA_64>(Node), MemOperand(sp, SlotOffset));
break;
}
default: LOGMAN_MSG_A("Unhandled SpillRegister size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled FillRegister size: {}", OpSize);
}
} else if (Op->Class == FEXCore::IR::FPRClass) {
switch (OpSize) {
@@ -516,10 +524,10 @@ DEF_OP(FillRegister) {
ldr(GetDst(Node), MemOperand(sp, SlotOffset));
break;
}
default: LOGMAN_MSG_A("Unhandled SpillRegister size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled FillRegister size: {}", OpSize);
}
} else {
LOGMAN_MSG_A("Unhandled FillRegister class: %d", Op->Class.Val);
LOGMAN_MSG_A_FMT("Unhandled FillRegister class: {}", Op->Class.Val);
}
}
@@ -539,7 +547,7 @@ MemOperand Arm64JITCore::GenerateMemOperand(uint8_t AccessSize, aarch64::Registe
return MemOperand(Base);
} else {
if (OffsetScale != 1 && OffsetScale != AccessSize) {
LOGMAN_MSG_A("Unhandled GenerateMemOperand OffsetScale: %d", OffsetScale);
LOGMAN_MSG_A_FMT("Unhandled GenerateMemOperand OffsetScale: {}", OffsetScale);
}
uint64_t Const;
if (IsInlineConstant(Offset, &Const)) {
@@ -551,7 +559,7 @@ MemOperand Arm64JITCore::GenerateMemOperand(uint8_t AccessSize, aarch64::Registe
case IR::MEM_OFFSET_UXTW.Val: return MemOperand(Base, RegOffset.W(), Extend::UXTW, (int)std::log2(OffsetScale) );
case IR::MEM_OFFSET_SXTW.Val: return MemOperand(Base, RegOffset.W(), Extend::SXTW, (int)std::log2(OffsetScale) );
default: LOGMAN_MSG_A("Unhandled GenerateMemOperand OffsetType: %d", OffsetType.Val); break;
default: LOGMAN_MSG_A_FMT("Unhandled GenerateMemOperand OffsetType: {}", OffsetType.Val); break;
}
}
}
@@ -580,7 +588,7 @@ DEF_OP(LoadMem) {
case 8:
ldr(Dst, MemSrc);
break;
default: LOGMAN_MSG_A("Unhandled LoadMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled LoadMem size: {}", Op->Size);
}
}
else {
@@ -601,7 +609,7 @@ DEF_OP(LoadMem) {
case 16:
ldr(Dst, MemSrc);
break;
default: LOGMAN_MSG_A("Unhandled LoadMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled LoadMem size: {}", Op->Size);
}
}
}
@@ -612,7 +620,7 @@ DEF_OP(LoadMemTSO) {
auto MemSrc = MemOperand(GetReg<RA_64>(Op->Header.Args[0].ID()));
if (!Op->Offset.IsInvalid()) {
LOGMAN_MSG_A("LoadMemTSO: No offset allowed");
LOGMAN_MSG_A_FMT("LoadMemTSO: No offset allowed");
}
if (SupportsRCPC && Op->Class == FEXCore::IR::GPRClass) {
@@ -635,7 +643,7 @@ DEF_OP(LoadMemTSO) {
case 8:
ldapr(Dst, MemSrc);
break;
default: LOGMAN_MSG_A("Unhandled LoadMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled LoadMemTSO size: {}", Op->Size);
}
nop();
}
@@ -660,7 +668,7 @@ DEF_OP(LoadMemTSO) {
case 8:
ldar(Dst, MemSrc);
break;
default: LOGMAN_MSG_A("Unhandled LoadMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled LoadMemTSO size: {}", Op->Size);
}
nop();
}
@@ -681,7 +689,7 @@ DEF_OP(LoadMemTSO) {
case 16:
ldr(Dst, MemSrc);
break;
default: LOGMAN_MSG_A("Unhandled LoadMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled LoadMemTSO size: {}", Op->Size);
}
dmb(InnerShareable, BarrierAll);
}
@@ -708,7 +716,7 @@ DEF_OP(StoreMem) {
case 8:
str(GetReg<RA_64>(Op->Header.Args[1].ID()), MemSrc);
break;
default: LOGMAN_MSG_A("Unhandled StoreMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled StoreMem size: {}", Op->Size);
}
}
else {
@@ -729,7 +737,7 @@ DEF_OP(StoreMem) {
case 16:
str(Src, MemSrc);
break;
default: LOGMAN_MSG_A("Unhandled StoreMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled StoreMem size: {}", Op->Size);
}
}
}
@@ -739,7 +747,7 @@ DEF_OP(StoreMemTSO) {
auto MemSrc = MemOperand(GetReg<RA_64>(Op->Header.Args[0].ID()));
if (!Op->Offset.IsInvalid()) {
LOGMAN_MSG_A("StoreMemTSO: No offset allowed");
LOGMAN_MSG_A_FMT("StoreMemTSO: No offset allowed");
}
if (Op->Class == FEXCore::IR::GPRClass) {
@@ -759,7 +767,7 @@ DEF_OP(StoreMemTSO) {
case 8:
stlr(GetReg<RA_64>(Op->Header.Args[1].ID()), MemSrc);
break;
default: LOGMAN_MSG_A("Unhandled StoreMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled StoreMemTSO size: {}", Op->Size);
}
nop();
}
@@ -783,7 +791,7 @@ DEF_OP(StoreMemTSO) {
case 16:
str(Src, MemSrc);
break;
default: LOGMAN_MSG_A("Unhandled StoreMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled StoreMemTSO size: {}", Op->Size);
}
dmb(InnerShareable, BarrierAll);
}
@@ -795,7 +803,7 @@ DEF_OP(ParanoidLoadMemTSO) {
auto MemSrc = MemOperand(GetReg<RA_64>(Op->Header.Args[0].ID()));
if (!Op->Offset.IsInvalid()) {
LOGMAN_MSG_A("LoadMemTSO: No offset allowed");
LOGMAN_MSG_A_FMT("ParanoidLoadMemTSO: No offset allowed");
}
if (Op->Class == FEXCore::IR::GPRClass) {
@@ -806,7 +814,6 @@ DEF_OP(ParanoidLoadMemTSO) {
}
else {
auto Dst = GetReg<RA_64>(Node);
nop();
switch (Op->Size) {
case 2:
ldarh(Dst, MemSrc);
@@ -817,31 +824,24 @@ DEF_OP(ParanoidLoadMemTSO) {
case 8:
ldar(Dst, MemSrc);
break;
default: LOGMAN_MSG_A("Unhandled LoadMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled ParanoidLoadMemTSO size: {}", Op->Size);
}
nop();
}
}
else {
auto Dst = GetDst(Node);
switch (Op->Size) {
case 2:
nop();
ldarh(TMP1, MemSrc);
nop();
fmov(Dst, TMP1);
ldarh(TMP1.W(), MemSrc);
fmov(Dst.H(), TMP1.W());
break;
case 4:
nop();
ldar(TMP1.W(), MemSrc);
nop();
fmov(Dst, TMP1);
fmov(Dst.S(), TMP1.W());
break;
case 8:
nop();
ldar(TMP1, MemSrc);
nop();
fmov(Dst, TMP1);
fmov(Dst.D(), TMP1);
break;
case 16:
nop();
@@ -850,7 +850,7 @@ DEF_OP(ParanoidLoadMemTSO) {
mov(Dst.V2D(), 0, TMP1);
mov(Dst.V2D(), 1, TMP2);
break;
default: LOGMAN_MSG_A("Unhandled LoadMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled ParanoidLoadMemTSO size: {}", Op->Size);
}
}
}
@@ -860,7 +860,7 @@ DEF_OP(ParanoidStoreMemTSO) {
auto MemSrc = MemOperand(GetReg<RA_64>(Op->Header.Args[0].ID()));
if (!Op->Offset.IsInvalid()) {
LOGMAN_MSG_A("StoreMemTSO: No offset allowed");
LOGMAN_MSG_A_FMT("ParanoidStoreMemTSO: No offset allowed");
}
if (Op->Class == FEXCore::IR::GPRClass) {
@@ -869,7 +869,6 @@ DEF_OP(ParanoidStoreMemTSO) {
stlrb(GetReg<RA_64>(Op->Header.Args[1].ID()), MemSrc);
}
else {
nop();
switch (Op->Size) {
case 2:
stlrh(GetReg<RA_64>(Op->Header.Args[1].ID()), MemSrc);
@@ -880,37 +879,30 @@ DEF_OP(ParanoidStoreMemTSO) {
case 8:
stlr(GetReg<RA_64>(Op->Header.Args[1].ID()), MemSrc);
break;
default: LOGMAN_MSG_A("Unhandled StoreMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled ParanoidStoreMemTSO size: {}", Op->Size);
}
nop();
}
}
else {
auto Src = GetSrc(Op->Header.Args[1].ID());
if (Op->Size == 1) {
// 8bit load is always aligned to natural alignment
mov(TMP1, Src.V4S(), 0);
mov(TMP1.W(), Src.V16B(), 0);
stlrb(TMP1, MemSrc);
}
else {
switch (Op->Size) {
case 2:
mov(TMP1, Src.V4S(), 0);
nop();
mov(TMP1.W(), Src.V8H(), 0);
stlrh(TMP1, MemSrc);
nop();
break;
case 4:
mov(TMP1, Src.V4S(), 0);
nop();
mov(TMP1.W(), Src.V4S(), 0);
stlr(TMP1.W(), MemSrc);
nop();
break;
case 8:
mov(TMP1, Src.V2D(), 0);
nop();
stlr(TMP1, MemSrc);
nop();
break;
case 16: {
// Move vector to GPRs
@@ -927,18 +919,18 @@ DEF_OP(ParanoidStoreMemTSO) {
cbnz(TMP3, &B); // < Overwritten with DMB
break;
}
default: LOGMAN_MSG_A("Unhandled StoreMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled ParanoidStoreMemTSO size: {}", Op->Size);
}
}
}
}
DEF_OP(VLoadMemElement) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(VStoreMemElement) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(CacheLineClear) {
@@ -8,11 +8,11 @@ $end_info$
namespace FEXCore::CPU {
static void PrintValue(uint64_t Value) {
LogMan::Msg::D("Value: 0x%lx", Value);
LogMan::Msg::DFmt("Value: 0x{:x}", Value);
}
static void PrintVectorValue(uint64_t Value, uint64_t ValueUpper) {
LogMan::Msg::D("Value: 0x%016lx'%016lx", ValueUpper, Value);
LogMan::Msg::DFmt("Value: 0x{:016x}'{:016x}", ValueUpper, Value);
}
using namespace vixl;
@@ -31,7 +31,7 @@ DEF_OP(Fence) {
case IR::Fence_Store.Val:
dmb(FullSystem, BarrierWrites);
break;
default: LOGMAN_MSG_A("Unknown Fence: %d", Op->Fence); break;
default: LOGMAN_MSG_A_FMT("Unknown Fence: {}", Op->Fence); break;
}
}
@@ -69,7 +69,7 @@ DEF_OP(Break) {
br(TMP1);
break;
}
default: LOGMAN_MSG_A("Unknown Break reason: %d", Op->Reason);
default: LOGMAN_MSG_A_FMT("Unknown Break reason: {}", Op->Reason);
}
}
@@ -26,7 +26,7 @@ DEF_OP(ExtractElementPair) {
mov (GetReg<RA_64>(Node), Regs[Op->Element]);
break;
}
default: LOGMAN_MSG_A("Unknown Size"); break;
default: LOGMAN_MSG_A_FMT("Unknown Size"); break;
}
}
@@ -52,7 +52,7 @@ DEF_OP(CreateElementPair) {
RegTmp = x0;
break;
}
default: LOGMAN_MSG_A("Unknown Size"); break;
default: LOGMAN_MSG_A_FMT("Unknown Size"); break;
}
if (Dst.first.GetCode() != RegSecond.GetCode()) {
+132 -132
View File
@@ -22,7 +22,7 @@ DEF_OP(VectorZero) {
eor(GetDst(Node).V16B(), GetDst(Node).V16B(), GetDst(Node).V16B());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", OpSize); break;
}
}
@@ -43,47 +43,47 @@ DEF_OP(VectorImm) {
}
DEF_OP(CreateVector2) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(CreateVector4) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(SplatVector2) {
auto Op = IROp->C<IR::IROp_SplatVector2>();
auto Op = IROp->C<IR::IROp_SplatVector2>();
uint8_t OpSize = IROp->Size;
LOGMAN_THROW_A(OpSize <= 16, "Can't handle a vector of size: %d", OpSize);
LOGMAN_THROW_A_FMT(OpSize <= 16, "Can't handle a vector of size: {}", OpSize);
uint8_t ElementSize = OpSize / 2;
uint8_t ElementSize = OpSize / 2;
switch (ElementSize) {
case 4:
dup(GetDst(Node).V4S(), GetSrc(Op->Header.Args[0].ID()).V4S(), 0);
break;
case 8:
dup(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), 0);
break;
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.Size); break;
}
switch (ElementSize) {
case 4:
dup(GetDst(Node).V4S(), GetSrc(Op->Header.Args[0].ID()).V4S(), 0);
break;
case 8:
dup(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), 0);
break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.Size); break;
}
}
DEF_OP(SplatVector4) {
auto Op = IROp->C<IR::IROp_SplatVector4>();
auto Op = IROp->C<IR::IROp_SplatVector4>();
uint8_t OpSize = IROp->Size;
LOGMAN_THROW_A(OpSize <= 16, "Can't handle a vector of size: %d", OpSize);
LOGMAN_THROW_A_FMT(OpSize <= 16, "Can't handle a vector of size: {}", OpSize);
uint8_t ElementSize = OpSize / 4;
uint8_t ElementSize = OpSize / 4;
switch (ElementSize) {
case 4:
dup(GetDst(Node).V4S(), GetSrc(Op->Header.Args[0].ID()).V4S(), 0);
break;
case 8:
dup(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), 0);
break;
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.Size); break;
}
switch (ElementSize) {
case 4:
dup(GetDst(Node).V4S(), GetSrc(Op->Header.Args[0].ID()).V4S(), 0);
break;
case 8:
dup(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), 0);
break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.Size); break;
}
}
DEF_OP(VMov) {
@@ -118,7 +118,7 @@ DEF_OP(VMov) {
mov(GetDst(Node).V16B(), GetSrc(Op->Header.Args[0].ID()).V16B());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", OpSize); break;
}
}
@@ -161,7 +161,7 @@ DEF_OP(VAdd) {
add(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -184,7 +184,7 @@ DEF_OP(VSub) {
sub(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -207,7 +207,7 @@ DEF_OP(VUQAdd) {
uqadd(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -230,7 +230,7 @@ DEF_OP(VUQSub) {
uqsub(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -253,7 +253,7 @@ DEF_OP(VSQAdd) {
sqadd(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -276,7 +276,7 @@ DEF_OP(VSQSub) {
sqsub(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -298,7 +298,7 @@ DEF_OP(VAddP) {
addp(GetDst(Node).V2S(), GetSrc(Op->Header.Args[0].ID()).V2S(), GetSrc(Op->Header.Args[1].ID()).V2S());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -319,7 +319,7 @@ DEF_OP(VAddP) {
addp(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -338,7 +338,7 @@ DEF_OP(VAddV) {
case 8:
addp(GetDst(Node).VCast(OpSize * 8, 1), GetSrc(Op->Header.Args[0].ID()).VCast(OpSize * 8, Elements));
break;
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -353,7 +353,7 @@ DEF_OP(VUMinV) {
case 4:
uminv(GetDst(Node).VCast(Op->Header.ElementSize * 8, 1), GetSrc(Op->Header.Args[0].ID()).VCast(OpSize * 8, Elements));
break;
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -368,7 +368,7 @@ DEF_OP(VURAvg) {
urhadd(GetDst(Node).V8H(), GetSrc(Op->Header.Args[0].ID()).V8H(), GetSrc(Op->Header.Args[1].ID()).V8H());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -383,7 +383,7 @@ DEF_OP(VAbs) {
abs(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -395,7 +395,7 @@ DEF_OP(VAbs) {
case 8:
abs(GetDst(Node).VCast(OpSize * 8, Elements), GetSrc(Op->Header.Args[0].ID()).VCast(OpSize * 8, Elements));
break;
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -410,7 +410,7 @@ DEF_OP(VPopcount) {
cnt(GetDst(Node).V8B(), GetSrc(Op->Header.Args[0].ID()).V8B());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -419,7 +419,7 @@ DEF_OP(VPopcount) {
case 1:
cnt(GetDst(Node).V16B(), GetSrc(Op->Header.Args[0].ID()).V16B());
break;
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -438,7 +438,7 @@ DEF_OP(VFAdd) {
fadd(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), GetSrc(Op->Header.Args[1].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -452,7 +452,7 @@ DEF_OP(VFAdd) {
fadd(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -468,7 +468,7 @@ DEF_OP(VFAddP) {
faddp(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -486,7 +486,7 @@ DEF_OP(VFSub) {
fsub(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), GetSrc(Op->Header.Args[1].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -500,7 +500,7 @@ DEF_OP(VFSub) {
fsub(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -519,7 +519,7 @@ DEF_OP(VFMul) {
fmul(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), GetSrc(Op->Header.Args[1].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -533,7 +533,7 @@ DEF_OP(VFMul) {
fmul(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -552,7 +552,7 @@ DEF_OP(VFDiv) {
fdiv(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), GetSrc(Op->Header.Args[1].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -566,7 +566,7 @@ DEF_OP(VFDiv) {
fdiv(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -587,7 +587,7 @@ DEF_OP(VFMin) {
fcsel(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), GetSrc(Op->Header.Args[1].ID()).D(), Condition::mi);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -607,7 +607,7 @@ DEF_OP(VFMin) {
mov(GetDst(Node).V2D(), VTMP2.V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -628,7 +628,7 @@ DEF_OP(VFMax) {
fcsel(GetDst(Node).D(), GetSrc(Op->Header.Args[1].ID()).D(), GetSrc(Op->Header.Args[0].ID()).D(), Condition::mi);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -648,7 +648,7 @@ DEF_OP(VFMax) {
mov(GetDst(Node).V2D(), VTMP2.V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -669,7 +669,7 @@ DEF_OP(VFRecp) {
fdiv(GetDst(Node).D(), VTMP1.D(), GetSrc(Op->Header.Args[0].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -685,7 +685,7 @@ DEF_OP(VFRecp) {
fdiv(GetDst(Node).V2D(), VTMP1.V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -704,7 +704,7 @@ DEF_OP(VFSqrt) {
fsqrt(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -718,7 +718,7 @@ DEF_OP(VFSqrt) {
fsqrt(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -741,7 +741,7 @@ DEF_OP(VFRSqrt) {
fdiv(GetDst(Node).D(), VTMP1.D(), VTMP2.D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -759,7 +759,7 @@ DEF_OP(VFRSqrt) {
fdiv(GetDst(Node).V2D(), VTMP1.V2D(), VTMP2.V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -779,7 +779,7 @@ DEF_OP(VNeg) {
case 8:
neg(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
default: LOGMAN_MSG_A("Unsupported Not size: %d", IROp->Size);
default: LOGMAN_MSG_A_FMT("Unsupported VNeg size: {}", IROp->Size);
}
}
@@ -792,7 +792,7 @@ DEF_OP(VFNeg) {
case 8:
fneg(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
default: LOGMAN_MSG_A("Unsupported Not size: %d", IROp->Size);
default: LOGMAN_MSG_A_FMT("Unsupported VFNeg size: {}", IROp->Size);
}
}
@@ -823,7 +823,7 @@ DEF_OP(VUMin) {
mov(GetDst(Node).V2D(), VTMP2.V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -849,7 +849,7 @@ DEF_OP(VSMin) {
mov(GetDst(Node).V2D(), VTMP2.V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -875,7 +875,7 @@ DEF_OP(VUMax) {
mov(GetDst(Node).V2D(), VTMP2.V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -901,7 +901,7 @@ DEF_OP(VSMax) {
mov(GetDst(Node).V2D(), VTMP2.V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -922,7 +922,7 @@ DEF_OP(VZip) {
zip1(GetDst(Node).V2S(), GetSrc(Op->Header.Args[0].ID()).V2S(), GetSrc(Op->Header.Args[1].ID()).V2S());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -943,7 +943,7 @@ DEF_OP(VZip) {
zip1(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -965,7 +965,7 @@ DEF_OP(VZip2) {
zip2(GetDst(Node).V2S(), GetSrc(Op->Header.Args[0].ID()).V2S(), GetSrc(Op->Header.Args[1].ID()).V2S());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -986,7 +986,7 @@ DEF_OP(VZip2) {
zip2(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1008,7 +1008,7 @@ DEF_OP(VUnZip) {
uzp1(GetDst(Node).V2S(), GetSrc(Op->Header.Args[0].ID()).V2S(), GetSrc(Op->Header.Args[1].ID()).V2S());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1029,7 +1029,7 @@ DEF_OP(VUnZip) {
uzp1(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1051,7 +1051,7 @@ DEF_OP(VUnZip2) {
uzp2(GetDst(Node).V2S(), GetSrc(Op->Header.Args[0].ID()).V2S(), GetSrc(Op->Header.Args[1].ID()).V2S());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1072,7 +1072,7 @@ DEF_OP(VUnZip2) {
uzp2(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1105,7 +1105,7 @@ DEF_OP(VCMPEQ) {
cmeq(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), GetSrc(Op->Header.Args[1].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1127,7 +1127,7 @@ DEF_OP(VCMPEQ) {
cmeq(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1146,7 +1146,7 @@ DEF_OP(VCMPEQZ) {
cmeq(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), 0);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1168,7 +1168,7 @@ DEF_OP(VCMPEQZ) {
cmeq(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), 0);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1187,7 +1187,7 @@ DEF_OP(VCMPGT) {
cmgt(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), GetSrc(Op->Header.Args[1].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1209,7 +1209,7 @@ DEF_OP(VCMPGT) {
cmgt(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1228,7 +1228,7 @@ DEF_OP(VCMPGTZ) {
cmgt(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), 0);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1250,7 +1250,7 @@ DEF_OP(VCMPGTZ) {
cmgt(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), 0);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1269,7 +1269,7 @@ DEF_OP(VCMPLTZ) {
cmlt(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), 0);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1291,7 +1291,7 @@ DEF_OP(VCMPLTZ) {
cmlt(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), 0);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1310,7 +1310,7 @@ DEF_OP(VFCMPEQ) {
fcmeq(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), GetSrc(Op->Header.Args[1].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1328,7 +1328,7 @@ DEF_OP(VFCMPEQ) {
fcmeq(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1347,7 +1347,7 @@ DEF_OP(VFCMPNEQ) {
fcmeq(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), GetSrc(Op->Header.Args[1].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
mvn(GetDst(Node).V8B(), GetDst(Node).V8B());
}
@@ -1366,7 +1366,7 @@ DEF_OP(VFCMPNEQ) {
fcmeq(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
mvn(GetDst(Node).V16B(), GetDst(Node).V16B());
}
@@ -1386,7 +1386,7 @@ DEF_OP(VFCMPLT) {
fcmgt(GetDst(Node).D(), GetSrc(Op->Header.Args[1].ID()).D(), GetSrc(Op->Header.Args[0].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1404,7 +1404,7 @@ DEF_OP(VFCMPLT) {
fcmgt(GetDst(Node).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1423,7 +1423,7 @@ DEF_OP(VFCMPGT) {
fcmgt(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).D(), GetSrc(Op->Header.Args[1].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1441,7 +1441,7 @@ DEF_OP(VFCMPGT) {
fcmgt(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1460,7 +1460,7 @@ DEF_OP(VFCMPLE) {
fcmge(GetDst(Node).D(), GetSrc(Op->Header.Args[1].ID()).D(), GetSrc(Op->Header.Args[0].ID()).D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1478,7 +1478,7 @@ DEF_OP(VFCMPLE) {
fcmge(GetDst(Node).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1501,7 +1501,7 @@ DEF_OP(VFCMPORD) {
orr(GetDst(Node).V8B(), VTMP1.V8B(), VTMP2.V8B());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1525,7 +1525,7 @@ DEF_OP(VFCMPORD) {
orr(GetDst(Node).V16B(), VTMP1.V16B(), VTMP2.V16B());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1550,7 +1550,7 @@ DEF_OP(VFCMPUNO) {
mvn(GetDst(Node).V8B(), GetDst(Node).V8B());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -1577,21 +1577,21 @@ DEF_OP(VFCMPUNO) {
mvn(GetDst(Node).V16B(), GetDst(Node).V16B());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
DEF_OP(VUShl) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(VUShr) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(VSShr) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(VUShlS) {
@@ -1618,7 +1618,7 @@ DEF_OP(VUShlS) {
ushl(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), VTMP1.V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1650,7 +1650,7 @@ DEF_OP(VUShrS) {
ushl(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), VTMP1.V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1682,7 +1682,7 @@ DEF_OP(VSShrS) {
sshl(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), VTMP1.V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1713,7 +1713,7 @@ DEF_OP(VInsElement) {
mov(reg.V2D(), Op->DestIdx, GetSrc(Op->Header.Args[1].ID()).V2D(), Op->SrcIdx);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
if (GetDst(Node).GetCode() != reg.GetCode()) {
@@ -1748,7 +1748,7 @@ DEF_OP(VInsScalarElement) {
mov(reg.V2D(), Op->DestIdx, GetSrc(Op->Header.Args[1].ID()).V2D(), 0);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
if (GetDst(Node).GetCode() != reg.GetCode()) {
@@ -1772,7 +1772,7 @@ DEF_OP(VExtractElement) {
case 8:
mov(GetDst(Node).D(), GetSrc(Op->Header.Args[0].ID()).V2D(), Op->Index);
break;
default: LOGMAN_MSG_A("Unhandled ExtractElementSize: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled VExtractElement element size: {}", OpSize);
}
}
@@ -1791,7 +1791,7 @@ DEF_OP(VDupElement) {
case 8:
dup(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), Op->Index);
break;
default: LOGMAN_MSG_A("Unhandled DupElementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unhandled VDupElement element size: {}", Op->Header.ElementSize);
}
}
@@ -1906,7 +1906,7 @@ DEF_OP(VUShrI) {
ushr(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), Op->BitShift);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1931,7 +1931,7 @@ DEF_OP(VSShrI) {
sshr(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), std::min((uint8_t)(Op->Header.ElementSize * 8 - 1), Op->BitShift));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1959,7 +1959,7 @@ DEF_OP(VShlI) {
shl(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), Op->BitShift);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1980,7 +1980,7 @@ DEF_OP(VUShrNI) {
shrn(GetDst(Node).V2S(), GetSrc(Op->Header.Args[0].ID()).V2D(), Op->BitShift);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -2000,7 +2000,7 @@ DEF_OP(VUShrNI2) {
shrn2(VTMP1.V4S(), GetSrc(Op->Header.Args[1].ID()).V2D(), Op->BitShift);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
mov(GetDst(Node), VTMP1);
@@ -2023,7 +2023,7 @@ DEF_OP(VSXTL) {
case 8:
sxtl(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2S());
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
@@ -2039,7 +2039,7 @@ DEF_OP(VSXTL2) {
case 8:
sxtl2(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V4S());
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
@@ -2055,7 +2055,7 @@ DEF_OP(VUXTL) {
case 8:
uxtl(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2S());
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
@@ -2071,7 +2071,7 @@ DEF_OP(VUXTL2) {
case 8:
uxtl2(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V4S());
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
@@ -2087,7 +2087,7 @@ DEF_OP(VSQXTN) {
case 4:
sqxtn(GetDst(Node).V2S(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
@@ -2109,7 +2109,7 @@ DEF_OP(VSQXTN2) {
sqxtn(VTMP2.V2S(), GetSrc(Op->Header.Args[1].ID()).V2D());
ins(VTMP1.V4S(), 1, VTMP2.V4S(), 0);
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
else {
@@ -2123,7 +2123,7 @@ DEF_OP(VSQXTN2) {
case 4:
sqxtn2(VTMP1.V4S(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
mov(GetDst(Node), VTMP1);
@@ -2141,7 +2141,7 @@ DEF_OP(VSQXTUN) {
case 4:
sqxtun(GetDst(Node).V2S(), GetSrc(Op->Header.Args[0].ID()).V2D());
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
@@ -2163,7 +2163,7 @@ DEF_OP(VSQXTUN2) {
sqxtun(VTMP2.V2S(), GetSrc(Op->Header.Args[1].ID()).V2D());
ins(VTMP1.V4S(), 1, VTMP2.V4S(), 0);
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
else {
@@ -2177,7 +2177,7 @@ DEF_OP(VSQXTUN2) {
case 4:
sqxtun2(VTMP1.V4S(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
mov(GetDst(Node), VTMP1);
@@ -2202,7 +2202,7 @@ DEF_OP(VMul) {
mul(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2D(), GetSrc(Op->Header.Args[1].ID()).V2D());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -2221,7 +2221,7 @@ DEF_OP(VUMull) {
umull(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2S(), GetSrc(Op->Header.Args[1].ID()).V2S());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize >> 1); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize >> 1); break;
}
}
@@ -2240,7 +2240,7 @@ DEF_OP(VSMull) {
smull(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2S(), GetSrc(Op->Header.Args[1].ID()).V2S());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize >> 1); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize >> 1); break;
}
}
@@ -2259,7 +2259,7 @@ DEF_OP(VUMull2) {
umull2(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V4S(), GetSrc(Op->Header.Args[1].ID()).V4S());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize >> 1); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize >> 1); break;
}
}
@@ -2278,7 +2278,7 @@ DEF_OP(VSMull2) {
smull2(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V4S(), GetSrc(Op->Header.Args[1].ID()).V4S());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize >> 1); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize >> 1); break;
}
}
@@ -2297,7 +2297,7 @@ DEF_OP(VUABDL) {
uabdl(GetDst(Node).V2D(), GetSrc(Op->Header.Args[0].ID()).V2S(), GetSrc(Op->Header.Args[1].ID()).V2S());
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize >> 1); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize >> 1); break;
}
}
@@ -2314,7 +2314,7 @@ DEF_OP(VTBL1) {
tbl(GetDst(Node).V16B(), GetSrc(Op->Header.Args[0].ID()).V16B(), GetSrc(Op->Header.Args[1].ID()).V16B());
break;
}
default: LOGMAN_MSG_A("Unknown OpSize: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown OpSize: {}", OpSize); break;
}
}
+42 -35
View File
@@ -5,7 +5,14 @@ $end_info$
*/
#include "Interface/Core/JIT/x86_64/JITClass.h"
#include "Interface/IR/Passes/RegisterAllocationPass.h"
#include <FEXCore/IR/IR.h>
#include <FEXCore/Utils/LogManager.h>
#include <array>
#include <stdint.h>
#include <utility>
#include <xbyak/xbyak.h>
namespace FEXCore::CPU {
#define DEF_OP(x) void X86JITCore::Op_##x(FEXCore::IR::IROp_Header *IROp, uint32_t Node)
@@ -20,7 +27,7 @@ DEF_OP(TruncElementPair) {
mov(Dst.second, Src.second);
break;
}
default: LOGMAN_MSG_A("Unhandled Truncation size: %d", Op->Size); break;
default: LOGMAN_MSG_A_FMT("Unhandled Truncation size: {}", Op->Size); break;
}
}
@@ -70,7 +77,7 @@ DEF_OP(Add) {
case 8:
add(rax, Const);
break;
default: LOGMAN_MSG_A("Unhandled Add size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled Add size: {}", OpSize);
break;
}
} else {
@@ -81,7 +88,7 @@ DEF_OP(Add) {
case 8:
add(rax, GetSrc<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unhandled Add size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled Add size: {}", OpSize);
break;
}
}
@@ -103,7 +110,7 @@ DEF_OP(Sub) {
case 8:
sub(rax, Const);
break;
default: LOGMAN_MSG_A("Unhandled Sub size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled Sub size: {}", OpSize);
break;
}
} else {
@@ -114,7 +121,7 @@ DEF_OP(Sub) {
case 8:
sub(rax, GetSrc<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unhandled Sub size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled Sub size: {}", OpSize);
break;
}
}
@@ -136,7 +143,7 @@ DEF_OP(Neg) {
Src = GetSrc<RA_64>(Op->Header.Args[0].ID());
Dst = GetDst<RA_64>(Node);
break;
default: LOGMAN_MSG_A("Unhandled Neg size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled Neg size: {}", OpSize);
break;
}
mov(Dst, Src);
@@ -160,7 +167,7 @@ DEF_OP(Mul) {
imul(rax, GetSrc<RA_64>(Op->Header.Args[1].ID()));
mov(Dst, rax);
break;
default: LOGMAN_MSG_A("Unknown Sext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown Mul size: {}", OpSize);
}
}
@@ -179,7 +186,7 @@ DEF_OP(UMul) {
mul(GetSrc<RA_64>(Op->Header.Args[1].ID()));
mov(GetDst<RA_64>(Node), rax);
break;
default: LOGMAN_MSG_A("Unknown Sext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown UMul size: {}", OpSize);
}
}
@@ -218,7 +225,7 @@ DEF_OP(Div) {
mov(GetDst<RA_64>(Node), rax);
break;
}
default: LOGMAN_MSG_A("Unknown UDIV Size: %d", Size); break;
default: LOGMAN_MSG_A_FMT("Unknown DIV Size: {}", Size); break;
}
}
@@ -261,7 +268,7 @@ DEF_OP(UDiv) {
mov(GetDst<RA_64>(Node), rax);
break;
}
default: LOGMAN_MSG_A("Unknown UDIV OpSize: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown UDIV OpSize: {}", OpSize); break;
}
}
@@ -298,7 +305,7 @@ DEF_OP(Rem) {
mov(GetDst<RA_64>(Node), rdx);
break;
}
default: LOGMAN_MSG_A("Unknown UDIV Size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Rem Size: {}", OpSize); break;
}
}
@@ -341,7 +348,7 @@ DEF_OP(URem) {
mov(GetDst<RA_64>(Node), rdx);
break;
}
default: LOGMAN_MSG_A("Unknown UDIV OpSize: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown URem OpSize: {}", OpSize); break;
}
}
@@ -360,7 +367,7 @@ DEF_OP(MulH) {
imul(GetSrc<RA_64>(Op->Header.Args[1].ID()));
mov(GetDst<RA_64>(Node), rdx);
break;
default: LOGMAN_MSG_A("Unknown Sext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown MulH size: {}", OpSize);
}
}
@@ -379,7 +386,7 @@ DEF_OP(UMulH) {
mul(GetSrc<RA_64>(Op->Header.Args[1].ID()));
mov(GetDst<RA_64>(Node), rdx);
break;
default: LOGMAN_MSG_A("Unknown Sext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown UMulH size: {}", OpSize);
}
}
@@ -441,7 +448,7 @@ DEF_OP(Lshl) {
mov(GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
shl(GetDst<RA_64>(Node), Const);
break;
default: LOGMAN_MSG_A("Unknown LSHL Size: %d\n", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown LSHL Size: {}\n", OpSize); break;
};
} else {
mov(rcx, GetSrc<RA_64>(Op->Header.Args[1].ID()));
@@ -456,7 +463,7 @@ DEF_OP(Lshl) {
mov(GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
shl(GetDst<RA_64>(Node), cl);
break;
default: LOGMAN_MSG_A("Unknown LSHL Size: %d\n", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown LSHL Size: {}\n", OpSize); break;
};
}
}
@@ -488,7 +495,7 @@ DEF_OP(Lshr) {
mov(GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
shr(GetDst<RA_64>(Node), Const);
break;
default: LOGMAN_MSG_A("Unknown Size: %d\n", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Size: {}\n", OpSize); break;
};
} else {
@@ -512,7 +519,7 @@ DEF_OP(Lshr) {
mov(GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
shr(GetDst<RA_64>(Node), cl);
break;
default: LOGMAN_MSG_A("Unknown Size: %d\n", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Size: {}\n", OpSize); break;
};
}
}
@@ -546,7 +553,7 @@ DEF_OP(Ashr) {
mov(GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
sar(GetDst<RA_64>(Node), Const);
break;
default: LOGMAN_MSG_A("Unknown ASHR Size: %d\n", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown ASHR Size: {}\n", OpSize); break;
};
} else {
@@ -571,7 +578,7 @@ DEF_OP(Ashr) {
mov(GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
sar(GetDst<RA_64>(Node), cl);
break;
default: LOGMAN_MSG_A("Unknown ASHR Size: %d\n", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown ASHR Size: {}\n", OpSize); break;
};
}
}
@@ -596,7 +603,7 @@ DEF_OP(Ror) {
ror(rax, Const);
break;
}
default: LOGMAN_MSG_A("Unknown ROR Size: %d\n", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown ROR Size: {}\n", OpSize); break;
}
} else {
mov (rcx, GetSrc<RA_64>(Op->Header.Args[1].ID()));
@@ -612,7 +619,7 @@ DEF_OP(Ror) {
ror(rax, cl);
break;
}
default: LOGMAN_MSG_A("Unknown ROR Size: %d\n", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown ROR Size: {}\n", OpSize); break;
}
}
mov(GetDst<RA_64>(Node), rax);
@@ -668,7 +675,7 @@ DEF_OP(LDiv) {
mov(GetDst<RA_64>(Node), rax);
break;
}
default: LOGMAN_MSG_A("Unknown LDIV OpSize: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown LDIV OpSize: {}", OpSize); break;
}
}
@@ -700,7 +707,7 @@ DEF_OP(LUDiv) {
mov(GetDst<RA_64>(Node), rax);
break;
}
default: LOGMAN_MSG_A("Unknown LUDIV OpSize: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown LUDIV OpSize: {}", OpSize); break;
}
}
@@ -732,7 +739,7 @@ DEF_OP(LRem) {
mov(GetDst<RA_64>(Node), rdx);
break;
}
default: LOGMAN_MSG_A("Unknown LREM OpSize: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown LREM OpSize: {}", OpSize); break;
}
}
@@ -764,7 +771,7 @@ DEF_OP(LURem) {
mov(GetDst<RA_64>(Node), rdx);
break;
}
default: LOGMAN_MSG_A("Unknown LUDIV OpSize: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown LUREM OpSize: {}", OpSize); break;
}
}
@@ -829,7 +836,7 @@ DEF_OP(FindMSB) {
case 8:
bsr(GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unknown OpSize: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unknown FindMSB OpSize: {}", OpSize);
}
}
@@ -853,7 +860,7 @@ DEF_OP(FindTrailingZeros) {
mov(rax, 0x40);
cmovz(GetDst<RA_64>(Node), rax);
break;
default: LOGMAN_MSG_A("Unknown size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown FindTrailingZeros size: {}", OpSize); break;
}
}
@@ -876,7 +883,7 @@ DEF_OP(CountLeadingZeroes) {
lzcnt(GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown CountLeadingZeros size: {}", OpSize); break;
}
}
else {
@@ -915,7 +922,7 @@ DEF_OP(CountLeadingZeroes) {
mov(GetDst<RA_64>(Node), rax);
break;
}
default: LOGMAN_MSG_A("Unknown size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown CountLeadingZeros size: {}", OpSize); break;
}
}
}
@@ -937,7 +944,7 @@ DEF_OP(Rev) {
mov (GetDst<RA_64>(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()));
bswap(GetDst<RA_64>(Node).cvt64());
break;
default: LOGMAN_MSG_A("Unknown REV size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown REV size: {}", OpSize); break;
}
}
@@ -970,7 +977,7 @@ DEF_OP(Bfi) {
DEF_OP(Bfe) {
auto Op = IROp->C<IR::IROp_Bfe>();
LOGMAN_THROW_A(IROp->Size <= 8, "OpSize is too large for BFE: %d", IROp->Size);
LOGMAN_THROW_A_FMT(IROp->Size <= 8, "OpSize is too large for BFE: {}", IROp->Size);
auto Dst = GetDst<RA_64>(Node);
@@ -1071,7 +1078,7 @@ DEF_OP(Select) {
if (is_const_true || is_const_false) {
if (is_const_false != true || is_const_true != true || const_true != 1 || const_false != 0) {
LOGMAN_MSG_A("Select: Unsupported compare inline parameters");
LOGMAN_MSG_A_FMT("Select: Unsupported compare inline parameters");
}
(this->*SetCC)(al);
movzx(Dst, al);
@@ -1102,7 +1109,7 @@ DEF_OP(VExtractToGPR) {
pextrq(GetDst<RA_64>(Node), GetSrc(Op->Header.Args[0].ID()), Op->Idx);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -5,7 +5,14 @@ $end_info$
*/
#include "Interface/Core/JIT/x86_64/JITClass.h"
#include "Interface/IR/Passes/RegisterAllocationPass.h"
#include <FEXCore/IR/IR.h>
#include <FEXCore/Utils/LogManager.h>
#include <array>
#include <stdint.h>
#include <utility>
#include <xbyak/xbyak.h>
namespace FEXCore::CPU {
#define DEF_OP(x) void X86JITCore::Op_##x(FEXCore::IR::IROp_Header *IROp, uint32_t Node)
@@ -55,7 +62,7 @@ DEF_OP(CASPair) {
mov(Dst.second, rdx);
break;
}
default: LOGMAN_MSG_A("Unsupported: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported: {}", OpSize);
}
}
@@ -104,7 +111,7 @@ DEF_OP(CAS) {
mov (GetDst<RA_64>(Node), rax);
break;
}
default: LOGMAN_MSG_A("Unsupported: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unsupported: {}", OpSize);
}
}
@@ -127,7 +134,7 @@ DEF_OP(AtomicAdd) {
case 8:
add(qword [MemReg], GetSrc<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unhandled AtomicAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicAdd size: {}", Op->Size);
}
}
@@ -149,7 +156,7 @@ DEF_OP(AtomicSub) {
case 8:
sub(qword [MemReg], GetSrc<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unhandled AtomicAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicAdd size: {}", Op->Size);
}
}
@@ -171,7 +178,7 @@ DEF_OP(AtomicAnd) {
case 8:
and_(qword [MemReg], GetSrc<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unhandled AtomicAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicAdd size: {}", Op->Size);
}
}
@@ -193,7 +200,7 @@ DEF_OP(AtomicOr) {
case 8:
or_(qword [MemReg], GetSrc<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unhandled AtomicAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicAdd size: {}", Op->Size);
}
}
@@ -215,7 +222,7 @@ DEF_OP(AtomicXor) {
case 8:
xor_(qword [MemReg], GetSrc<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unhandled AtomicAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicAdd size: {}", Op->Size);
}
}
@@ -246,7 +253,7 @@ DEF_OP(AtomicSwap) {
lock();
xchg(qword [MemReg], GetDst<RA_64>(Node));
break;
default: LOGMAN_MSG_A("Unhandled AtomicAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicSwap size: {}", Op->Size);
}
}
@@ -279,7 +286,7 @@ DEF_OP(AtomicFetchAdd) {
xadd(qword [MemReg], rcx);
mov(GetDst<RA_64>(Node), rcx);
break;
default: LOGMAN_MSG_A("Unhandled AtomicFetchAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicFetchAdd size: {}", Op->Size);
}
}
@@ -316,7 +323,7 @@ DEF_OP(AtomicFetchSub) {
xadd(qword [MemReg], rcx);
mov(GetDst<RA_64>(Node), rcx);
break;
default: LOGMAN_MSG_A("Unhandled AtomicFetchAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicFetchSub size: {}", Op->Size);
}
}
@@ -394,7 +401,7 @@ DEF_OP(AtomicFetchAnd) {
mov(GetDst<RA_64>(Node), TMP3.cvt64());
break;
}
default: LOGMAN_MSG_A("Unhandled AtomicFetchAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicFetchAnd size: {}", Op->Size);
}
}
@@ -471,7 +478,7 @@ DEF_OP(AtomicFetchOr) {
mov(GetDst<RA_64>(Node), TMP3.cvt64());
break;
}
default: LOGMAN_MSG_A("Unhandled AtomicFetchAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicFetchOr size: {}", Op->Size);
}
}
@@ -548,7 +555,83 @@ DEF_OP(AtomicFetchXor) {
mov(GetDst<RA_64>(Node), TMP3.cvt64());
break;
}
default: LOGMAN_MSG_A("Unhandled AtomicFetchAdd size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled AtomicFetchXor size: {}", Op->Size);
}
}
DEF_OP(AtomicFetchNeg) {
auto Op = IROp->C<IR::IROp_AtomicFetchNeg>();
Xbyak::Reg MemReg = GetSrc<RA_64>(Op->Header.Args[0].ID());
switch (Op->Size) {
case 1: {
mov(TMP1.cvt8(), byte [MemReg]);
Label Loop;
L(Loop);
mov(TMP2.cvt8(), TMP1.cvt8());
mov(TMP3.cvt8(), TMP1.cvt8());
neg(TMP2.cvt8());
// Updates RAX with the value from memory
lock(); cmpxchg(byte [MemReg], TMP2.cvt8());
jne(Loop);
// Result is the previous value from memory, which is currently in TMP3
movzx(GetDst<RA_64>(Node), TMP3.cvt8());
break;
}
case 2: {
mov(TMP1.cvt16(), word [MemReg]);
Label Loop;
L(Loop);
mov(TMP2.cvt16(), TMP1.cvt16());
mov(TMP3.cvt16(), TMP1.cvt16());
neg(TMP2.cvt16());
// Updates RAX with the value from memory
lock(); cmpxchg(word [MemReg], TMP2.cvt16());
jne(Loop);
// Result is the previous value from memory, which is currently in TMP3
movzx(GetDst<RA_64>(Node), TMP3.cvt16());
break;
}
case 4: {
mov(TMP1.cvt32(), dword [MemReg]);
Label Loop;
L(Loop);
mov(TMP2.cvt32(), TMP1.cvt32());
mov(TMP3.cvt32(), TMP1.cvt32());
neg(TMP2.cvt32());
// Updates RAX with the value from memory
lock(); cmpxchg(dword [MemReg], TMP2.cvt32());
jne(Loop);
// Result is the previous value from memory, which is currently in TMP3
mov(GetDst<RA_32>(Node), TMP3.cvt32());
break;
}
case 8: {
mov(TMP1.cvt64(), qword [MemReg]);
Label Loop;
L(Loop);
mov(TMP2.cvt64(), TMP1.cvt64());
mov(TMP3.cvt64(), TMP1.cvt64());
neg(TMP2.cvt64());
// Updates RAX with the value from memory
lock(); cmpxchg(qword [MemReg], TMP2.cvt64());
jne(Loop);
// Result is the previous value from memory, which is currently in TMP3
mov(GetDst<RA_64>(Node), TMP3.cvt64());
break;
}
default: LOGMAN_MSG_A_FMT("Unhandled AtomicFetchNeg size: {}", Op->Size);
}
}
@@ -568,6 +651,7 @@ void X86JITCore::RegisterAtomicHandlers() {
REGISTER_OP(ATOMICFETCHAND, AtomicFetchAnd);
REGISTER_OP(ATOMICFETCHOR, AtomicFetchOr);
REGISTER_OP(ATOMICFETCHXOR, AtomicFetchXor);
REGISTER_OP(ATOMICFETCHNEG, AtomicFetchNeg);
#undef REGISTER_OP
}
}
@@ -4,25 +4,41 @@ tags: backend|x86-64
$end_info$
*/
#include "Interface/Context/Context.h"
#include "Interface/Core/CPUID.h"
#include "Interface/Core/Dispatcher/Dispatcher.h"
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/JIT/x86_64/JITClass.h"
#include "Interface/IR/Passes/RegisterAllocationPass.h"
#include "Interface/HLE/Thunks/Thunks.h"
#include <FEXCore/Core/CPUID.h>
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/Core/X86Enums.h>
#include <FEXCore/Debug/InternalThreadState.h>
#include <FEXCore/HLE/SyscallHandler.h>
#include <Interface/HLE/Thunks/Thunks.h>
#include <FEXCore/IR/IR.h>
#include <FEXCore/Utils/LogManager.h>
#include <array>
#include <memory>
#include <stddef.h>
#include <stdint.h>
#include <unordered_map>
#include <utility>
#include <xbyak/xbyak.h>
namespace FEXCore::CPU {
#define DEF_OP(x) void X86JITCore::Op_##x(FEXCore::IR::IROp_Header *IROp, uint32_t Node)
DEF_OP(GuestCallDirect) {
LogMan::Msg::D("Unimplemented");
LogMan::Msg::DFmt("Unimplemented");
}
DEF_OP(GuestCallIndirect) {
LogMan::Msg::D("Unimplemented");
LogMan::Msg::DFmt("Unimplemented");
}
DEF_OP(GuestReturn) {
LogMan::Msg::D("Unimplemented");
LogMan::Msg::DFmt("Unimplemented");
}
DEF_OP(SignalReturn) {
@@ -5,7 +5,13 @@ $end_info$
*/
#include "Interface/Core/JIT/x86_64/JITClass.h"
#include "Interface/IR/Passes/RegisterAllocationPass.h"
#include <FEXCore/IR/IR.h>
#include <FEXCore/Utils/LogManager.h>
#include <array>
#include <stdint.h>
#include <xbyak/xbyak.h>
namespace FEXCore::CPU {
@@ -31,7 +37,7 @@ DEF_OP(VInsGPR) {
pinsrq(GetDst(Node), GetSrc<RA_64>(Op->Header.Args[1].ID()), Op->Index);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -52,7 +58,7 @@ DEF_OP(VCastFromGPR) {
case 8:
vmovq(GetDst(Node), GetSrc<RA_64>(Op->Header.Args[0].ID()).cvt64());
break;
default: LOGMAN_MSG_A("Unknown castGPR element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown VCastFromGPR element size: {}", Op->Header.ElementSize);
}
}
@@ -91,7 +97,7 @@ DEF_OP(Float_FToF) {
cvtsd2ss(GetDst(Node), GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown FCVT sizes: 0x%x", Conv);
default: LOGMAN_MSG_A_FMT("Unknown Float_FToF sizes: 0x{:x}", Conv);
}
}
@@ -113,7 +119,7 @@ DEF_OP(Vector_SToF) {
cvtsi2sd(xmm15, rax);
movlhps(GetDst(Node), xmm15);
break;
default: LOGMAN_MSG_A("Unknown castGPR element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Vector_SToF element size: {}", Op->Header.ElementSize);
}
}
@@ -126,7 +132,7 @@ DEF_OP(Vector_FToZS) {
case 8:
cvttpd2dq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unknown castGPR element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Vector_FToZS element size: {}", Op->Header.ElementSize);
}
}
@@ -139,7 +145,7 @@ DEF_OP(Vector_FToS) {
case 8:
cvtpd2dq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unknown castGPR element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Vector_FToS element size: {}", Op->Header.ElementSize);
}
}
@@ -156,7 +162,7 @@ DEF_OP(Vector_FToF) {
cvtpd2ps(GetDst(Node), GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Conversion Type : 0%04x", Conv); break;
default: LOGMAN_MSG_A_FMT("Unknown Vector_FToF conversion type : 0x{:04x}", Conv); break;
}
}
@@ -5,7 +5,12 @@ $end_info$
*/
#include "Interface/Core/JIT/x86_64/JITClass.h"
#include "Interface/IR/Passes/RegisterAllocationPass.h"
#include <FEXCore/IR/IR.h>
#include <array>
#include <stdint.h>
#include <xbyak/xbyak.h>
namespace FEXCore::CPU {
#define DEF_OP(x) void X86JITCore::Op_##x(FEXCore::IR::IROp_Header *IROp, uint32_t Node)
@@ -5,7 +5,12 @@ $end_info$
*/
#include "Interface/Core/JIT/x86_64/JITClass.h"
#include "Interface/IR/Passes/RegisterAllocationPass.h"
#include <FEXCore/IR/IR.h>
#include <array>
#include <stdint.h>
#include <xbyak/xbyak.h>
namespace FEXCore::CPU {
+40 -24
View File
@@ -6,21 +6,38 @@ $end_info$
*/
#include "Interface/Context/Context.h"
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/Dispatcher/Dispatcher.h"
#include "Interface/Core/Dispatcher/X86Dispatcher.h"
#include "Interface/Core/Interpreter/InterpreterOps.h"
#include "Interface/Core/JIT/x86_64/JITClass.h"
#include "Interface/Core/InternalThreadState.h"
#include "Interface/IR/PassManager.h"
#include "Interface/IR/Passes/RegisterAllocationPass.h"
#include <FEXCore/Core/X86Enums.h>
#include <FEXCore/Core/UContext.h>
#include <FEXCore/Core/CPUBackend.h>
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/Core/SignalDelegator.h>
#include <FEXCore/Debug/InternalThreadState.h>
#include <FEXCore/IR/IR.h>
#include <FEXCore/IR/IntrusiveIRList.h>
#include <FEXCore/IR/RegisterAllocationData.h>
#include <FEXCore/Utils/Allocator.h>
#include <FEXCore/Utils/LogManager.h>
#include <cmath>
#include <algorithm>
#include <array>
#include <bits/types/stack_t.h>
#include <memory>
#include <stddef.h>
#include <stdint.h>
#include <signal.h>
#include "Interface/Core/Interpreter/InterpreterOps.h"
#include <sys/mman.h>
#include <tuple>
#include <unordered_map>
#include <utility>
#include <vector>
#include <xbyak/xbyak.h>
// #define DEBUG_RA 1
// #define DEBUG_CYCLES
@@ -36,7 +53,7 @@ CodeBuffer AllocateNewCodeBuffer(size_t Size) {
PROT_READ | PROT_WRITE | PROT_EXEC,
MAP_PRIVATE | MAP_ANONYMOUS,
-1, 0));
LOGMAN_THROW_A(Buffer.Ptr != reinterpret_cast<uint8_t*>(~0ULL), "Couldn't allocate code buffer");
LOGMAN_THROW_A_FMT(Buffer.Ptr != reinterpret_cast<uint8_t*>(~0ULL), "Couldn't allocate code buffer");
return Buffer;
}
@@ -54,9 +71,9 @@ void X86JITCore::CopyNecessaryDataForCompileThread(CPUBackend *Original) {
}
void X86JITCore::PushRegs() {
for (auto &Xmm : RAXMM_x) {
sub(rsp, 16);
movaps(ptr[rsp], Xmm);
sub(rsp, 16 * RAXMM_x.size());
for (size_t i = 0; i < RAXMM_x.size(); ++i) {
movaps(ptr[rsp + i * 16], RAXMM_x[i]);
}
for (auto &Reg : RA64)
@@ -75,18 +92,18 @@ void X86JITCore::PopRegs() {
for (uint32_t i = RA64.size(); i > 0; --i)
pop(RA64[i - 1]);
for (uint32_t i = RAXMM_x.size(); i > 0; --i) {
movaps(RAXMM_x[i - 1], ptr[rsp]);
add(rsp, 16);
for (size_t i = 0; i < RAXMM_x.size(); ++i) {
movaps(RAXMM_x[i], ptr[rsp + i * 16]);
}
add(rsp, 16 * RAXMM_x.size());
}
void X86JITCore::Op_Unhandled(FEXCore::IR::IROp_Header *IROp, uint32_t Node) {
FallbackInfo Info;
if (!InterpreterOps::GetFallbackHandler(IROp, &Info)) {
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
auto Name = FEXCore::IR::GetName(IROp->Op);
LOGMAN_MSG_A("Unhandled IR Op: %s", std::string(Name).c_str());
LOGMAN_MSG_A_FMT("Unhandled IR Op: {}", FEXCore::IR::GetName(IROp->Op));
#endif
} else {
switch(Info.ABI) {
@@ -285,8 +302,7 @@ void X86JITCore::Op_Unhandled(FEXCore::IR::IROp_Header *IROp, uint32_t Node) {
case FABI_UNKNOWN:
default:
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
auto Name = FEXCore::IR::GetName(IROp->Op);
LOGMAN_MSG_A("Unhandled IR Fallback abi: %s %d", std::string(Name).c_str(), Info.ABI);
LOGMAN_MSG_A_FMT("Unhandled IR Fallback ABI: {} {}", FEXCore::IR::GetName(IROp->Op), Info.ABI);
#endif
break;
}
@@ -360,7 +376,7 @@ X86JITCore::X86JITCore(FEXCore::Context::Context *ctx, FEXCore::Core::InternalTh
return Core->Dispatcher->HandleGuestSignal(Signal, info, ucontext, GuestAction, GuestStack);
};
for (uint32_t Signal = 0; Signal < SignalDelegator::MAX_SIGNALS; ++Signal) {
for (uint32_t Signal = 0; Signal <= SignalDelegator::MAX_SIGNALS; ++Signal) {
CTX->SignalDelegation->RegisterHostSignalHandlerForGuest(Signal, GuestSignalHandler);
}
}
@@ -419,7 +435,7 @@ void X86JITCore::ClearCache() {
IR::PhysicalRegister X86JITCore::GetPhys(uint32_t Node) const {
auto PhyReg = RAData->GetNodeRegister(Node);
LOGMAN_THROW_A(PhyReg.Raw != 255, "Couldn't Allocate register for node: ssa%d. Class: %d", Node, PhyReg.Class);
LOGMAN_THROW_A_FMT(PhyReg.Raw != 255, "Couldn't Allocate register for node: ssa{}. Class: {}", Node, PhyReg.Class);
return PhyReg;
}
@@ -568,7 +584,7 @@ std::tuple<X86JITCore::SetCC, X86JITCore::CMovCC, X86JITCore::JCC> X86JITCore::G
case FEXCore::IR::COND_VS:
case FEXCore::IR::COND_VC:
default:
LOGMAN_MSG_A("Unsupported compare type");
LOGMAN_MSG_A_FMT("Unsupported compare type");
break;
}
@@ -611,7 +627,7 @@ void *X86JITCore::CompileCode(uint64_t Entry, [[maybe_unused]] FEXCore::IR::IRLi
L(RunBlock);
}
LOGMAN_THROW_A(RAData != nullptr, "Needs RA");
LOGMAN_THROW_A_FMT(RAData != nullptr, "Needs RA");
SpillSlots = RAData->SpillSlots();
@@ -669,7 +685,7 @@ void *X86JITCore::CompileCode(uint64_t Entry, [[maybe_unused]] FEXCore::IR::IRLi
{
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
auto BlockIROp = BlockHeader->CW<IROp_CodeBlock>();
LOGMAN_THROW_A(BlockIROp->Header.Op == IR::OP_CODEBLOCK, "IR type failed to be a code block");
LOGMAN_THROW_A_FMT(BlockIROp->Header.Op == IR::OP_CODEBLOCK, "IR type failed to be a code block");
#endif
uint32_t Node = IR->GetID(BlockNode);
@@ -721,7 +737,7 @@ void *X86JITCore::CompileCode(uint64_t Entry, [[maybe_unused]] FEXCore::IR::IRLi
Inst << "Reg" << GetPhys(ArgNode) << (i + 1 == NumArgs ? "" : ", ");
}
LogMan::Msg::D("%s", Inst.str().c_str());
LogMan::Msg::DFmt("{}", Inst.str());
}
#endif
uint32_t ID = IR->GetID(CodeNode);
@@ -6,7 +6,6 @@ $end_info$
#pragma once
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/BlockSamplingData.h"
#include "Interface/Core/Dispatcher/Dispatcher.h"
@@ -260,6 +259,7 @@ private:
DEF_OP(AtomicFetchAnd);
DEF_OP(AtomicFetchOr);
DEF_OP(AtomicFetchXor);
DEF_OP(AtomicFetchNeg);
///< Branch ops
DEF_OP(GuestCallDirect);
@@ -5,9 +5,15 @@ $end_info$
*/
#include "Interface/Core/JIT/x86_64/JITClass.h"
#include "Interface/IR/Passes/RegisterAllocationPass.h"
#include <cmath>
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/IR/IR.h>
#include <FEXCore/Utils/LogManager.h>
#include <array>
#include <stddef.h>
#include <stdint.h>
#include <xbyak/xbyak.h>
namespace FEXCore::CPU {
@@ -36,10 +42,10 @@ DEF_OP(LoadContext) {
}
break;
case 16: {
LOGMAN_MSG_A("Invalid GPR load of size 16");
LOGMAN_MSG_A_FMT("Invalid GPR load of size 16");
}
break;
default: LOGMAN_MSG_A("Unhandled LoadContext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled LoadContext size: {}", OpSize);
}
}
else {
@@ -69,7 +75,7 @@ DEF_OP(LoadContext) {
movups(GetDst(Node), xword [STATE + Op->Offset]);
}
break;
default: LOGMAN_MSG_A("Unhandled LoadContext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled LoadContext size: {}", OpSize);
}
}
}
@@ -98,9 +104,9 @@ DEF_OP(StoreContext) {
}
break;
case 16:
LogMan::Msg::D("Invalid store size of 16");
LogMan::Msg::DFmt("Invalid store size of 16");
break;
default: LOGMAN_MSG_A("Unhandled StoreContext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled StoreContext size: {}", OpSize);
}
}
else {
@@ -129,7 +135,7 @@ DEF_OP(StoreContext) {
movups(xword [STATE + Op->Offset], GetSrc(Op->Header.Args[0].ID()));
}
break;
default: LOGMAN_MSG_A("Unhandled StoreContext size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled StoreContext size: {}", OpSize);
}
}
}
@@ -160,17 +166,18 @@ DEF_OP(LoadContextIndexed) {
mov(GetDst<RA_64>(Node), qword [rax + index * Op->Stride]);
break;
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed size: %d", Op->Size);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed size: {}", Op->Size);
break;
}
break;
}
case 16:
LOGMAN_MSG_A("Invalid Class load of size 16");
LOGMAN_MSG_A_FMT("Invalid Class load of size 16");
break;
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed stride: %d", Op->Stride);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed stride: {}", Op->Stride);
break;
}
}
else {
switch (Op->Stride) {
@@ -195,7 +202,8 @@ DEF_OP(LoadContextIndexed) {
vmovq(GetDst(Node), qword [rax + index * Op->Stride]);
break;
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed size: %d", Op->Size);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed size: {}", Op->Size);
break;
}
break;
}
@@ -223,12 +231,14 @@ DEF_OP(LoadContextIndexed) {
movups(GetDst(Node), xword [STATE + rax]);
break;
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed size: %d", Op->Size);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed size: {}", Op->Size);
break;
}
break;
}
default:
LOGMAN_MSG_A("Unhandled LoadContextIndexed stride: %d", Op->Stride);
LOGMAN_MSG_A_FMT("Unhandled LoadContextIndexed stride: {}", Op->Stride);
break;
}
}
}
@@ -248,13 +258,14 @@ DEF_OP(StoreContextIndexed) {
case 4:
case 8: {
if (!(size == 1 || size == 2 || size == 4 || size == 8)) {
LOGMAN_MSG_A("Unhandled StoreContextIndexed size: %d", Op->Size);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed size: {}", Op->Size);
}
mov(AddressFrame(Op->Size * 8) [rax + index * Op->Stride], value);
break;
}
default:
LOGMAN_MSG_A("Unhandled StoreContextIndexed stride: %d", Op->Stride);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed stride: {}", Op->Stride);
break;
}
}
else {
@@ -279,7 +290,8 @@ DEF_OP(StoreContextIndexed) {
vmovq(AddressFrame(Op->Size * 8) [rax + index * Op->Stride], value);
break;
default:
LOGMAN_MSG_A("Unhandled StoreContextIndexed size: %d", size);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed size: {}", size);
break;
}
break;
}
@@ -307,12 +319,14 @@ DEF_OP(StoreContextIndexed) {
movups(xword [STATE + rax], value);
break;
default:
LOGMAN_MSG_A("Unhandled StoreContextIndexed size: %d", size);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed size: {}", size);
break;
}
break;
}
default:
LOGMAN_MSG_A("Unhandled StoreContextIndexed stride: %d", Op->Stride);
LOGMAN_MSG_A_FMT("Unhandled StoreContextIndexed stride: {}", Op->Stride);
break;
}
}
}
@@ -340,7 +354,7 @@ DEF_OP(SpillRegister) {
mov(qword [rsp + SlotOffset], GetSrc<RA_64>(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unhandled SpillRegister size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled SpillRegister size: {}", OpSize);
}
} else if (Op->Class == FEXCore::IR::FPRClass) {
switch (OpSize) {
@@ -356,10 +370,10 @@ DEF_OP(SpillRegister) {
movaps(xword [rsp + SlotOffset], GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unhandled SpillRegister size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled SpillRegister size: {}", OpSize);
}
} else {
LOGMAN_MSG_A("Unhandled SpillRegister class: %d", Op->Class.Val);
LOGMAN_MSG_A_FMT("Unhandled SpillRegister class: {}", Op->Class.Val);
}
@@ -388,7 +402,7 @@ DEF_OP(FillRegister) {
mov(GetDst<RA_64>(Node), qword [rsp + SlotOffset]);
break;
}
default: LOGMAN_MSG_A("Unhandled FillRegister size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled FillRegister size: {}", OpSize);
}
} else if (Op->Class == FEXCore::IR::FPRClass) {
switch (OpSize) {
@@ -404,10 +418,10 @@ DEF_OP(FillRegister) {
movaps(GetDst(Node), xword [rsp + SlotOffset]);
break;
}
default: LOGMAN_MSG_A("Unhandled FillRegister size: %d", OpSize);
default: LOGMAN_MSG_A_FMT("Unhandled FillRegister size: {}", OpSize);
}
} else {
LOGMAN_MSG_A("Unhandled FillRegister class: %d", Op->Class.Val);
LOGMAN_MSG_A_FMT("Unhandled FillRegister class: {}", Op->Class.Val);
}
}
@@ -430,11 +444,11 @@ Xbyak::RegExp X86JITCore::GenerateModRM(Xbyak::Reg Base, IR::OrderedNodeWrapper
return Base;
} else {
if (OffsetScale != 1 && OffsetScale != 2 && OffsetScale != 4 && OffsetScale != 8) {
LOGMAN_MSG_A("Unhandled GenerateModRM OffsetScale: %d", OffsetScale);
LOGMAN_MSG_A_FMT("Unhandled GenerateModRM OffsetScale: {}", OffsetScale);
}
if (OffsetType != IR::MEM_OFFSET_SXTX) {
LOGMAN_MSG_A("Unhandled GenerateModRM OffsetType: %d", OffsetType.Val);
LOGMAN_MSG_A_FMT("Unhandled GenerateModRM OffsetType: {}", OffsetType.Val);
}
uint64_t Const;
@@ -475,7 +489,7 @@ DEF_OP(LoadMem) {
mov(Dst, qword [MemPtr]);
}
break;
default: LOGMAN_MSG_A("Unhandled LoadMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled LoadMem size: {}", Op->Size);
}
}
else
@@ -511,7 +525,7 @@ DEF_OP(LoadMem) {
}
}
break;
default: LOGMAN_MSG_A("Unhandled LoadMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled LoadMem size: {}", Op->Size);
}
}
}
@@ -537,7 +551,7 @@ DEF_OP(StoreMem) {
case 8:
mov(qword [MemPtr], GetSrc<RA_64>(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unhandled StoreMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled StoreMem size: {}", Op->Size);
}
}
else {
@@ -560,17 +574,17 @@ DEF_OP(StoreMem) {
else
movups(xword [MemPtr], GetSrc(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unhandled StoreMem size: %d", Op->Size);
default: LOGMAN_MSG_A_FMT("Unhandled StoreMem size: {}", Op->Size);
}
}
}
DEF_OP(VLoadMemElement) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(VStoreMemElement) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(CacheLineClear) {
@@ -4,16 +4,26 @@ tags: backend|x86-64
$end_info$
*/
#include "Interface/Context/Context.h"
#include "Interface/Core/Dispatcher/Dispatcher.h"
#include "Interface/Core/JIT/x86_64/JITClass.h"
#include "Interface/IR/Passes/RegisterAllocationPass.h"
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/Utils/LogManager.h>
#include <FEXCore/IR/IR.h>
#include <array>
#include <stddef.h>
#include <stdint.h>
#include <xbyak/xbyak.h>
namespace FEXCore::CPU {
static void PrintValue(uint64_t Value) {
LogMan::Msg::D("Value: 0x%lx", Value);
LogMan::Msg::DFmt("Value: 0x{:x}", Value);
}
static void PrintVectorValue(uint64_t Value, uint64_t ValueUpper) {
LogMan::Msg::D("Value: 0x%016lx'%016lx", ValueUpper, Value);
LogMan::Msg::DFmt("Value: 0x{:016x}'{:016x}", ValueUpper, Value);
}
#define DEF_OP(x) void X86JITCore::Op_##x(FEXCore::IR::IROp_Header *IROp, uint32_t Node)
@@ -30,7 +40,7 @@ DEF_OP(Fence) {
case IR::Fence_Store.Val:
sfence();
break;
default: LOGMAN_MSG_A("Unknown Fence: %d", Op->Fence); break;
default: LOGMAN_MSG_A_FMT("Unknown Fence: {}", Op->Fence); break;
}
}
@@ -83,7 +93,7 @@ DEF_OP(Break) {
}
break;
}
default: LOGMAN_MSG_A("Unknown Break reason: %d", Op->Reason);
default: LOGMAN_MSG_A_FMT("Unknown Break reason: {}", Op->Reason);
}
}
@@ -5,7 +5,13 @@ $end_info$
*/
#include "Interface/Core/JIT/x86_64/JITClass.h"
#include "Interface/IR/Passes/RegisterAllocationPass.h"
#include <FEXCore/Utils/LogManager.h>
#include <FEXCore/IR/IR.h>
#include <array>
#include <stdint.h>
#include <utility>
namespace FEXCore::CPU {
@@ -25,7 +31,7 @@ DEF_OP(ExtractElementPair) {
mov (GetDst<RA_64>(Node), Regs[Op->Element]);
break;
}
default: LOGMAN_MSG_A("Unknown Size"); break;
default: LOGMAN_MSG_A_FMT("Unknown Size"); break;
}
}
@@ -51,7 +57,7 @@ DEF_OP(CreateElementPair) {
RegTmp = rax;
break;
}
default: LOGMAN_MSG_A("Unknown Size"); break;
default: LOGMAN_MSG_A_FMT("Unknown Size"); break;
}
if (Dst.first != RegSecond) {
+108 -102
View File
@@ -5,8 +5,14 @@ $end_info$
*/
#include "Interface/Core/JIT/x86_64/JITClass.h"
#include "Interface/IR/Passes/RegisterAllocationPass.h"
#include <FEXCore/IR/IR.h>
#include <FEXCore/Utils/LogManager.h>
#include <array>
#include <stddef.h>
#include <stdint.h>
#include <xbyak/xbyak.h>
namespace FEXCore::CPU {
@@ -62,24 +68,24 @@ DEF_OP(VectorImm) {
}
DEF_OP(CreateVector2) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(CreateVector4) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(SplatVector) {
auto Op = IROp->C<IR::IROp_SplatVector2>();
uint8_t OpSize = IROp->Size;
LOGMAN_THROW_A(OpSize <= 16, "Can't handle a vector of size: %d", OpSize);
LOGMAN_THROW_A_FMT(OpSize <= 16, "Can't handle a vector of size: {}", OpSize);
uint8_t Elements = 0;
switch (Op->Header.Op) {
case IR::OP_SPLATVECTOR4: Elements = 4; break;
case IR::OP_SPLATVECTOR2: Elements = 2; break;
default: LOGMAN_MSG_A("Uknown Splat size"); break;
default: LOGMAN_MSG_A_FMT("Unknown Splat size"); break;
}
uint8_t ElementSize = OpSize / Elements;
@@ -92,7 +98,7 @@ DEF_OP(SplatVector) {
case 8:
movddup(GetDst(Node), GetSrc(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.Size); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.Size); break;
}
}
@@ -130,7 +136,7 @@ DEF_OP(VMov) {
movaps(GetDst(Node), GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", OpSize); break;
}
}
@@ -176,7 +182,7 @@ DEF_OP(VAdd) {
vpaddq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -199,7 +205,7 @@ DEF_OP(VSub) {
vpsubq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -214,7 +220,7 @@ DEF_OP(VUQAdd) {
vpaddusw(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -229,7 +235,7 @@ DEF_OP(VUQSub) {
vpsubusw(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -244,7 +250,7 @@ DEF_OP(VSQAdd) {
vpaddsw(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -259,7 +265,7 @@ DEF_OP(VSQSub) {
vpsubsw(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -296,7 +302,7 @@ DEF_OP(VAddP) {
case 4:
vphaddd(GetDst(Node), xmm15, xmm14);
break;
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -325,7 +331,7 @@ DEF_OP(VAddP) {
case 4:
vphaddd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -357,7 +363,7 @@ DEF_OP(VAddV) {
pinsrd(xmm15, eax, 0);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
movaps(Dest, xmm15);
@@ -376,7 +382,7 @@ DEF_OP(VUMinV) {
pinsrw(Dest, eax, 1);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -391,7 +397,7 @@ DEF_OP(VURAvg) {
vpavgw(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -414,7 +420,7 @@ DEF_OP(VAbs) {
vpabsq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -438,7 +444,7 @@ DEF_OP(VPopcount) {
}
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
movaps(Dest, xmm15);
@@ -459,7 +465,7 @@ DEF_OP(VFAdd) {
vaddsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -473,7 +479,7 @@ DEF_OP(VFAdd) {
vaddpd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -487,7 +493,7 @@ DEF_OP(VFAddP) {
case 8:
vhaddpd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -506,7 +512,7 @@ DEF_OP(VFSub) {
vsubsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -520,7 +526,7 @@ DEF_OP(VFSub) {
vsubpd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -540,7 +546,7 @@ DEF_OP(VFMul) {
vmulsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -554,7 +560,7 @@ DEF_OP(VFMul) {
vmulpd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -574,7 +580,7 @@ DEF_OP(VFDiv) {
vdivsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -588,7 +594,7 @@ DEF_OP(VFDiv) {
vdivpd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -608,7 +614,7 @@ DEF_OP(VFMin) {
vminsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -622,7 +628,7 @@ DEF_OP(VFMin) {
vminpd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -642,7 +648,7 @@ DEF_OP(VFMax) {
vmaxsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -656,7 +662,7 @@ DEF_OP(VFMax) {
vmaxpd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -674,7 +680,7 @@ DEF_OP(VFRecp) {
vdivss(GetDst(Node), xmm15, GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -687,7 +693,7 @@ DEF_OP(VFRecp) {
vdivps(GetDst(Node), xmm15, GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -707,7 +713,7 @@ DEF_OP(VFSqrt) {
vsqrtsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -721,7 +727,7 @@ DEF_OP(VFSqrt) {
vsqrtpd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -747,7 +753,7 @@ DEF_OP(VFRSqrt) {
divsd(GetDst(Node), xmm15);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -761,7 +767,7 @@ DEF_OP(VFRSqrt) {
divps(GetDst(Node), xmm15);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -786,7 +792,7 @@ DEF_OP(VNeg) {
vpsubq(GetDst(Node), xmm15, GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -807,7 +813,7 @@ DEF_OP(VFNeg) {
vxorpd(GetDst(Node), xmm15, GetSrc(Op->Header.Args[0].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -830,7 +836,7 @@ DEF_OP(VUMin) {
pinsrq(GetDst(Node), TMP2, 0);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -847,7 +853,7 @@ DEF_OP(VUMin) {
vpminud(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -867,7 +873,7 @@ DEF_OP(VSMin) {
vpminsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -886,7 +892,7 @@ DEF_OP(VUMax) {
vpmaxud(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -905,7 +911,7 @@ DEF_OP(VSMax) {
vpmaxsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -930,7 +936,7 @@ DEF_OP(VZip) {
punpcklqdq(xmm15, GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
movapd(GetDst(Node), xmm15);
}
@@ -957,7 +963,7 @@ DEF_OP(VZip2) {
vpunpckhdq(GetDst(Node), xmm15, xmm14);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
else {
@@ -978,7 +984,7 @@ DEF_OP(VZip2) {
punpckhqdq(xmm15, GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
movapd(GetDst(Node), xmm15);
}
@@ -989,7 +995,7 @@ DEF_OP(VUnZip) {
uint8_t OpSize = IROp->Size;
if (OpSize == 8) {
LOGMAN_MSG_A("Unsupported registersize on VunZip");
LOGMAN_MSG_A_FMT("Unsupported register size on VUnZip");
}
else {
switch (Op->Header.ElementSize) {
@@ -1031,7 +1037,7 @@ DEF_OP(VUnZip) {
0b0'0);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1042,7 +1048,7 @@ DEF_OP(VUnZip2) {
if (OpSize == 8) {
LOGMAN_MSG_A("Unsupported registersize on VunZip");
LOGMAN_MSG_A_FMT("Unsupported register size on VUnZip2");
}
else {
switch (Op->Header.ElementSize) {
@@ -1084,7 +1090,7 @@ DEF_OP(VUnZip2) {
0b1'1);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
}
@@ -1113,7 +1119,7 @@ DEF_OP(VCMPEQ) {
case 8:
vpcmpeqq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
@@ -1134,7 +1140,7 @@ DEF_OP(VCMPEQZ) {
case 8:
vpcmpeqq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), xmm15);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
@@ -1154,7 +1160,7 @@ DEF_OP(VCMPGT) {
case 8:
vpcmpgtq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
@@ -1175,7 +1181,7 @@ DEF_OP(VCMPGTZ) {
case 8:
vpcmpgtq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), xmm15);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
@@ -1196,7 +1202,7 @@ DEF_OP(VCMPLTZ) {
case 8:
vpcmpgtq(GetDst(Node), xmm15, GetSrc(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
@@ -1212,7 +1218,7 @@ DEF_OP(VFCMPEQ) {
case 8:
vcmpsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 0);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
else {
@@ -1223,7 +1229,7 @@ DEF_OP(VFCMPEQ) {
case 8:
vcmppd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 0);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
}
@@ -1240,7 +1246,7 @@ DEF_OP(VFCMPNEQ) {
case 8:
vcmpsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 4);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
@@ -1252,7 +1258,7 @@ DEF_OP(VFCMPNEQ) {
case 8:
vcmppd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 4);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
}
@@ -1269,7 +1275,7 @@ DEF_OP(VFCMPLT) {
case 8:
vcmpsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 1);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
else {
@@ -1280,7 +1286,7 @@ DEF_OP(VFCMPLT) {
case 8:
vcmppd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 1);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
}
@@ -1297,7 +1303,7 @@ DEF_OP(VFCMPGT) {
case 8:
vcmpsd(GetDst(Node), GetSrc(Op->Header.Args[1].ID()), GetSrc(Op->Header.Args[0].ID()), 1);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
else {
@@ -1308,7 +1314,7 @@ DEF_OP(VFCMPGT) {
case 8:
vcmppd(GetDst(Node), GetSrc(Op->Header.Args[1].ID()), GetSrc(Op->Header.Args[0].ID()), 1);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
}
@@ -1325,7 +1331,7 @@ DEF_OP(VFCMPLE) {
case 8:
vcmpsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 2);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
else {
@@ -1336,7 +1342,7 @@ DEF_OP(VFCMPLE) {
case 8:
vcmppd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 2);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
}
@@ -1353,7 +1359,7 @@ DEF_OP(VFCMPORD) {
case 8:
vcmpsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 7);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
else {
@@ -1364,7 +1370,7 @@ DEF_OP(VFCMPORD) {
case 8:
vcmppd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 7);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
}
@@ -1381,7 +1387,7 @@ DEF_OP(VFCMPUNO) {
case 8:
vcmpsd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 3);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
else {
@@ -1392,21 +1398,21 @@ DEF_OP(VFCMPUNO) {
case 8:
vcmppd(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()), 3);
break;
default: LOGMAN_MSG_A("Unsupported elementSize: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unsupported element size: {}", Op->Header.ElementSize);
}
}
}
DEF_OP(VUShl) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(VUShr) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(VSShr) {
LOGMAN_MSG_A("Unimplemented");
LOGMAN_MSG_A_FMT("Unimplemented");
}
DEF_OP(VUShlS) {
@@ -1425,7 +1431,7 @@ DEF_OP(VUShlS) {
vpsllq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1445,7 +1451,7 @@ DEF_OP(VUShrS) {
vpsrlq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1462,7 +1468,7 @@ DEF_OP(VSShrS) {
break;
}
case 8: // Doesn't exist on x86
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1495,7 +1501,7 @@ DEF_OP(VInsElement) {
pinsrq(xmm15, rax, Op->DestIdx);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
movapd(GetDst(Node), xmm15);
@@ -1530,7 +1536,7 @@ DEF_OP(VInsScalarElement) {
pinsrq(xmm15, rax, Op->DestIdx);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
movapd(GetDst(Node), xmm15);
@@ -1560,7 +1566,7 @@ DEF_OP(VExtractElement) {
pinsrq(GetDst(Node), rax, 0);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1613,7 +1619,7 @@ DEF_OP(VDupElement) {
(Op->Index << 1));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1666,7 +1672,7 @@ DEF_OP(VUShrI) {
psrlq(GetDst(Node), Op->BitShift);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1704,7 +1710,7 @@ DEF_OP(VSShrI) {
}
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1724,7 +1730,7 @@ DEF_OP(VShlI) {
psllq(GetDst(Node), Op->BitShift);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1754,7 +1760,7 @@ DEF_OP(VUShrNI) {
mov(rcx, 0x80'80'80'80'80'80'80'80); // Upper
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
vmovq(xmm15, rax);
@@ -1790,7 +1796,7 @@ DEF_OP(VUShrNI2) {
mov(rcx, 0x0B'0A'09'08'03'02'01'00); // Upper
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
vmovq(xmm15, rax);
@@ -1817,7 +1823,7 @@ DEF_OP(VSXTL) {
case 8:
pmovsxdq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
@@ -1836,7 +1842,7 @@ DEF_OP(VSXTL2) {
case 8:
pmovsxdq(GetDst(Node), GetDst(Node));
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
@@ -1852,7 +1858,7 @@ DEF_OP(VUXTL) {
case 8:
pmovzxdq(GetDst(Node), GetSrc(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
@@ -1871,7 +1877,7 @@ DEF_OP(VUXTL2) {
case 8:
pmovzxdq(GetDst(Node), GetDst(Node));
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
}
@@ -1884,7 +1890,7 @@ DEF_OP(VSQXTN) {
case 2:
packssdw(xmm15, GetSrc(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
psrldq(xmm15, 8);
movaps(GetDst(Node), xmm15);
@@ -1903,7 +1909,7 @@ DEF_OP(VSQXTN2) {
case 2:
packssdw(xmm15, GetSrc(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
if (OpSize == 8) {
@@ -1921,7 +1927,7 @@ DEF_OP(VSQXTUN) {
case 2:
packusdw(xmm15, GetSrc(Op->Header.Args[0].ID()));
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
psrldq(xmm15, 8);
movaps(GetDst(Node), xmm15);
@@ -1940,7 +1946,7 @@ DEF_OP(VSQXTUN2) {
case 2:
packusdw(xmm15, GetSrc(Op->Header.Args[1].ID()));
break;
default: LOGMAN_MSG_A("Unknown element size: %d", Op->Header.ElementSize);
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize);
}
if (OpSize == 8) {
psrldq(xmm15, OpSize / 2);
@@ -1960,7 +1966,7 @@ DEF_OP(VMul) {
vpmulld(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -1991,7 +1997,7 @@ DEF_OP(VUMull) {
vpmuludq(GetDst(Node), xmm14, xmm15);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -2026,7 +2032,7 @@ DEF_OP(VSMull) {
vpmuldq(GetDst(Node), xmm14, xmm15);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -2062,7 +2068,7 @@ DEF_OP(VUMull2) {
vpmuludq(GetDst(Node), xmm14, xmm15);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -2102,7 +2108,7 @@ DEF_OP(VSMull2) {
vpmuldq(GetDst(Node), xmm14, xmm15);
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -2123,7 +2129,7 @@ DEF_OP(VUABDL) {
vpabsd(GetDst(Node), GetDst(Node));
break;
}
default: LOGMAN_MSG_A("Unknown Element Size: %d", Op->Header.ElementSize); break;
default: LOGMAN_MSG_A_FMT("Unknown Element Size: {}", Op->Header.ElementSize); break;
}
}
@@ -2141,7 +2147,7 @@ DEF_OP(VTBL1) {
vpshufb(GetDst(Node), GetSrc(Op->Header.Args[0].ID()), GetSrc(Op->Header.Args[1].ID()));
break;
}
default: LOGMAN_MSG_A("Unknown OpSize: %d", OpSize); break;
default: LOGMAN_MSG_A_FMT("Unknown OpSize: {}", OpSize); break;
}
}
+4 -3
View File
@@ -5,10 +5,11 @@ desc: Stores information about blocks, and provides C++ implementations to looku
$end_info$
*/
#include "Interface/Context/Context.h"
#include "Interface/Core/Core.h"
#include "Interface/Core/LookupCache.h"
#include <FEXCore/Utils/Allocator.h>
#include <FEXCore/Utils/LogManager.h>
#include "Interface/Context/Context.h"
#include "Interface/Core/LookupCache.h"
#include <sys/mman.h>
+9 -1
View File
@@ -1,10 +1,18 @@
#pragma once
#include "Interface/Context/Context.h"
#include <FEXCore/Utils/LogManager.h>
#include <cstdint>
#include <functional>
#include <map>
#include <stddef.h>
#include <utility>
#include <vector>
namespace FEXCore {
namespace Context {
struct Context;
}
class LookupCache {
public:
File diff suppressed because it is too large. Load diff
+33 -11
View File
@@ -3,7 +3,8 @@
#include "Interface/Core/Frontend.h"
#include "Interface/Context/Context.h"
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/Config/Config.h>
#include <FEXCore/Core/Context.h>
#include <FEXCore/Debug/X86Tables.h>
#include <FEXCore/IR/IntrusiveIRList.h>
#include <FEXCore/IR/IR.h>
@@ -12,9 +13,10 @@
#include <FEXCore/Utils/LogManager.h>
#include <cstdint>
#include <functional>
#include <map>
#include <set>
#include <stddef.h>
#include <utility>
#include <vector>
namespace FEXCore::IR {
class Pass;
@@ -270,15 +272,15 @@ public:
template<size_t ElementSize, bool HalfSize, bool Low>
void PSHUFDOp(OpcodeArgs);
void MOVDOp(OpcodeArgs);
template<size_t ElementSize, bool Scalar, uint32_t SrcIndex>
template<size_t ElementSize>
void PSRLDOp(OpcodeArgs);
template<size_t ElementSize>
void PSRLI(OpcodeArgs);
template<size_t ElementSize>
void PSLLI(OpcodeArgs);
template<size_t ElementSize, bool Scalar, uint32_t SrcIndex>
template<size_t ElementSize>
void PSLL(OpcodeArgs);
template<size_t ElementSize, bool Scalar, uint32_t SrcIndex>
template<size_t ElementSize>
void PSRAOp(OpcodeArgs);
void PSRLDQ(OpcodeArgs);
void PSLLDQ(OpcodeArgs);
@@ -299,9 +301,9 @@ public:
void Vector_CVT_Float_To_Float(OpcodeArgs);
template<size_t SrcElementSize, bool Narrow, bool HostRoundingMode>
void Vector_CVT_Float_To_Int(OpcodeArgs);
template<size_t SrcElementSize, bool Signed, bool Widen>
template<size_t SrcElementSize, bool Widen>
void MMX_To_XMM_Vector_CVT_Int_To_Float(OpcodeArgs);
template<size_t SrcElementSize, bool Narrow, bool HostRoundingMode>
template<size_t SrcElementSize, bool HostRoundingMode>
void XMM_To_MMX_Vector_CVT_Float_To_Int(OpcodeArgs);
void MASKMOVOp(OpcodeArgs);
void MOVBetweenGPR_FPR(OpcodeArgs);
@@ -380,6 +382,8 @@ public:
void X87FRSTOR(OpcodeArgs);
void X87FXAM(OpcodeArgs);
void X87FCMOV(OpcodeArgs);
void X87EMMS(OpcodeArgs);
void X87FFREE(OpcodeArgs);
void FXCH(OpcodeArgs);
@@ -501,9 +505,19 @@ private:
uint8_t GetSrcSize(FEXCore::X86Tables::DecodedOp Op) const;
template<unsigned BitOffset>
void SetRFLAG(OrderedNode *Value);
void SetRFLAG(OrderedNode *Value, unsigned BitOffset);
OrderedNode *GetRFLAG(unsigned BitOffset);
void SetRFLAG(OrderedNode *Value) {
flagsOp = FLAGS_OP_NONE;
_StoreFlag(_Bfe(1, 0, Value), BitOffset);
}
void SetRFLAG(OrderedNode *Value, unsigned BitOffset) {
flagsOp = FLAGS_OP_NONE;
_StoreFlag(_Bfe(1, 0, Value), BitOffset);
}
OrderedNode *GetRFLAG(unsigned BitOffset) {
return _LoadFlag(BitOffset);
}
OrderedNode *SelectCC(uint8_t OP, OrderedNode *TrueValue, OrderedNode *FalseValue);
@@ -526,6 +540,14 @@ private:
void GenerateFlags_RotateLeftImmediate(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, uint64_t Shift);
OrderedNode * GetX87Top();
enum X87Tag {
TAG_VALID = 0b00,
TAG_ZERO = 0b01,
TAG_SPECIAL = 0b10,
TAG_EMPTY = 0b11
};
void SetX87TopTag(OrderedNode *Value, uint32_t Tag);
OrderedNode *GetX87FTW(OrderedNode *Value);
void SetX87Top(OrderedNode *Value);
bool DestIsLockedMem(FEXCore::X86Tables::DecodedOp Op) const {
@@ -0,0 +1,63 @@
/*
$info$
tags: frontend|x86-to-ir, opcodes|dispatcher-implementations
desc: Handles x86/64 Crypto instructions to IR
$end_info$
*/
#include <FEXCore/Debug/X86Tables.h>
#include <FEXCore/IR/IREmitter.h>
#include <FEXCore/Utils/LogManager.h>
#include "Interface/Core/OpcodeDispatcher.h"
#include <stdint.h>
namespace FEXCore::IR {
class OrderedNode;
#define OpcodeArgs [[maybe_unused]] FEXCore::X86Tables::DecodedOp Op
void OpDispatchBuilder::AESImcOp(OpcodeArgs) {
OrderedNode *Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
auto Res = _VAESImc(Src);
StoreResult(FPRClass, Op, Res, -1);
}
void OpDispatchBuilder::AESEncOp(OpcodeArgs) {
OrderedNode *Dest = LoadSource(FPRClass, Op, Op->Dest, Op->Flags, -1);
OrderedNode *Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
auto Res = _VAESEnc(Dest, Src);
StoreResult(FPRClass, Op, Res, -1);
}
void OpDispatchBuilder::AESEncLastOp(OpcodeArgs) {
OrderedNode *Dest = LoadSource(FPRClass, Op, Op->Dest, Op->Flags, -1);
OrderedNode *Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
auto Res = _VAESEncLast(Dest, Src);
StoreResult(FPRClass, Op, Res, -1);
}
void OpDispatchBuilder::AESDecOp(OpcodeArgs) {
OrderedNode *Dest = LoadSource(FPRClass, Op, Op->Dest, Op->Flags, -1);
OrderedNode *Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
auto Res = _VAESDec(Dest, Src);
StoreResult(FPRClass, Op, Res, -1);
}
void OpDispatchBuilder::AESDecLastOp(OpcodeArgs) {
OrderedNode *Dest = LoadSource(FPRClass, Op, Op->Dest, Op->Flags, -1);
OrderedNode *Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
auto Res = _VAESDecLast(Dest, Src);
StoreResult(FPRClass, Op, Res, -1);
}
void OpDispatchBuilder::AESKeyGenAssist(OpcodeArgs) {
OrderedNode *Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
LOGMAN_THROW_A(Op->Src[1].IsLiteral(), "Src1 needs to be literal here");
uint64_t RCON = Op->Src[1].Data.Literal.Value;
auto Res = _VAESKeyGenAssist(Src, RCON);
StoreResult(FPRClass, Op, Res, -1);
}
}
@@ -0,0 +1,818 @@
/*
$info$
tags: frontend|x86-to-ir, opcodes|dispatcher-implementations
desc: Handles x86/64 flag generation
$end_info$
*/
#include "Interface/Context/Context.h"
#include "Interface/Core/OpcodeDispatcher.h"
#include <FEXCore/Core/X86Enums.h>
#include <FEXCore/Config/Config.h>
#include <FEXCore/Debug/X86Tables.h>
#include <FEXCore/Utils/LogManager.h>
#include <FEXCore/IR/IR.h>
#include <array>
#include <cstdint>
namespace FEXCore::IR {
constexpr std::array<uint32_t, 17> FlagOffsets = {
FEXCore::X86State::RFLAG_CF_LOC,
FEXCore::X86State::RFLAG_PF_LOC,
FEXCore::X86State::RFLAG_AF_LOC,
FEXCore::X86State::RFLAG_ZF_LOC,
FEXCore::X86State::RFLAG_SF_LOC,
FEXCore::X86State::RFLAG_TF_LOC,
FEXCore::X86State::RFLAG_IF_LOC,
FEXCore::X86State::RFLAG_DF_LOC,
FEXCore::X86State::RFLAG_OF_LOC,
FEXCore::X86State::RFLAG_IOPL_LOC,
FEXCore::X86State::RFLAG_NT_LOC,
FEXCore::X86State::RFLAG_RF_LOC,
FEXCore::X86State::RFLAG_VM_LOC,
FEXCore::X86State::RFLAG_AC_LOC,
FEXCore::X86State::RFLAG_VIF_LOC,
FEXCore::X86State::RFLAG_VIP_LOC,
FEXCore::X86State::RFLAG_ID_LOC,
};
void OpDispatchBuilder::SetPackedRFLAG(bool Lower8, OrderedNode *Src) {
uint8_t NumFlags = FlagOffsets.size();
if (Lower8) {
NumFlags = 5;
}
auto OneConst = _Constant(1);
for (int i = 0; i < NumFlags; ++i) {
auto Tmp = _And(_Lshr(Src, _Constant(FlagOffsets[i])), OneConst);
SetRFLAG(Tmp, FlagOffsets[i]);
}
}
OrderedNode *OpDispatchBuilder::GetPackedRFLAG(bool Lower8) {
OrderedNode *Original = _Constant(2);
uint8_t NumFlags = FlagOffsets.size();
if (Lower8) {
NumFlags = 5;
}
for (int i = 0; i < NumFlags; ++i) {
OrderedNode *Flag = _LoadFlag(FlagOffsets[i]);
Flag = _Bfe(4, 32, 0, Flag);
Flag = _Lshl(Flag, _Constant(FlagOffsets[i]));
Original = _Or(Original, Flag);
}
return Original;
}
void OpDispatchBuilder::GenerateFlags_ADC(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2, OrderedNode *CF) {
auto Size = GetSrcSize(Op) * 8;
// AF
{
OrderedNode *AFRes = _Xor(_Xor(Src1, Src2), Res);
AFRes = _Bfe(1, 4, AFRes);
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(AFRes);
}
// SF
{
auto SignBitConst = _Constant(GetSrcSize(Op) * 8 - 1);
auto LshrOp = _Lshr(Res, SignBitConst);
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(LshrOp);
}
// PF
if (!CTX->Config.ABINoPF) {
auto PopCountOp = _Popcount(_And(Res, _Constant(0xFF)));
auto XorOp = _Xor(PopCountOp, _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(SelectOp);
}
// CF
// Unsigned
{
auto SelectOpLT = _Select(FEXCore::IR::COND_ULT, Res, Src2, _Constant(1), _Constant(0));
auto SelectOpLE = _Select(FEXCore::IR::COND_ULE, Res, Src2, _Constant(1), _Constant(0));
auto SelectCF = _Select(FEXCore::IR::COND_EQ, CF, _Constant(1), SelectOpLE, SelectOpLT);
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(SelectCF);
}
// OF
// Signed
{
auto NegOne = _Constant(~0ULL);
auto XorOp1 = _Xor(_Xor(Src1, Src2), NegOne);
auto XorOp2 = _Xor(Res, Src1);
OrderedNode *AndOp1 = _And(XorOp1, XorOp2);
switch (Size) {
case 8:
AndOp1 = _Bfe(1, 7, AndOp1);
break;
case 16:
AndOp1 = _Bfe(1, 15, AndOp1);
break;
case 32:
AndOp1 = _Bfe(1, 31, AndOp1);
break;
case 64:
AndOp1 = _Bfe(1, 63, AndOp1);
break;
default: LOGMAN_MSG_A("Unknown BFESize: %d", Size); break;
}
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(AndOp1);
}
}
void OpDispatchBuilder::GenerateFlags_SBB(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2, OrderedNode *CF) {
// AF
{
OrderedNode *AFRes = _Xor(_Xor(Src1, Src2), Res);
AFRes = _Bfe(1, 4, AFRes);
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(AFRes);
}
// SF
{
auto SignBitConst = _Constant(GetSrcSize(Op) * 8 - 1);
auto LshrOp = _Lshr(Res, SignBitConst);
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(LshrOp);
}
// PF
if (!CTX->Config.ABINoPF) {
auto PopCountOp = _Popcount(_And(Res, _Constant(0xFF)));
auto XorOp = _Xor(PopCountOp, _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(SelectOp);
}
// CF
// Unsigned
{
auto SelectOpLT = _Select(FEXCore::IR::COND_UGT, Res, Src1, _Constant(1), _Constant(0));
auto SelectOpLE = _Select(FEXCore::IR::COND_UGE, Res, Src1, _Constant(1), _Constant(0));
auto SelectCF = _Select(FEXCore::IR::COND_EQ, CF, _Constant(1), SelectOpLE, SelectOpLT);
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(SelectCF);
}
// OF
// Signed
{
auto XorOp1 = _Xor(Src1, Src2);
auto XorOp2 = _Xor(Res, Src1);
OrderedNode *AndOp1 = _And(XorOp1, XorOp2);
switch (GetSrcSize(Op)) {
case 1:
AndOp1 = _Bfe(1, 7, AndOp1);
break;
case 2:
AndOp1 = _Bfe(1, 15, AndOp1);
break;
case 4:
AndOp1 = _Bfe(1, 31, AndOp1);
break;
case 8:
AndOp1 = _Bfe(1, 63, AndOp1);
break;
default: LOGMAN_MSG_A("Unknown BFESize: %d", GetSrcSize(Op)); break;
}
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(AndOp1);
}
}
void OpDispatchBuilder::GenerateFlags_SUB(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2, bool UpdateCF) {
// AF
{
OrderedNode *AFRes = _Xor(_Xor(Src1, Src2), Res);
AFRes = _Bfe(1, 4, AFRes);
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(AFRes);
}
// SF
{
auto SignBitConst = _Constant(GetSrcSize(Op) * 8 - 1);
auto LshrOp = _Lshr(Res, SignBitConst);
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(LshrOp);
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// ZF
{
auto ZeroConst = _Constant(0);
auto OneConst = _Constant(1);
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, ZeroConst, OneConst, ZeroConst);
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(SelectOp);
}
// CF
if (UpdateCF) {
auto ZeroConst = _Constant(0);
auto OneConst = _Constant(1);
auto SelectOp = _Select(FEXCore::IR::COND_ULT,
Src1, Src2, OneConst, ZeroConst);
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(SelectOp);
}
// OF
{
auto XorOp1 = _Xor(Src1, Src2);
auto XorOp2 = _Xor(Res, Src1);
OrderedNode *FinalAnd = _And(XorOp1, XorOp2);
FinalAnd = _Bfe(1, GetSrcSize(Op) * 8 - 1, FinalAnd);
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(FinalAnd);
}
}
void OpDispatchBuilder::GenerateFlags_ADD(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2, bool UpdateCF) {
// AF
{
OrderedNode *AFRes = _Xor(_Xor(Src1, Src2), Res);
AFRes = _Bfe(1, 4, AFRes);
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(AFRes);
}
// SF
{
auto SignBitConst = _Constant(GetSrcSize(Op) * 8 - 1);
auto LshrOp = _Lshr(Res, SignBitConst);
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(LshrOp);
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(SelectOp);
}
// CF
if (UpdateCF) {
auto SelectOp = _Select(FEXCore::IR::COND_ULT, Res, Src2, _Constant(1), _Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(SelectOp);
}
// OF
{
auto NegOne = _Constant(~0ULL);
auto XorOp1 = _Xor(_Xor(Src1, Src2), NegOne);
auto XorOp2 = _Xor(Res, Src1);
OrderedNode *AndOp1 = _And(XorOp1, XorOp2);
switch (GetSrcSize(Op)) {
case 1:
AndOp1 = _Bfe(1, 7, AndOp1);
break;
case 2:
AndOp1 = _Bfe(1, 15, AndOp1);
break;
case 4:
AndOp1 = _Bfe(1, 31, AndOp1);
break;
case 8:
AndOp1 = _Bfe(1, 63, AndOp1);
break;
default: LOGMAN_MSG_A("Unknown BFESize: %d", GetSrcSize(Op)); break;
}
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(AndOp1);
}
}
void OpDispatchBuilder::GenerateFlags_MUL(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *High) {
// PF/AF/ZF/SF
// Undefined
{
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(_Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(_Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(_Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(_Constant(0));
}
// CF/OF
{
// CF and OF are set if the result of the operation can't be fit in to the destination register
// If the value can fit then the top bits will be zero
auto SignBit = _Sbfe(1, GetSrcSize(Op) * 8 - 1, Res);
auto SelectOp = _Select(FEXCore::IR::COND_EQ, High, SignBit, _Constant(0), _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(SelectOp);
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(SelectOp);
}
}
void OpDispatchBuilder::GenerateFlags_UMUL(FEXCore::X86Tables::DecodedOp Op, OrderedNode *High) {
// AF/SF/PF/ZF
// Undefined
{
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(_Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(_Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(_Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(_Constant(0));
}
// CF/OF
{
// CF and OF are set if the result of the operation can't be fit in to the destination register
// The result register will be all zero if it can't fit due to how multiplication behaves
auto SelectOp = _Select(FEXCore::IR::COND_EQ, High, _Constant(0), _Constant(0), _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(SelectOp);
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(SelectOp);
}
}
void OpDispatchBuilder::GenerateFlags_Logical(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2) {
// AF
{
// Undefined
// Set to zero anyway
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(_Constant(0));
}
// SF
{
auto SignBitConst = _Constant(GetSrcSize(Op) * 8 - 1);
auto LshrOp = _Lshr(Res, SignBitConst);
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(LshrOp);
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(SelectOp);
}
// CF/OF
{
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(_Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(_Constant(0));
}
}
#define COND_FLAG_SET(cond, flag, newflag) \
auto oldflag = GetRFLAG(FEXCore::X86State::flag);\
auto newval = _Select(FEXCore::IR::COND_EQ, cond, _Constant(0), oldflag, newflag);\
SetRFLAG<FEXCore::X86State::flag>(newval);
void OpDispatchBuilder::GenerateFlags_ShiftLeft(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2) {
// CF
{
// Extract the last bit shifted in to CF
auto Size = _Constant(GetSrcSize(Op) * 8);
auto ShiftAmt = _Sub(Size, Src2);
auto LastBit = _And(_Lshr(Src1, ShiftAmt), _Constant(1));
COND_FLAG_SET(Src2, RFLAG_CF_LOC, LastBit);
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
COND_FLAG_SET(Src2, RFLAG_PF_LOC, XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// AF
{
// Undefined
// Set to zero anyway
COND_FLAG_SET(Src2, RFLAG_AF_LOC, _Constant(0));
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
COND_FLAG_SET(Src2, RFLAG_ZF_LOC, SelectOp);
}
// SF
{
auto val = _Bfe(1, GetSrcSize(Op) * 8 - 1, Res);
COND_FLAG_SET(Src2, RFLAG_SF_LOC, val);
}
// OF
{
// In the case of left shift. OF is only set from the result of <Top Source Bit> XOR <Top Result Bit>
// When Shift > 1 then OF is undefined
auto val = _Bfe(1, GetSrcSize(Op) * 8 - 1, _Xor(Src1, Res));
COND_FLAG_SET(Src2, RFLAG_OF_LOC, val);
}
}
void OpDispatchBuilder::GenerateFlags_ShiftRight(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2) {
// CF
{
// Extract the last bit shifted in to CF
auto ShiftAmt = _Sub(Src2, _Constant(1));
auto LastBit = _And(_Lshr(Src1, ShiftAmt), _Constant(1));
COND_FLAG_SET(Src2, RFLAG_CF_LOC, LastBit);
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
COND_FLAG_SET(Src2, RFLAG_PF_LOC, XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// AF
{
// Undefined
// Set to zero anyway
COND_FLAG_SET(Src2, RFLAG_AF_LOC, _Constant(0));
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
COND_FLAG_SET(Src2, RFLAG_ZF_LOC, SelectOp);
}
// SF
{
auto val =_Bfe(1, GetSrcSize(Op) * 8 - 1, Res);
COND_FLAG_SET(Src2, RFLAG_SF_LOC, val);
}
// OF
{
// Only defined when Shift is 1 else undefined
// OF flag is set if a sign change occurred
auto val = _Bfe(1, GetSrcSize(Op) * 8 - 1, _Xor(Src1, Res));
COND_FLAG_SET(Src2, RFLAG_OF_LOC, val);
}
}
void OpDispatchBuilder::GenerateFlags_SignShiftRight(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2) {
// CF
{
// Extract the last bit shifted in to CF
auto ShiftAmt = _Sub(Src2, _Constant(1));
auto LastBit = _And(_Lshr(Src1, ShiftAmt), _Constant(1));
COND_FLAG_SET(Src2, RFLAG_CF_LOC, LastBit);
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
COND_FLAG_SET(Src2, RFLAG_PF_LOC, XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// AF
{
// Undefined
// Set to zero anyway
COND_FLAG_SET(Src2, RFLAG_AF_LOC, _Constant(0));
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
COND_FLAG_SET(Src2, RFLAG_ZF_LOC, SelectOp);
}
// SF
{
auto SignBitConst = _Constant(GetSrcSize(Op) * 8 - 1);
auto LshrOp = _Lshr(Res, SignBitConst);
COND_FLAG_SET(Src2, RFLAG_SF_LOC, LshrOp);
}
// OF
{
COND_FLAG_SET(Src2, RFLAG_OF_LOC, _Constant(0));
}
}
void OpDispatchBuilder::GenerateFlags_ShiftLeftImmediate(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, uint64_t Shift) {
// No flags changed if shift is zero
if (Shift == 0) return;
// CF
{
// Extract the last bit shifted in to CF
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(_Bfe(1, GetSrcSize(Op) * 8 - Shift, Src1));
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// AF
{
// Undefined
// Set to zero anyway
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(_Constant(0));
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(SelectOp);
}
// SF
{
auto LshrOp = _Bfe(1, GetSrcSize(Op) * 8 - 1, Res);
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(LshrOp);
// OF
// In the case of left shift. OF is only set from the result of <Top Source Bit> XOR <Top Result Bit>
if (Shift == 1) {
auto SourceBit = _Bfe(1, GetSrcSize(Op) * 8 - 1, Src1);
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(_Xor(SourceBit, LshrOp));
}
}
}
void OpDispatchBuilder::GenerateFlags_SignShiftRightImmediate(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, uint64_t Shift) {
// No flags changed if shift is zero
if (Shift == 0) return;
// CF
{
// Extract the last bit shifted in to CF
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(_Bfe(1, Shift-1, Src1));
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// AF
{
// Undefined
// Set to zero anyway
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(_Constant(0));
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(SelectOp);
}
// SF
{
auto SignBitConst = _Constant(GetSrcSize(Op) * 8 - 1);
auto LshrOp = _Lshr(Res, SignBitConst);
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(LshrOp);
// OF
// Only defined when Shift is 1 else undefined
// Only is set if the top bit was set to 1 when shifted
// So it is set to same value as SF
if (Shift == 1) {
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(_Constant(0));
}
}
}
void OpDispatchBuilder::GenerateFlags_ShiftRightImmediate(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, uint64_t Shift) {
// No flags changed if shift is zero
if (Shift == 0) return;
// CF
{
// Extract the last bit shifted in to CF
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(_Bfe(1, Shift-1, Src1));
}
// PF
if (!CTX->Config.ABINoPF) {
auto EightBitMask = _Constant(0xFF);
auto PopCountOp = _Popcount(_And(Res, EightBitMask));
auto XorOp = _Xor(PopCountOp, _Constant(1));
SetRFLAG<FEXCore::X86State::RFLAG_PF_LOC>(XorOp);
} else {
_InvalidateFlags(1UL << FEXCore::X86State::RFLAG_PF_LOC);
}
// AF
{
// Undefined
// Set to zero anyway
SetRFLAG<FEXCore::X86State::RFLAG_AF_LOC>(_Constant(0));
}
// ZF
{
auto SelectOp = _Select(FEXCore::IR::COND_EQ,
Res, _Constant(0), _Constant(1), _Constant(0));
SetRFLAG<FEXCore::X86State::RFLAG_ZF_LOC>(SelectOp);
}
// SF
{
auto SignBitConst = _Constant(GetSrcSize(Op) * 8 - 1);
auto LshrOp = _Lshr(Res, SignBitConst);
SetRFLAG<FEXCore::X86State::RFLAG_SF_LOC>(LshrOp);
}
// OF
{
// Only defined when Shift is 1 else undefined
// Is set to the MSB of the original value
if (Shift == 1) {
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(_Bfe(1, GetSrcSize(Op) * 8 - 1, Src1));
}
}
}
void OpDispatchBuilder::GenerateFlags_RotateRight(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2) {
auto OpSize = GetSrcSize(Op) * 8;
// Extract the last bit shifted in to CF
auto NewCF = _Bfe(1, OpSize - 1, Res);
// CF
{
auto OldCF = GetRFLAG(FEXCore::X86State::RFLAG_CF_LOC);
auto CF = _Select(FEXCore::IR::COND_EQ, Src2, _Constant(0), OldCF, NewCF);
// Extract the last bit shifted in to CF
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(CF);
}
// OF
{
auto OldOF = GetRFLAG(FEXCore::X86State::RFLAG_OF_LOC);
// OF is set to the XOR of the new CF bit and the most significant bit of the result
auto NewOF = _Xor(_Bfe(1, OpSize - 2, Res), NewCF);
// If shift == 0, don't update flags
auto OF = _Select(FEXCore::IR::COND_EQ, Src2, _Constant(0), OldOF, NewOF);
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(OF);
}
}
void OpDispatchBuilder::GenerateFlags_RotateLeft(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, OrderedNode *Src2) {
auto OpSize = GetSrcSize(Op) * 8;
// Extract the last bit shifted in to CF
//auto Size = _Constant(GetSrcSize(Res) * 8);
//auto ShiftAmt = _Sub(Size, Src2);
auto NewCF = _Bfe(1, 0, Res);
// CF
{
auto OldCF = GetRFLAG(FEXCore::X86State::RFLAG_CF_LOC);
auto CF = _Select(FEXCore::IR::COND_EQ, Src2, _Constant(0), OldCF, NewCF);
// Extract the last bit shifted in to CF
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(CF);
}
// OF
{
auto OldOF = GetRFLAG(FEXCore::X86State::RFLAG_OF_LOC);
// OF is set to the XOR of the new CF bit and the most significant bit of the result
auto NewOF = _Xor(_Bfe(1, OpSize - 1, Res), NewCF);
auto OF = _Select(FEXCore::IR::COND_EQ, Src2, _Constant(0), OldOF, NewOF);
// If shift == 0, don't update flags
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(OF);
}
}
void OpDispatchBuilder::GenerateFlags_RotateRightImmediate(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, uint64_t Shift) {
if (Shift == 0) return;
auto OpSize = GetSrcSize(Op) * 8;
auto NewCF = _Bfe(1, OpSize - Shift, Src1);
// CF
{
// Extract the last bit shifted in to CF
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(NewCF);
}
// OF
{
if (Shift == 1) {
// OF is set to the XOR of the new CF bit and the most significant bit of the result
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(_Xor(_Bfe(1, OpSize - 1, Res), NewCF));
}
}
}
void OpDispatchBuilder::GenerateFlags_RotateLeftImmediate(FEXCore::X86Tables::DecodedOp Op, OrderedNode *Res, OrderedNode *Src1, uint64_t Shift) {
if (Shift == 0) return;
auto OpSize = GetSrcSize(Op) * 8;
// CF
{
// Extract the last bit shifted in to CF
SetRFLAG<FEXCore::X86State::RFLAG_CF_LOC>(_Bfe(1, Shift, Src1));
}
// OF
{
if (Shift == 1) {
// OF is the top two MSBs XOR'd together
SetRFLAG<FEXCore::X86State::RFLAG_OF_LOC>(_Xor(_Bfe(1, OpSize - 1, Src1), _Bfe(1, OpSize - 2, Src1)));
}
}
}
}
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,113 @@
#include <FEXCore/Core/SignalDelegator.h>
#include <FEXCore/Utils/LogManager.h>
#include <unistd.h>
#include <signal.h>
namespace FEXCore {
struct ThreadState {
FEXCore::Core::InternalThreadState *Thread{};
};
thread_local ThreadState ThreadData{};
static bool IsSynchronous(int Signal) {
switch (Signal) {
case SIGBUS:
case SIGFPE:
case SIGILL:
case SIGSEGV:
case SIGTRAP:
return true;
default: break;
};
return false;
}
/**
* @brief Masks signals from the signal mask
*
* @param how Argument to sigmask. SIG_{BLOCK, SETMASK, UNBLOCK}
* @param Signal Which signal to set or -1 to sweep through them all
*/
static void MaskSignals(int how, int Signal = -1) {
// If we have a helper thread, we need to mask a significant amount of signals so the an errant thread doesn't receive a signal that it shouldn't
sigset_t SignalSet{};
sigemptyset(&SignalSet);
if (Signal == -1) {
for (int i = 0; i <= SignalDelegator::MAX_SIGNALS; ++i) {
// If it is a synchronous signal then don't ignore it
if (IsSynchronous(i)) {
continue;
}
// Add this signal to the ignore list
sigaddset(&SignalSet, i);
}
}
else {
sigaddset(&SignalSet, Signal);
}
// Be warned, a thread will inherit the signal mask if created from this thread
int Result = pthread_sigmask(how, &SignalSet, nullptr);
if (Result != 0) {
LogMan::Msg::E("Couldn't register thread to mask signals");
}
}
void SignalDelegator::MaskThreadSignals() {
MaskSignals(SIG_BLOCK);
}
FEXCore::Core::InternalThreadState *SignalDelegator::GetTLSThread() {
return ThreadData.Thread;
}
void SignalDelegator::RegisterTLSState(FEXCore::Core::InternalThreadState *Thread) {
ThreadData.Thread = Thread;
RegisterFrontendTLSState(Thread);
}
void SignalDelegator::UninstallTLSState(FEXCore::Core::InternalThreadState *Thread) {
UninstallFrontendTLSState(Thread);
ThreadData.Thread = nullptr;
}
void SignalDelegator::RegisterHostSignalHandler(int Signal, HostSignalDelegatorFunction Func, bool Required) {
SetHostSignalHandler(Signal, Func, Required);
FrontendRegisterHostSignalHandler(Signal, Func, Required);
}
void SignalDelegator::RegisterFrontendHostSignalHandler(int Signal, HostSignalDelegatorFunction Func, bool Required) {
SetFrontendHostSignalHandler(Signal, Func, Required);
FrontendRegisterFrontendHostSignalHandler(Signal, Func, Required);
}
void SignalDelegator::HandleSignal(int Signal, void *Info, void *UContext) {
// Let the host take first stab at handling the signal
auto Thread = GetTLSThread();
HostSignalHandler &Handler = HostHandlers[Signal];
if (!Thread) {
LogMan::Msg::E("[%d] Thread has received a signal and hasn't registered itself with the delegate! Programming error!", ::gettid());
}
else {
if (Handler.Handler &&
Handler.Handler(Thread, Signal, Info, UContext)) {
// If the host handler handled the fault then we can continue now
return;
}
if (Handler.FrontendHandler &&
Handler.FrontendHandler(Thread, Signal, Info, UContext)) {
return;
}
// Now let the frontend handle the signal
// It's clearly a guest signal and this ends up being an OS specific issue
HandleGuestSignal(Thread, Signal, Info, UContext);
}
}
}
@@ -0,0 +1,285 @@
// This is the vsyscall page for x86_64 guest code
// This was compiled with nasm with the following source then exported to binary
//BITS 64;
//
//align 4096, db 0xcc
// ; __NR_gettimeofday
// mov rax, 96
// syscall
// ret
//
//align 1024, db 0xcc
// ; __NR_time
// mov rax, 201
// syscall
// ret
//
//align 1024, db 0xcc
// ; __NR_getcpu
// mov rax, 309
// syscall
// ret
//
//align 4096, db 0xcc
// We only want one of these pages per FEX process
// One page
const static uint8_t VSyscallData[0x1000] = {
0xB8, 0x60, 0x00, 0x00, 0x00, 0x0F, 0x05, 0xC3, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xB8, 0xC9, 0x00, 0x00, 0x00, 0x0F, 0x05, 0xC3, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xB8, 0x35, 0x01, 0x00, 0x00, 0x0F, 0x05, 0xC3, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC,
0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC, 0xCC
};
@@ -114,8 +114,6 @@ void InstallDebugInfo() {
GenerateDebugTable(PrimaryInstGroupOps, PrimaryGroupOpTable);
GenerateDebugTable(SecondInstGroupOps, SecondaryExtensionOpTable);
LogMan::Msg::D("Installing debug info");
}
}
#endif
+6 -3
View File
@@ -6,12 +6,15 @@ $end_info$
*/
#include "Interface/Core/X86HelperGen.h"
#include <FEXCore/Config/Config.h>
#include <FEXCore/Utils/Allocator.h>
#include <cstdint>
#include <cstring>
#include <stdlib.h>
#include <vector>
#include <sys/mman.h>
#include <bits/mman-map-flags-generic.h>
namespace FEXCore {
constexpr size_t CODE_SIZE = 0x1000;
@@ -50,14 +53,14 @@ void* X86GeneratedCode::AllocateGuestCodeSpace(size_t Size) {
// We need to have the sigret handler in the lower 32bits of memory space
// Scan top down and try to allocate a location
for (size_t Location = 0xFFFF'E000; Location != 0x0; Location -= 0x1000) {
void *Ptr = FEXCore::Allocator::mmap(reinterpret_cast<void*>(Location), Size, PROT_READ | PROT_WRITE, MAP_FIXED_NOREPLACE | MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
void *Ptr = ::mmap(reinterpret_cast<void*>(Location), Size, PROT_READ | PROT_WRITE, MAP_FIXED_NOREPLACE | MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
if (Ptr != MAP_FAILED &&
reinterpret_cast<uintptr_t>(Ptr) >= LOCATION_MAX) {
// Failed to map in the lower 32bits
// Try again
// Can happen in the case that host kernel ignores MAP_FIXED_NOREPLACE
FEXCore::Allocator::munmap(Ptr, Size);
::munmap(Ptr, Size);
continue;
}
+1 -1
View File
@@ -5,8 +5,8 @@ $end_info$
*/
#pragma once
#include <FEXCore/Config/Config.h>
#include <stddef.h>
#include <stdint.h>
namespace FEXCore {
-7
View File
@@ -5,14 +5,8 @@ tags: frontend|x86-tables
$end_info$
*/
#include <FEXCore/Utils/LogManager.h>
#include <FEXCore/Core/Context.h>
#include <FEXCore/Debug/X86Tables.h>
#include <array>
#include <cstdint>
#include <tuple>
#include <vector>
namespace FEXCore::X86Tables {
@@ -108,7 +102,6 @@ void InitializeInfoTables(Context::OperatingMode Mode) {
#ifndef NDEBUG
X86InstDebugInfo::InstallDebugInfo();
LogMan::Msg::D("X86Tables had %ld total insts, and %ld labeled as understood", Total, NumInsts);
#endif
}
@@ -7,6 +7,9 @@ $end_info$
#include "Interface/Core/X86Tables/X86Tables.h"
#include <FEXCore/Core/Context.h>
#include <FEXCore/Debug/X86Tables.h>
#include <iterator>
namespace FEXCore::X86Tables {
using namespace InstFlags;
@@ -6,6 +6,10 @@ $end_info$
#include "Interface/Core/X86Tables/X86Tables.h"
#include <FEXCore/Debug/X86Tables.h>
#include <iterator>
namespace FEXCore::X86Tables {
using namespace InstFlags;
@@ -6,6 +6,10 @@ $end_info$
#include "Interface/Core/X86Tables/X86Tables.h"
#include <FEXCore/Debug/X86Tables.h>
#include <iterator>
namespace FEXCore::X86Tables {
using namespace InstFlags;
@@ -5,6 +5,10 @@ $end_info$
*/
#include "Interface/Core/X86Tables/X86Tables.h"
#include <FEXCore/Debug/X86Tables.h>
#include <iterator>
#include <stdint.h>
namespace FEXCore::X86Tables {
using namespace InstFlags;
@@ -6,6 +6,12 @@ $end_info$
#include "Interface/Core/X86Tables/X86Tables.h"
#include <FEXCore/Debug/X86Tables.h>
#include <FEXCore/Core/Context.h>
#include <iterator>
#include <stdint.h>
namespace FEXCore::X86Tables {
using namespace InstFlags;
@@ -6,6 +6,11 @@ $end_info$
#include "Interface/Core/X86Tables/X86Tables.h"
#include <FEXCore/Core/Context.h>
#include <FEXCore/Debug/X86Tables.h>
#include <iterator>
namespace FEXCore::X86Tables {
using namespace InstFlags;
@@ -6,6 +6,11 @@ $end_info$
#include "Interface/Core/X86Tables/X86Tables.h"
#include <FEXCore/Debug/X86Tables.h>
#include <iterator>
#include <stdint.h>
namespace FEXCore::X86Tables {
using namespace InstFlags;
@@ -6,6 +6,10 @@ $end_info$
#include "Interface/Core/X86Tables/X86Tables.h"
#include <FEXCore/Debug/X86Tables.h>
#include <iterator>
namespace FEXCore::X86Tables {
using namespace InstFlags;
@@ -4,7 +4,12 @@ tags: frontend|x86-tables
$end_info$
*/
#include "Interface/Core/X86Tables/X86Tables.h"
#include "Interface/Core/X86Tables/X86Tables.h"
#include <FEXCore/Core/Context.h>
#include <FEXCore/Debug/X86Tables.h>
#include <iterator>
namespace FEXCore::X86Tables {
using namespace InstFlags;
@@ -27,7 +32,7 @@ void InitializeSecondaryTables(Context::OperatingMode Mode) {
{0x0B, 1, X86InstInfo{"UD2", TYPE_INST, FLAGS_DEBUG | FLAGS_BLOCK_END | FLAGS_NO_OVERLAY, 0, nullptr}},
{0x0C, 1, X86InstInfo{"", TYPE_INVALID, FLAGS_NO_OVERLAY, 0, nullptr}},
{0x0D, 1, X86InstInfo{"", TYPE_GROUP_P, FLAGS_MODRM | FLAGS_NO_OVERLAY, 0, nullptr}},
{0x0E, 1, X86InstInfo{"FEMMS", TYPE_INST, FLAGS_BLOCK_END | FLAGS_NO_OVERLAY, 0, nullptr}},
{0x0E, 1, X86InstInfo{"FEMMS", TYPE_INST, FLAGS_NO_OVERLAY, 0, nullptr}},
{0x0F, 1, X86InstInfo{"", TYPE_3DNOW_TABLE, FLAGS_NO_OVERLAY, 0, nullptr}},
{0x10, 1, X86InstInfo{"MOVUPS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
@@ -6,6 +6,9 @@ $end_info$
#include "Interface/Core/X86Tables/X86Tables.h"
#include <FEXCore/Debug/X86Tables.h>
#include <iterator>
namespace FEXCore::X86Tables {
using namespace InstFlags;
@@ -6,6 +6,7 @@ $end_info$
#pragma once
#include <FEXCore/Debug/X86Tables.h>
#include <FEXCore/Core/Context.h>
#include <FEXCore/Utils/LogManager.h>
@@ -113,5 +114,7 @@ static inline void GenerateX87Table(X86InstInfo *FinalTable, U16U8InfoStruct con
}
};
void InitializeInfoTables(Context::OperatingMode Mode);
}
@@ -6,6 +6,9 @@ $end_info$
#include "Interface/Core/X86Tables/X86Tables.h"
#include <FEXCore/Debug/X86Tables.h>
#include <iterator>
namespace FEXCore::X86Tables {
using namespace InstFlags;
@@ -6,6 +6,11 @@ $end_info$
#include "Interface/Core/X86Tables/X86Tables.h"
#include <FEXCore/Debug/X86Tables.h>
#include <iterator>
#include <stdint.h>
namespace FEXCore::X86Tables {
using namespace InstFlags;
+10 -8
View File
@@ -5,21 +5,23 @@ tags: glue|thunks
$end_info$
*/
#include <FEXCore/Config/Config.h>
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/Debug/InternalThreadState.h>
#include <FEXCore/Utils/LogManager.h>
#include <FEXCore/IR/IR.h>
#include "Thunks.h"
#include "stdio.h"
#include <dlfcn.h>
#include <string>
#include <map>
#include <array>
#include <Interface/Context/Context.h>
#include "Interface/Core/InternalThreadState.h"
#include "FEXCore/Core/X86Enums.h"
#include <mutex>
#include <map>
#include <memory>
#include <shared_mutex>
#include <stdint.h>
#include <string>
#include <utility>
struct LoadlibArgs {
const char *Name;
@@ -51,7 +53,7 @@ namespace FEXCore {
Thread->CurrentFrame->State.gregs[FEXCore::X86State::REG_RDI] = (uintptr_t)arg0;
Thread->CurrentFrame->State.gregs[FEXCore::X86State::REG_RSI] = (uintptr_t)arg1;
Thread->CTX->HandleCallback((uintptr_t)callback);
Thread->CTX->HandleCallback(Thread, (uintptr_t)callback);
}
static void LoadLib(void *ArgsV) {
+5 -2
View File
@@ -5,12 +5,15 @@ $end_info$
*/
#pragma once
#include <FEXCore/IR/IR.h>
namespace FEXCore::Core {
struct InternalThreadState;
}
namespace FEXCore::IR {
struct SHA256Sum;
}
namespace FEXCore {
typedef void ThunkedFunction(void* ArgsRv);
@@ -22,4 +25,4 @@ namespace FEXCore {
static ThunkHandler* Create();
};
};
};
+18
View File
@@ -1462,6 +1462,24 @@
]
},
"AtomicFetchNeg": {
"HasSideEffects": true,
"Desc": ["Atomic integer fetch and two's complement negate",
"Dest is the value prior to operating on the value in memory"
],
"OpClass": "Atomic",
"HasDest": true,
"DestClass": "GPR",
"DestSize": "Size",
"SSAArgs": "1",
"SSANames": [
"Addr"
],
"Args": [
"uint8_t", "Size"
]
},
"VExtractToGPR": {
"Desc": ["Extracts an element from a vector and places it in a GPR",
"The element that is extracted from the vector is zero extended to the GPR size"
+7 -2
View File
@@ -7,9 +7,14 @@ $end_info$
#include <FEXCore/IR/IR.h>
#include <FEXCore/IR/IntrusiveIRList.h>
#include <FEXCore/Utils/LogManager.h>
#include "Interface/IR/Passes/RegisterAllocationPass.h"
#include <FEXCore/IR/RegisterAllocationData.h>
#include <algorithm>
#include <array>
#include <ostream>
#include <stdint.h>
#include <string>
#include <string_view>
#include <iomanip>
namespace FEXCore::IR {
+10 -2
View File
@@ -5,7 +5,15 @@ tags: ir|emitter
$end_info$
*/
#include <FEXCore/IR/IR.h>
#include <FEXCore/IR/IREmitter.h>
#include <FEXCore/IR/IntrusiveIRList.h>
#include <FEXCore/Utils/LogManager.h>
#include <array>
#include <stdint.h>
#include <string.h>
#include <vector>
namespace FEXCore::IR {
void IREmitter::ResetWorkingList() {
@@ -82,7 +90,7 @@ IREmitter::IRPair<IROp_CodeBlock> IREmitter::CreateNewCodeBlockAfter(OrderedNode
if (insertAfter) {
LinkCodeBlocks(insertAfter, CodeNode);
} else {
LOGMAN_THROW_A(CurrentCodeBlock != nullptr, "CurrentCodeBlock must not be null here");
LOGMAN_THROW_A_FMT(CurrentCodeBlock != nullptr, "CurrentCodeBlock must not be null here");
// Find last block
auto LastBlock = CurrentCodeBlock;
@@ -101,7 +109,7 @@ IREmitter::IRPair<IROp_CodeBlock> IREmitter::CreateNewCodeBlockAfter(OrderedNode
void IREmitter::SetCurrentCodeBlock(OrderedNode *Node) {
CurrentCodeBlock = Node;
LOGMAN_THROW_A(Node->Op(DualListData.DataBegin())->Op == OP_CODEBLOCK, "Node wasn't codeblock. It was '%s'", std::string(IR::GetName(Node->Op(DualListData.DataBegin())->Op)).c_str());
LOGMAN_THROW_A_FMT(Node->Op(DualListData.DataBegin())->Op == OP_CODEBLOCK, "Node wasn't codeblock. It was '{}'", IR::GetName(Node->Op(DualListData.DataBegin())->Op));
SetWriteCursor(Node->Op(DualListData.DataBegin())->CW<IROp_CodeBlock>()->Begin.GetNode(DualListData.ListBegin()));
}
+17 -10
View File
@@ -5,16 +5,24 @@ tags: ir|parser
$end_info$
*/
#include <string>
#include <vector>
#include <istream>
#include <unordered_map>
#include <FEXCore/IR/IR.h>
#include <FEXCore/IR/IntrusiveIRList.h>
#include <FEXCore/IR/IREmitter.h>
#include <FEXCore/Utils/LogManager.h>
#include <algorithm>
#include <array>
#include <cstdint>
#include <errno.h>
#include <memory>
#include <stdio.h>
#include <stdlib.h>
#include <string>
#include <string_view>
#include <utility>
#include <vector>
#include <istream>
#include <unordered_map>
namespace FEXCore::IR {
namespace {
@@ -70,8 +78,6 @@ std::string DecodeErrorToString(DecodeFailure Failure) {
return "Unknown Error";
}
std::unordered_map<std::string_view, FEXCore::IR::IROps> NameToOpMap;
class IRParser: public FEXCore::IR::IREmitter {
public:
template<typename Type>
@@ -299,9 +305,10 @@ class IRParser: public FEXCore::IR::IREmitter {
std::unordered_map<std::string, OrderedNode*> SSANameMapper;
std::vector<LineDefinition> Defs;
LineDefinition *CurrentDef{};
std::unordered_map<std::string_view, FEXCore::IR::IROps> NameToOpMap;
IRParser(std::istream *text) {
InitializeStaticTables();
InitializeNameMap();
std::string TmpLine;
while (!text->eof()) {
@@ -630,7 +637,7 @@ class IRParser: public FEXCore::IR::IREmitter {
return true;
}
void InitializeStaticTables() {
void InitializeNameMap() {
if (NameToOpMap.empty()) {
for (FEXCore::IR::IROps Op = FEXCore::IR::IROps::OP_DUMMY;
Op <= FEXCore::IR::IROps::OP_LAST;
+1
View File
@@ -13,6 +13,7 @@ $end_info$
#include <FEXCore/Config/Config.h>
namespace FEXCore::IR {
class IREmitter;
void PassManager::AddDefaultPasses(bool InlineConstants, bool StaticRegisterAllocation) {
FEX_CONFIG_OPT(DisablePasses, O0);
+4 -4
View File
@@ -7,11 +7,10 @@ $end_info$
#pragma once
#include <FEXCore/Config/Config.h>
#include <FEXCore/IR/IntrusiveIRList.h>
#include <FEXCore/IR/IREmitter.h>
#include <functional>
#include <memory>
#include <utility>
#include <vector>
namespace FEXCore::HLE {
@@ -19,8 +18,9 @@ class SyscallHandler;
}
namespace FEXCore::IR {
class OpDispatchBuilder;
class SyscallOptimization;
class PassManager;
class IREmitter;
class RegisterAllocationPass;
using ShouldExitHandler = std::function<void(void)>;
Loaded 100 of 288 files, more files were not shown because too many files have changed in this diff. Show more