Compare commits

..
382 Commits
Author SHA1 Message Date
Ryan Houdek 70d4a436cf Docs: Update for release FEX-2301 2023-01-06 07:53:17 -08:00
Ryan Houdek ec55ecdb31 Merge pull request #2290 from Sonicadvance1/new_arm_emitter
Create a new ARM64 Emitter and move JIT over to it.
2023-01-05 13:50:25 -08:00
Tony Wasserka 12b866c276 Merge pull request #2308 from neobrain/refactor_thunkdb_loading
ThunkDB: Clean up database loading
2023-01-05 14:56:35 +01:00
Tony Wasserka 1038ba7060 ThunksDB: Disable error message in 32-bit mode 2023-01-05 14:45:13 +01:00
Tony Wasserka 9bde513161 Thunks: Simplify state carried around while setting up ThunkOverlays 2023-01-05 12:15:09 +01:00
Tony Wasserka 2e3f77c43f ThunksDB: Clean up initial DB loading 2023-01-05 12:15:09 +01:00
Tony Wasserka 0d8de6463b Thunks: Move LoadThunkDatabase out of the header file 2023-01-05 12:15:08 +01:00
Tony Wasserka 98349ee485 ThunksDB: Clean up string replacement logic 2023-01-05 12:15:08 +01:00
Tony Wasserka e486833d75 Merge pull request #2307 from neobrain/fix_thunkdb_libnames
ThunksDB: Fix misspelt guest library names
2023-01-04 15:43:51 +01:00
Ryan Houdek 8a38999c7a RAData: Fixes uninitialized members 2023-01-04 05:30:01 -08:00
Ryan Houdek 2d0b61fd9d Emitter: Adds unit tests
Every operation that the emitter supports is tested in the unit tests.
For the most part uses vixl's dissassembler and a string comparison to
ensure the emitter is outputing what we expect.

For operations that are PC-relative we instead use a bit-exact test to
ensure it is outputting what we care about. This is because vixl
helpfully outputs the PC that the operation is acting on. Since these
locations aren't static in memory, the PC moves around per test.
2023-01-04 05:30:01 -08:00
Ryan Houdek ffd9bb547d Arm64: Convert ARM Emitter over to new emitter
Not yet complete. Missing a full SVE implementation and needs
testing/validation.
2023-01-04 05:30:01 -08:00
Ryan Houdek 7a6ef8821f Arm64: Adds new ARM emitter
Still needs more work.
Missing operations, cleanup, validation
Notably SVE is missing large chunks.
2023-01-04 05:30:01 -08:00
Tony Wasserka 64387bf00d Thunks: Make failure to find a guest thunk library a critical error 2023-01-03 18:15:50 +01:00
Tony Wasserka 8cc7e55394 ThunksDB: Fix misspelt guest library names 2023-01-03 18:15:50 +01:00
Ryan Houdek 874c1da1b5 External: Update vixl 2023-01-02 01:38:17 -08:00
Ryan Houdek 3904a5264f Merge pull request #2306 from lioncash/perm
OpcodeDispatcher: Handle immediate variants of VPERMILPD/VPERMILPS
2022-12-31 21:31:59 -08:00
lioncash b95c1719c3 OpcodeDispatcher: Handle VPERMILPS (immediate) 2023-01-01 05:15:25 +00:00
lioncash dfb3f31453 OpcodeDispatcher: Handle VPERMILPD (immediate) 2023-01-01 04:59:48 +00:00
Ryan Houdek c6297edac0 Merge pull request #2305 from lioncash/mask
OpcodeDispatcher: Handle VMASKMOVDQU
2022-12-31 20:31:20 -08:00
lioncash 8031f76642 OpcodeDispatcher: Handle VMASKMOVDQU 2023-01-01 04:18:46 +00:00
Ryan Houdek 4786ddc44c Merge pull request #2304 from lioncash/sub
OpcodeDispatcher: Handle VPHSUBD/VPHSUBW
2022-12-31 19:23:43 -08:00
lioncash ae8a5fa98d OpcodeDispatcher: Handle VPHSUBD 2023-01-01 03:10:21 +00:00
lioncash 6914598f9a OpcodeDispatcher: Handle VPHSUBW 2023-01-01 02:42:05 +00:00
lioncash 450aedc8b6 x86_64: Fix 256-bit UnZip/UnZip2
We weren't swapping the elements so that the operation acts like the two
vectors are concatenated.
2023-01-01 02:42:05 +00:00
lioncash 1e221210a2 OpcodeDispatcher: Move PHSUB impl to helper function
This will be used for the AVX variants.
2023-01-01 02:42:03 +00:00
Ryan Houdek 58ec2b2d7f Merge pull request #2303 from lioncash/swizz
OpcodeDispatcher: Zip elements instead of for loop insertion in PHSUB
2022-12-31 16:09:13 -08:00
lioncash 438adf2f45 x86_64/VectorOps: Handle OpSize==8 case in VUnZip/VUnZip2
Allows the x86 side of things to execute the new codepath in PHSUB
2022-12-31 23:30:59 +00:00
lioncash 9707e9a4df OpcodeDispatcher: Zip elements instead of for loop in PHSUB
Makes this much nicer for 128-bit and soon-to-be 256-bit vectors.
2022-12-31 23:30:38 +00:00
Ryan Houdek 9b8c92e275 Merge pull request #2302 from lioncash/dpp
OpcodeDispatcher: Handle VDPPD/VDPPS
2022-12-31 15:01:52 -08:00
Ryan Houdek 6caf764b7c Merge pull request #2301 from lioncash/insps
OpcodeDispatcher: Handle VINSERTPS
2022-12-31 14:59:11 -08:00
Ryan Houdek faa81f241b Merge pull request #2300 from lioncash/msk
OpcodeDispatcher: Handle VMOVMSKPD/VMOVMSKPS
2022-12-31 14:34:44 -08:00
lioncash 769c548ba4 OpcodeDispatcher: Handle VDPPD
x86 just doesn't have a 256-bit version of this op.
2022-12-31 21:13:20 +00:00
lioncash dae1676e4a OpcodeDispatcher: Handle VDPPS 2022-12-31 21:01:48 +00:00
lioncash 74526d1f02 OpcodeDispatcher: Move DPP op impl into helper function
This will be used to implement the AVX variants.
2022-12-31 21:01:26 +00:00
lioncash 8c005db81c OpcodeDispatcher: Handle VINSERTPS 2022-12-31 20:23:22 +00:00
lioncash 32b70c8590 OpcodeDispatcher: Move InsertPS impl to helper function
This will be reused for VINSERTPS
2022-12-31 20:23:18 +00:00
lioncash bdda14eb75 OpcodeDispatcher: Handle VMOVMSKPD 2022-12-31 19:24:14 +00:00
lioncash b8c0b0c267 OpcodeDispatcher: Handle VMOVMSKPS 2022-12-31 19:24:11 +00:00
lioncash ad7dc6ca0a OpcodeDispatcher: Move ADDSUB table entries
Makes them numerically ordered.
2022-12-31 19:07:19 +00:00
Ryan Houdek 64cd377e37 Merge pull request #2299 from lioncash/ph
OpcodeDispatcher: Handle VPUNPCKHBW/VPUNPCKHWD/VPUNPCKHDQ/VPUNPCKHQDQ
2022-12-30 17:18:14 -08:00
lioncash 45d7564716 OpcodeDispatcher: Handle VPUNPCKHBW 2022-12-30 14:07:30 +00:00
lioncash c585bae85d OpcodeDispatcher: Handle VPUNPCKHWD 2022-12-30 14:01:18 +00:00
lioncash d07383fa73 OpcodeDispatcher: Handle VPUNPCKHQDQ 2022-12-30 13:54:45 +00:00
lioncash e1cdcf0651 OpcodeDispatcher: Handle VPUNPCKHDQ 2022-12-30 13:46:53 +00:00
Ryan Houdek 138f1fc844 Merge pull request #2298 from lioncash/hps
OpcodeDispatcher: Handle VUNPCKHPD/VUNPCKHPS
2022-12-30 05:31:48 -08:00
lioncash ae69aa9a81 OpcodeDispatcher: Handle VUNPCKHPD 2022-12-30 13:18:33 +00:00
lioncash 6341ac6814 OpcodeDispatcher: Handle VUNPCKHPS 2022-12-30 13:05:01 +00:00
Ryan Houdek 6bc1c3fc30 Merge pull request #2297 from lioncash/lps
OpcodeDispatcher: Handle VPUNPCKLBW/VPUNPCKLWD/VPUNPCKLDQ/VPUNPCKLQDQ
2022-12-29 14:49:11 -08:00
lioncash d91f2ed6b0 OpcodeDispatcher: Handle VPUNPCKLBW 2022-12-29 16:36:45 +00:00
lioncash 7b30a241c0 OpcodeDispatcher: Handle VPUNPCKLWD 2022-12-29 16:30:12 +00:00
lioncash aaf8e3757d OpcodeDispatcher: Handle VPUNPCKLQDQ 2022-12-29 16:21:56 +00:00
lioncash d9c49c4ce1 OpcodeDispatcher: Handle VPUNPCKLDQ 2022-12-29 16:19:09 +00:00
Ryan Houdek 4560c5b73c Merge pull request #2296 from lioncash/lps
OpcodeDispatcher: Handle VUNPCKLPD/VUNPCKLPS
2022-12-29 08:01:23 -08:00
lioncash 9d05c8a67b OpcodeDispatcher: Handle VUNPCKLPD 2022-12-29 15:12:36 +00:00
lioncash be9578551a OpcodeDispatcher: Handle VUNPCKLPS 2022-12-29 15:00:13 +00:00
Ryan Houdek 4a884802f8 Merge pull request #2295 from lioncash/cvt
OpcodeDispatcher: Handle VCVTSS2SI/VCVTTSS2SI/VCVTSD2SI/VCVTTSD2SI
2022-12-29 03:14:54 -08:00
lioncash 75a01ed2b6 OpcodeDispatcher: Handle VCVTTSD2SI 2022-12-29 10:49:03 +00:00
lioncash 3ebe141032 OpcodeDispatcher: Handle VCVTSD2SI 2022-12-29 10:43:10 +00:00
lioncash 31e332bd61 OpcodeDispatcher: Handle VCVTTSS2SI 2022-12-29 10:26:19 +00:00
lioncash 764324d557 OpcodeDispatcher: Handle VCVTSS2SI 2022-12-29 10:16:12 +00:00
Ryan Houdek f37938576d Merge pull request #2292 from lioncash/cvt
OpcodeDispatcher: Handle VCVTPD2DQ/VCVTTPD2DQ/VCVTPS2DQ/VCVTTPS2DQ
2022-12-28 17:16:46 -08:00
Ryan Houdek 16969fcdad Merge pull request #2293 from neobrain/fix_thunks_ide_integration
Thunks: Fix IDE integration
2022-12-28 17:14:10 -08:00
Tony Wasserka 0dfe141d70 Thunks: Fix IDE integration 2022-12-28 17:45:05 +01:00
lioncash 2a7795fe2c OpcodeDispatcher: Handle VCVTTPD2DQ 2022-12-28 12:01:53 +00:00
lioncash b00b41b8fa OpcodeDispatcher: Handle VCVTPD2DQ 2022-12-28 11:54:28 +00:00
lioncash 39396789b1 Interpreter/ConversionOps: Prevent out of bounds/excessive element handling in Vector_FToF
Previously this could access more elements than it needs to.

In the case where we pass in a 128-bit vector and perform a 128-bit
operation, only two doubles should be handled, but in this case will
actually try and handle four double elements.

Consider converting doubles within a vector into floats:

e.g. _Vector_FToF(16, 4, Src, 8)

16 is our OpSize
4 is our DestElementSize
Src is our input vector
8 is the SrcElementSize

(OpSize << 1) / Op->SrcElementSize becomes:
(16 << 1) / 8 ->
32 / 8 ->
4

What we actually want it 2 here, we're indexing 128-bit vectors out of
its bounds (not a problem now, since we assume up to 256-bit in the
interpreter).

Similarly with 256-bit vectors

_Vector_FToF(32, 4, Src, 8)

would become:

64 / 8 -> 8

where we actually want 4, since we only have 4 64-bit elements in a
256-bit vector.

This corrects this in the interpreter by just special-casing the 64-bit
calculation.
2022-12-28 11:53:29 +00:00
lioncash 38a2886a59 OpcodeDispatcher: Handle VCVTTPS2DQ 2022-12-28 07:54:16 +00:00
lioncash bd8e1a80f6 OpcodeDispatcher: Handle VCVTPS2DQ 2022-12-28 07:45:37 +00:00
Ryan Houdek b7358b4926 Merge pull request #2261 from Sonicadvance1/optimize_lookup_pmr_map
LookupCache: Use a PMR map for our Blocklinks with monotonic allocator
2022-12-23 09:55:28 -08:00
Ryan Houdek 0e0f3f9290 LookupCache: Use a PMR map for our Blocklinks with monotonic allocator
We generate a /lot/ of block links which causes our shutdown time to
take a while for this map. For short running applications this shutdown
time can take a statistically significant amount of time, and this has
been haunting us for a while.

Using a monotonic buffer resource, we can cut the dellocation time down
to effectively zero on cache clear and shutdown.
For short running applications this basically means we clear their
shutdown time, for long running applications this means a cache clear is
less painful.

I've measured up to 2ms , but more often this takes ~0.5ms before
optimization. Now it is consistently ~0.25ms on a short running
application.
Times will vary /greatly/ depending on how full it has filled this map.
2022-12-23 09:41:54 -08:00
lioncash dfa113dcdb OpcodeDispatcher: Move Vector_CVT_Float_To_Int to helper function
This can be reused for the AVX variant.
2022-12-23 17:33:15 +00:00
Mai bf7d0f7ed9 Merge pull request #2287 from Sonicadvance1/rename_getreg
Arm64: Rename GetSrcPair, GetDst, and GetSrc
2022-12-22 08:40:10 +00:00
Ryan Houdek 82adc2f931 Merge pull request #2288 from lioncash/hrsw
OpcodeDispatcher: Handle VPMULHRSW
2022-12-22 00:38:45 -08:00
lioncash 94cb2ddae7 OpcodeDispatcher: Handle VPMULHRSW 2022-12-22 08:21:35 +00:00
lioncash 0496f8d5c1 OpcodeDispatcher: Move PMULHRSW impl to helper 2022-12-22 08:13:43 +00:00
Ryan Houdek 4a3af8d7f9 Merge pull request #2286 from lioncash/mulhw
OpcodeDispatcher: Handle VPMULHW/VPMULHUW
2022-12-22 00:06:53 -08:00
Ryan Houdek 37a9588855 Arm64: Rename GetSrcPair to GetRegPair
This matches the other handler's names and isn't only for source
registers, but also destination registers.
2022-12-22 00:03:41 -08:00
Ryan Houdek 24f72b5f30 Arm64: Rename GetDst & GetSrc to GetVReg
These only resolved to getting vector registers now and having
duplicating handlers for it was just confusing.
2022-12-21 23:59:21 -08:00
lioncash d927c4a903 OpcodeDispatcher: Handle VPMULHUW 2022-12-22 07:52:27 +00:00
lioncash 12afe95602 OpcodeDispatcher: Handle VPMULHW 2022-12-22 07:52:24 +00:00
lioncash 7e715b9e04 OpcodeDispatcher: Move PMULHW impl to helper
This can be used with AVX variants.
2022-12-22 07:33:26 +00:00
Ryan Houdek 9d58514f57 Merge pull request #2285 from lioncash/phmin
OpcodeDispatcher: Handle VPHMINPOSUW
2022-12-21 19:02:35 -08:00
lioncash 4f9402e5dd OpcodeDispatcher: Handle VPHMINPOSUW 2022-12-22 02:46:08 +00:00
lioncash 789093d158 OpcodeDispatcher: Move VPHMINPOSUW impl to separate function
This will be used with the AVX variant.
2022-12-22 02:38:31 +00:00
Ryan Houdek 33e8f21ac7 Merge pull request #2284 from lioncash/pmull
OpcodeDispatcher: Handle VPMULDQ/VPMULUDQ
2022-12-21 18:33:02 -08:00
lioncash d672528e62 OpcodeDispatcher: Handle VPMULUDQ 2022-12-22 02:18:37 +00:00
lioncash d7c959090d OpcodeDispatcher: Handle VPMULDQ 2022-12-22 02:08:53 +00:00
lioncash 0a4846a524 OpcodeDispatcher: Factor PMULLOp impl to regular function
This can be reused for the AVX implementations.
2022-12-22 01:30:06 +00:00
Ryan Houdek cecda7bbb6 Merge pull request #2283 from lioncash/cmpss
OpcodeDispatcher: Handle VCMPSD/VCMPSS
2022-12-21 16:58:01 -08:00
lioncash 2d9cb65d5c OpcodeDispatcher: Handle VCMPSD 2022-12-21 20:46:07 +00:00
lioncash e21002e0d7 OpcodeDispatcher: Handle VCMPSS 2022-12-21 20:39:13 +00:00
Ryan Houdek ce351282f2 Merge pull request #2282 from lioncash/debug
OpcodeDispatcher: Remove lingering debug log from VPFCMPOp
2022-12-21 00:53:13 -08:00
lioncash ab41856328 OpcodeDispatcher: Remove lingering debug log from VPFCMPOp 2022-12-21 08:40:27 +00:00
Ryan Houdek 345e9b97bf Merge pull request #2281 from lioncash/assert
OpcodeDispatcher: Convert runtime assert to static_assert in SHUFOps
2022-12-21 00:23:24 -08:00
Ryan Houdek 0c651dd5f8 Merge pull request #2280 from lioncash/cmp
OpcodeDispatcher: Handle VCMPPD/VCMPPS
2022-12-21 00:10:14 -08:00
lioncash f850a02d3f OpcodeDispatcher: Convert runtime assert to static_assert in SHUFOps 2022-12-21 08:05:32 +00:00
lioncash 983b53a0c2 OpcodeDispatcher: Handle VCMPPD 2022-12-21 06:12:14 +00:00
lioncash 10a6b5794b OpcodeDispatcher: Handle VCMPPS 2022-12-21 05:57:24 +00:00
lioncash 2c5aceb9b6 OpcodeDispatcher: Factor out VFCMPOp impl into a regular function
This can be used with AVX implementations.
2022-12-21 05:27:10 +00:00
Ryan Houdek 1668db046d Merge pull request #2279 from lioncash/psrldq
OpcodeDispatcher: Handle VPSRLDQ
2022-12-20 21:06:57 -08:00
lioncash 825e921940 OpcodeDispatcher: Handle VPSRLDQ 2022-12-21 04:51:49 +00:00
Ryan Houdek 515b3e485b Merge pull request #2278 from lioncash/shift
OpcodeDispatcher: Remove unnecessary usage of VMov in VPSLLDQOp
2022-12-20 20:30:08 -08:00
lioncash c06f0b7cb3 OpcodeDispatcher: Remove usage of VMov in VPSLLDQOp
The extract operation essentially does this for us.
2022-12-21 04:13:26 +00:00
Ryan Houdek 4aed60ee3d Merge pull request #2277 from lioncash/cvt
OpcodeDispatcher: Handle VCVTDQ2PD/VCVTDQ2PS
2022-12-20 20:02:12 -08:00
lioncash da9f7ec31f OpcodeDispatcher: Handle VCVTDQ2PD 2022-12-21 03:45:00 +00:00
lioncash 8ec932fc4c OpcodeDispatcher: Handle VCVTDQ2PS 2022-12-21 03:44:56 +00:00
lioncash 6f35a23161 x86_64/ConversionOps: Don't clear upper lane in Vector_SToF
If Dst and Vector are the same, this will obliterate the top lane data
that needs to be operated on in the 128-bit case.
2022-12-21 03:40:38 +00:00
lioncash bd70af9724 OpcodeDispatcher: Move Vector_CVT_Int_To_Float impl to regular function
This can be used with the AVX implementation as well.
2022-12-21 02:23:04 +00:00
lioncash bdba062f72 VEXTables: Amend VCVTTPS2DQ name 2022-12-21 02:17:31 +00:00
Ryan Houdek 60a2fb163c Merge pull request #2276 from lioncash/lldq
OpcodeDispatcher: Handle VPSLLDQ
2022-12-20 18:12:15 -08:00
lioncash d591b1ed8c Interpreter: Prevent overrun with 256-bit VExtr 2022-12-21 01:51:25 +00:00
lioncash 3bae4a225c OpcodeDispatcher: Handle VPSLLDQ 2022-12-21 01:46:17 +00:00
Ryan Houdek d0cb329608 Merge pull request #2275 from lioncash/sha1
OpcodeDispatcher: Simplify SHA1MSG1 implementation
2022-12-20 12:49:41 -08:00
lioncash bb80e7d45c OpcodeDispatcher: Simplify SHA1MSG1 implementation
We can just arrange the elements into a vector and XOR them all at once
2022-12-20 19:49:08 +00:00
Ryan Houdek 72a3b18279 Merge pull request #2274 from lioncash/right
OpcodeDispatcher: Handle immediate variants of VPSRLD/VPSRLQ/VPSRLW
2022-12-20 11:38:13 -08:00
lioncash 109ed7d112 OpcodeDispatcher: Handle VPSRLQ (immediate) 2022-12-20 19:24:58 +00:00
lioncash 133a644231 OpcodeDispatcher: Handle VPSRLD (immediate) 2022-12-20 19:12:16 +00:00
lioncash 666f8bfbd9 OpcodeDispatcher: Handle VPSRLW (immediate) 2022-12-20 19:07:47 +00:00
Ryan Houdek 1800451251 Merge pull request #2273 from lioncash/keygen
OpcodeDispatcher: Handle 128-bit AVX AES instructions
2022-12-20 10:52:53 -08:00
Ryan Houdek 1d9218224f Merge pull request #2272 from lioncash/psra
OpcodeDispatcher: Handle immediate variants of VPSRAD/VPSRAW
2022-12-20 10:50:22 -08:00
lioncash 7931bd1004 OpcodeDispatcher: Handle VAESDECLAST (128-bit) 2022-12-20 17:34:17 +00:00
lioncash 58978dd047 OpcodeDispatcher: Handle VAESDEC (128-bit) 2022-12-20 17:26:19 +00:00
lioncash 25fb243ac7 OpcodeDispatcher: Handle VAESENCLAST (128-bit) 2022-12-20 17:11:47 +00:00
lioncash 84f1e7ad4c OpcodeDispatcher: Handle VAESENC (128-bit)
Only 128-bit is required to be handled by base-level AVX.

The VAES feature flag indicates support for 256-bit VAESENC
2022-12-20 16:58:01 +00:00
lioncash 3fb5835453 OpcodeDispatcher: Handle VAESIMC
VAESIMC behaves exactly like AESIMC, except the upper lane of the vector
is always cleared.
2022-12-20 16:49:03 +00:00
lioncash bcb6726b22 OpcodeDispatcher: Handle VAESKEYGENASSIST
This does the exact same thing as AESKEYGENASSIST, except that the upper
lane gets cleared.
2022-12-20 16:15:42 +00:00
lioncash 2bed562eb6 OpcodeDispatcher: Extract AESKeyGenAssist impl to helper function
This can be reused for the AVX variant.
2022-12-20 16:01:25 +00:00
lioncash bae7209224 OpcodeDispatcher: Handle VPSRAD (immediate) 2022-12-20 15:49:38 +00:00
lioncash b53f8944ac OpcodeDispatcher: Handle VPSRAW (immediate) 2022-12-20 15:40:49 +00:00
Mai 03a061339a Merge pull request #2269 from Sonicadvance1/vixl_disassembler
Arm64: Enables debug option for disassembling the JIT code
2022-12-20 01:25:51 +00:00
Ryan Houdek 0d7c086b69 Arm64: Enables debug option for disassembling the JIT code
This is useful as a debug option and will be useful to have in upstream
while comparing output between current vixl emitter and the new emitter.

With this in place I can easily do binary comparisons to see where I
have mistakes in the new emitter.

We don't want this enabled in release builds as it is a debug feature.
This has already caught a bunch of mistakes, so make it easier by
upstreaming.
It'll likely be useful in the future as well when we are inspecting code
running in the vixl simulator.
2022-12-18 14:56:40 -08:00
Ryan Houdek b958fa39a5 External: Update vixl 2022-12-18 14:54:52 -08:00
Mai 2b6a020c4c Merge pull request #2260 from Sonicadvance1/optimize_lookup_map
LookupCache: Optimize cache clearing and allocation
2022-12-17 22:43:22 +00:00
Ryan Houdek 6e733bfc22 Merge pull request #2268 from lioncash/upack
OpcodeDispatcher: Handle VPACKUSDW/VPACKUSWB
2022-12-16 20:00:54 -08:00
lioncash 873d63002a OpcodeDispatcher: Handle VPACKUSDW 2022-12-17 03:43:06 +00:00
lioncash bb6a0f39f5 OpcodeDispatcher: Handle VPACKUSWB 2022-12-17 03:31:40 +00:00
lioncash 392e6ae424 OpcodeDispatcher: Factor out PACKUSOp impl into a regular function
We can use this for the AVX instructions too.
2022-12-17 03:19:28 +00:00
Ryan Houdek 01d22849cf Merge pull request #2267 from lioncash/pack
OpcodeDispatcher: Handle VPACKSSDW/VPACKSSWB
2022-12-16 19:16:28 -08:00
lioncash 0537f2d014 OpcodeDispatcher: Handle VPACKSSDW 2022-12-17 03:01:33 +00:00
lioncash f57debeb29 OpcodeDispatcher: Handle VPACKSSWB 2022-12-17 02:42:09 +00:00
lioncash 4ac031df59 OpcodeDispatcher: Move PACKSSOp impl to a regular function
We can reuse it with AVX versions.
2022-12-17 02:13:26 +00:00
Ryan Houdek 78b53bfa49 Merge pull request #2266 from lioncash/arith
OpcodeDispatcher: Handle vector versions of VPSRA{D, W}
2022-12-16 18:05:05 -08:00
lioncash c53fb7d697 OpcodeDispatcher: Handle VPSRAD (vector) 2022-12-17 01:51:42 +00:00
lioncash a1a52450cb OpcodeDispatcher: Handle VPSRAW (vector) 2022-12-17 01:40:25 +00:00
Ryan Houdek fabf453046 Merge pull request #2265 from lioncash/pextrw
OpcodeDispatcher: Handle remaining PEXTRW opcode
2022-12-16 17:35:33 -08:00
lioncash 68916ae2d9 OpcodeDispatcher: Move PSRAOp implementation to regular function
We can reuse this with the AVX variant.
2022-12-17 01:23:02 +00:00
lioncash bf56b7b2da OpcodeDispatcher: Handle remaining PEXTRW opcode 2022-12-17 01:14:22 +00:00
Ryan Houdek 905eb015c0 Merge pull request #2264 from lioncash/addsub
OpcodeHandler: Handle VADDSUBP{D, S}
2022-12-16 16:54:35 -08:00
lioncash 858f13e76a OpcodeDispatcher: Handle VADDSUBPD 2022-12-17 00:41:25 +00:00
lioncash 169d7bbf50 OpcodeDispatcher: Handle VADDSUBPS 2022-12-17 00:29:29 +00:00
lioncash 31c8d4acac OpcodeDispatcher: Factor out ADDSUB impl into regular function
We can reuse this with the AVX versions
2022-12-17 00:16:38 +00:00
lioncash 8291e600fa OpcodeDispatcher: Simplify ADDSUBPOp
Rather than looping vectors, we can interleave them together directly
with IR ops.
2022-12-17 00:11:51 +00:00
Ryan Houdek b26e4109fa Merge pull request #2263 from lioncash/mull
OpcodeDispatcher: Handle VPMULL{D, B}
2022-12-16 15:40:53 -08:00
lioncash dcc218a168 OpcodeDispatcher: Handle VPMULLD 2022-12-16 23:20:25 +00:00
lioncash 49b9b18b4a OpcodeDispatcher: Handle VPMULLW 2022-12-16 23:07:06 +00:00
Ryan Houdek ad3bf189c0 Merge pull request #2262 from lioncash/rlog
OpcodeDispatcher: Handle vector variants of VPSRL{D, Q, W}
2022-12-16 14:56:56 -08:00
lioncash 47b21fa758 OpcodeDispatcher: Handle VPSRLQ (vector)
Also mark VPMOVMSKB as UNDEC, since it's not implemented yet.
2022-12-16 22:18:40 +00:00
lioncash b6e82965df OpcodeDispatcher: Handle VPSRLD (vector) 2022-12-16 22:09:52 +00:00
lioncash 8dc8785340 OpcodeDispatcher: Handle VPSRLW (vector) 2022-12-16 22:00:53 +00:00
lioncash c710ab60b0 OpcodeDispatcher: Factor out PSRLDOp implementation to regular function
This will be used with the AVX variants of the shifts also
2022-12-16 21:43:11 +00:00
Ryan Houdek c86ba7646c Merge pull request #2259 from lioncash/pextr
OpcodeDispatcher: Handle VPEXTR{B, D, Q, W}/VEXTRACTPS
2022-12-16 11:25:38 -08:00
Ryan Houdek c1e301a5ed Merge pull request #2257 from lioncash/limm
OpcodeDispatcher: Handle immediate variants of VPSLL{D, Q, W}
2022-12-16 11:23:16 -08:00
Ryan Houdek cad0dc6848 LookupCache: Optimize cache clearing and allocation
Use one large allocation for all levels of the cache so they are
virtually contiguous.
This allows us to clear the cache entirely by using a single madvise
instead of three. Which ends up being quite a bit nicer.
2022-12-16 11:08:39 -08:00
Mai 0ebb15c732 Merge pull request #2258 from Sonicadvance1/fixed_syscall_spill
Arm64: Inline Syscall spill optimization
2022-12-16 18:48:08 +00:00
lioncash 37c743b616 OpcodeDispatcher: Handle VPSLLQ (immediate) 2022-12-16 18:37:27 +00:00
lioncash c810ae4018 OpcodeDispatcher: Handle VPSLLD (immediate) 2022-12-16 18:37:27 +00:00
lioncash d3481c8271 OpcodeDispatcher: Handle VPSLLW (immediate) 2022-12-16 18:37:27 +00:00
lioncash 7c1e152441 OpcodeDispatcher: Extract PSLLI impl to regular function
This will be reused for the AVX variants.
2022-12-16 18:37:20 +00:00
lioncash f11ac8674d OpcodeDispatcher: Handle VEXTRACTPS 2022-12-16 18:13:55 +00:00
Ryan Houdek 1fecf89bfc Arm64: Inline Syscall spill optimization
This was likely an issue with signals racing to the spill handler, which
we have fixed bugs with over the past few months.

This means we don't need to spill all SRA GPR registers anymore, at most
we need to spill three registers that intersect with syscall arguments.
2022-12-16 10:04:16 -08:00
lioncash 21ad0fa334 OpcodeDispatcher: Handle VPEXTRQ
VPEXTRQ uses VEX.W to handle size differencing, since it shares an
encoding spot with VPEXTRD, so we need to handle that a little
differently.
2022-12-16 18:02:24 +00:00
lioncash 3429815103 OpcodeDispatcher: Handle VPEXTRD 2022-12-16 17:33:01 +00:00
lioncash 559ff1582e OpcodeDispatcher: Handle VPEXTRW 2022-12-16 17:29:16 +00:00
lioncash 2e973ae079 OpcodeDispatcher: Handle VPEXTRB 2022-12-16 14:37:47 +00:00
Mai 1ab4471ef9 Merge pull request #2255 from Sonicadvance1/optimize_sve_spillfill
Arm64: Optimize SVE register spilling and filling
2022-12-16 13:19:05 +00:00
Ryan Houdek 40e073c8b2 Arm64: Optimize SVE register spilling and filling
Causes the dispatcher to drop from 4476 bytes down to 3900 for
SVE-256bit supporting targets.

This is done by significantly reducing SVE loadstore ops. Going from 8
instructions per 4 registers, down to 2 instructions.

This is done by switching from 1 register loadstore instructions up to 4
register loadstore instructions. Which should significantly improve
performance on future SVE platforms.

Filling and Spilling to the context is still using the old code path
because SVE doesn't offer non-interleaving loadstores.
Spilling and filling on the stack is fine because we don't need to match
context state.
2022-12-16 00:25:50 -08:00
Ryan Houdek 58fab721b3 Merge pull request #2254 from lioncash/logical
OpcodeDispatcher: Handle vector variants of VPSLL{D, Q, W}
2022-12-15 22:52:05 -08:00
lioncash 8fac21e43f OpcodeDispatcher: Handle VPSLLQ (vector) 2022-12-16 06:34:00 +00:00
lioncash d9a1e97bc1 OpcodeDispatcher: Handle VPSLLD (vector) 2022-12-16 06:34:00 +00:00
lioncash 848f1a2f78 OpcodeDispatcher: Handle VPSLLW (vector) 2022-12-16 06:34:00 +00:00
lioncash 7b8a46d934 OpcodeDispatcher: Move PSLL impl into a regular function 2022-12-16 06:33:58 +00:00
Mai 9a8852f9b6 Merge pull request #2250 from Sonicadvance1/optimize_spilling_filling
Arm64: Optimizing spilling and filling
2022-12-16 04:47:22 +00:00
Mai 65e8bf9d72 Merge pull request #2253 from Sonicadvance1/single_page_dispatcher
Arm64: Reduce dispatcher to 1 page
2022-12-16 04:44:55 +00:00
Ryan Houdek 344ec33ba5 Merge pull request #2252 from lioncash/fadd
Arm64/VectorOps: Simplify FADDP result merging
2022-12-15 20:37:11 -08:00
Ryan Houdek 5dc7dfacb3 Arm64: Reduce dispatcher to 1 page
We currently only use 2236 bytes, no need for two pages.
Once #2250 is merged we will use 1716 bytes
2022-12-15 20:33:28 -08:00
lioncash 122aa8a69a Arm64/VectorOps: Simplify FADDP result merging
Keeps the implementation similarly in sync with VAddP.
2022-12-16 04:19:46 +00:00
Ryan Houdek 8ce6c08152 Merge pull request #2251 from lioncash/hadd
OpcodeDispatcher: Handle VPHADDW/VPHADDD
2022-12-15 20:11:07 -08:00
Ryan Houdek 1beb791d52 Arm64: Optimizing spilling and filling
Just makes these a little more optimal when jumping out of the JIT.

Noticed these while working on the new emitter.
2022-12-15 20:04:16 -08:00
lioncash 27c0d4a9f5 OpcodeDispatcher: Handle VPHADDD 2022-12-16 03:28:57 +00:00
lioncash dd4ba7562f OpcodeDispatcher: Handle VPHADDW 2022-12-16 03:28:57 +00:00
lioncash bd9d8e8fe5 x86_64: Correct handling for 128-bit/256-bit VAddP
Makes the behavior consistent with the ARM JIT.
2022-12-16 03:28:57 +00:00
lioncash c7ac204322 Arm64/VectorOps: Simplify VAddP merging operation
We can just merge the two results together instead of shifting to the
left and then ORing together.
2022-12-16 03:28:52 +00:00
Ryan Houdek 4c013c867f Merge pull request #2249 from lioncash/clear
Crypto: Explicitly clear upper lane with VPCLMULQDQ
2022-12-15 17:33:57 -08:00
lioncash 5e634fcbc9 Crypto: Explicitly clear upper lane with VPCLMULQDQ
Ensures the 128-bit case will be handled when extending for 256-bit
2022-12-16 01:08:48 +00:00
Ryan Houdek 91c00d2cb6 Merge pull request #2248 from lioncash/acc
X86Tables: Restrict CVTDQ2PD and CVTTSD2SI to 64-bit memory accesses
2022-12-15 16:48:42 -08:00
lioncash e985dcdb22 X86Tables: Restrict CVTTSD2SI src to 64 bit
When accessing memory, this should only be doing a 64-bit access, rather
than a 128-bit one.
2022-12-15 23:59:15 +00:00
lioncash ee9778480d X86Tables: Restrict CVTDQ2PD src to 64 bit
When accessing memory, this should only be doing a 64-bit access, rather
than a 128-bit one.
2022-12-15 23:46:48 +00:00
Mai 048daa4579 Merge pull request #2244 from Sonicadvance1/move_to_header
ARM64: Moves RA functions to header
2022-12-15 23:13:32 +00:00
Ryan Houdek 6ae8a1e55f ARM64: Moves RA functions to header
These are just some basic address calculations and a load, we want these
to be inlined as much as possible.
2022-12-15 15:00:33 -08:00
Ryan Houdek dc2eaf6511 Merge pull request #2246 from lioncash/extend
OpcodeDispatcher: Handle VPMOVSXB{D, W, Q}/VPMOVSXW{D, Q}/VPMOVSXDQ/VPMOVZXB{D, W, Q}/VPMOVZXW{D, Q}/VPMOVZXDQ
2022-12-15 14:19:28 -08:00
Ryan Houdek 0e233a96f0 Merge pull request #2247 from lioncash/roundacc
OpcodeDispatcher: Narrow memory access with scalar rounding operations
2022-12-15 14:17:49 -08:00
lioncash ba5fafcd7f OpcodeDispatcher: Narrow memory access with scalar rounding operations
These should only be accessing a 32-bit or 64-bit portion of memory
depending on single or double precision variants are used. Previously
we'd be doing a full 128-bit load.
2022-12-15 19:42:37 +00:00
lioncash b12503fe32 OpcodeDispatcher: Handle VPMOVSXDQ 2022-12-15 18:10:38 +00:00
lioncash aa63c7b94d OpcodeDispatcher: Handle VPMOVSXWQ 2022-12-15 18:08:00 +00:00
lioncash cccbb7f595 OpcodeDispatcher: Handle VPMOVSXWD 2022-12-15 18:01:43 +00:00
lioncash ce12ed60ae OpcodeDispatcher: Handle VPMOVSXBQ 2022-12-15 17:58:42 +00:00
lioncash d7eab5f787 OpcodeDispatcher: Handle VPMOVSXBD 2022-12-15 17:54:51 +00:00
lioncash 21537a3636 OpcodeDispatcher: Handle VPMOVSXBW 2022-12-15 17:50:58 +00:00
lioncash 588a2611a7 OpcodeDispatcher: Handle VPMOVZXDQ 2022-12-15 17:45:14 +00:00
lioncash 2895a09101 OpcodeDispatcher: Handle VPMOVZXWQ 2022-12-15 17:41:15 +00:00
lioncash 5c8d40d9be OpcodeDispatcher: Handle VPMOVZXWD 2022-12-15 17:37:51 +00:00
lioncash b4079cfea3 OpcodeDispatcher: Handle VPMOVZXBQ 2022-12-15 17:32:18 +00:00
lioncash 2b5570a910 OpcodeDispatcher: Handle VPMOVZXBD 2022-12-15 17:28:35 +00:00
lioncash 6bb0c5b24c OpcodeDispatcher: Handle VPMOVZXBW 2022-12-15 17:18:49 +00:00
lioncash bc31f98f16 OpcodeDispatcher: Move ExtendVectorElements impl to regular function
This can be reused for the AVX versions.
2022-12-15 17:11:02 +00:00
Ryan Houdek 4b891d6147 Merge pull request #2245 from lioncash/split
OpcodeDispatcher: Move template impl to regular function where applicable
2022-12-14 18:18:43 -08:00
lioncash 58c3e20bd1 OpcodeDispatcher: Move template impl to regular function where applicable
Reduces the amount of code size generated by the specializations.

Only targets ones that are heavily templated like the generic op helper
functions.
2022-12-15 01:54:12 +00:00
Ryan Houdek d5f3a091d0 Merge pull request #2216 from Sonicadvance1/32bit_host_thunk_support
Initial 32-bit host thunk feature support
2022-12-14 12:05:37 -08:00
Ryan Houdek a14e03f35d Update guest thunk lib register usage comment 2022-12-14 11:40:33 -08:00
Ryan Houdek 5c1789952e GuestThunks: Disable stack protector on 32-bit 2022-12-14 11:29:19 -08:00
Ryan Houdek f5809f24f7 GuestLibs: Fixes accidental guest lib setting 2022-12-14 11:29:19 -08:00
Ryan Houdek 122a9114a3 Thunks: 32-bit host library support 2022-12-14 11:29:19 -08:00
Ryan Houdek d8f226b460 Support 32-bit thunks ABI 2022-12-14 11:29:19 -08:00
Ryan Houdek 7171c5ae39 Support 32-bit thunksdb 2022-12-14 11:29:19 -08:00
Ryan Houdek 798a78534a Support Indirect thunk callback with mm0 as custom ABI 2022-12-14 11:24:18 -08:00
Ryan Houdek ae4a04b560 Fix incorrect THUNK_ABI prefix 2022-12-14 11:24:18 -08:00
Ryan Houdek 1971c8d505 32bit host thunk lib config path support 2022-12-14 11:24:18 -08:00
Ryan Houdek 1ca356371d Merge pull request #2242 from lioncash/round
OpcodeDispatcher: Handle VROUNDS{D, S}/VROUNDP{D, S}
2022-12-13 23:00:51 -08:00
lioncash 27ea6096a2 OpcodeDispatcher: Handle VROUNDSD 2022-12-14 06:41:36 +00:00
lioncash 2244dd9847 OpcodeDispatcher: Handle VROUNDSS 2022-12-14 06:34:58 +00:00
lioncash ca2f4bd468 OpcodeDispatcher: Handle VROUNDPD 2022-12-14 06:28:17 +00:00
lioncash 6b5c94be23 OpcodeDispatcher: Handle VROUNDPS 2022-12-14 06:27:59 +00:00
lioncash 779dc48d8d OpcodeDispatcher: Factor out VectorRound into VectorRoundImpl
This will be used in following commits for the AVX versions that use
this.
2022-12-14 05:52:16 +00:00
Ryan Houdek 4b2164768f Merge pull request #2241 from lioncash/ins
OpcodeDispatcher: Handle VINSERTF128/VINSERTI128
2022-12-13 20:45:44 -08:00
lioncash 90828aeb11 OpcodeDispatcher: Handle VINSERTI128 2022-12-14 04:26:42 +00:00
lioncash fe7c6da1e2 OpcodeDispatcher: Handle VINSERTF128 2022-12-14 04:24:04 +00:00
Ryan Houdek f3d0fa6f60 Merge pull request #2240 from lioncash/perm2
OpcodeDispatcher: Handle VPERM2F128/VPERM2I128
2022-12-13 19:57:31 -08:00
lioncash a9ad0d081c OpcodeDispatcher: Handle VPERM2I128 2022-12-14 03:41:29 +00:00
lioncash 54885bec32 OpcodeDispatcher: Handle VPERM2F128 2022-12-14 03:41:22 +00:00
Ryan Houdek e8aa79bea9 Merge pull request #2239 from lioncash/dec
Frontend: Handle 256-bit destination sizes directly
2022-12-13 19:01:18 -08:00
Ryan Houdek 60a45615df Merge pull request #2238 from lioncash/permq
OpcodeDispatcher: Handle VPERMQ/VPERMPD
2022-12-13 17:54:21 -08:00
lioncash 8a961bfcc5 VEXTables: Specify VPERMQ/VPERMPD as 256-bit
The AVX versions of these operands only operate on 256-bit ymm
registers, so we can specify this directly to be a little more
self-documenting.
2022-12-14 01:51:36 +00:00
lioncash d6ab7a4f97 Frontend: Handle 256-bit destination sizes directly
Previously the only time we'd promote to a 256-bit size is if the VEX.L
bit was set in the 128-bit path.

Allow specifying 256-bit sizes directly.
2022-12-14 01:50:05 +00:00
lioncash 7114fb3293 OpcodeDispatcher: Handle VPERMPD 2022-12-14 01:37:27 +00:00
lioncash 8a87aff730 OpcodeDispatcher: Handle VPERMQ 2022-12-14 01:30:55 +00:00
Ryan Houdek ded257c92f Merge pull request #2237 from lioncash/hadd
OpcodeDispatcher: Handle VHADDP{D, S}
2022-12-13 16:00:13 -08:00
lioncash c5b4719793 OpcodeDispatcher: Handle VHADDPD 2022-12-13 23:45:42 +00:00
lioncash 0f6201108f OpcodeDispatcher: Handle VHADDPS 2022-12-13 23:45:38 +00:00
lioncash b589dce7f5 x86_64/VectorOps: Make VFADDP behavior consistent with ARMv8
We need to swap the second and third results to be consistent with ARM.
Also fixes the mistake where I used vhaddpd instead of vhaddps in the
single-precision 256-bit case.
2022-12-13 23:34:12 +00:00
Ryan Houdek 9de5840f7a Merge pull request #2236 from lioncash/max
OpcodeDispatcher: Handle VPMAXS{B, D, W}/VPMAXU{B, D, W}
2022-12-13 14:11:17 -08:00
lioncash c98fffd33d OpcodeDispatcher: Handle VPMAXSD 2022-12-13 21:49:40 +00:00
lioncash de3777cc78 OpcodeDispatcher: Handle VPMAXSW 2022-12-13 21:47:17 +00:00
lioncash dd640e7a3d OpcodeDispatcher: Handle VPMAXSB 2022-12-13 21:44:46 +00:00
lioncash d53ddb73bf OpcodeDispatcher: Handle VPMAXUD 2022-12-13 21:40:29 +00:00
lioncash 25e9333abb OpcodeDispatcher: Handle VPMAXUW 2022-12-13 21:38:52 +00:00
lioncash 85766dd074 OpcodeDispatcher: Handle VPMAXUB 2022-12-13 21:36:47 +00:00
Ryan Houdek 40bab6b58e Merge pull request #2235 from lioncash/min
OpcodeDispatcher: Handle VPMINS{B, D, W}/VPMINU{B, D, W}
2022-12-13 13:31:14 -08:00
lioncash b0e0a2a165 OpcodeDispatcher: Handle VPMINSD 2022-12-13 20:51:14 +00:00
lioncash 0efcb912b5 OpcodeDispatcher: Handle VPMINSW 2022-12-13 20:49:08 +00:00
lioncash 9d0cc58737 OpcodeDispatcher: Handle VPMINSB 2022-12-13 20:47:00 +00:00
lioncash b671ed57ef OpcodeDispatcher: Handle VPMINUD 2022-12-13 20:41:53 +00:00
lioncash a2a44d188a OpcodeDispatcher: Handle VPMINUW 2022-12-13 20:39:55 +00:00
lioncash 364064536b OpcodeDispatcher: Handle VPMINUB 2022-12-13 20:36:51 +00:00
Mai 98a454169d Merge pull request #2234 from lioncash/sadd
OpcodeDispatcher: Handle VPADDS{B, W}/VPSUBS{B, W}
2022-12-13 20:25:25 +00:00
lioncash 6d44370f28 OpcodeDispatcher: Handle VPSUBSW 2022-12-13 18:52:55 +00:00
lioncash 273e2977a8 OpcodeDispatcher: Handle VPSUBSB 2022-12-13 18:50:02 +00:00
lioncash 7264b07d4f OpcodeDispatcher: Handle VPADDSW 2022-12-13 18:42:11 +00:00
lioncash 92351e7f33 OpcodeDispatcher: Handle VPADDSB 2022-12-13 18:40:08 +00:00
Ryan Houdek 757602bb1e Merge pull request #2233 from lioncash/uadd
OpcodeDispatcher: Handle VPADDUS{B, W}/VPSUBUS{B, W}
2022-12-13 10:25:59 -08:00
lioncash 6aaffec67f OpcodeDispatcher: Handle VPSUBUSW 2022-12-13 18:05:46 +00:00
lioncash 6f474cedd3 OpcodeDispatcher: Handle VPSUBUSB 2022-12-13 18:03:13 +00:00
lioncash d9176114c5 OpcodeDispatcher: Handle VPADDUSW 2022-12-13 18:03:07 +00:00
lioncash 287cee5b41 OpcodeDispatcher: Handle VPADDUSB 2022-12-13 18:02:55 +00:00
Mai a90067fb1e Merge pull request #2232 from lioncash/psub
OpcodeDispatcher: Handle VPSUB{B, D, Q, W}
2022-12-13 17:41:49 +00:00
lioncash 90d23098db OpcoodeDispatcher: Handle VPSUBQ 2022-12-13 06:01:19 +00:00
lioncash 384a09bbf1 OpcoodeDispatcher: Handle VPSUBD 2022-12-13 05:58:37 +00:00
lioncash 2fd29c47d4 OpcoodeDispatcher: Handle VPSUBW 2022-12-13 05:58:34 +00:00
lioncash e8aa8d89ec OpcoodeDispatcher: Handle VPSUBB 2022-12-13 05:47:50 +00:00
Ryan Houdek 1bc013d5f0 Merge pull request #2231 from lioncash/psign
OpcodeDispatcher: Handle VPSIGN{B, D, W}
2022-12-12 21:43:59 -08:00
lioncash 469ff91311 OpcodeDispatcher: Handle VPSIGND 2022-12-13 05:24:35 +00:00
lioncash ef14c411ce OpcodeDispatcher: Handle VPSIGNW 2022-12-13 05:20:12 +00:00
lioncash c2c5d176e4 OpcodeDispatcher: Handle VPSIGNB 2022-12-13 05:13:26 +00:00
lioncash 2328430d2e OpcodeDispatcher: Factor PSIGN handling into helper
This will allow us to use this with VEX and non-VEX variants without
needing to insert the upper-lane clearing for 128-bit variants into the
non-VEX path.

That, and this also allows us to not need to add additional template
arguments
2022-12-13 05:13:17 +00:00
Ryan Houdek a07a533640 Merge pull request #2230 from lioncash/div
OpcodeDispatcher: Handle VDIVP{D, S}/VDIVS{D, S}
2022-12-12 20:40:26 -08:00
lioncash 8c59e3e9e2 OpcodeDispatcher: Handle VDIVSD 2022-12-13 04:28:19 +00:00
lioncash fed861fa6b OpcodeDispatcher: Handle VDIVSS 2022-12-13 04:22:00 +00:00
lioncash ce9969ee8f OpcodeDispatcher: Handle VDIVPD 2022-12-13 04:16:33 +00:00
lioncash 9330ca41ea OpcodeDispatcher: Handle VDIVPS 2022-12-13 04:12:32 +00:00
Ryan Houdek eefcea49f4 Merge pull request #2229 from lioncash/mul
OpcodeDispatcher: Handle VMULP{D, S}/VMULS{D, S}
2022-12-12 20:03:43 -08:00
lioncash ed1b060494 OpcodeDispatcher: Handle VMULSD 2022-12-13 03:46:23 +00:00
lioncash 6db165e24a OpcodeDispatcher: Handle VMULSS 2022-12-13 03:42:08 +00:00
lioncash 58d20f199e OpcodeDispatcher: Handle VMULPD 2022-12-13 03:34:40 +00:00
lioncash 437ab47ae7 OpcodeDispatcher: Handle VMULPS 2022-12-13 03:29:35 +00:00
Ryan Houdek d6b137e6b7 Merge pull request #2228 from lioncash/min
OpcodeDispatcher: Handle VMAXP{D, S}/VMAXS{D, S}/VMINP{D, S}/VMINS{D, S}
2022-12-12 19:23:02 -08:00
lioncash e1de89af79 OpcodeDispatcher: Handle VMAXSD 2022-12-13 03:07:55 +00:00
lioncash 42d24c21e1 OpcodeDispatcher: Handle VMAXSS 2022-12-13 03:07:55 +00:00
lioncash 3590f090c7 OpcodeDispatcher: Handle VMAXPD 2022-12-13 03:07:55 +00:00
lioncash b7e177c11c OpcodeDispatcher: Handle VMAXPS 2022-12-13 03:07:55 +00:00
lioncash 92f92ddbbe OpcodeDispatcher: Handle VMINSD 2022-12-13 03:07:55 +00:00
lioncash f8d851b9b5 OpcodeDispatcher: Handle VMINSS 2022-12-13 03:07:55 +00:00
lioncash 1689742e96 OpcodeDispatcher: Handle VMINPD 2022-12-13 03:07:52 +00:00
lioncash 462b6b8c1c OpcodeDispatcher: Handle VMINPS 2022-12-13 01:48:57 +00:00
Ryan Houdek db90390179 Merge pull request #2227 from lioncash/sub
OpcodeDispatcher: Handle VSUBP{D, S}/ VSUBS{D, S}
2022-12-12 13:50:31 -08:00
lioncash e15fa66225 OpcodeDispatcher: Handle VSUBSD 2022-12-12 21:37:41 +00:00
lioncash 04a1fa6dc2 OpcodeDispatcher: Handle VSUBSS 2022-12-12 21:23:37 +00:00
lioncash f5a337a142 OpcodeDispatcher: Handle VSUBPD 2022-12-12 21:09:53 +00:00
lioncash 2b9d0314ce OpcodeDispatcher: Handle VSUBPS 2022-12-12 21:05:30 +00:00
Ryan Houdek 293734408c Merge pull request #2225 from lioncash/rcps
OpcodeDispatcher: Handle VRCPPS/VRCPSS
2022-12-12 12:04:46 -08:00
Ryan Houdek 03fbb923b3 Merge pull request #2226 from lioncash/lddqu
OpcodeDispatcher: Handle VLDDQU
2022-12-12 12:03:53 -08:00
lioncash 39f0c8542f OpcodeDispatcher: Handle VRCPSS 2022-12-12 19:48:21 +00:00
lioncash 6877d5b3ec OpcodeDispatcher: Handle VRCPPS 2022-12-12 19:48:21 +00:00
lioncash a6c30b35dc OpcodeDispatcher: Handle VLDDQU 2022-12-12 19:41:29 +00:00
Ryan Houdek a57f3a6264 Merge pull request #2224 from lioncash/abs
OpcodeDispatcher: Handle VPABS{B, D, W}
2022-12-12 11:28:07 -08:00
lioncash fa0ff71ddf OpcodeDispatcher: Handle VPABSD 2022-12-12 18:47:10 +00:00
lioncash c91ccf2cbe OpcodeDispatcher: Handle VPABSW 2022-12-12 18:47:10 +00:00
lioncash 41df5f816d OpcodeDispatcher: Handle VPABSB 2022-12-12 18:47:10 +00:00
Ryan Houdek 573896d0b7 Merge pull request #2223 from lioncash/pcmp
OpcodeDispatcher: Handle VPCMPEQ{B, D, Q, W}/VPCMPGT{B, D, Q, W}
2022-12-12 10:32:40 -08:00
lioncash 36a6264571 OpcodeDispatcher: Handle VPCMPEQQ 2022-12-12 18:00:38 +00:00
lioncash 12f01bc93a OpcodeDispatcher: Handle VPCMPEQD 2022-12-12 17:55:07 +00:00
lioncash 777b2c7966 OpcodeDispatcher: Handle VPCMPEQW 2022-12-12 17:51:36 +00:00
lioncash f0141f124d OpcodeDispatcher: Handle VPCMPEQB 2022-12-12 17:42:29 +00:00
lioncash 283b178285 OpcodeDispatcher: Handle VPCMPGTQ 2022-12-12 17:29:36 +00:00
lioncash d3a5eef08a OpcodeDispatcher: Handle VPCMPGTD 2022-12-12 17:13:46 +00:00
lioncash 1bac33ff44 OpcodeDispatcher: Handle VPCMPGTW 2022-12-12 17:13:46 +00:00
lioncash 327a6f52fd OpcodeDispatcher: Handle VPCMPGTB 2022-12-12 17:13:46 +00:00
Ryan Houdek 4f313f5d40 Merge pull request #2219 from Sonicadvance1/handle_pf_write
Dispatcher: Calculate REG_ERR correctly using ARM ESR_EL1
2022-12-12 09:02:27 -08:00
Ryan Houdek b42b4e03a4 Merge pull request #2218 from Sonicadvance1/GOT_optimization
OpCodeDispatcher: Optimize a case of GOT calculation
2022-12-12 09:02:18 -08:00
Ryan Houdek ab14375a03 Merge pull request #2222 from lioncash/rsqrt
OpcodeDispatcher: Handle VRSQRTSS/VRSQRTPS
2022-12-12 09:02:04 -08:00
Ryan Houdek ace90aac95 Merge pull request #2221 from lioncash/pbroad
OpcodeDispatcher: Handle VPBROADCAST{B, D, Q, W}/VBROADCASTI128
2022-12-12 09:01:56 -08:00
lioncash c4c93f5bfe OpcodeDispatcher: Handle VRSQRTSS 2022-12-12 16:30:34 +00:00
lioncash 3504ba068e OpcodeDispatcher: Handle VRSQRTPS 2022-12-12 16:11:16 +00:00
lioncash 88b88c9cd3 OpcodeDispatcher: Handle VBROADCASTI128 2022-12-12 15:51:12 +00:00
lioncash e99928990e OpcodeDispatcher: Handle VPBROADCASTQ 2022-12-12 15:41:41 +00:00
lioncash 6733f83471 OpcodeDispatcher: Handle VPBROADCASTD 2022-12-12 15:37:59 +00:00
lioncash a14cce27a4 OpcodeDispatcher: Handle VPBROADCASTW 2022-12-12 15:34:17 +00:00
lioncash 04d5b53389 OpcodeDispatcher: Handle VPBROADCASTB 2022-12-12 15:31:17 +00:00
Ryan Houdek a6b0181cd4 OpCodeDispatcher: Optimize a case of GOT calculation
32-bit GOT calculation needs to do a call+pop to do get the EIP on
32-bit. LEA doesn't work because it there is no EIP relative ops like on
x86-64.

This causes a terrible block split on every GOT calculation without the
optimization in place.

Now the block can continue through this weird GOT calculation.

This will be worthwhile for our 32-bit thunks where for some reason the
GOT calculation can't be removed. The GOT is calculated even though it
isn't used.
2022-12-10 02:50:48 -08:00
Ryan Houdek 3afd5691a4 unittests: Adds unit test to test ERR 2022-12-09 15:44:22 -08:00
Ryan Houdek 82ad26307c Dispatcher: Calculate REG_ERR correctly using ARM ESR_EL1
On Fault then ARM will return information about the fault in ESR_EL1 to
the user.

We need to decode what ESR_EL1 means in the context of the fault to get
the flags we care about.

The flags we care about specifically are PF_USER and PF_WRITE.
PF_PROT would have been interesting but I didn't see when this gets
returned to the user. ARM makes the difference if the page is unmapped
or "mapped" with PROT_NONE. x86 doesn't make the distinction here.

This should fix an issue that a user was hitting.
2022-12-09 15:44:11 -08:00
Mai dc9737a394 Merge pull request #2217 from Sonicadvance1/fix_global_app_config
Config: Fixes global application configs
2022-12-09 18:02:08 +00:00
Ryan Houdek eaef06d14e Config: Fixes global application configs
Accidentally was checking for SteamID layer types twice, rather than
global.

Fixes steamwebhelper config not getting loaded from global config.
2022-12-09 08:26:51 -08:00
Ryan Houdek 2123868a42 Merge pull request #2215 from lioncash/broadcast
OpcodeDispatcher: Handle VBROADCASTSD/VBROADCASTSD/VBROADCASTF128
2022-12-07 19:50:19 -08:00
lioncash b891999a7f OpcodeDispatcher: Handle VBROADCASTF128 2022-12-08 03:18:58 +00:00
lioncash a53fd07bda OpcodeDispatcher: Handle VBROADCASTSD 2022-12-08 02:58:12 +00:00
lioncash 8f213b75be OpcodeDispatcher: Handle VBROADCASTSS 2022-12-08 02:40:36 +00:00
Ryan Houdek b73aeb8902 Merge pull request #2214 from lioncash/stmxcsr
OpcodeDispatcher: Handle VLDMXCSR/VSTMXCSR
2022-12-07 17:07:07 -08:00
lioncash d642c1a646 OpcodeDispatcher: Handle VLDMXCSR/VSTMXCSR 2022-12-08 00:42:13 +00:00
Ryan Houdek d965ae03c4 Merge pull request #2210 from lioncash/sqrt
OpcodeDispatcher: Handle VSQRTPD/VSQRTPS/VSQRTSD/VSQRTSS
2022-12-07 14:53:16 -08:00
lioncash e42de0b645 OpcodeDispatcher: Handle VSQRTSD 2022-12-07 22:42:16 +00:00
lioncash 9ef5247dd7 OpcodeDispatcher: Handle VSQRTSS 2022-12-07 22:42:16 +00:00
lioncash 25428cb28c OpcodeDispatcher: Handle VSQRTPD 2022-12-07 22:42:16 +00:00
lioncash 2125949d6d OpcodeDispatcher: Handle VSQRTPS 2022-12-07 22:42:16 +00:00
Ryan Houdek 7ac21e794d Merge pull request #2212 from lioncash/comiss
OpcodeDispatcher: Handle VCOMISD/VCOMISS/VUCOMISD/VUCOMISS
2022-12-07 14:38:49 -08:00
lioncash 4aa0f3d0a4 OpcodeDispatcher: Handle VCOMISD 2022-12-07 22:06:25 +00:00
lioncash 740c983f65 OpcodeDispatcher: Handle VCOMISS 2022-12-07 22:06:25 +00:00
lioncash 83bccc0032 OpcodeDispatcher: Handle VUCOMISD 2022-12-07 22:06:22 +00:00
Ryan Houdek a98920d4e4 Merge pull request #2211 from lioncash/avg
OpcodeDispatcher: Handle VPAVGB/VPAVGW
2022-12-07 13:37:58 -08:00
lioncash d1ab636df1 OpcodeDispatcher: Handle VUCOMISS 2022-12-07 21:22:19 +00:00
lioncash 26b629833e OpcodeDispatcher: Handle VPAVGW 2022-12-07 21:07:43 +00:00
lioncash 95964f8dd8 OpcodeDispatcher: Handle VPAVGB 2022-12-07 21:07:40 +00:00
Ryan Houdek 94ae2e3a9c Merge pull request #2209 from lioncash/ins
IR: Handle 128-bit VInsElement with SVE
2022-12-07 11:08:47 -08:00
lioncash eae33b0c50 IR: Handle 128-bit VInsElement with SVE
Currently VDupElement allows duplicating 128-bit elements in 256-bit
vectors with SVE, so we can extend VInsElement to have similar behavior.
2022-12-07 18:41:45 +00:00
Ryan Houdek e9aa368a62 Merge pull request #2208 from lioncash/zero
OpcodeDispatcher: Explicitly zero upper lanes
2022-12-07 10:13:53 -08:00
lioncash 5a37786da7 OpcodeDispatcher: Explicitly zero upper lanes
Makes our intent to zero-extend the upper lanes explicit and lets us
remove a special case in the LoadSource implementation.

This also makes things a little nicer since we're not hardcoding 32 byte
stores.
2022-12-07 17:22:20 +00:00
Ryan Houdek 5ac44baa2c Merge pull request #2207 from lioncash/adds
OpcodeDispatcher: Handle VADDSD/VADDSS
2022-12-07 09:19:59 -08:00
lioncash 4cf3805950 OpcodeDispatcher: Handle VADDSD 2022-12-07 16:43:47 +00:00
lioncash 1f5a1826a6 OpcodeDispatcher: Handle VADDSS 2022-12-07 16:43:44 +00:00
Ryan Houdek bf86df7a66 Merge pull request #2206 from lioncash/haddp
OpcodeDispatcher: Merge HADDP/PHADD into VectorALUOp
2022-12-07 07:39:19 -08:00
lioncash 6a63ae2d9c OpcodeDispatcher: Merge PHADD into VectorALUOp 2022-12-07 15:18:19 +00:00
lioncash a7a1e2abd3 OpcodeDispatcher: Merge HADDP into VectorALUOp 2022-12-07 15:14:10 +00:00
Ryan Houdek 3322f8b890 Merge pull request #2205 from lioncash/pavg
OpcodeDispatcher: Merge PAVGOp with VectorALUOp
2022-12-07 07:08:48 -08:00
lioncash 047ae13c98 OpcodeDispatcher: Merge PAVGOp with VectorALUOp
This can be merged into it, considering it only has one IR op.
2022-12-07 14:51:08 +00:00
Ryan Houdek 9eaa45f922 Merge pull request #2202 from lioncash/addmerge
OpcodeDispatcher: Merge PADDQOp, PSUBQOp, PADDSOp, PSUBSOp with VectorALUOp
2022-12-05 17:48:59 -08:00
lioncash 3d5e0c5832 OpcodeDispatcher: Merge PSUBSOp with VectorALUOp 2022-12-06 01:31:26 +00:00
lioncash 71a36763df OpcodeDispatcher: Merge PADDSOp with VectorALUOp 2022-12-06 01:27:31 +00:00
lioncash 07bd3137ef OpcodeDispatcher: Merge PSUBQOp with VectorALUOp 2022-12-06 01:21:33 +00:00
lioncash c2b40b4dd3 OpcodeDispather: Merge PADDQOp with VectorALUOp 2022-12-06 01:13:12 +00:00
Ryan Houdek 4b16718602 Merge pull request #2201 from lioncash/bic
OpcodeDispatcher: Merge ANDNOp with VectorALUROp
2022-12-05 15:15:31 -08:00
Ryan Houdek 2bf7e09862 Merge pull request #2200 from lioncash/bic
OpcodeDispatcher: Simplify VANDN
2022-12-05 14:31:33 -08:00
lioncash db74e46fc3 OpcodeDispatcher: Merge ANDNOp with VectorALUROp
Now that ANDNOp is reduced to one IR op, we can merge it with
VectorALUROp
2022-12-05 22:28:39 +00:00
lioncash 13bba59ba6 OpcodeDispatcher: Simplify ANDNOp and VANDNOp
We can just use VBic here to simplify everything.
2022-12-05 22:15:15 +00:00
272 changed files with 45006 additions and 7717 deletions

No files matched your search

+1
View File
@@ -29,6 +29,7 @@ option(ENABLE_INTERPRETER "Enables FEX's Interpreter" FALSE)
option(ENABLE_CCACHE "Enables ccache for compile caching" TRUE)
option(ENABLE_TERMUX_BUILD "Forces building for Termux on a non-Termux build machine" FALSE)
option(ENABLE_VIXL_SIMULATOR "Forces the FEX JIT to use the VIXL simulator" FALSE)
option(ENABLE_VIXL_DISASSEMBLER "Enables debug disassembler output with VIXL" FALSE)
option(ENABLE_FEXCORE_PROFILER "Enables use of the FEXCore timeline profiling capabilities" FALSE)
set (FEXCORE_PROFILER_BACKEND "gpuvis" CACHE STRING "Set which backend you want to use for the FEXCore profiler")
+69 -69
View File
@@ -6,10 +6,10 @@
"X11"
],
"Overlay": [
"@PREFIX_LIB@/x86_64-linux-gnu/libGL.so",
"@PREFIX_LIB@/x86_64-linux-gnu/libGL.so.1",
"@PREFIX_LIB@/x86_64-linux-gnu/libGL.so.1.2.0",
"@PREFIX_LIB@/x86_64-linux-gnu/libGL.so.1.7.0"
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libGL.so",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libGL.so.1",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libGL.so.1.2.0",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libGL.so.1.7.0"
]
},
"GLESv2": {
@@ -18,17 +18,17 @@
"X11"
],
"Overlay": [
"@PREFIX_LIB@/x86_64-linux-gnu/libGLESv2.so",
"@PREFIX_LIB@/x86_64-linux-gnu/libGLESv2.so.2",
"@PREFIX_LIB@/x86_64-linux-gnu/libGLESv2.so.2.0.0"
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libGLESv2.so",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libGLESv2.so.2",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libGLESv2.so.2.0.0"
]
},
"X11": {
"Library": "libX11-guest.so",
"Overlay": [
"@PREFIX_LIB@/x86_64-linux-gnu/libX11.so",
"@PREFIX_LIB@/x86_64-linux-gnu/libX11.so.6",
"@PREFIX_LIB@/x86_64-linux-gnu/libX11.so.6.4.0"
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libX11.so",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libX11.so.6",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libX11.so.6.4.0"
]
},
"Vulkan": {
@@ -37,8 +37,8 @@
"xcb"
],
"Overlay": [
"@PREFIX_LIB@/x86_64-linux-gnu/libvulkan.so",
"@PREFIX_LIB@/x86_64-linux-gnu/libvulkan.so.1",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libvulkan.so",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libvulkan.so.1",
"@HOME@/.local/share/Steam/ubuntu12_32/steam-runtime/pinned_libs_64/libvulkan.so.1"
],
"Comment": [
@@ -48,129 +48,129 @@
"xcb": {
"Library": "libxcb-guest.so",
"Overlay": [
"@PREFIX_LIB@/x86_64-linux-gnu/libxcb.so",
"@PREFIX_LIB@/x86_64-linux-gnu/libxcb.so.1",
"@PREFIX_LIB@/x86_64-linux-gnu/libxcb.so.1.1.0"
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libxcb.so",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libxcb.so.1",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libxcb.so.1.1.0"
]
},
"xcb-dri2": {
"Library": "libxcb_dri2-guest.so",
"Library": "libxcb-dri2-guest.so",
"Overlay": [
"@PREFIX_LIB@/x86_64-linux-gnu/libxcb-dri2.so",
"@PREFIX_LIB@/x86_64-linux-gnu/libxcb-dri2.so.0",
"@PREFIX_LIB@/x86_64-linux-gnu/libxcb-dri2.so.0.0.0"
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libxcb-dri2.so",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libxcb-dri2.so.0",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libxcb-dri2.so.0.0.0"
]
},
"xcb-dri3": {
"Library": "libxcb_dri3-guest.so",
"Library": "libxcb-dri3-guest.so",
"Overlay": [
"@PREFIX_LIB@/x86_64-linux-gnu/libxcb-dri3.so",
"@PREFIX_LIB@/x86_64-linux-gnu/libxcb-dri3.so.0",
"@PREFIX_LIB@/x86_64-linux-gnu/libxcb-dri3.so.0.0.0"
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libxcb-dri3.so",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libxcb-dri3.so.0",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libxcb-dri3.so.0.0.0"
]
},
"xcb-xfixes": {
"Library": "libxcb_xfixes-guest.so",
"Library": "libxcb-xfixes-guest.so",
"Overlay": [
"@PREFIX_LIB@/x86_64-linux-gnu/libxcb-xfixes.so",
"@PREFIX_LIB@/x86_64-linux-gnu/libxcb-xfixes.so.0",
"@PREFIX_LIB@/x86_64-linux-gnu/libxcb-xfixes.so.0.0.0"
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libxcb-xfixes.so",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libxcb-xfixes.so.0",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libxcb-xfixes.so.0.0.0"
]
},
"xcb-shm": {
"Library": "libxcb_shm-guest.so",
"Library": "libxcb-shm-guest.so",
"Overlay": [
"@PREFIX_LIB@/x86_64-linux-gnu/libxcb-shm.so",
"@PREFIX_LIB@/x86_64-linux-gnu/libxcb-shm.so.0",
"@PREFIX_LIB@/x86_64-linux-gnu/libxcb-shm.so.0.0.0"
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libxcb-shm.so",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libxcb-shm.so.0",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libxcb-shm.so.0.0.0"
]
},
"xcb-sync": {
"Library": "libxcb_sync-guest.so",
"Library": "libxcb-sync-guest.so",
"Overlay": [
"@PREFIX_LIB@/x86_64-linux-gnu/libxcb-sync.so",
"@PREFIX_LIB@/x86_64-linux-gnu/libxcb-sync.so.1",
"@PREFIX_LIB@/x86_64-linux-gnu/libxcb-sync.so.1.0.0"
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libxcb-sync.so",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libxcb-sync.so.1",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libxcb-sync.so.1.0.0"
]
},
"xcb-randr": {
"Library": "libxcb_randr-guest.so",
"Library": "libxcb-randr-guest.so",
"Overlay": [
"@PREFIX_LIB@/x86_64-linux-gnu/libxcb-randr.so",
"@PREFIX_LIB@/x86_64-linux-gnu/libxcb-randr.so.0",
"@PREFIX_LIB@/x86_64-linux-gnu/libxcb-randr.so.0.1.0"
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libxcb-randr.so",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libxcb-randr.so.0",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libxcb-randr.so.0.1.0"
]
},
"xcb-present": {
"Library": "libxcb_present-guest.so",
"Library": "libxcb-present-guest.so",
"Overlay": [
"@PREFIX_LIB@/x86_64-linux-gnu/libxcb-present.so",
"@PREFIX_LIB@/x86_64-linux-gnu/libxcb-present.so.0",
"@PREFIX_LIB@/x86_64-linux-gnu/libxcb-present.so.0.0.0"
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libxcb-present.so",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libxcb-present.so.0",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libxcb-present.so.0.0.0"
]
},
"xcb-glx": {
"Library": "libxcb_glx-guest.so",
"Library": "libxcb-glx-guest.so",
"Overlay": [
"@PREFIX_LIB@/x86_64-linux-gnu/libxcb-glx.so",
"@PREFIX_LIB@/x86_64-linux-gnu/libxcb-glx.so.0",
"@PREFIX_LIB@/x86_64-linux-gnu/libxcb-glx.so.0.0.0"
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libxcb-glx.so",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libxcb-glx.so.0",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libxcb-glx.so.0.0.0"
]
},
"xshmfence": {
"Library": "libshmfence-guest.so",
"Library": "libxshmfence-guest.so",
"Overlay": [
"@PREFIX_LIB@/x86_64-linux-gnu/libxshmfence.so",
"@PREFIX_LIB@/x86_64-linux-gnu/libxshmfence.so.1",
"@PREFIX_LIB@/x86_64-linux-gnu/libxshmfence.so.1.0.0"
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libxshmfence.so",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libxshmfence.so.1",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libxshmfence.so.1.0.0"
]
},
"drm": {
"Library": "libdrm-guest.so",
"Overlay": [
"@PREFIX_LIB@/x86_64-linux-gnu/libdrm.so",
"@PREFIX_LIB@/x86_64-linux-gnu/libdrm.so.2",
"@PREFIX_LIB@/x86_64-linux-gnu/libdrm.so.2.4.0"
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libdrm.so",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libdrm.so.2",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libdrm.so.2.4.0"
]
},
"asound": {
"Library": "libasound-guest.so",
"Overlay": [
"@PREFIX_LIB@/x86_64-linux-gnu/libasound.so",
"@PREFIX_LIB@/x86_64-linux-gnu/libasound.so.2",
"@PREFIX_LIB@/x86_64-linux-gnu/libasound.so.2.0.0"
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libasound.so",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libasound.so.2",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libasound.so.2.0.0"
]
},
"Xrender": {
"Library": "libXrender-guest.so",
"Overlay": [
"@PREFIX_LIB@/x86_64-linux-gnu/libXrender.so",
"@PREFIX_LIB@/x86_64-linux-gnu/libXrender.so.1",
"@PREFIX_LIB@/x86_64-linux-gnu/libXrender.so.1.3.0"
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libXrender.so",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libXrender.so.1",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libXrender.so.1.3.0"
]
},
"Xext": {
"Library": "libXext-guest.so",
"Overlay": [
"@PREFIX_LIB@/x86_64-linux-gnu/libXext.so",
"@PREFIX_LIB@/x86_64-linux-gnu/libXext.so.6",
"@PREFIX_LIB@/x86_64-linux-gnu/libXext.so.6.4.0"
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libXext.so",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libXext.so.6",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libXext.so.6.4.0"
]
},
"Xfixes": {
"Library": "libXfixes-guest.so",
"Overlay": [
"@PREFIX_LIB@/x86_64-linux-gnu/libXfixes.so",
"@PREFIX_LIB@/x86_64-linux-gnu/libXfixes.so.3",
"@PREFIX_LIB@/x86_64-linux-gnu/libXfixes.so.3.1.0"
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libXfixes.so",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libXfixes.so.3",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libXfixes.so.3.1.0"
]
},
"OpenCL": {
"Library" : "libOpenCL-guest.so",
"Overlay": [
"@PREFIX_LIB@/x86_64-linux-gnu/libOpenCL.so",
"@PREFIX_LIB@/x86_64-linux-gnu/libOpenCL.so.1",
"@PREFIX_LIB@/x86_64-linux-gnu/libOpenCL.so.1.0.0"
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libOpenCL.so",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libOpenCL.so.1",
"@PREFIX_LIB@/@PREFIX_ARCH@-linux-gnu/libOpenCL.so.1.0.0"
]
},
"":{}
+4
View File
@@ -82,3 +82,7 @@ add_subdirectory(Source/)
install (DIRECTORY include/FEXCore ${CMAKE_BINARY_DIR}/include/FEXCore
DESTINATION include
COMPONENT Development)
if (BUILD_TESTS)
add_subdirectory(unittests/)
endif()
+4
View File
@@ -182,6 +182,10 @@ if (ENABLE_VIXL_SIMULATOR)
list(APPEND DEFINES -DVIXL_SIMULATOR=1 -DVIXL_INCLUDE_SIMULATOR_AARCH64=1)
endif()
if (ENABLE_VIXL_DISASSEMBLER)
list(APPEND DEFINES -DVIXL_DISASSEMBLER=1)
endif()
if (ENABLE_JIT_X86_64)
list(APPEND SRCS
Interface/Core/JIT/x86_64/JIT.cpp
+1 -1
View File
@@ -686,7 +686,7 @@ namespace JSON {
AppLoader::AppLoader(const std::string& Filename, FEXCore::Config::LayerType Type)
: FEXCore::Config::OptionMapper(Type) {
const bool Global = Type == FEXCore::Config::LayerType::LAYER_GLOBAL_STEAM_APP ||
Type == FEXCore::Config::LayerType::LAYER_LOCAL_STEAM_APP;
Type == FEXCore::Config::LayerType::LAYER_GLOBAL_APP;
Config = FEXCore::Config::GetApplicationConfig(Filename, Global);
// Immediately load so we can reload the meta layer
@@ -91,6 +91,13 @@
"Folder to find the guest-side thunking libraries."
]
},
"ThunkHostLibs32": {
"Type": "str",
"Default": "@CMAKE_INSTALL_PREFIX@/lib/fex-emu/HostThunks_32/",
"Desc": [
"Folder to find the 32-bit host-side thunking libraries."
]
},
"ThunkGuestLibs32": {
"Type": "str",
"Default": "@CMAKE_INSTALL_PREFIX@/share/fex-emu/GuestThunks_32/",
+1
View File
@@ -105,6 +105,7 @@ namespace FEXCore::Context {
FEX_CONFIG_OPT(MaxInstPerBlock, MAXINST);
FEX_CONFIG_OPT(RootFSPath, ROOTFS);
FEX_CONFIG_OPT(ThunkHostLibsPath, THUNKHOSTLIBS);
FEX_CONFIG_OPT(ThunkHostLibsPath32, THUNKHOSTLIBS32);
FEX_CONFIG_OPT(ThunkConfigFile, THUNKCONFIG);
FEX_CONFIG_OPT(DumpIR, DUMPIR);
FEX_CONFIG_OPT(StaticRegisterAllocation, SRA);
@@ -1,7 +1,6 @@
#include "Interface/Core/ArchHelpers/Arm64.h"
#include "Interface/Core/ArchHelpers/MContext.h"
#include <aarch64/cpu-aarch64.h>
#include "Interface/Core/ArchHelpers/CodeEmitter/Buffer.h"
#include <FEXCore/Utils/EnumUtils.h>
#include <FEXCore/Utils/LogManager.h>
@@ -572,7 +571,7 @@ bool HandleAtomicVectorStore(void *_ucontext, void *_info, uint32_t Instr) {
PC[1] = STP;
PC[2] = DMB;
// Back up one instruction and have another go
vixl::aarch64::CPU::EnsureIAndDCacheCoherency(&PC[0], 16);
FEXCore::ARMEmitter::Buffer::ClearICache(&PC[0], 16);
return true;
}
}
@@ -2311,7 +2310,7 @@ bool HandleSIGBUS(bool ParanoidTSO, int Signal, void *info, void *ucontext) {
return false;
}
vixl::aarch64::CPU::EnsureIAndDCacheCoherency(&PC[-1], 16);
FEXCore::ARMEmitter::Buffer::ClearICache(&PC[-1], 16);
return true;
}
return false;
@@ -1,4 +1,5 @@
#include "Interface/Core/ArchHelpers/Arm64Emitter.h"
#include "Interface/Core/ArchHelpers/CodeEmitter/Emitter.h"
#include "Interface/Core/Dispatcher/Dispatcher.h"
#include "Interface/Context/Context.h"
#include "Interface/HLE/Thunks/Thunks.h"
@@ -20,38 +21,24 @@ namespace FEXCore::CPU {
// We want vixl to not allocate a default buffer. Jit and dispatcher will manually create one.
Arm64Emitter::Arm64Emitter(FEXCore::Context::Context *ctx, size_t size)
: vixl::aarch64::Assembler(size ? (byte*)FEXCore::Allocator::mmap(nullptr, size, PROT_READ | PROT_WRITE | PROT_EXEC, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0) : reinterpret_cast<byte*>(~0ULL),
size,
vixl::aarch64::PositionDependentCode)
: Emitter(size ? (uint8_t*)FEXCore::Allocator::mmap(nullptr, size, PROT_READ | PROT_WRITE | PROT_EXEC, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0) : nullptr, size)
, EmitterCTX {ctx} {
CPU.SetUp();
#ifdef VIXL_SIMULATOR
auto Features = vixl::CPUFeatures::All();
#else
auto Features = vixl::CPUFeatures::InferFromOS();
if (ctx->HostFeatures.SupportsAtomics) {
// Hypervisor can hide this on the c630?
Features.Combine(vixl::CPUFeatures::Feature::kLORegions);
}
#endif
SetCPUFeatures(Features);
}
Arm64Emitter::~Arm64Emitter() {
auto CodeBuffer = GetBuffer();
if (CodeBuffer->GetCapacity()) {
FEXCore::Allocator::munmap(CodeBuffer->GetStartAddress<void*>(), CodeBuffer->GetCapacity());
auto BufferSize = GetBufferSize();
if (BufferSize) {
FEXCore::Allocator::munmap(GetBufferBase(), BufferSize);
}
}
void Arm64Emitter::LoadConstant(vixl::aarch64::Register Reg, uint64_t Constant, bool NOPPad) {
bool Is64Bit = Reg.IsX();
void Arm64Emitter::LoadConstant(ARMEmitter::Size s, ARMEmitter::Register Reg, uint64_t Constant, bool NOPPad) {
bool Is64Bit = s == ARMEmitter::Size::i64Bit;
int Segments = Is64Bit ? 4 : 2;
if (Is64Bit && ((~Constant)>> 16) == 0) {
movn(Reg, (~Constant) & 0xFFFF);
movn(s, Reg, (~Constant) & 0xFFFF);
if (NOPPad) {
nop(); nop(); nop();
@@ -98,17 +85,17 @@ void Arm64Emitter::LoadConstant(vixl::aarch64::Register Reg, uint64_t Constant,
else {
// Need to use ADRP + ADD
adrp(Reg, AlignedOffset >> 12);
add(Reg, Reg, Constant & 0xFFF);
add(s, Reg, Reg, Constant & 0xFFF);
NumMoves = 2;
}
}
}
else {
movz(Reg, (Constant) & 0xFFFF, 0);
movz(s, Reg, (Constant) & 0xFFFF, 0);
for (int i = 1; i < Segments; ++i) {
uint16_t Part = (Constant >> (i * 16)) & 0xFFFF;
if (Part) {
movk(Reg, Part, i * 16);
movk(s, Reg, Part, i * 16);
++NumMoves;
}
}
@@ -124,126 +111,143 @@ void Arm64Emitter::LoadConstant(vixl::aarch64::Register Reg, uint64_t Constant,
void Arm64Emitter::PushCalleeSavedRegisters() {
// We need to save pairs of registers
// We save r19-r30
MemOperand PairOffset(sp, -16, PreIndex);
const std::array<std::pair<vixl::aarch64::Register, vixl::aarch64::Register>, 6> CalleeSaved = {{
{x19, x20},
{x21, x22},
{x23, x24},
{x25, x26},
{x27, x28},
{x29, x30},
const std::array<std::pair<ARMEmitter::XRegister, ARMEmitter::XRegister>, 6> CalleeSaved = {{
{ARMEmitter::XReg::x19, ARMEmitter::XReg::x20},
{ARMEmitter::XReg::x21, ARMEmitter::XReg::x22},
{ARMEmitter::XReg::x23, ARMEmitter::XReg::x24},
{ARMEmitter::XReg::x25, ARMEmitter::XReg::x26},
{ARMEmitter::XReg::x27, ARMEmitter::XReg::x28},
{ARMEmitter::XReg::x29, ARMEmitter::XReg::x30},
}};
for (auto &RegPair : CalleeSaved) {
stp(RegPair.first, RegPair.second, PairOffset);
stp<ARMEmitter::IndexType::PRE>(RegPair.first, RegPair.second, ARMEmitter::Reg::rsp, -16);
}
// Additionally we need to store the lower 64bits of v8-v15
// Here's a fun thing, we can use two ST4 instructions to store everything
// We just need a single sub to sp before that
const std::array<
std::tuple<vixl::aarch64::VRegister,
vixl::aarch64::VRegister,
vixl::aarch64::VRegister,
vixl::aarch64::VRegister>, 2> FPRs = {{
{v8, v9, v10, v11},
{v12, v13, v14, v15},
std::tuple<ARMEmitter::DRegister,
ARMEmitter::DRegister,
ARMEmitter::DRegister,
ARMEmitter::DRegister>, 2> FPRs = {{
{ARMEmitter::DReg::d8, ARMEmitter::DReg::d9, ARMEmitter::DReg::d10, ARMEmitter::DReg::d11},
{ARMEmitter::DReg::d12, ARMEmitter::DReg::d13, ARMEmitter::DReg::d14, ARMEmitter::DReg::d15},
}};
uint32_t VectorSaveSize = sizeof(uint64_t) * 8;
sub(sp, sp, VectorSaveSize);
sub(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::rsp, ARMEmitter::Reg::rsp, VectorSaveSize);
// SP supporting move
// We just saved x19 so it is safe
add(x19, sp, 0);
add(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r19, ARMEmitter::Reg::rsp, 0);
MemOperand QuadOffset(x19, 32, PostIndex);
for (auto &RegQuad : FPRs) {
st4(std::get<0>(RegQuad).D(),
std::get<1>(RegQuad).D(),
std::get<2>(RegQuad).D(),
std::get<3>(RegQuad).D(),
st4(ARMEmitter::SubRegSize::i64Bit,
std::get<0>(RegQuad),
std::get<1>(RegQuad),
std::get<2>(RegQuad),
std::get<3>(RegQuad),
0,
QuadOffset);
ARMEmitter::Reg::r19,
32);
}
}
void Arm64Emitter::PopCalleeSavedRegisters() {
const std::array<
std::tuple<vixl::aarch64::VRegister,
vixl::aarch64::VRegister,
vixl::aarch64::VRegister,
vixl::aarch64::VRegister>, 2> FPRs = {{
{v12, v13, v14, v15},
{v8, v9, v10, v11},
std::tuple<ARMEmitter::DRegister,
ARMEmitter::DRegister,
ARMEmitter::DRegister,
ARMEmitter::DRegister>, 2> FPRs = {{
{ARMEmitter::DReg::d12, ARMEmitter::DReg::d13, ARMEmitter::DReg::d14, ARMEmitter::DReg::d15},
{ARMEmitter::DReg::d8, ARMEmitter::DReg::d9, ARMEmitter::DReg::d10, ARMEmitter::DReg::d11},
}};
MemOperand QuadOffset(sp, 32, PostIndex);
for (auto &RegQuad : FPRs) {
ld4(std::get<0>(RegQuad).D(),
std::get<1>(RegQuad).D(),
std::get<2>(RegQuad).D(),
std::get<3>(RegQuad).D(),
ld4(ARMEmitter::SubRegSize::i64Bit,
std::get<0>(RegQuad),
std::get<1>(RegQuad),
std::get<2>(RegQuad),
std::get<3>(RegQuad),
0,
QuadOffset);
ARMEmitter::Reg::rsp,
32);
}
MemOperand PairOffset(sp, 16, PostIndex);
const std::array<std::pair<vixl::aarch64::Register, vixl::aarch64::Register>, 6> CalleeSaved = {{
{x29, x30},
{x27, x28},
{x25, x26},
{x23, x24},
{x21, x22},
{x19, x20},
const std::array<std::pair<ARMEmitter::XRegister, ARMEmitter::XRegister>, 6> CalleeSaved = {{
{ARMEmitter::XReg::x29, ARMEmitter::XReg::x30},
{ARMEmitter::XReg::x27, ARMEmitter::XReg::x28},
{ARMEmitter::XReg::x25, ARMEmitter::XReg::x26},
{ARMEmitter::XReg::x23, ARMEmitter::XReg::x24},
{ARMEmitter::XReg::x21, ARMEmitter::XReg::x22},
{ARMEmitter::XReg::x19, ARMEmitter::XReg::x20},
}};
for (auto &RegPair : CalleeSaved) {
ldp(RegPair.first, RegPair.second, PairOffset);
ldp<ARMEmitter::IndexType::POST>(RegPair.first, RegPair.second, ARMEmitter::Reg::rsp, 16);
}
}
void Arm64Emitter::SpillStaticRegs(bool FPRs, uint32_t GPRSpillMask, uint32_t FPRSpillMask) {
if (StaticRegisterAllocation()) {
for (size_t i = 0; i < SRA64.size(); i+=2) {
auto Reg1 = SRA64[i];
auto Reg2 = SRA64[i+1];
if (((1U << Reg1.GetCode()) & GPRSpillMask) &&
((1U << Reg2.GetCode()) & GPRSpillMask)) {
stp(Reg1, Reg2, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[i])));
}
else if (((1U << Reg1.GetCode()) & GPRSpillMask)) {
str(Reg1, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[i])));
}
else if (((1U << Reg2.GetCode()) & GPRSpillMask)) {
str(Reg2, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[i+1])));
}
if (!StaticRegisterAllocation()) {
return;
}
for (size_t i = 0; i < SRA64.size(); i+=2) {
auto Reg1 = SRA64[i];
auto Reg2 = SRA64[i+1];
if (((1U << Reg1.Idx()) & GPRSpillMask) &&
((1U << Reg2.Idx()) & GPRSpillMask)) {
stp<ARMEmitter::IndexType::OFFSET>(Reg1.X(), Reg2.X(), STATE.R(), offsetof(FEXCore::Core::CpuStateFrame, State.gregs[i]));
}
else if (((1U << Reg1.Idx()) & GPRSpillMask)) {
str(Reg1.X(), STATE.R(), offsetof(FEXCore::Core::CpuStateFrame, State.gregs[i]));
}
else if (((1U << Reg2.Idx()) & GPRSpillMask)) {
str(Reg2.X(), STATE.R(), offsetof(FEXCore::Core::CpuStateFrame, State.gregs[i+1]));
}
}
if (FPRs) {
if (EmitterCTX->HostFeatures.SupportsAVX) {
for (size_t i = 0; i < SRAFPR.size(); i++) {
const auto Reg = SRAFPR[i];
if (FPRs) {
if (EmitterCTX->HostFeatures.SupportsAVX) {
for (size_t i = 0; i < SRAFPR.size(); i++) {
const auto Reg = SRAFPR[i];
if (((1U << Reg.GetCode()) & FPRSpillMask) != 0) {
mov(TMP4, offsetof(Core::CpuStateFrame, State.xmm.avx.data[i][0]));
st1b(Reg.Z().VnB(), PRED_TMP_32B, SVEMemOperand(STATE, TMP4));
}
if (((1U << Reg.Idx()) & FPRSpillMask) != 0) {
mov(ARMEmitter::Size::i64Bit, TMP4.R(), offsetof(Core::CpuStateFrame, State.xmm.avx.data[i][0]));
st1b<ARMEmitter::SubRegSize::i8Bit>(Reg, PRED_TMP_32B, STATE.R(), TMP4.R());
}
} else {
}
} else {
if (GPRSpillMask && FPRSpillMask == ~0U) {
// Optimize the common case where we can spill four registers per instruction
auto TmpReg = SRA64[__builtin_ffs(GPRSpillMask)];
// Load the sse offset in to the temporary register
add(ARMEmitter::Size::i64Bit, TmpReg, STATE.R(), offsetof(FEXCore::Core::CpuStateFrame, State.xmm.sse.data[0][0]));
for (size_t i = 0; i < SRAFPR.size(); i += 4) {
const auto Reg1 = SRAFPR[i];
const auto Reg2 = SRAFPR[i + 1];
const auto Reg3 = SRAFPR[i + 2];
const auto Reg4 = SRAFPR[i + 3];
st1<ARMEmitter::SubRegSize::i64Bit>(Reg1.Q(), Reg2.Q(), Reg3.Q(), Reg4.Q(), TmpReg, 64);
}
}
else {
for (size_t i = 0; i < SRAFPR.size(); i += 2) {
const auto Reg1 = SRAFPR[i];
const auto Reg2 = SRAFPR[i + 1];
if (((1U << Reg1.GetCode()) & FPRSpillMask) &&
((1U << Reg2.GetCode()) & FPRSpillMask)) {
stp(Reg1.Q(), Reg2.Q(), MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.xmm.sse.data[i][0])));
if (((1U << Reg1.Idx()) & FPRSpillMask) &&
((1U << Reg2.Idx()) & FPRSpillMask)) {
stp<ARMEmitter::IndexType::OFFSET>(Reg1.Q(), Reg2.Q(), STATE.R(), offsetof(FEXCore::Core::CpuStateFrame, State.xmm.sse.data[i][0]));
}
else if (((1U << Reg1.GetCode()) & FPRSpillMask)) {
str(Reg1.Q(), MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.xmm.sse.data[i][0])));
else if (((1U << Reg1.Idx()) & FPRSpillMask)) {
str(Reg1.Q(), STATE.R(), offsetof(FEXCore::Core::CpuStateFrame, State.xmm.sse.data[i][0]));
}
else if (((1U << Reg2.GetCode()) & FPRSpillMask)) {
str(Reg2.Q(), MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.xmm.sse.data[i+1][0])));
else if (((1U << Reg2.Idx()) & FPRSpillMask)) {
str(Reg2.Q(), STATE.R(), offsetof(FEXCore::Core::CpuStateFrame, State.xmm.sse.data[i+1][0]));
}
}
}
@@ -252,126 +256,137 @@ void Arm64Emitter::SpillStaticRegs(bool FPRs, uint32_t GPRSpillMask, uint32_t FP
}
void Arm64Emitter::FillStaticRegs(bool FPRs, uint32_t GPRFillMask, uint32_t FPRFillMask) {
if (StaticRegisterAllocation()) {
for (size_t i = 0; i < SRA64.size(); i+=2) {
auto Reg1 = SRA64[i];
auto Reg2 = SRA64[i+1];
if (((1U << Reg1.GetCode()) & GPRFillMask) &&
((1U << Reg2.GetCode()) & GPRFillMask)) {
ldp(Reg1, Reg2, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[i])));
}
else if (((1U << Reg1.GetCode()) & GPRFillMask)) {
ldr(Reg1, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[i])));
}
else if (((1U << Reg2.GetCode()) & GPRFillMask)) {
ldr(Reg2, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[i+1])));
}
}
if (!StaticRegisterAllocation()) {
return;
}
if (FPRs) {
if (EmitterCTX->HostFeatures.SupportsAVX) {
// Set up predicate registers.
// We don't bother spilling these in SpillStaticRegs,
// since all that matters is we restore them on a fill.
// It's not a concern if they get trounced by something else.
ptrue(PRED_TMP_16B.VnB(), SVE_VL16);
ptrue(PRED_TMP_32B.VnB(), SVE_VL32);
if (FPRs) {
if (EmitterCTX->HostFeatures.SupportsAVX) {
// Set up predicate registers.
// We don't bother spilling these in SpillStaticRegs,
// since all that matters is we restore them on a fill.
// It's not a concern if they get trounced by something else.
ptrue<ARMEmitter::SubRegSize::i8Bit>(PRED_TMP_16B, ARMEmitter::PredicatePattern::SVE_VL16);
ptrue<ARMEmitter::SubRegSize::i8Bit>(PRED_TMP_32B, ARMEmitter::PredicatePattern::SVE_VL32);
for (size_t i = 0; i < SRAFPR.size(); i++) {
const auto Reg = SRAFPR[i];
if (((1U << Reg.GetCode()) & FPRFillMask) != 0) {
mov(TMP4, offsetof(Core::CpuStateFrame, State.xmm.avx.data[i][0]));
ld1b(Reg.Z().VnB(), PRED_TMP_32B.Zeroing(), SVEMemOperand(STATE, TMP4));
}
for (size_t i = 0; i < SRAFPR.size(); i++) {
const auto Reg = SRAFPR[i];
if (((1U << Reg.Idx()) & FPRFillMask) != 0) {
mov(ARMEmitter::Size::i64Bit, TMP4.R(), offsetof(Core::CpuStateFrame, State.xmm.avx.data[i][0]));
ld1b<ARMEmitter::SubRegSize::i8Bit>(Reg, PRED_TMP_32B, STATE.R(), TMP4.R());
}
} else {
}
} else {
if (GPRFillMask && FPRFillMask == ~0U) {
// Optimize the common case where we can fill four registers per instruction.
// Use one of the filling static registers before we fill it.
auto TmpReg = SRA64[__builtin_ffs(GPRFillMask)];
// Load the sse offset in to the temporary register
add(ARMEmitter::Size::i64Bit, TmpReg, STATE.R(), offsetof(FEXCore::Core::CpuStateFrame, State.xmm.sse.data[0][0]));
for (size_t i = 0; i < SRAFPR.size(); i += 4) {
const auto Reg1 = SRAFPR[i];
const auto Reg2 = SRAFPR[i + 1];
const auto Reg3 = SRAFPR[i + 2];
const auto Reg4 = SRAFPR[i + 3];
ld1<ARMEmitter::SubRegSize::i64Bit>(Reg1.Q(), Reg2.Q(), Reg3.Q(), Reg4.Q(), TmpReg, 64);
}
}
else {
for (size_t i = 0; i < SRAFPR.size(); i += 2) {
const auto Reg1 = SRAFPR[i];
const auto Reg2 = SRAFPR[i + 1];
if (((1U << Reg1.GetCode()) & FPRFillMask) &&
((1U << Reg2.GetCode()) & FPRFillMask)) {
ldp(Reg1.Q(), Reg2.Q(), MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.xmm.sse.data[i][0])));
if (((1U << Reg1.Idx()) & FPRFillMask) &&
((1U << Reg2.Idx()) & FPRFillMask)) {
ldp<ARMEmitter::IndexType::OFFSET>(Reg1.Q(), Reg2.Q(), STATE.R(), offsetof(FEXCore::Core::CpuStateFrame, State.xmm.sse.data[i][0]));
}
else if (((1U << Reg1.GetCode()) & FPRFillMask)) {
ldr(Reg1.Q(), MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.xmm.sse.data[i][0])));
else if (((1U << Reg1.Idx()) & FPRFillMask)) {
ldr(Reg1.Q(), STATE.R(), offsetof(FEXCore::Core::CpuStateFrame, State.xmm.sse.data[i][0]));
}
else if (((1U << Reg2.GetCode()) & FPRFillMask)) {
ldr(Reg2.Q(), MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.xmm.sse.data[i+1][0])));
else if (((1U << Reg2.Idx()) & FPRFillMask)) {
ldr(Reg2.Q(), STATE.R(), offsetof(FEXCore::Core::CpuStateFrame, State.xmm.sse.data[i+1][0]));
}
}
}
}
}
for (size_t i = 0; i < SRA64.size(); i+=2) {
auto Reg1 = SRA64[i];
auto Reg2 = SRA64[i+1];
if (((1U << Reg1.Idx()) & GPRFillMask) &&
((1U << Reg2.Idx()) & GPRFillMask)) {
ldp<ARMEmitter::IndexType::OFFSET>(Reg1.X(), Reg2.X(), STATE.R(), offsetof(FEXCore::Core::CpuStateFrame, State.gregs[i]));
}
else if ((1U << Reg1.Idx()) & GPRFillMask) {
ldr(Reg1.X(), STATE.R(), offsetof(FEXCore::Core::CpuStateFrame, State.gregs[i]));
}
else if ((1U << Reg2.Idx()) & GPRFillMask) {
ldr(Reg2.X(), STATE.R(), offsetof(FEXCore::Core::CpuStateFrame, State.gregs[i+1]));
}
}
}
void Arm64Emitter::PushDynamicRegsAndLR() {
void Arm64Emitter::PushDynamicRegsAndLR(FEXCore::ARMEmitter::Register TmpReg) {
const auto CanUseSVE = EmitterCTX->HostFeatures.SupportsAVX;
const auto GPRSize = (RA64.size() + 1) * Core::CPUState::GPR_REG_SIZE;
const auto GPRSize = 1 * Core::CPUState::GPR_REG_SIZE;
const auto FPRRegSize = CanUseSVE ? Core::CPUState::XMM_AVX_REG_SIZE
: Core::CPUState::XMM_SSE_REG_SIZE;
const auto FPRSize = RAFPR.size() * FPRRegSize;
const uint64_t SPOffset = AlignUp(GPRSize + FPRSize, 16);
sub(sp, sp, SPOffset);
int i = 0;
sub(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::rsp, ARMEmitter::Reg::rsp, SPOffset);
// rsp capable move
add(ARMEmitter::Size::i64Bit, TmpReg, ARMEmitter::Reg::rsp, 0);
if (CanUseSVE) {
for (const auto& RA : RAFPR) {
mov(TMP4, i * 8);
st1b(RA.Z().VnB(), PRED_TMP_32B, SVEMemOperand(sp, TMP4));
i += 4;
for (size_t i = 0; i < RAFPR.size(); i += 4) {
const auto Reg1 = RAFPR[i];
const auto Reg2 = RAFPR[i + 1];
const auto Reg3 = RAFPR[i + 2];
const auto Reg4 = RAFPR[i + 3];
st4b(Reg1, Reg2, Reg3, Reg4, PRED_TMP_32B, TmpReg, 0);
add(ARMEmitter::Size::i64Bit, TmpReg, TmpReg, 32 * 4);
}
} else {
for (const auto& RA : RAFPR) {
str(RA.Q(), MemOperand(sp, i * 8));
i += 2;
static_assert(RAFPR.size() % 4 == 0, "Needs to have multiple of 4 FPRs for RA");
for (size_t i = 0; i < RAFPR.size(); i += 4) {
const auto Reg1 = RAFPR[i];
const auto Reg2 = RAFPR[i + 1];
const auto Reg3 = RAFPR[i + 2];
const auto Reg4 = RAFPR[i + 3];
st1<ARMEmitter::SubRegSize::i64Bit>(Reg1.Q(), Reg2.Q(), Reg3.Q(), Reg4.Q(), TmpReg, 64);
}
}
#if 0 // All GPRs should be caller saved
for (const auto& RA : RA64) {
str(RA, MemOperand(sp, i * 8));
i++;
}
#endif
str(lr, MemOperand(sp, i * 8));
str(ARMEmitter::XReg::lr, TmpReg, 0);
}
void Arm64Emitter::PopDynamicRegsAndLR() {
const auto CanUseSVE = EmitterCTX->HostFeatures.SupportsAVX;
const auto GPRSize = (RA64.size() + 1) * Core::CPUState::GPR_REG_SIZE;
const auto FPRRegSize = CanUseSVE ? Core::CPUState::XMM_AVX_REG_SIZE
: Core::CPUState::XMM_SSE_REG_SIZE;
const auto FPRSize = RAFPR.size() * FPRRegSize;
const uint64_t SPOffset = AlignUp(GPRSize + FPRSize, 16);
int i = 0;
if (CanUseSVE) {
for (const auto& RA : RAFPR) {
mov(TMP4, i * 8);
ld1b(RA.Z().VnB(), PRED_TMP_32B.Zeroing(), SVEMemOperand(sp, TMP4));
i += 4;
for (size_t i = 0; i < RAFPR.size(); i += 4) {
const auto Reg1 = RAFPR[i];
const auto Reg2 = RAFPR[i + 1];
const auto Reg3 = RAFPR[i + 2];
const auto Reg4 = RAFPR[i + 3];
ld4b(Reg1, Reg2, Reg3, Reg4, PRED_TMP_32B, ARMEmitter::Reg::rsp);
add(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::rsp, ARMEmitter::Reg::rsp, 32 * 4);
}
} else {
for (const auto& RA : RAFPR) {
ldr(RA.Q(), MemOperand(sp, i * 8));
i += 2;
for (size_t i = 0; i < RAFPR.size(); i += 4) {
const auto Reg1 = RAFPR[i];
const auto Reg2 = RAFPR[i + 1];
const auto Reg3 = RAFPR[i + 2];
const auto Reg4 = RAFPR[i + 3];
ld1<ARMEmitter::SubRegSize::i64Bit>(Reg1.Q(), Reg2.Q(), Reg3.Q(), Reg4.Q(), ARMEmitter::Reg::rsp, 64);
}
}
#if 0 // All GPRs should be caller saved
for (const auto& RA : RA64) {
ldr(RA, MemOperand(sp, i * 8));
i++;
}
#endif
ldr(lr, MemOperand(sp, i * 8));
add(sp, sp, SPOffset);
ldr<ARMEmitter::IndexType::POST>(ARMEmitter::XReg::lr, ARMEmitter::Reg::rsp, 16);
}
void Arm64Emitter::Align16B() {
@@ -1,5 +1,9 @@
#pragma once
#include "FEXCore/Utils/EnumUtils.h"
#include "Interface/Core/ArchHelpers/CodeEmitter/Emitter.h"
#include "Interface/Core/ArchHelpers/CodeEmitter/Registers.h"
#include "Interface/Core/Dispatcher/Dispatcher.h"
#include "Interface/Core/ObjectCache/Relocations.h"
@@ -8,6 +12,9 @@
#include <aarch64/cpu-aarch64.h>
#include <aarch64/operands-aarch64.h>
#include <platform-vixl.h>
#ifdef VIXL_DISASSEMBLER
#include <aarch64/disasm-aarch64.h>
#endif
#ifdef VIXL_SIMULATOR
#include <aarch64/simulator-aarch64.h>
#include <aarch64/simulator-constants-aarch64.h>
@@ -21,78 +28,70 @@
#include <utility>
namespace FEXCore::CPU {
using namespace vixl;
using namespace vixl::aarch64;
// All but x29 are caller saved
const std::array<aarch64::Register, 16> SRA64 = {
x4, x5, x6, x7, x8, x9, x10, x11,
x12, x18, x17, x16, x15, x14, x13, x29
constexpr std::array<FEXCore::ARMEmitter::Register, 16> SRA64 = {
FEXCore::ARMEmitter::Reg::r4, FEXCore::ARMEmitter::Reg::r5, FEXCore::ARMEmitter::Reg::r6, FEXCore::ARMEmitter::Reg::r7, FEXCore::ARMEmitter::Reg::r8, FEXCore::ARMEmitter::Reg::r9, FEXCore::ARMEmitter::Reg::r10, FEXCore::ARMEmitter::Reg::r11,
FEXCore::ARMEmitter::Reg::r12, FEXCore::ARMEmitter::Reg::r18, FEXCore::ARMEmitter::Reg::r17, FEXCore::ARMEmitter::Reg::r16, FEXCore::ARMEmitter::Reg::r15, FEXCore::ARMEmitter::Reg::r14, FEXCore::ARMEmitter::Reg::r13, FEXCore::ARMEmitter::Reg::r29
};
// All are callee saved
const std::array<aarch64::Register, 9> RA64 = {
x20, x21, x22, x23, x24, x25, x26, x27,
x19
constexpr std::array<FEXCore::ARMEmitter::Register, 9> RA64 = {
FEXCore::ARMEmitter::Reg::r20, FEXCore::ARMEmitter::Reg::r21, FEXCore::ARMEmitter::Reg::r22, FEXCore::ARMEmitter::Reg::r23, FEXCore::ARMEmitter::Reg::r24, FEXCore::ARMEmitter::Reg::r25, FEXCore::ARMEmitter::Reg::r26, FEXCore::ARMEmitter::Reg::r27,
FEXCore::ARMEmitter::Reg::r19
};
const std::array<std::pair<aarch64::Register, aarch64::Register>, 4> RA64Pair = {{
{x20, x21},
{x22, x23},
{x24, x25},
{x26, x27},
}};
const std::array<std::pair<aarch64::Register, aarch64::Register>, 4> RA32Pair = {{
{w20, w21},
{w22, w23},
{w24, w25},
{w26, w27},
constexpr std::array<std::pair<FEXCore::ARMEmitter::Register, FEXCore::ARMEmitter::Register>, 4> RA64Pair = {{
{FEXCore::ARMEmitter::Reg::r20, FEXCore::ARMEmitter::Reg::r21},
{FEXCore::ARMEmitter::Reg::r22, FEXCore::ARMEmitter::Reg::r23},
{FEXCore::ARMEmitter::Reg::r24, FEXCore::ARMEmitter::Reg::r25},
{FEXCore::ARMEmitter::Reg::r26, FEXCore::ARMEmitter::Reg::r27},
}};
// All are caller saved
const std::array<aarch64::VRegister, 16> SRAFPR = {
v16, v17, v18, v19, v20, v21, v22, v23,
v24, v25, v26, v27, v28, v29, v30, v31
constexpr std::array<FEXCore::ARMEmitter::VRegister, 16> SRAFPR = {
FEXCore::ARMEmitter::VReg::v16, FEXCore::ARMEmitter::VReg::v17, FEXCore::ARMEmitter::VReg::v18, FEXCore::ARMEmitter::VReg::v19, FEXCore::ARMEmitter::VReg::v20, FEXCore::ARMEmitter::VReg::v21, FEXCore::ARMEmitter::VReg::v22, FEXCore::ARMEmitter::VReg::v23,
FEXCore::ARMEmitter::VReg::v24, FEXCore::ARMEmitter::VReg::v25, FEXCore::ARMEmitter::VReg::v26, FEXCore::ARMEmitter::VReg::v27, FEXCore::ARMEmitter::VReg::v28, FEXCore::ARMEmitter::VReg::v29, FEXCore::ARMEmitter::VReg::v30, FEXCore::ARMEmitter::VReg::v31
};
// v8..v15 = (lower 64bits) Callee saved
const std::array<aarch64::VRegister, 12> RAFPR = {
/*v0, v1, v2, v3,*/v4, v5, v6, v7, // v0 ~ v3 are used as temps
v8, v9, v10, v11, v12, v13, v14, v15
constexpr std::array<FEXCore::ARMEmitter::VRegister, 12> RAFPR = {
/*FEXCore::ARMEmitter::VReg::v0, FEXCore::ARMEmitter::VReg::v1, FEXCore::ARMEmitter::VReg::v2, FEXCore::ARMEmitter::VReg::v3,*/FEXCore::ARMEmitter::VReg::v4, FEXCore::ARMEmitter::VReg::v5, FEXCore::ARMEmitter::VReg::v6, FEXCore::ARMEmitter::VReg::v7, // FEXCore::ARMEmitter::VReg::v0 ~ FEXCore::ARMEmitter::VReg::v3 are used as temps
FEXCore::ARMEmitter::VReg::v8, FEXCore::ARMEmitter::VReg::v9, FEXCore::ARMEmitter::VReg::v10, FEXCore::ARMEmitter::VReg::v11, FEXCore::ARMEmitter::VReg::v12, FEXCore::ARMEmitter::VReg::v13, FEXCore::ARMEmitter::VReg::v14, FEXCore::ARMEmitter::VReg::v15
};
// Contains the address to the currently available CPU state
#define STATE x28
constexpr auto STATE = FEXCore::ARMEmitter::XReg::x28;
// GPR temporaries. Only x3 can be used across spill boundaries
// so if these ever need to change, be very careful about that.
#define TMP1 x0
#define TMP2 x1
#define TMP3 x2
#define TMP4 x3
constexpr auto TMP1 = FEXCore::ARMEmitter::XReg::x0;
constexpr auto TMP2 = FEXCore::ARMEmitter::XReg::x1;
constexpr auto TMP3 = FEXCore::ARMEmitter::XReg::x2;
constexpr auto TMP4 = FEXCore::ARMEmitter::XReg::x3;
// Vector temporaries
#define VTMP1 v1
#define VTMP2 v2
#define VTMP3 v3
constexpr auto VTMP1 = FEXCore::ARMEmitter::VReg::v0;
constexpr auto VTMP2 = FEXCore::ARMEmitter::VReg::v1;
constexpr auto VTMP3 = FEXCore::ARMEmitter::VReg::v2;
constexpr auto VTMP4 = FEXCore::ARMEmitter::VReg::v3;
// Predicate register temporaries (used when AVX support is enabled)
// PRED_TMP_16B indicates a predicate register that indicates the first 16 bytes set to 1.
// PRED_TMP_32B indicates a predicate register that indicates the first 32 bytes set to 1.
#define PRED_TMP_16B p6
#define PRED_TMP_32B p7
constexpr FEXCore::ARMEmitter::PRegister PRED_TMP_16B = FEXCore::ARMEmitter::PReg::p6;
constexpr FEXCore::ARMEmitter::PRegister PRED_TMP_32B = FEXCore::ARMEmitter::PReg::p7;
// This class contains common emitter utility functions that can
// be used by both Arm64 JIT and ARM64 Dispatcher
class Arm64Emitter : public vixl::aarch64::Assembler {
class Arm64Emitter : public FEXCore::ARMEmitter::Emitter {
protected:
Arm64Emitter(FEXCore::Context::Context *ctx, size_t size);
~Arm64Emitter();
FEXCore::Context::Context *EmitterCTX;
vixl::aarch64::CPU CPU;
void LoadConstant(vixl::aarch64::Register Reg, uint64_t Constant, bool NOPPad = false);
void LoadConstant(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register Reg, uint64_t Constant, bool NOPPad = false);
// NOTE: These functions WILL clobber the register TMP4 if AVX support is enabled
// and FPRs are being spilled or filled. If only GPRs are spilled/filled, then
@@ -106,13 +105,14 @@ protected:
// We can't guarantee only the lower 64bits are used so flush everything
static constexpr uint32_t CALLER_FPR_MASK = ~0U;
void PushDynamicRegsAndLR();
void PushDynamicRegsAndLR(FEXCore::ARMEmitter::Register TmpReg);
void PopDynamicRegsAndLR();
void PushCalleeSavedRegisters();
void PopCalleeSavedRegisters();
void Align16B();
#ifdef VIXL_SIMULATOR
// Generates a vixl simulator runtime call.
//
@@ -124,61 +124,64 @@ protected:
// 2) Simulator wrapper handler
// 3) Function to call
// 4) Style of the function call (Call versus tail-call)
template<typename R, typename... P>
void GenerateRuntimeCall(R (*Function)(P...)) {
uintptr_t SimulatorWrapperAddress = reinterpret_cast<uintptr_t>(
&(Simulator::RuntimeCallStructHelper<R, P...>::Wrapper));
&(vixl::aarch64::Simulator::RuntimeCallStructHelper<R, P...>::Wrapper));
uintptr_t FunctionAddress = reinterpret_cast<uintptr_t>(Function);
hlt(kRuntimeCallOpcode);
hlt(vixl::aarch64::kRuntimeCallOpcode);
// Simulator wrapper address pointer.
dc(SimulatorWrapperAddress);
dc64(SimulatorWrapperAddress);
// Runtime function address to call
dc(FunctionAddress);
dc64(FunctionAddress);
// Call type
dc32(kCallRuntime);
dc32(vixl::aarch64::kCallRuntime);
}
template<typename R, typename... P>
void GenerateIndirectRuntimeCall(vixl::aarch64::Register Reg) {
void GenerateIndirectRuntimeCall(ARMEmitter::Register Reg) {
uintptr_t SimulatorWrapperAddress = reinterpret_cast<uintptr_t>(
&(Simulator::RuntimeCallStructHelper<R, P...>::Wrapper));
&(vixl::aarch64::Simulator::RuntimeCallStructHelper<R, P...>::Wrapper));
hlt(kIndirectRuntimeCallOpcode);
hlt(vixl::aarch64::kIndirectRuntimeCallOpcode);
// Simulator wrapper address pointer.
dc(SimulatorWrapperAddress);
dc64(SimulatorWrapperAddress);
// Register that contains the function to call
dc(Reg.GetCode());
dc32(Reg.Idx());
// Call type
dc32(kCallRuntime);
dc32(vixl::aarch64::kCallRuntime);
}
template<>
void GenerateIndirectRuntimeCall<float, __uint128_t>(vixl::aarch64::Register Reg) {
void GenerateIndirectRuntimeCall<float, __uint128_t>(ARMEmitter::Register Reg) {
uintptr_t SimulatorWrapperAddress = reinterpret_cast<uintptr_t>(
&(Simulator::RuntimeCallStructHelper<float, __uint128_t>::Wrapper));
&(vixl::aarch64::Simulator::RuntimeCallStructHelper<float, __uint128_t>::Wrapper));
hlt(kIndirectRuntimeCallOpcode);
hlt(vixl::aarch64::kIndirectRuntimeCallOpcode);
// Simulator wrapper address pointer.
dc(SimulatorWrapperAddress);
dc64(SimulatorWrapperAddress);
// Register that contains the function to call
dc(Reg.GetCode());
dc32(Reg.Idx());
// Call type
dc32(kCallRuntime);
dc32(vixl::aarch64::kCallRuntime);
}
#endif
#ifdef VIXL_DISASSEMBLER
vixl::aarch64::PrintDisassembler Disasm {stderr};
#endif
FEX_CONFIG_OPT(StaticRegisterAllocation, SRA);
};
@@ -0,0 +1,912 @@
/* ALU instruction emitters.
*
* Almost all of these operations have `ARMEmitter::Size` as their first argument.
* This allows both 32-bit and 64-bit selection of how that instruction is going to operate.
*
* Some emitter operations explicitly use `XRegister` or `WRegister`.
* This is usually due to the instruction only supporting one operating size.
* Although in some cases is a minor convenience without any performance implications.
*
* FEX-Emu ALU operations usually have a 32-bit or 64-bit operating size encoded in the IR operation,
* This allows FEX to use a single helper function which decodes to both handlers.
*/
public:
// PC relative
void adr(FEXCore::ARMEmitter::Register rd, uint32_t Imm) {
constexpr uint32_t Op = 0b0001'0000 << 24;
DataProcessing_PCRel_Imm(Op, rd, Imm);
}
void adr(FEXCore::ARMEmitter::Register rd, BackwardLabel const* Label) {
int32_t Imm = static_cast<int32_t>(Label->Location - GetCursorAddress<uint8_t*>());
LOGMAN_THROW_A_FMT(Imm >= -1048576 && Imm <= 1048575, "Unscaled offset too large");
constexpr uint32_t Op = 0b0001'0000 << 24;
DataProcessing_PCRel_Imm(Op, rd, Imm);
}
void adr(FEXCore::ARMEmitter::Register rd, ForwardLabel *Label) {
Label->Insts.emplace_back(ForwardLabel::Instructions{ .Location = GetCursorAddress<uint8_t*>(), .Type = ForwardLabel::Instructions::InstType::ADR });
constexpr uint32_t Op = 0b0001'0000 << 24;
DataProcessing_PCRel_Imm(Op, rd, 0);
}
void adr(FEXCore::ARMEmitter::Register rd, BiDirectionalLabel *Label) {
if (Label->Backward.Location) {
adr(rd, &Label->Backward);
}
else {
adr(rd, &Label->Forward);
}
}
void adrp(FEXCore::ARMEmitter::Register rd, uint32_t Imm) {
constexpr uint32_t Op = 0b1001'0000 << 24;
DataProcessing_PCRel_Imm(Op, rd, Imm);
}
void adrp(FEXCore::ARMEmitter::Register rd, BackwardLabel const* Label) {
int64_t Imm = reinterpret_cast<int64_t>(Label->Location) - (GetCursorAddress<int64_t>() & ~0xFFFLL);
LOGMAN_THROW_A_FMT(Imm >= -4294967296 && Imm <= 4294963200 && (Imm & 0xFFF) == 0, "Unscaled offset too large");
constexpr uint32_t Op = 0b1001'0000 << 24;
DataProcessing_PCRel_Imm(Op, rd, Imm);
}
void adrp(FEXCore::ARMEmitter::Register rd, ForwardLabel *Label) {
Label->Insts.emplace_back(ForwardLabel::Instructions{ .Location = GetCursorAddress<uint8_t*>(), .Type = ForwardLabel::Instructions::InstType::ADRP });
constexpr uint32_t Op = 0b1001'0000 << 24;
DataProcessing_PCRel_Imm(Op, rd, 0);
}
void adrp(FEXCore::ARMEmitter::Register rd, BiDirectionalLabel *Label) {
if (Label->Backward.Location) {
adrp(rd, &Label->Backward);
}
else {
adrp(rd, &Label->Forward);
}
}
// Add/subtract immediate
void add(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, uint32_t Imm, bool LSL12 = false) {
constexpr uint32_t Op = 0b0001'0001'0 << 23;
DataProcessing_AddSub_Imm(Op, s, rd, rn, Imm, LSL12);
}
void adds(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, uint32_t Imm, bool LSL12 = false) {
constexpr uint32_t Op = 0b0011'0001'0 << 23;
DataProcessing_AddSub_Imm(Op, s, rd, rn, Imm, LSL12);
}
void sub(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, uint32_t Imm, bool LSL12 = false) {
constexpr uint32_t Op = 0b0101'0001'0 << 23;
DataProcessing_AddSub_Imm(Op, s, rd, rn, Imm, LSL12);
}
void cmp(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rn, uint32_t Imm, bool LSL12 = false) {
constexpr uint32_t Op = 0b0111'0001'0 << 23;
DataProcessing_AddSub_Imm(Op, s, FEXCore::ARMEmitter::Reg::rsp, rn, Imm, LSL12);
}
void subs(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, uint32_t Imm, bool LSL12 = false) {
constexpr uint32_t Op = 0b0111'0001'0 << 23;
DataProcessing_AddSub_Imm(Op, s, rd, rn, Imm, LSL12);
}
// Logical immediate
void and_(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, uint64_t Imm) {
uint32_t n, immr, imms;
[[maybe_unused]] const auto IsImm = vixl::aarch64::Assembler::IsImmLogical(Imm,
RegSizeInBits(s),
&n,
&imms,
&immr);
LOGMAN_THROW_A_FMT(IsImm, "Couldn't encode immediate to logical op");
and_(s, rd, rn, n, immr, imms);
}
void bic(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, uint64_t Imm) {
and_(s, rd, rn, ~Imm);
}
void ands(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, uint64_t Imm) {
uint32_t n, immr, imms;
[[maybe_unused]] const auto IsImm = vixl::aarch64::Assembler::IsImmLogical(Imm,
RegSizeInBits(s),
&n,
&imms,
&immr);
LOGMAN_THROW_A_FMT(IsImm, "Couldn't encode immediate to logical op");
ands(s, rd, rn, n, immr, imms);
}
void bics(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, uint64_t Imm) {
ands(s, rd, rn, ~Imm);
}
void orr(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, uint64_t Imm) {
uint32_t n, immr, imms;
[[maybe_unused]] const auto IsImm = vixl::aarch64::Assembler::IsImmLogical(Imm,
RegSizeInBits(s),
&n,
&imms,
&immr);
LOGMAN_THROW_A_FMT(IsImm, "Couldn't encode immediate to logical op");
orr(s, rd, rn, n, immr, imms);
}
void eor(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, uint64_t Imm) {
uint32_t n, immr, imms;
[[maybe_unused]] const auto IsImm = vixl::aarch64::Assembler::IsImmLogical(Imm,
RegSizeInBits(s),
&n,
&imms,
&immr);
LOGMAN_THROW_A_FMT(IsImm, "Couldn't encode immediate to logical op");
eor(s, rd, rn, n, immr, imms);
}
// Move wide immediate
void movn(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, uint32_t Imm, uint32_t Offset = 0) {
LOGMAN_THROW_A_FMT((Imm & 0xFFFF0000U) == 0, "Upper bits of move wide not valid");
LOGMAN_THROW_A_FMT((Offset % 16) == 0, "Offset must be 16bit aligned");
constexpr uint32_t Op = 0b001'0010'100 << 21;
DataProcessing_MoveWide(Op, s, rd, Imm, Offset >> 4);
}
void mov(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, uint32_t Imm) {
movz(s, rd, Imm, 0);
}
void mov(FEXCore::ARMEmitter::XRegister rd, uint32_t Imm) {
movz(FEXCore::ARMEmitter::Size::i64Bit, rd.R(), Imm, 0);
}
void mov(FEXCore::ARMEmitter::WRegister rd, uint32_t Imm) {
movz(FEXCore::ARMEmitter::Size::i32Bit, rd.R(), Imm, 0);
}
void movz(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, uint32_t Imm, uint32_t Offset = 0) {
LOGMAN_THROW_A_FMT((Imm & 0xFFFF0000U) == 0, "Upper bits of move wide not valid");
LOGMAN_THROW_A_FMT((Offset % 16) == 0, "Offset must be 16bit aligned");
constexpr uint32_t Op = 0b101'0010'100 << 21;
DataProcessing_MoveWide(Op, s, rd, Imm, Offset >> 4);
}
void movk(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, uint32_t Imm, uint32_t Offset = 0) {
LOGMAN_THROW_A_FMT((Imm & 0xFFFF0000U) == 0, "Upper bits of move wide not valid");
LOGMAN_THROW_A_FMT((Offset % 16) == 0, "Offset must be 16bit aligned");
constexpr uint32_t Op = 0b111'0010'100 << 21;
DataProcessing_MoveWide(Op, s, rd, Imm, Offset >> 4);
}
void movn(FEXCore::ARMEmitter::XRegister rd, uint32_t Imm, uint32_t Offset = 0) {
movn(FEXCore::ARMEmitter::Size::i64Bit, rd.R(), Imm, Offset);
}
void movz(FEXCore::ARMEmitter::XRegister rd, uint32_t Imm, uint32_t Offset = 0) {
movz(FEXCore::ARMEmitter::Size::i64Bit, rd.R(), Imm, Offset);
}
void movk(FEXCore::ARMEmitter::XRegister rd, uint32_t Imm, uint32_t Offset = 0) {
movk(FEXCore::ARMEmitter::Size::i64Bit, rd.R(), Imm, Offset);
}
void movn(FEXCore::ARMEmitter::WRegister rd, uint32_t Imm, uint32_t Offset = 0) {
movn(FEXCore::ARMEmitter::Size::i32Bit, rd.R(), Imm, Offset);
}
void movz(FEXCore::ARMEmitter::WRegister rd, uint32_t Imm, uint32_t Offset = 0) {
movz(FEXCore::ARMEmitter::Size::i32Bit, rd.R(), Imm, Offset);
}
void movk(FEXCore::ARMEmitter::WRegister rd, uint32_t Imm, uint32_t Offset = 0) {
movk(FEXCore::ARMEmitter::Size::i32Bit, rd.R(), Imm, Offset);
}
// Bitfield
void sxtb(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn) {
sbfm(s, rd, rn, 0, 7);
}
void sxth(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn) {
sbfm(s, rd, rn, 0, 15);
}
void sxtw(FEXCore::ARMEmitter::XRegister rd, FEXCore::ARMEmitter::XRegister rn) {
sbfm(ARMEmitter::Size::i64Bit, rd, rn, 0, 31);
}
void sbfx(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, uint32_t lsb, uint32_t width) {
LOGMAN_THROW_A_FMT(width > 0, "sbfx needs width > 0");
LOGMAN_THROW_A_FMT((lsb + width) <= RegSizeInBits(s), "Tried to sbfx a region larger than the register");
sbfm(s, rd, rn, lsb, lsb + width - 1);
}
void asr(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, uint32_t shift) {
LOGMAN_THROW_A_FMT(shift <= RegSizeInBits(s), "Tried to asr a region larger than the register");
sbfm(s, rd, rn, shift, RegSizeInBits(s) - 1);
}
void uxtb(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn) {
ubfm(s, rd, rn, 0, 7);
}
void uxth(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn) {
ubfm(s, rd, rn, 0, 15);
}
void uxtw(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn) {
ubfm(s, rd, rn, 0, 31);
}
void ubfm(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, uint32_t immr, uint32_t imms) {
constexpr uint32_t Op = 0b0101'0011'00 << 22;
DataProcessing_Logical_Imm(Op, s, rd, rn, s == ARMEmitter::Size::i64Bit, immr, imms);
}
void lsl(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, uint32_t shift) {
const auto RegSize = RegSizeInBits(s);
LOGMAN_THROW_A_FMT(shift < RegSize, "Tried to asr a region larger than the register");
ubfm(s, rd, rn, (RegSize - shift) % RegSize, RegSize - shift - 1);
}
void lsr(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, uint32_t shift) {
const auto RegSize = RegSizeInBits(s);
LOGMAN_THROW_A_FMT(shift < RegSize, "Tried to asr a region larger than the register");
ubfm(s, rd, rn, shift, RegSize - 1);
}
void ubfx(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, uint32_t lsb, uint32_t width) {
LOGMAN_THROW_A_FMT(width > 0, "ubfx needs width > 0");
LOGMAN_THROW_A_FMT((lsb + width) <= RegSizeInBits(s), "Tried to ubfx a region larger than the register");
ubfm(s, rd, rn, lsb, lsb + width - 1);
}
void bfi(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, uint32_t lsb, uint32_t width) {
const auto RegSize = RegSizeInBits(s);
LOGMAN_THROW_A_FMT(width > 0, "sbfx needs width > 0");
LOGMAN_THROW_A_FMT((lsb + width) <= RegSize, "Tried to sbfx a region larger than the register");
bfm(s, rd, rn, (RegSize - lsb) & (RegSize - 1), width - 1);
}
// Extract
void extr(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm, uint32_t Imm) {
constexpr uint32_t Op = 0b001'0011'100 << 21;
LOGMAN_THROW_A_FMT(Imm < RegSizeInBits(s), "Tried to extr a region larger than the register");
DataProcessing_Extract(Op, s, rd, rn, rm, Imm);
}
void ror(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, uint32_t Imm) {
LOGMAN_THROW_A_FMT(Imm < RegSizeInBits(s), "Tried to extr a region larger than the register");
extr(s, rd, rn, rn, Imm);
}
// Data processing - 2 source
void udiv(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm) {
constexpr uint32_t Op = (0b001'1010'110U << 21) |
(0b0000'10U << 10);
DataProcessing_2Source(Op, s, rd, rn, rm);
}
void sdiv(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm) {
constexpr uint32_t Op = (0b001'1010'110U << 21) |
(0b0000'11U << 10);
DataProcessing_2Source(Op, s, rd, rn, rm);
}
void lslv(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm) {
constexpr uint32_t Op = (0b001'1010'110U << 21) |
(0b0010'00U << 10);
DataProcessing_2Source(Op, s, rd, rn, rm);
}
void lsrv(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm) {
constexpr uint32_t Op = (0b001'1010'110U << 21) |
(0b0010'01U << 10);
DataProcessing_2Source(Op, s, rd, rn, rm);
}
void asrv(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm) {
constexpr uint32_t Op = (0b001'1010'110U << 21) |
(0b0010'10U << 10);
DataProcessing_2Source(Op, s, rd, rn, rm);
}
void rorv(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm) {
constexpr uint32_t Op = (0b001'1010'110U << 21) |
(0b0010'11U << 10);
DataProcessing_2Source(Op, s, rd, rn, rm);
}
void crc32b(FEXCore::ARMEmitter::WRegister rd, FEXCore::ARMEmitter::WRegister rn, FEXCore::ARMEmitter::WRegister rm) {
constexpr uint32_t Op = (0b001'1010'110U << 21) |
(0b0100'00U << 10);
DataProcessing_2Source(Op, ARMEmitter::Size::i32Bit, rd, rn, rm);
}
void crc32h(FEXCore::ARMEmitter::WRegister rd, FEXCore::ARMEmitter::WRegister rn, FEXCore::ARMEmitter::WRegister rm) {
constexpr uint32_t Op = (0b001'1010'110U << 21) |
(0b0100'01U << 10);
DataProcessing_2Source(Op, ARMEmitter::Size::i32Bit, rd, rn, rm);
}
void crc32w(FEXCore::ARMEmitter::WRegister rd, FEXCore::ARMEmitter::WRegister rn, FEXCore::ARMEmitter::WRegister rm) {
constexpr uint32_t Op = (0b001'1010'110U << 21) |
(0b0100'10U << 10);
DataProcessing_2Source(Op, ARMEmitter::Size::i32Bit, rd, rn, rm);
}
void crc32cb(FEXCore::ARMEmitter::WRegister rd, FEXCore::ARMEmitter::WRegister rn, FEXCore::ARMEmitter::WRegister rm) {
constexpr uint32_t Op = (0b001'1010'110U << 21) |
(0b0101'00U << 10);
DataProcessing_2Source(Op, ARMEmitter::Size::i32Bit, rd, rn, rm);
}
void crc32ch(FEXCore::ARMEmitter::WRegister rd, FEXCore::ARMEmitter::WRegister rn, FEXCore::ARMEmitter::WRegister rm) {
constexpr uint32_t Op = (0b001'1010'110U << 21) |
(0b0101'01U << 10);
DataProcessing_2Source(Op, ARMEmitter::Size::i32Bit, rd, rn, rm);
}
void crc32cw(FEXCore::ARMEmitter::WRegister rd, FEXCore::ARMEmitter::WRegister rn, FEXCore::ARMEmitter::WRegister rm) {
constexpr uint32_t Op = (0b001'1010'110U << 21) |
(0b0101'10U << 10);
DataProcessing_2Source(Op, ARMEmitter::Size::i32Bit, rd, rn, rm);
}
void subp(FEXCore::ARMEmitter::XRegister rd, FEXCore::ARMEmitter::XRegister rn, FEXCore::ARMEmitter::XRegister rm) {
constexpr uint32_t Op = (0b001'1010'110U << 21) |
(0b0000'00U << 10);
DataProcessing_2Source(Op, FEXCore::ARMEmitter::Size::i64Bit, rd, rn, rm);
}
void irg(FEXCore::ARMEmitter::XRegister rd, FEXCore::ARMEmitter::XRegister rn, FEXCore::ARMEmitter::XRegister rm) {
constexpr uint32_t Op = (0b001'1010'110U << 21) |
(0b0001'00U << 10);
DataProcessing_2Source(Op, FEXCore::ARMEmitter::Size::i64Bit, rd, rn, rm);
}
void gmi(FEXCore::ARMEmitter::XRegister rd, FEXCore::ARMEmitter::XRegister rn, FEXCore::ARMEmitter::XRegister rm) {
constexpr uint32_t Op = (0b001'1010'110U << 21) |
(0b0001'01U << 10);
DataProcessing_2Source(Op, FEXCore::ARMEmitter::Size::i64Bit, rd, rn, rm);
}
void pacga(FEXCore::ARMEmitter::XRegister rd, FEXCore::ARMEmitter::XRegister rn, FEXCore::ARMEmitter::XRegister rm) {
constexpr uint32_t Op = (0b001'1010'110U << 21) |
(0b0011'00U << 10);
DataProcessing_2Source(Op, FEXCore::ARMEmitter::Size::i64Bit, rd, rn, rm);
}
void crc32x(FEXCore::ARMEmitter::XRegister rd, FEXCore::ARMEmitter::XRegister rn, FEXCore::ARMEmitter::XRegister rm) {
constexpr uint32_t Op = (0b001'1010'110U << 21) |
(0b0100'11U << 10);
DataProcessing_2Source(Op, FEXCore::ARMEmitter::Size::i64Bit, rd, rn, rm);
}
void crc32cx(FEXCore::ARMEmitter::XRegister rd, FEXCore::ARMEmitter::XRegister rn, FEXCore::ARMEmitter::XRegister rm) {
constexpr uint32_t Op = (0b001'1010'110U << 21) |
(0b0101'11U << 10);
DataProcessing_2Source(Op, FEXCore::ARMEmitter::Size::i64Bit, rd, rn, rm);
}
void subps(FEXCore::ARMEmitter::XRegister rd, FEXCore::ARMEmitter::XRegister rn, FEXCore::ARMEmitter::XRegister rm) {
constexpr uint32_t Op = (0b011'1010'110U << 21) |
(0b0000'00U << 10);
DataProcessing_2Source(Op, FEXCore::ARMEmitter::Size::i64Bit, rd, rn, rm);
}
// Data processing - 1 source
void rbit(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn) {
constexpr uint32_t Op = (0b101'1010'110U << 21) |
(0b0'0000U << 16) |
(0b0000'00U << 10);
DataProcessing_1Source(Op, s, rd, rn);
}
void rev16(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn) {
constexpr uint32_t Op = (0b101'1010'110U << 21) |
(0b0'0000U << 16) |
(0b0000'01U << 10);
DataProcessing_1Source(Op, s, rd, rn);
}
void rev(FEXCore::ARMEmitter::WRegister rd, FEXCore::ARMEmitter::WRegister rn) {
constexpr uint32_t Op = (0b101'1010'110U << 21) |
(0b0'0000U << 16) |
(0b0000'10U << 10);
DataProcessing_1Source(Op, FEXCore::ARMEmitter::Size::i32Bit, rd, rn);
}
void rev32(FEXCore::ARMEmitter::XRegister rd, FEXCore::ARMEmitter::XRegister rn) {
constexpr uint32_t Op = (0b101'1010'110U << 21) |
(0b0'0000U << 16) |
(0b0000'10U << 10);
DataProcessing_1Source(Op, FEXCore::ARMEmitter::Size::i64Bit, rd, rn);
}
void clz(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn) {
constexpr uint32_t Op = (0b101'1010'110U << 21) |
(0b0'0000U << 16) |
(0b0001'00U << 10);
DataProcessing_1Source(Op, s, rd, rn);
}
void cls(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn) {
constexpr uint32_t Op = (0b101'1010'110U << 21) |
(0b0'0000U << 16) |
(0b0001'01U << 10);
DataProcessing_1Source(Op, s, rd, rn);
}
void rev(FEXCore::ARMEmitter::XRegister rd, FEXCore::ARMEmitter::XRegister rn) {
constexpr uint32_t Op = (0b101'1010'110U << 21) |
(0b0'0000U << 16) |
(0b0000'11U << 10);
DataProcessing_1Source(Op, FEXCore::ARMEmitter::Size::i64Bit, rd, rn);
}
void rev(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn) {
uint32_t Op = (0b101'1010'110U << 21) |
(0b0'0000U << 16) |
(0b0000'10U << 10) |
(s == ARMEmitter::Size::i64Bit ? (1U << 10) : 0);
DataProcessing_1Source(Op, s, rd, rn);
}
// TODO: PAUTH
// Logical - shifted register
void mov(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn) {
orr(s, rd, FEXCore::ARMEmitter::Reg::zr, rn, ARMEmitter::ShiftType::LSL, 0);
}
void mov(FEXCore::ARMEmitter::XRegister rd, FEXCore::ARMEmitter::XRegister rn) {
orr(FEXCore::ARMEmitter::Size::i64Bit, rd.R(), FEXCore::ARMEmitter::Reg::zr, rn.R(), ARMEmitter::ShiftType::LSL, 0);
}
void mov(FEXCore::ARMEmitter::WRegister rd, FEXCore::ARMEmitter::WRegister rn) {
orr(FEXCore::ARMEmitter::Size::i32Bit, rd.R(), FEXCore::ARMEmitter::Reg::zr, rn.R(), ARMEmitter::ShiftType::LSL, 0);
}
void mvn(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::ShiftType Shift = FEXCore::ARMEmitter::ShiftType::LSL, uint32_t amt = 0) {
orn(s, rd, FEXCore::ARMEmitter::Reg::zr, rn, Shift, amt);
}
void and_(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm, FEXCore::ARMEmitter::ShiftType Shift = FEXCore::ARMEmitter::ShiftType::LSL, uint32_t amt = 0) {
constexpr uint32_t Op = 0b000'1010'000U << 21;
DataProcessing_Shifted_Reg(Op, s, rd, rn, rm, Shift, amt);
}
void ands(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm, FEXCore::ARMEmitter::ShiftType Shift = FEXCore::ARMEmitter::ShiftType::LSL, uint32_t amt = 0) {
constexpr uint32_t Op = 0b110'1010'000U << 21;
DataProcessing_Shifted_Reg(Op, s, rd, rn, rm, Shift, amt);
}
void bic(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm, FEXCore::ARMEmitter::ShiftType Shift = FEXCore::ARMEmitter::ShiftType::LSL, uint32_t amt = 0) {
constexpr uint32_t Op = 0b000'1010'001U << 21;
DataProcessing_Shifted_Reg(Op, s, rd, rn, rm, Shift, amt);
}
void bics(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm, FEXCore::ARMEmitter::ShiftType Shift = FEXCore::ARMEmitter::ShiftType::LSL, uint32_t amt = 0) {
constexpr uint32_t Op = 0b110'1010'001U << 21;
DataProcessing_Shifted_Reg(Op, s, rd, rn, rm, Shift, amt);
}
void orr(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm, FEXCore::ARMEmitter::ShiftType Shift = FEXCore::ARMEmitter::ShiftType::LSL, uint32_t amt = 0) {
constexpr uint32_t Op = 0b010'1010'000U << 21;
DataProcessing_Shifted_Reg(Op, s, rd, rn, rm, Shift, amt);
}
void orn(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm, FEXCore::ARMEmitter::ShiftType Shift = FEXCore::ARMEmitter::ShiftType::LSL, uint32_t amt = 0) {
constexpr uint32_t Op = 0b010'1010'001U << 21;
DataProcessing_Shifted_Reg(Op, s, rd, rn, rm, Shift, amt);
}
void eor(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm, FEXCore::ARMEmitter::ShiftType Shift = FEXCore::ARMEmitter::ShiftType::LSL, uint32_t amt = 0) {
constexpr uint32_t Op = 0b100'1010'000U << 21;
DataProcessing_Shifted_Reg(Op, s, rd, rn, rm, Shift, amt);
}
void eon(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm, FEXCore::ARMEmitter::ShiftType Shift = FEXCore::ARMEmitter::ShiftType::LSL, uint32_t amt = 0) {
constexpr uint32_t Op = 0b100'1010'001U << 21;
DataProcessing_Shifted_Reg(Op, s, rd, rn, rm, Shift, amt);
}
// AddSub - shifted register
void add(FEXCore::ARMEmitter::XRegister rd, FEXCore::ARMEmitter::XRegister rn, FEXCore::ARMEmitter::XRegister rm, FEXCore::ARMEmitter::ShiftType Shift = FEXCore::ARMEmitter::ShiftType::LSL, uint32_t amt = 0) {
add(ARMEmitter::Size::i64Bit, rd.R(), rn.R(), rm.R(), Shift, amt);
}
void adds(FEXCore::ARMEmitter::XRegister rd, FEXCore::ARMEmitter::XRegister rn, FEXCore::ARMEmitter::XRegister rm, FEXCore::ARMEmitter::ShiftType Shift = FEXCore::ARMEmitter::ShiftType::LSL, uint32_t amt = 0) {
adds(ARMEmitter::Size::i64Bit, rd.R(), rn.R(), rm.R(), Shift, amt);
}
void sub(FEXCore::ARMEmitter::XRegister rd, FEXCore::ARMEmitter::XRegister rn, FEXCore::ARMEmitter::XRegister rm, FEXCore::ARMEmitter::ShiftType Shift = FEXCore::ARMEmitter::ShiftType::LSL, uint32_t amt = 0) {
sub(ARMEmitter::Size::i64Bit, rd.R(), rn.R(), rm.R(), Shift, amt);
}
void neg(FEXCore::ARMEmitter::XRegister rd, FEXCore::ARMEmitter::XRegister rm, FEXCore::ARMEmitter::ShiftType Shift = FEXCore::ARMEmitter::ShiftType::LSL, uint32_t amt = 0) {
sub(rd, FEXCore::ARMEmitter::XReg::zr, rm, Shift, amt);
}
void cmp(FEXCore::ARMEmitter::XRegister rn, FEXCore::ARMEmitter::XRegister rm, FEXCore::ARMEmitter::ShiftType Shift = FEXCore::ARMEmitter::ShiftType::LSL, uint32_t amt = 0) {
subs(ARMEmitter::Size::i64Bit, FEXCore::ARMEmitter::Reg::rsp, rn.R(), rm.R(), Shift, amt);
}
void subs(FEXCore::ARMEmitter::XRegister rd, FEXCore::ARMEmitter::XRegister rn, FEXCore::ARMEmitter::XRegister rm, FEXCore::ARMEmitter::ShiftType Shift = FEXCore::ARMEmitter::ShiftType::LSL, uint32_t amt = 0) {
subs(ARMEmitter::Size::i64Bit, rd.R(), rn.R(), rm.R(), Shift, amt);
}
void negs(FEXCore::ARMEmitter::XRegister rd, FEXCore::ARMEmitter::XRegister rm, FEXCore::ARMEmitter::ShiftType Shift = FEXCore::ARMEmitter::ShiftType::LSL, uint32_t amt = 0) {
subs(rd, FEXCore::ARMEmitter::XReg::zr, rm, Shift, amt);
}
void add(FEXCore::ARMEmitter::WRegister rd, FEXCore::ARMEmitter::WRegister rn, FEXCore::ARMEmitter::WRegister rm, FEXCore::ARMEmitter::ShiftType Shift = FEXCore::ARMEmitter::ShiftType::LSL, uint32_t amt = 0) {
add(ARMEmitter::Size::i32Bit, rd.R(), rn.R(), rm.R(), Shift, amt);
}
void adds(FEXCore::ARMEmitter::WRegister rd, FEXCore::ARMEmitter::WRegister rn, FEXCore::ARMEmitter::WRegister rm, FEXCore::ARMEmitter::ShiftType Shift = FEXCore::ARMEmitter::ShiftType::LSL, uint32_t amt = 0) {
adds(ARMEmitter::Size::i32Bit, rd.R(), rn.R(), rm.R(), Shift, amt);
}
void sub(FEXCore::ARMEmitter::WRegister rd, FEXCore::ARMEmitter::WRegister rn, FEXCore::ARMEmitter::WRegister rm, FEXCore::ARMEmitter::ShiftType Shift = FEXCore::ARMEmitter::ShiftType::LSL, uint32_t amt = 0) {
sub(ARMEmitter::Size::i32Bit, rd.R(), rn.R(), rm.R(), Shift, amt);
}
void neg(FEXCore::ARMEmitter::WRegister rd, FEXCore::ARMEmitter::WRegister rm, FEXCore::ARMEmitter::ShiftType Shift = FEXCore::ARMEmitter::ShiftType::LSL, uint32_t amt = 0) {
sub(rd, FEXCore::ARMEmitter::WReg::zr, rm, Shift, amt);
}
void cmp(FEXCore::ARMEmitter::WRegister rn, FEXCore::ARMEmitter::WRegister rm, FEXCore::ARMEmitter::ShiftType Shift = FEXCore::ARMEmitter::ShiftType::LSL, uint32_t amt = 0) {
subs(ARMEmitter::Size::i32Bit, FEXCore::ARMEmitter::Reg::rsp, rn.R(), rm.R(), Shift, amt);
}
void subs(FEXCore::ARMEmitter::WRegister rd, FEXCore::ARMEmitter::WRegister rn, FEXCore::ARMEmitter::WRegister rm, FEXCore::ARMEmitter::ShiftType Shift = FEXCore::ARMEmitter::ShiftType::LSL, uint32_t amt = 0) {
subs(ARMEmitter::Size::i32Bit, rd.R(), rn.R(), rm.R(), Shift, amt);
}
void negs(FEXCore::ARMEmitter::WRegister rd, FEXCore::ARMEmitter::WRegister rm, FEXCore::ARMEmitter::ShiftType Shift = FEXCore::ARMEmitter::ShiftType::LSL, uint32_t amt = 0) {
subs(rd, FEXCore::ARMEmitter::WReg::zr, rm, Shift, amt);
}
void add(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm, FEXCore::ARMEmitter::ShiftType Shift = FEXCore::ARMEmitter::ShiftType::LSL, uint32_t amt = 0) {
LOGMAN_THROW_AA_FMT(Shift != FEXCore::ARMEmitter::ShiftType::ROR, "Doesn't support ROR");
constexpr uint32_t Op = 0b000'1011'000U << 21;
DataProcessing_Shifted_Reg(Op, s, rd, rn, rm, Shift, amt);
}
void adds(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm, FEXCore::ARMEmitter::ShiftType Shift = FEXCore::ARMEmitter::ShiftType::LSL, uint32_t amt = 0) {
LOGMAN_THROW_AA_FMT(Shift != FEXCore::ARMEmitter::ShiftType::ROR, "Doesn't support ROR");
constexpr uint32_t Op = 0b010'1011'000U << 21;
DataProcessing_Shifted_Reg(Op, s, rd, rn, rm, Shift, amt);
}
void sub(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm, FEXCore::ARMEmitter::ShiftType Shift = FEXCore::ARMEmitter::ShiftType::LSL, uint32_t amt = 0) {
LOGMAN_THROW_AA_FMT(Shift != FEXCore::ARMEmitter::ShiftType::ROR, "Doesn't support ROR");
constexpr uint32_t Op = 0b100'1011'000U << 21;
DataProcessing_Shifted_Reg(Op, s, rd, rn, rm, Shift, amt);
}
void neg(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rm, FEXCore::ARMEmitter::ShiftType Shift = FEXCore::ARMEmitter::ShiftType::LSL, uint32_t amt = 0) {
sub(s, rd, FEXCore::ARMEmitter::Reg::zr, rm, Shift, amt);
}
void cmp(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm, FEXCore::ARMEmitter::ShiftType Shift = FEXCore::ARMEmitter::ShiftType::LSL, uint32_t amt = 0) {
subs(s, FEXCore::ARMEmitter::Reg::zr, rn, rm, Shift, amt);
}
void subs(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm, FEXCore::ARMEmitter::ShiftType Shift = FEXCore::ARMEmitter::ShiftType::LSL, uint32_t amt = 0) {
LOGMAN_THROW_AA_FMT(Shift != FEXCore::ARMEmitter::ShiftType::ROR, "Doesn't support ROR");
constexpr uint32_t Op = 0b110'1011'000U << 21;
DataProcessing_Shifted_Reg(Op, s, rd, rn, rm, Shift, amt);
}
void negs(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rm, FEXCore::ARMEmitter::ShiftType Shift = FEXCore::ARMEmitter::ShiftType::LSL, uint32_t amt = 0) {
subs(s, rd, FEXCore::ARMEmitter::Reg::zr, rm, Shift, amt);
}
// AddSub - extended register
void add(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm, FEXCore::ARMEmitter::ExtendedType Option, uint32_t Shift = 0) {
LOGMAN_THROW_AA_FMT(Shift <= 4, "Shift amount is too large");
constexpr uint32_t Op = 0b000'1011'001U << 21;
DataProcessing_Extended_Reg(Op, s, rd, rn, rm, Option, Shift);
}
void adds(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm, FEXCore::ARMEmitter::ExtendedType Option, uint32_t Shift = 0) {
constexpr uint32_t Op = 0b010'1011'001U << 21;
DataProcessing_Extended_Reg(Op, s, rd, rn, rm, Option, Shift);
}
void sub(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm, FEXCore::ARMEmitter::ExtendedType Option, uint32_t Shift = 0) {
constexpr uint32_t Op = 0b100'1011'001U << 21;
DataProcessing_Extended_Reg(Op, s, rd, rn, rm, Option, Shift);
}
void subs(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm, FEXCore::ARMEmitter::ExtendedType Option, uint32_t Shift = 0) {
constexpr uint32_t Op = 0b110'1011'001U << 21;
DataProcessing_Extended_Reg(Op, s, rd, rn, rm, Option, Shift);
}
void cmp(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm, FEXCore::ARMEmitter::ExtendedType Option, uint32_t Shift = 0) {
constexpr uint32_t Op = 0b110'1011'001U << 21;
DataProcessing_Extended_Reg(Op, s, FEXCore::ARMEmitter::Reg::zr, rn, rm, Option, Shift);
}
// AddSub - with carry
void adc(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm) {
constexpr uint32_t Op = 0b0001'1010'000U << 21;
DataProcessing_Extended_Reg(Op, s, rd, rn, rm, FEXCore::ARMEmitter::ExtendedType::UXTB, 0);
}
void adcs(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm) {
constexpr uint32_t Op = 0b0011'1010'000U << 21;
DataProcessing_Extended_Reg(Op, s, rd, rn, rm, FEXCore::ARMEmitter::ExtendedType::UXTB, 0);
}
void sbc(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm) {
constexpr uint32_t Op = 0b0101'1010'000U << 21;
DataProcessing_Extended_Reg(Op, s, rd, rn, rm, FEXCore::ARMEmitter::ExtendedType::UXTB, 0);
}
void sbcs(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm) {
constexpr uint32_t Op = 0b0111'1010'000U << 21;
DataProcessing_Extended_Reg(Op, s, rd, rn, rm, FEXCore::ARMEmitter::ExtendedType::UXTB, 0);
}
// Rotate right into flags
// TODO
// Evaluate into flags
// TODO
// Conditional compare - register
void ccmn(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm, FEXCore::ARMEmitter::StatusFlags flags, FEXCore::ARMEmitter::Condition Cond) {
constexpr uint32_t Op = 0b0011'1010'010 << 21;
ConditionalCompare(Op, 0, 0b00, 0, s, rn, rm, flags, Cond);
}
void ccmp(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm, FEXCore::ARMEmitter::StatusFlags flags, FEXCore::ARMEmitter::Condition Cond) {
constexpr uint32_t Op = 0b0011'1010'010 << 21;
ConditionalCompare(Op, 1, 0b00, 0, s, rn, rm, flags, Cond);
}
// Conditional compare - immediate
void ccmn(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rn, uint32_t rm, FEXCore::ARMEmitter::StatusFlags flags, FEXCore::ARMEmitter::Condition Cond) {
LOGMAN_THROW_A_FMT((rm & ~0b1'1111) == 0, "Comparison imm too large");
constexpr uint32_t Op = 0b0011'1010'010 << 21;
ConditionalCompare(Op, 0, 0b10, 0, s, rn, rm, flags, Cond);
}
void ccmp(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rn, uint32_t rm, FEXCore::ARMEmitter::StatusFlags flags, FEXCore::ARMEmitter::Condition Cond) {
LOGMAN_THROW_A_FMT((rm & ~0b1'1111) == 0, "Comparison imm too large");
constexpr uint32_t Op = 0b0011'1010'010 << 21;
ConditionalCompare(Op, 1, 0b10, 0, s, rn, rm, flags, Cond);
}
// Conditional select
void csel(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm, FEXCore::ARMEmitter::Condition Cond) {
constexpr uint32_t Op = 0b0001'1010'100 << 21;
ConditionalCompare(Op, 0, 0b00, s, rd, rn, rm, Cond);
}
void cset(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Condition Cond) {
constexpr uint32_t Op = 0b0001'1010'100 << 21;
ConditionalCompare(Op, 0, 0b01, s, rd, FEXCore::ARMEmitter::Reg::zr, FEXCore::ARMEmitter::Reg::zr, static_cast<FEXCore::ARMEmitter::Condition>(FEXCore::ToUnderlying(Cond) ^ FEXCore::ToUnderlying(FEXCore::ARMEmitter::Condition::CC_NE)));
}
void csinc(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm, FEXCore::ARMEmitter::Condition Cond) {
constexpr uint32_t Op = 0b0001'1010'100 << 21;
ConditionalCompare(Op, 0, 0b01, s, rd, rn, rm, Cond);
}
void csinv(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm, FEXCore::ARMEmitter::Condition Cond) {
constexpr uint32_t Op = 0b0001'1010'100 << 21;
ConditionalCompare(Op, 1, 0b00, s, rd, rn, rm, Cond);
}
void csneg(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm, FEXCore::ARMEmitter::Condition Cond) {
constexpr uint32_t Op = 0b0001'1010'100 << 21;
ConditionalCompare(Op, 1, 0b01, s, rd, rn, rm, Cond);
}
// Data processing - 3 source
void madd(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm, FEXCore::ARMEmitter::Register ra) {
constexpr uint32_t Op = 0b001'1011'000U << 21;
DataProcessing_3Source(Op, 0, s, rd, rn, rm, ra);
}
void mul(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm) {
madd(s, rd, rn, rm, FEXCore::ARMEmitter::Reg::zr);
}
void msub(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm, FEXCore::ARMEmitter::Register ra) {
constexpr uint32_t Op = 0b001'1011'000U << 21;
DataProcessing_3Source(Op, 1, s, rd, rn, rm, ra);
}
void mneg(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm) {
msub(s, rd, rn, rm, FEXCore::ARMEmitter::Reg::zr);
}
void smaddl(FEXCore::ARMEmitter::XRegister rd, FEXCore::ARMEmitter::WRegister rn, FEXCore::ARMEmitter::WRegister rm, FEXCore::ARMEmitter::XRegister ra) {
constexpr uint32_t Op = 0b001'1011'001U << 21;
DataProcessing_3Source(Op, 0, FEXCore::ARMEmitter::Size::i64Bit, rd, rn, rm, ra);
}
void smull(FEXCore::ARMEmitter::XRegister rd, FEXCore::ARMEmitter::WRegister rn, FEXCore::ARMEmitter::WRegister rm) {
smaddl(rd, rn, rm, FEXCore::ARMEmitter::Reg::zr);
}
void smsubl(FEXCore::ARMEmitter::XRegister rd, FEXCore::ARMEmitter::WRegister rn, FEXCore::ARMEmitter::WRegister rm, FEXCore::ARMEmitter::XRegister ra) {
constexpr uint32_t Op = 0b001'1011'001U << 21;
DataProcessing_3Source(Op, 1, FEXCore::ARMEmitter::Size::i64Bit, rd, rn, rm, ra);
}
void smnegl(FEXCore::ARMEmitter::XRegister rd, FEXCore::ARMEmitter::WRegister rn, FEXCore::ARMEmitter::WRegister rm) {
smsubl(rd, rn, rm, FEXCore::ARMEmitter::Reg::zr);
}
void smulh(FEXCore::ARMEmitter::XRegister rd, FEXCore::ARMEmitter::XRegister rn, FEXCore::ARMEmitter::XRegister rm) {
constexpr uint32_t Op = 0b001'1011'010U << 21;
DataProcessing_3Source(Op, 0, FEXCore::ARMEmitter::Size::i64Bit, rd, rn, rm, FEXCore::ARMEmitter::Reg::zr);
}
void umaddl(FEXCore::ARMEmitter::XRegister rd, FEXCore::ARMEmitter::WRegister rn, FEXCore::ARMEmitter::WRegister rm, FEXCore::ARMEmitter::XRegister ra) {
constexpr uint32_t Op = 0b001'1011'101U << 21;
DataProcessing_3Source(Op, 0, FEXCore::ARMEmitter::Size::i64Bit, rd, rn, rm, ra);
}
void umull(FEXCore::ARMEmitter::XRegister rd, FEXCore::ARMEmitter::WRegister rn, FEXCore::ARMEmitter::WRegister rm) {
umaddl(rd, rn, rm, FEXCore::ARMEmitter::Reg::zr);
}
void umsubl(FEXCore::ARMEmitter::XRegister rd, FEXCore::ARMEmitter::WRegister rn, FEXCore::ARMEmitter::WRegister rm, FEXCore::ARMEmitter::XRegister ra) {
constexpr uint32_t Op = 0b001'1011'101U << 21;
DataProcessing_3Source(Op, 1, FEXCore::ARMEmitter::Size::i64Bit, rd, rn, rm, ra);
}
void umnegl(FEXCore::ARMEmitter::XRegister rd, FEXCore::ARMEmitter::WRegister rn, FEXCore::ARMEmitter::WRegister rm) {
umsubl(rd, rn, rm, FEXCore::ARMEmitter::Reg::zr);
}
void umulh(FEXCore::ARMEmitter::XRegister rd, FEXCore::ARMEmitter::XRegister rn, FEXCore::ARMEmitter::XRegister rm) {
constexpr uint32_t Op = 0b001'1011'110U << 21;
DataProcessing_3Source(Op, 0, FEXCore::ARMEmitter::Size::i64Bit, rd, rn, rm, FEXCore::ARMEmitter::Reg::zr);
}
private:
void and_(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, uint32_t n, uint32_t immr, uint32_t imms) {
constexpr uint32_t Op = 0b001'0010'00 << 22;
DataProcessing_Logical_Imm(Op, s, rd, rn, n, immr, imms);
}
void ands(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, uint32_t n, uint32_t immr, uint32_t imms) {
constexpr uint32_t Op = 0b111'0010'00 << 22;
DataProcessing_Logical_Imm(Op, s, rd, rn, n, immr, imms);
}
void orr(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, uint32_t n, uint32_t immr, uint32_t imms) {
constexpr uint32_t Op = 0b011'0010'00 << 22;
DataProcessing_Logical_Imm(Op, s, rd, rn, n, immr, imms);
}
void eor(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, uint32_t n, uint32_t immr, uint32_t imms) {
constexpr uint32_t Op = 0b101'0010'00 << 22;
DataProcessing_Logical_Imm(Op, s, rd, rn, n, immr, imms);
}
void sbfm(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, uint32_t immr, uint32_t imms) {
constexpr uint32_t Op = 0b0001'0011'00 << 22;
DataProcessing_Logical_Imm(Op, s, rd, rn, s == ARMEmitter::Size::i64Bit, immr, imms);
}
void bfm(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, uint32_t immr, uint32_t imms) {
constexpr uint32_t Op = 0b0011'0011'00 << 22;
DataProcessing_Logical_Imm(Op, s, rd, rn, s == ARMEmitter::Size::i64Bit, immr, imms);
}
// 4.1.64 - Data processing - Immediate
void DataProcessing_PCRel_Imm(uint32_t Op, FEXCore::ARMEmitter::Register rd, uint32_t Imm) {
// Ensure the immediate is masked.
Imm &= 0b1'1111'1111'1111'1111'1111U;
uint32_t Instr = Op;
Instr |= (Imm & 0b11) << 29;
Instr |= (Imm >> 2) << 5;
Instr |= Encode_rd(rd);
dc32(Instr);
}
void DataProcessing_AddSub_Imm(uint32_t Op, FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, uint32_t Imm, bool LSL12) {
bool TooLarge = (Imm & ~0b1111'1111'1111U) != 0;
if (TooLarge && !LSL12 && ((Imm >> 12) & ~0b1111'1111'1111U) == 0) {
// We can convert an immediate
TooLarge = false;
LSL12 = true;
Imm >>= 12;
}
LOGMAN_THROW_AA_FMT(TooLarge == false, "Imm amount too large: 0x{:x}", Imm);
const uint32_t SF = s == FEXCore::ARMEmitter::Size::i64Bit ? (1U << 31) : 0;
uint32_t Instr = Op;
Instr |= SF;
Instr |= LSL12 << 22;
Instr |= Imm << 10;
Instr |= Encode_rn(rn);
Instr |= Encode_rd(rd);
dc32(Instr);
}
// Move Wide
void DataProcessing_MoveWide(uint32_t Op, FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, uint32_t Imm, uint32_t Offset) {
const uint32_t SF = s == FEXCore::ARMEmitter::Size::i64Bit ? (1U << 31) : 0;
uint32_t Instr = Op;
Instr |= SF;
Instr |= Imm << 5;
Instr |= Offset << 21;
Instr |= Encode_rd(rd);
dc32(Instr);
}
// Logical immediate
void DataProcessing_Logical_Imm(uint32_t Op, FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, uint32_t n, uint32_t immr, uint32_t imms) {
const uint32_t SF = s == FEXCore::ARMEmitter::Size::i64Bit ? (1U << 31) : 0;
uint32_t Instr = Op;
Instr |= SF;
Instr |= n << 22;
Instr |= immr << 16;
Instr |= imms << 10;
Instr |= Encode_rn(rn);
Instr |= Encode_rd(rd);
dc32(Instr);
}
void DataProcessing_Extract(uint32_t Op, FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm, uint32_t Imm) {
const uint32_t SF = s == FEXCore::ARMEmitter::Size::i64Bit ? (1U << 31) : 0;
// Current ARMv8 spec hardcodes SF == N for this class of instructions.
// Anythign else is undefined behaviour.
const uint32_t N = s == FEXCore::ARMEmitter::Size::i64Bit ? (1U << 22) : 0;
uint32_t Instr = Op;
Instr |= SF;
Instr |= N;
Instr |= Encode_rm(rm);
Instr |= Imm << 10;
Instr |= Encode_rn(rn);
Instr |= Encode_rd(rd);
dc32(Instr);
}
// Data-processing - 2 source
void DataProcessing_2Source(uint32_t Op, FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm) {
const uint32_t SF = s == FEXCore::ARMEmitter::Size::i64Bit ? (1U << 31) : 0;
uint32_t Instr = Op;
Instr |= SF;
Instr |= Encode_rm(rm);
Instr |= Encode_rn(rn);
Instr |= Encode_rd(rd);
dc32(Instr);
}
// Data processing - 1 source
template<typename T>
void DataProcessing_1Source(uint32_t Op, FEXCore::ARMEmitter::Size s, T rd, T rn) {
const uint32_t SF = s == FEXCore::ARMEmitter::Size::i64Bit ? (1U << 31) : 0;
uint32_t Instr = Op;
Instr |= SF;
Instr |= Encode_rn(rn);
Instr |= Encode_rd(rd);
dc32(Instr);
}
// AddSub - shifted register
void DataProcessing_Shifted_Reg(uint32_t Op, FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm, FEXCore::ARMEmitter::ShiftType Shift, uint32_t amt) {
LOGMAN_THROW_AA_FMT((amt & ~0b11'1111U) == 0, "Shift amount too large");
const uint32_t SF = s == FEXCore::ARMEmitter::Size::i64Bit ? (1U << 31) : 0;
uint32_t Instr = Op;
Instr |= SF;
Instr |= FEXCore::ToUnderlying(Shift) << 22;
Instr |= Encode_rm(rm);
Instr |= static_cast<uint32_t>(amt) << 10;
Instr |= Encode_rn(rn);
Instr |= Encode_rd(rd);
dc32(Instr);
}
// AddSub - extended register
void DataProcessing_Extended_Reg(uint32_t Op, FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm, FEXCore::ARMEmitter::ExtendedType Option, uint32_t Shift) {
const uint32_t SF = s == FEXCore::ARMEmitter::Size::i64Bit ? (1U << 31) : 0;
uint32_t Instr = Op;
Instr |= SF;
Instr |= Encode_rm(rm);
Instr |= FEXCore::ToUnderlying(Option) << 13;
Instr |= static_cast<uint32_t>(Shift) << 10;
Instr |= Encode_rn(rn);
Instr |= Encode_rd(rd);
dc32(Instr);
}
// Conditional compare - register
template<typename T>
void ConditionalCompare(uint32_t Op, uint32_t o1, uint32_t o2, uint32_t o3, FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rn, T rm, FEXCore::ARMEmitter::StatusFlags flags, FEXCore::ARMEmitter::Condition Cond) {
const uint32_t SF = s == FEXCore::ARMEmitter::Size::i64Bit ? (1U << 31) : 0;
uint32_t Instr = Op;
Instr |= SF;
Instr |= o1 << 30;
Instr |= Encode_rm(rm);
Instr |= FEXCore::ToUnderlying(Cond) << 12;
Instr |= o2 << 10;
Instr |= Encode_rn(rn);
Instr |= o3 << 4;
Instr |= FEXCore::ToUnderlying(flags);
dc32(Instr);
}
template<typename T>
void ConditionalCompare(uint32_t Op, uint32_t o1, uint32_t o2, FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, T rm, FEXCore::ARMEmitter::Condition Cond) {
const uint32_t SF = s == FEXCore::ARMEmitter::Size::i64Bit ? (1U << 31) : 0;
uint32_t Instr = Op;
Instr |= SF;
Instr |= o1 << 30;
Instr |= Encode_rm(rm);
Instr |= FEXCore::ToUnderlying(Cond) << 12;
Instr |= o2 << 10;
Instr |= Encode_rn(rn);
Instr |= Encode_rd(rd);
dc32(Instr);
}
// Data-processing - 3 source
void DataProcessing_3Source(uint32_t Op, uint32_t Op0, FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::Register rn, FEXCore::ARMEmitter::Register rm, FEXCore::ARMEmitter::Register ra) {
const uint32_t SF = s == FEXCore::ARMEmitter::Size::i64Bit ? (1U << 31) : 0;
uint32_t Instr = Op;
Instr |= SF;
Instr |= Encode_rm(rm);
Instr |= Op0 << 15;
Instr |= Encode_ra(ra);
Instr |= Encode_rn(rn);
Instr |= Encode_rd(rd);
dc32(Instr);
}
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,322 @@
/* Branch instruction emitters.
*
* Most of these instructions will use `BackwardLabel`, `ForwardLabel`, or `BiDirectionLabel` to determine where a branch targets.
*/
public:
// Branches, Exception Generating and System instructions
public:
// Conditional branch immediate
///< Branch conditional
void b(FEXCore::ARMEmitter::Condition Cond, uint32_t Imm) {
constexpr uint32_t Op = 0b0101'010 << 25;
Branch_Conditional(Op, 0, 0, Cond, Imm);
}
void b(FEXCore::ARMEmitter::Condition Cond, BackwardLabel const* Label) {
int32_t Imm = static_cast<int32_t>(Label->Location - GetCursorAddress<uint8_t*>());
LOGMAN_THROW_A_FMT(Imm >= -1048576 && Imm <= 1048575 && ((Imm & 0b11) == 0), "Unscaled offset too large");
constexpr uint32_t Op = 0b0101'010 << 25;
Branch_Conditional(Op, 0, 0, Cond, Imm >> 2);
}
void b(FEXCore::ARMEmitter::Condition Cond, ForwardLabel *Label) {
Label->Insts.emplace_back(ForwardLabel::Instructions{ .Location = GetCursorAddress<uint8_t*>(), .Type = ForwardLabel::Instructions::InstType::BC });
constexpr uint32_t Op = 0b0101'010 << 25;
Branch_Conditional(Op, 0, 0, Cond, 0);
}
void b(FEXCore::ARMEmitter::Condition Cond, BiDirectionalLabel *Label) {
if (Label->Backward.Location) {
b(Cond, &Label->Backward);
}
else {
b(Cond, &Label->Forward);
}
}
///< Branch consistent conditional
void bc(FEXCore::ARMEmitter::Condition Cond, uint32_t Imm) {
constexpr uint32_t Op = 0b0101'010 << 25;
Branch_Conditional(Op, 0, 1, Cond, Imm);
}
void bc(FEXCore::ARMEmitter::Condition Cond, BackwardLabel const* Label) {
int32_t Imm = static_cast<int32_t>(Label->Location - GetCursorAddress<uint8_t*>());
LOGMAN_THROW_A_FMT(Imm >= -1048576 && Imm <= 1048575 && ((Imm & 0b11) == 0), "Unscaled offset too large");
constexpr uint32_t Op = 0b0101'010 << 25;
Branch_Conditional(Op, 0, 1, Cond, Imm >> 2);
}
void bc(FEXCore::ARMEmitter::Condition Cond, ForwardLabel *Label) {
Label->Insts.emplace_back(ForwardLabel::Instructions{ .Location = GetCursorAddress<uint8_t*>(), .Type = ForwardLabel::Instructions::InstType::BC });
constexpr uint32_t Op = 0b0101'010 << 25;
Branch_Conditional(Op, 0, 1, Cond, 0);
}
void bc(FEXCore::ARMEmitter::Condition Cond, BiDirectionalLabel *Label) {
if (Label->Backward.Location) {
bc(Cond, &Label->Backward);
}
else {
bc(Cond, &Label->Forward);
}
}
// Unconditional branch register
void br(FEXCore::ARMEmitter::Register rn) {
constexpr uint32_t Op = 0b1101011 << 25 |
0b0'000 << 21 | // opc
0b1'1111 << 16 | // op2
0b0000'00 << 10 | // op3
0b0'0000; // op4
UnconditionalBranch(Op, rn);
}
void blr(FEXCore::ARMEmitter::Register rn) {
constexpr uint32_t Op = 0b1101011 << 25 |
0b0'001 << 21 | // opc
0b1'1111 << 16 | // op2
0b0000'00 << 10 | // op3
0b0'0000; // op4
UnconditionalBranch(Op, rn);
}
void ret(FEXCore::ARMEmitter::Register rn = FEXCore::ARMEmitter::Reg::r30) {
constexpr uint32_t Op = 0b1101011 << 25 |
0b0'010 << 21 | // opc
0b1'1111 << 16 | // op2
0b0000'00 << 10 | // op3
0b0'0000; // op4
UnconditionalBranch(Op, rn);
}
// Unconditional branch immediate
void b(uint32_t Imm) {
constexpr uint32_t Op = 0b0001'01 << 26;
UnconditionalBranch(Op, Imm);
}
void b(BackwardLabel const* Label) {
int32_t Imm = static_cast<int32_t>(Label->Location - GetCursorAddress<uint8_t*>());
LOGMAN_THROW_A_FMT(Imm >= -134217728 && Imm <= 134217724 && ((Imm & 0b11) == 0), "Unscaled offset too large");
constexpr uint32_t Op = 0b0001'01 << 26;
UnconditionalBranch(Op, Imm >> 2);
}
void b(ForwardLabel *Label) {
Label->Insts.emplace_back(ForwardLabel::Instructions{ .Location = GetCursorAddress<uint8_t*>(), .Type = ForwardLabel::Instructions::InstType::B });
constexpr uint32_t Op = 0b0001'01 << 26;
UnconditionalBranch(Op, 0);
}
void b(BiDirectionalLabel *Label) {
if (Label->Backward.Location) {
b(&Label->Backward);
}
else {
b(&Label->Forward);
}
}
void bl(uint32_t Imm) {
constexpr uint32_t Op = 0b1001'01 << 26;
UnconditionalBranch(Op, Imm);
}
void bl(BackwardLabel const* Label) {
int32_t Imm = static_cast<int32_t>(Label->Location - GetCursorAddress<uint8_t*>());
LOGMAN_THROW_A_FMT(Imm >= -134217728 && Imm <= 134217724 && ((Imm & 0b11) == 0), "Unscaled offset too large");
constexpr uint32_t Op = 0b1001'01 << 26;
UnconditionalBranch(Op, Imm >> 2);
}
void bl(ForwardLabel *Label) {
Label->Insts.emplace_back(ForwardLabel::Instructions{ .Location = GetCursorAddress<uint8_t*>(), .Type = ForwardLabel::Instructions::InstType::B });
constexpr uint32_t Op = 0b1001'01 << 26;
UnconditionalBranch(Op, 0);
}
void bl(BiDirectionalLabel *Label) {
if (Label->Backward.Location) {
bl(&Label->Backward);
}
else {
bl(&Label->Forward);
}
}
// Compare and branch
void cbz(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rt, uint32_t Imm) {
constexpr uint32_t Op = 0b0011'0100 << 24;
CompareAndBranch(Op, s, rt, Imm);
}
void cbz(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rt, BackwardLabel const* Label) {
int32_t Imm = static_cast<int32_t>(Label->Location - GetCursorAddress<uint8_t*>());
LOGMAN_THROW_A_FMT(Imm >= -1048576 && Imm <= 1048575 && ((Imm & 0b11) == 0), "Unscaled offset too large");
constexpr uint32_t Op = 0b0011'0100 << 24;
CompareAndBranch(Op, s, rt, Imm >> 2);
}
void cbz(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rt, ForwardLabel *Label) {
Label->Insts.emplace_back(ForwardLabel::Instructions{ .Location = GetCursorAddress<uint8_t*>(), .Type = ForwardLabel::Instructions::InstType::BC });
constexpr uint32_t Op = 0b0011'0100 << 24;
CompareAndBranch(Op, s, rt, 0);
}
void cbz(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rt, BiDirectionalLabel *Label) {
if (Label->Backward.Location) {
cbz(s, rt, &Label->Backward);
}
else {
cbz(s, rt, &Label->Forward);
}
}
void cbnz(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rt, uint32_t Imm) {
constexpr uint32_t Op = 0b0011'0101 << 24;
CompareAndBranch(Op, s, rt, Imm);
}
void cbnz(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rt, BackwardLabel const* Label) {
int32_t Imm = static_cast<int32_t>(Label->Location - GetCursorAddress<uint8_t*>());
LOGMAN_THROW_A_FMT(Imm >= -1048576 && Imm <= 1048575 && ((Imm & 0b11) == 0), "Unscaled offset too large");
constexpr uint32_t Op = 0b0011'0101 << 24;
CompareAndBranch(Op, s, rt, Imm >> 2);
}
void cbnz(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rt, ForwardLabel *Label) {
Label->Insts.emplace_back(ForwardLabel::Instructions{ .Location = GetCursorAddress<uint8_t*>(), .Type = ForwardLabel::Instructions::InstType::BC });
constexpr uint32_t Op = 0b0011'0101 << 24;
CompareAndBranch(Op, s, rt, 0);
}
void cbnz(FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rt, BiDirectionalLabel *Label) {
if (Label->Backward.Location) {
cbnz(s, rt, &Label->Backward);
}
else {
cbnz(s, rt, &Label->Forward);
}
}
// Test and branch immediate
void tbz(FEXCore::ARMEmitter::Register rt, uint32_t Bit, uint32_t Imm) {
constexpr uint32_t Op = 0b0011'0110 << 24;
TestAndBranch(Op, rt, Bit, Imm);
}
void tbz(FEXCore::ARMEmitter::Register rt, uint32_t Bit, BackwardLabel const* Label) {
int32_t Imm = static_cast<int32_t>(Label->Location - GetCursorAddress<uint8_t*>());
LOGMAN_THROW_A_FMT(Imm >= -32768 && Imm <= 32764 && ((Imm & 0b11) == 0), "Unscaled offset too large");
constexpr uint32_t Op = 0b0011'0110 << 24;
TestAndBranch(Op, rt, Bit, Imm >> 2);
}
void tbz(FEXCore::ARMEmitter::Register rt, uint32_t Bit, ForwardLabel *Label) {
Label->Insts.emplace_back(ForwardLabel::Instructions{ .Location = GetCursorAddress<uint8_t*>(), .Type = ForwardLabel::Instructions::InstType::TEST_BRANCH });
constexpr uint32_t Op = 0b0011'0110 << 24;
TestAndBranch(Op, rt, Bit, 0);
}
void tbz(FEXCore::ARMEmitter::Register rt, uint32_t Bit, BiDirectionalLabel *Label) {
if (Label->Backward.Location) {
tbz(rt, Bit, &Label->Backward);
}
else {
tbz(rt, Bit, &Label->Forward);
}
}
void tbnz(FEXCore::ARMEmitter::Register rt, uint32_t Bit, uint32_t Imm) {
constexpr uint32_t Op = 0b0011'0111 << 24;
TestAndBranch(Op, rt, Bit, Imm);
}
void tbnz(FEXCore::ARMEmitter::Register rt, uint32_t Bit, BackwardLabel const* Label) {
int32_t Imm = static_cast<int32_t>(Label->Location - GetCursorAddress<uint8_t*>());
LOGMAN_THROW_A_FMT(Imm >= -32768 && Imm <= 32764 && ((Imm & 0b11) == 0), "Unscaled offset too large");
constexpr uint32_t Op = 0b0011'0111 << 24;
TestAndBranch(Op, rt, Bit, Imm >> 2);
}
void tbnz(FEXCore::ARMEmitter::Register rt, uint32_t Bit, ForwardLabel *Label) {
Label->Insts.emplace_back(ForwardLabel::Instructions{ .Location = GetCursorAddress<uint8_t*>(), .Type = ForwardLabel::Instructions::InstType::TEST_BRANCH });
constexpr uint32_t Op = 0b0011'0111 << 24;
TestAndBranch(Op, rt, Bit, 0);
}
void tbnz(FEXCore::ARMEmitter::Register rt, uint32_t Bit, BiDirectionalLabel *Label) {
if (Label->Backward.Location) {
tbnz(rt, Bit, &Label->Backward);
}
else {
tbnz(rt, Bit, &Label->Forward);
}
}
private:
// Conditional branch immediate
void Branch_Conditional(uint32_t Op, uint32_t Op1, uint32_t Op0, FEXCore::ARMEmitter::Condition Cond, uint32_t Imm) {
uint32_t Instr = Op;
Instr |= Op1 << 24;
Instr |= (Imm & 0x7'FFFF) << 5;
Instr |= Op0 << 4;
Instr |= FEXCore::ToUnderlying(Cond);
dc32(Instr);
}
// Unconditional branch register
void UnconditionalBranch(uint32_t Op, FEXCore::ARMEmitter::Register rn) {
uint32_t Instr = Op;
Instr |= Encode_rn(rn);
dc32(Instr);
}
// Unconditional branch - immediate
void UnconditionalBranch(uint32_t Op, uint32_t Imm) {
uint32_t Instr = Op;
Instr |= Imm & 0x3FF'FFFF;
dc32(Instr);
}
// Compare and branch
void CompareAndBranch(uint32_t Op, FEXCore::ARMEmitter::Size s, FEXCore::ARMEmitter::Register rt, uint32_t Imm) {
const uint32_t SF = s == FEXCore::ARMEmitter::Size::i64Bit ? (1U << 31) : 0;
uint32_t Instr = Op;
Instr |= SF;
Instr |= (Imm & 0x7'FFFF) << 5;
Instr |= Encode_rt(rt);
dc32(Instr);
}
// Test and branch - immediate
void TestAndBranch(uint32_t Op, FEXCore::ARMEmitter::Register rt, uint32_t Bit, uint32_t Imm) {
uint32_t Instr = Op;
Instr |= (Bit >> 5) << 31;
Instr |= (Bit & 0b1'1111) << 19;
Instr |= (Imm & 0x3FFF) << 5;
Instr |= Encode_rt(rt);
dc32(Instr);
}
@@ -0,0 +1,100 @@
#pragma once
#include <cstddef>
#include <cstdint>
#include <cstring>
namespace FEXCore::ARMEmitter {
class Buffer {
public:
Buffer() {
SetBuffer(nullptr, 0);
}
Buffer(uint8_t* Base, uint64_t BaseSize) {
SetBuffer(Base, BaseSize);
}
void SetBuffer(uint8_t* Base, uint64_t BaseSize) {
BufferBase = Base;
CurrentOffset = BufferBase;
Size = BaseSize;
}
void dc8(uint8_t Data) {
decltype(Data) *Memory = reinterpret_cast<decltype(Data)*>(CurrentOffset);
*Memory = Data;
CurrentOffset += sizeof(Data);
}
void dc16(uint16_t Data) {
decltype(Data) *Memory = reinterpret_cast<decltype(Data)*>(CurrentOffset);
*Memory = Data;
CurrentOffset += sizeof(Data);
}
void dc32(uint32_t Data) {
decltype(Data) *Memory = reinterpret_cast<decltype(Data)*>(CurrentOffset);
*Memory = Data;
CurrentOffset += sizeof(Data);
}
void dc64(uint64_t Data) {
decltype(Data) *Memory = reinterpret_cast<decltype(Data)*>(CurrentOffset);
*Memory = Data;
CurrentOffset += sizeof(Data);
}
void EmitString(const char *String) {
const auto StringLength = strlen(String);
memcpy(CurrentOffset, String, StringLength);
CurrentOffset += StringLength;
}
void Align() {
// Align the buffer to instruction size
auto CurrentAlignment = reinterpret_cast<uint64_t>(CurrentOffset) & 0b11;
if (!CurrentAlignment) {
return;
}
CurrentOffset += 4 - CurrentAlignment;
}
template<typename T>
T GetCursorAddress() const {
return reinterpret_cast<T>(CurrentOffset);
}
static void ClearICache(void* Begin, std::size_t Length) {
__builtin___clear_cache(static_cast<char*>(Begin), static_cast<char*>(Begin) + Length);
}
size_t GetCursorOffset() const {
return static_cast<size_t>(CurrentOffset - BufferBase);
}
uint8_t *GetBufferBase() const {
return BufferBase;
}
void CursorIncrement(size_t Size) {
CurrentOffset += Size;
}
void SetCursorOffset(size_t Offset) {
CurrentOffset = BufferBase + Offset;
}
uint64_t GetBufferSize() const {
return Size;
}
protected:
void ResetBuffer() {
CurrentOffset = BufferBase;
}
uint8_t* BufferBase;
uint8_t* CurrentOffset;
uint64_t Size;
};
}
@@ -0,0 +1,721 @@
#pragma once
#include "Interface/Core/ArchHelpers/CodeEmitter/Buffer.h"
#include "Interface/Core/ArchHelpers/CodeEmitter/Registers.h"
#include <FEXCore/Utils/CompilerDefs.h>
#include <FEXCore/Utils/EnumUtils.h>
#include <FEXCore/Utils/LogManager.h>
#include <aarch64/assembler-aarch64.h>
#include <cstdint>
#include <utility>
#include <type_traits>
#include <vector>
/*
* Welcome to FEX-Emu's custom AArch64 emitter.
* This was written specifically to avoid the performance cost of the vixl emitter.
*
* There are some specific design constraints in this design to target a couple features:
* - High performance
* - Low CPU cache performance hit
* - Significantly reduced code footprint
* - Low number of branches
*
* These requirements are mostly achieved by removing a bunch of developer conveniences
* that vixl provides. The developer needs to take a lot of care to not shoot themselves in the foot.
*
* Misc design decisions:
* - Registers are encoded as basic uint32_t enums.
* - Converting between different registers is zero-cost.
* - Passing around as arguments are as cheap as registers
* - Contrast to vixl where every register requires living on the stack.
* - Registers can get encoded in to instructions with a simple `BFM` instruction.
*
* - Instructions are very simply emitted, allowing direct inlining most of the time.
* - These are simple enough that multiple back-to-back instructions get optimized to 128-bit load-store operations.
* - Contrast to vixl where pretty much no instruction emitter gets inlined.
*
* - Instruction emitters are /mostly/ unsized. Most instructions take a size argument first, which gets encoded
* directly in to the instruction.
* - Contrast to vixl where the register arguments are how the instructions determine operating size.
* - Size argument allows FEX to use `CSEL` to select a size at runtime, instead of branching.
* - Some instructions are explicitly sized based on register type. Read comments in the respective `inl` files to
* see why.
* Some scalar/vector operations are an example of this.
*
* - Almost zero helper functions.
* - Primary exception to this rule is load-store operations. These will use a helper to make
* it easier to select the correct load-store instruction. Mostly because these are a nightmare selecting
* the right instruction.
*/
namespace FEXCore::ARMEmitter {
/*
* This `Size` enum is used for most ALU operations.
* These follow the AArch64 encoding style in most cases.
*/
enum class Size : uint32_t {
i32Bit = 0,
i64Bit,
};
// This allows us to get the `Size` enum in bits.
template<Size size>
constexpr size_t RegSizeInBits() {
constexpr size_t RegSize[] = {
32, 64, 128,
};
return RegSize[FEXCore::ToUnderlying(size)];
}
[[maybe_unused]]
static inline size_t RegSizeInBits(Size size) {
constexpr size_t RegSize[] = {
32, 64, 128,
};
return RegSize[FEXCore::ToUnderlying(size)];
}
/* This `SubRegSize` enum is used for most ASIMD operations.
* These follow the AArch64 encoding style in most cases.
*/
enum class SubRegSize : uint32_t {
i8Bit = 0b00,
i16Bit = 0b01,
i32Bit = 0b10,
i64Bit = 0b11,
i128Bit = 0b100,
};
// This allows us to get the `SubRegSize` in bits.
template<SubRegSize size>
constexpr size_t SubRegSizeInBits() {
return (1 << FEXCore::ToUnderlying(size)) * 8;
}
[[maybe_unused]]
static inline size_t SubRegSizeInBits(SubRegSize size) {
return (1 << FEXCore::ToUnderlying(size)) * 8;
}
/* This `ScalarRegSize` enum is used for most scalar float
* operations.
*
* This is specifically duplicated from `SubRegSize` to have strongly
* typed functions.
*
* `ScalarRegSize` specifically doesn't have `i128Bit` because scalar operations
* can't operate at 128-bit.
*/
enum class ScalarRegSize : uint32_t {
i8Bit = 0b00,
i16Bit = 0b01,
i32Bit = 0b10,
i64Bit = 0b11,
};
// This allows us to get the `ScalarRegSize` in bits.
template<ScalarRegSize size>
constexpr size_t ScalarRegSizeInBits() {
return (1 << FEXCore::ToUnderlying(size)) * 8;
}
[[maybe_unused]]
static inline size_t ScalarRegSizeInBits(ScalarRegSize size) {
return (1 << FEXCore::ToUnderlying(size)) * 8;
}
/* This `VectorRegSizePair` union allows us to have an overlapping type
* to select a scalar operation or a vector depending on which operation
* we pass in.
* Useful in FEX's vector operations that behave as scalar or vector
* depending on various factors. But since the operation will have the sa,e
* element size, we want to choose the operation more easily
*/
union VectorRegSizePair {
ScalarRegSize Scalar;
SubRegSize Vector;
};
// This allows us to create a `VectorRegSizePair` union.
[[maybe_unused]]
static inline VectorRegSizePair ToVectorSizePair(SubRegSize size) {
return VectorRegSizePair {.Vector = size};
}
[[maybe_unused]]
static inline VectorRegSizePair ToVectorSizePair(ScalarRegSize size) {
return VectorRegSizePair {.Scalar = size};
}
// This `ShiftType` enum is used for ALU shift-register encoded instructions.
enum class ShiftType : uint32_t {
LSL = 0,
LSR,
ASR,
ROR,
};
// This `ExtendedType` enum is used for ALU extended-register encoded instructions.
enum class ExtendedType : uint32_t {
UXTB = 0b000,
UXTH = 0b001,
UXTW = 0b010,
UXTX = 0b011,
SXTB = 0b100,
SXTH = 0b101,
SXTW = 0b110,
SXTX = 0b111,
LSL_32 = UXTW,
LSL_64 = UXTX,
};
// This `Condition` enum is used for various conditional instructions.
enum class Condition : uint32_t {
// Meaning: Int - Float
CC_EQ = 0, // Equal - Equal
CC_NE, // Not Eq - Not Eq or unordered
CC_CS, // Carry set - Greater than, equal, or unordered
CC_CC, // Carry clear - Less than
CC_MI, // Minus/Negative - Less than
CC_PL, // Plus, positive or zero - GT, equal, or unordered
CC_VS, // Overflow - Unordered
CC_VC, // No Overflow - Ordered
CC_HI, // Unsigned higher - GT, or unordered
CC_LS, // Unsigned lower or same - LT or EQ
CC_GE, // Signed GT or EQ - GT or EQ
CC_LT, // Signed LT - LT or Unordered
CC_GT, // Signed GT - GT
CC_LE, // Signed LT or EQ - LT, EQ, or Unordered
CC_AL, // Always - Always
CC_NV, // Always - Always
// Aliases
CC_HS = CC_CS,
CC_LO = CC_CC,
};
/*
* This `StatusFlags` enum is used for conditional compare encoded instructions.
* These directly encode to the `nzcv` flags.
*/
enum class StatusFlags : uint32_t {
None = 0,
Flag_V = 0b0001,
Flag_C = 0b0010,
Flag_Z = 0b0100,
Flag_N = 0b1000,
Flag_NZCV = Flag_N | Flag_Z | Flag_C | Flag_V,
};
/*
* This `IndexType` enum is used for load-store instructions.
* Not all load-store instructions use this, so the user needs to be careful.
*/
enum class IndexType {
POST,
OFFSET,
PRE,
UNPRIVILEGED,
};
/* This `SVEMemOperand` class is used for the helper SVE load-store instructions.
* Load-store instructions are quite expressive, so having a helper that handles these differences is worth it.
*/
class SVEMemOperand final {
public:
SVEMemOperand(XRegister rn, XRegister rm = XReg::zr)
: rn {rn}
, MetaType {
.ScalarScalarType {
.Header = { .MemType = TYPE_SCALAR_SCALAR },
.rm = rm,
}
} {}
SVEMemOperand(XRegister rn, int32_t imm = 0)
: rn {rn}
, MetaType {
.ScalarImmType {
.Header = { .MemType = TYPE_SCALAR_IMM },
.Imm = imm,
}
} {}
Register rn;
enum Type {
TYPE_SCALAR_SCALAR,
TYPE_SCALAR_IMM,
TYPE_SCALAR_VECTOR,
TYPE_VECTOR_IMM,
};
struct HeaderStruct {
Type MemType;
};
union {
HeaderStruct Header;
struct {
HeaderStruct Header;
Register rm;
} ScalarScalarType;
struct {
HeaderStruct Header;
int32_t Imm;
} ScalarImmType;
struct {
HeaderStruct Header;
ZRegister zm;
// TODO: Implement support for modifier
} ScalarVectorType;
struct {
HeaderStruct Header;
// rn will be a ZRegister
int32_t Imm;
} VectorImmType;
} MetaType;
};
/* This `ExtendedMemOperand` class is used for the helper load-store instructions.
* Load-store instructions are quite expressive, so having a helper that handles these differences is worth it.
*/
class ExtendedMemOperand final {
public:
ExtendedMemOperand(XRegister rn, XRegister rm = XReg::zr, ExtendedType Option = ExtendedType::LSL_64, uint32_t Shift = 0)
: rn {rn}
, MetaType {
.ExtendedType {
.Header = { .MemType = TYPE_EXTENDED },
.rm = rm,
.Option = Option,
.Shift = Shift,
}
} {}
ExtendedMemOperand(XRegister rn, IndexType Index = IndexType::OFFSET, int32_t Imm = 0)
: rn {rn}
, MetaType {
.ImmType {
.Header = { .MemType = TYPE_IMM },
.Index = Index,
.Imm = Imm,
}
} {}
Register rn;
enum Type {
TYPE_EXTENDED,
TYPE_IMM,
};
struct HeaderStruct {
Type MemType;
};
union {
HeaderStruct Header;
struct {
HeaderStruct Header;
Register rm;
ExtendedType Option;
uint32_t Shift;
} ExtendedType;
struct {
HeaderStruct Header;
IndexType Index;
int32_t Imm;
} ImmType;
} MetaType;
};
template<uint32_t op0, uint32_t op1, uint32_t CRn, uint32_t CRm, uint32_t op2>
constexpr uint32_t GenSystemReg() {
return op0 << 19 |
op1 << 16 |
CRn << 12 |
CRm << 8 |
op2 << 5;
};
// This `SystemRegister` enum is used for the mrs/msr instructions.
enum class SystemRegister : uint32_t {
CTR_EL0 = GenSystemReg<0b11, 0b011, 0b0000, 0b0000, 0b001>(),
DCZID_EL0 = GenSystemReg<0b11, 0b011, 0b0000, 0b0000, 0b111>(),
TPIDR_EL0 = GenSystemReg<0b11, 0b011, 0b1101, 0b0000, 0b010>(),
RNDR = GenSystemReg<0b11, 0b011, 0b0010, 0b0100, 0b000>(),
RNDRRS = GenSystemReg<0b11, 0b011, 0b0010, 0b0100, 0b001>(),
NZCV = GenSystemReg<0b11, 0b011, 0b0100, 0b0010, 0b000>(),
FPCR = GenSystemReg<0b11, 0b011, 0b0100, 0b0100, 0b000>(),
CNTFRQ_EL0 = GenSystemReg<0b11, 0b011, 0b1110, 0b0000, 0b000>(),
CNTVCT_EL0 = GenSystemReg<0b11, 0b011, 0b1110, 0b0000, 0b010>(),
};
template<uint32_t op1, uint32_t CRm, uint32_t op2>
constexpr uint32_t GenDCReg() {
return op1 << 16 |
CRm << 8 |
op2 << 5;
};
// This `DataCacheOperation` enum is used for the dc instruction.
enum class DataCacheOperation : uint32_t {
IVAC = GenDCReg<0b000, 0b0110, 0b001>(),
ISW = GenDCReg<0b000, 0b0110, 0b010>(),
CSW = GenDCReg<0b000, 0b1010, 0b010>(),
CISW = GenDCReg<0b000, 0b1110, 0b010>(),
ZVA = GenDCReg<0b011, 0b0100, 0b001>(),
CVAC = GenDCReg<0b011, 0b1010, 0b001>(),
CVAU = GenDCReg<0b011, 0b1011, 0b001>(),
CIVAC = GenDCReg<0b011, 0b1110, 0b001>(),
// MTE2
IGVAC = GenDCReg<0b000, 0b0110, 0b011>(),
IGSW = GenDCReg<0b000, 0b0110, 0b100>(),
IGDVAC = GenDCReg<0b000, 0b0110, 0b101>(),
IGDSW = GenDCReg<0b000, 0b0110, 0b110>(),
CGSW = GenDCReg<0b000, 0b1010, 0b100>(),
CGDSW = GenDCReg<0b000, 0b1010, 0b110>(),
CIGSW = GenDCReg<0b000, 0b1110, 0b100>(),
CIGDSW = GenDCReg<0b000, 0b1110, 0b110>(),
// MTE
GVA = GenDCReg<0b011, 0b0100, 0b011>(),
GZVA = GenDCReg<0b011, 0b0100, 0b100>(),
CGVAC = GenDCReg<0b011, 0b1010, 0b011>(),
CGDVAC = GenDCReg<0b011, 0b1010, 0b101>(),
CGVAP = GenDCReg<0b011, 0b1100, 0b011>(),
CGDVAP = GenDCReg<0b011, 0b1100, 0b101>(),
CGVADP = GenDCReg<0b011, 0b1101, 0b011>(),
CGDVADP = GenDCReg<0b011, 0b1101, 0b101>(),
CIGVAC = GenDCReg<0b011, 0b1110, 0b011>(),
CIGDVAC = GenDCReg<0b011, 0b1110, 0b101>(),
// DPB
CVAP = GenDCReg<0b011, 0b1100, 0b001>(),
// DPB2
CVADP = GenDCReg<0b011, 0b1101, 0b001>(),
};
template<uint32_t CRm, uint32_t op2>
constexpr uint32_t GenHintBarrierReg() {
return CRm << 8 |
op2 << 5;
}
// This `HintRegister` enum is used for the hint instruction.
enum class HintRegister : uint32_t {
NOP = GenHintBarrierReg<0b0000, 0b000>(),
YIELD = GenHintBarrierReg<0b0000, 0b001>(),
WFE = GenHintBarrierReg<0b0000, 0b010>(),
WFI = GenHintBarrierReg<0b0000, 0b011>(),
SEV = GenHintBarrierReg<0b0000, 0b100>(),
SEVL = GenHintBarrierReg<0b0000, 0b101>(),
DGH = GenHintBarrierReg<0b0000, 0b110>(),
CSDB = GenHintBarrierReg<0b0010, 0b100>(),
};
// This `BarrierRegister` enum is used for the various barrier instructions.
enum class BarrierRegister : uint32_t {
CLREX = GenHintBarrierReg<0b0000, 0b010>(),
TCOMMIT = GenHintBarrierReg<0b0000, 0b011>(),
DSB = GenHintBarrierReg<0b0000, 0b100>(),
DMB = GenHintBarrierReg<0b0000, 0b101>(),
ISB = GenHintBarrierReg<0b0000, 0b110>(),
SB = GenHintBarrierReg<0b0000, 0b111>(),
};
// This `BarrierScope` enum is used for the dsb/dmb instructions.
enum class BarrierScope : uint32_t {
// Outer shareable
OSHLD = 0b0001,
OSHST = 0b0010,
OSH = 0b0011,
// Non shareable
NSHLD = 0b0101,
NSHST = 0b0110,
NSH = 0b0111,
// Inner shareable
ISHLD = 0b1001,
ISHST = 0b1010,
ISH = 0b1011,
// Full System visibility
LD = 0b1101,
ST = 0b1110,
SY = 0b1111,
};
// This `Prefetch` enum is used for prefetch instructions.
enum class Prefetch : uint32_t {
// Prefetch for load
PLDL1KEEP = 0b00000,
PLDL1STRM = 0b00001,
PLDL2KEEP = 0b00010,
PLDL2STRM = 0b00011,
PLDL3KEEP = 0b00100,
PLDL3STRM = 0b00101,
// Preload instructions
PLIL1KEEP = 0b01000,
PLIL1STRM = 0b01001,
PLIL2KEEP = 0b01010,
PLIL2STRM = 0b01011,
PLIL3KEEP = 0b01100,
PLIL3STRM = 0b01101,
// Preload for store
PSTL1KEEP = 0b10000,
PSTL1STRM = 0b10001,
PSTL2KEEP = 0b10010,
PSTL2STRM = 0b10011,
PSTL3KEEP = 0b10100,
PSTL3STRM = 0b10101,
};
// This `PredicatePattern` enun is used for some SVE instructions.
enum class PredicatePattern : uint32_t {
SVE_POW2 = 0b00000,
SVE_VL1 = 0b00001,
SVE_VL2 = 0b00010,
SVE_VL3 = 0b00011,
SVE_VL4 = 0b00100,
SVE_VL5 = 0b00101,
SVE_VL6 = 0b00110,
SVE_VL7 = 0b00111,
SVE_VL8 = 0b01000,
SVE_VL16 = 0b01001,
SVE_VL32 = 0b01010,
SVE_VL64 = 0b01011,
SVE_VL128 = 0b01100,
SVE_VL256 = 0b01101,
SVE_MUL4 = 0b11101,
SVE_MUL3 = 0b11110,
SVE_ALL = 0b11111,
};
/* This `BackwardLabel` struct used for retaining a location for PC-Relative instructions.
* This is specifically a label for a target that is logically `below` an instruction that uses it.
* Which means that a branch would jump backwards.
*/
struct BackwardLabel {
uint8_t *Location{};
};
/* This `ForwardLabel` struct used for retaining a location for PC-Relative instructions.
* This is specifically a label for a target that is logically `above` an instruction that uses it.
* Which means that a branch would jump forwards.
*
* This can be bound to multiple instructions, so it needs a vector for each bind instruction type.
*/
struct ForwardLabel {
struct Instructions {
enum class InstType {
ADR,
ADRP,
B,
BC,
TEST_BRANCH,
RELATIVE_LOAD,
};
uint8_t *Location{};
InstType Type;
};
std::vector<Instructions> Insts{};
};
/* This `BiDirectionalLabel` struct used for retaining a location for PC-Relative instructions.
* This is specifically a label for a target that is in either direction of an instruction that uses it.
* Which means a branch could jump backwards or forwards depending on situation.
*/
struct BiDirectionalLabel {
BackwardLabel Backward;
ForwardLabel Forward;
};
// This is an emitter that is designed around the smallest code bloat as possible.
// Eschewing most developer convenience in order to keep code as small as possible.
// Choices:
// - Size of ops passed as an argument rather than template to let the compiler use csel instead of branching.
// - Registers are unsized so they can be passed in a GPR and not need conversion operations
class Emitter : public FEXCore::ARMEmitter::Buffer {
public:
Emitter() = default;
Emitter(uint8_t* Base, uint64_t BaseSize)
: Buffer (Base, BaseSize) {
}
// Bind a backward label to an address.
// Address that is bound is the current emitter location.
void Bind(BackwardLabel *Label) {
LOGMAN_THROW_AA_FMT(Label->Location == nullptr, "Trying to bind a label twice");
Label->Location = GetCursorAddress<uint8_t*>();
}
// Bind a forward label to a location.
// This walks all the instructions in the label's vector.
// Then backpatching all instructions that have used the label.
template<bool WarnAboutEmpty = false>
void Bind(ForwardLabel *Label) {
if constexpr (WarnAboutEmpty) {
LOGMAN_THROW_A_FMT(Label->Insts.empty() == false, "Binding forward label that didn't have any instructions using it");
}
uint8_t *CurrentAddress = GetCursorAddress<uint8_t*>();
for (const auto &Inst : Label->Insts) {
// Patch up the instructions
switch (Inst.Type) {
case ForwardLabel::Instructions::InstType::ADR: {
uint32_t *Instruction = reinterpret_cast<uint32_t*>(Inst.Location);
int64_t Imm = reinterpret_cast<int64_t>(CurrentAddress) - reinterpret_cast<int64_t>(Instruction);
LOGMAN_THROW_A_FMT(Imm >= -1048576 && Imm <= 1048575, "Unscaled offset too large");
uint32_t InstMask = 0b11 << 29 | 0b1111'1111'1111'1111'111 << 5;
uint32_t Offset = static_cast<uint32_t>(Imm) & 0x3F'FFFF;
uint32_t Inst = *Instruction & ~InstMask;
Inst |= (Offset & 0b11) << 29;
Inst |= (Offset >> 2) << 5;
*Instruction = Inst;
break;
}
case ForwardLabel::Instructions::InstType::ADRP: {
uint32_t *Instruction = reinterpret_cast<uint32_t*>(Inst.Location);
int64_t Imm = reinterpret_cast<int64_t>(CurrentAddress) - reinterpret_cast<int64_t>(Instruction);
LOGMAN_THROW_A_FMT(Imm >= -4294967296 && Imm <= 4294963200 && (Imm & 0xFFF) == 0, "Unscaled offset too large");
Imm >>= 12;
uint32_t InstMask = 0b11 << 29 | 0b1111'1111'1111'1111'111 << 5;
uint32_t Offset = static_cast<uint32_t>(Imm) & 0x3F'FFFF;
uint32_t Inst = *Instruction & ~InstMask;
Inst |= (Offset & 0b11) << 29;
Inst |= (Offset >> 2) << 5;
*Instruction = Inst;
break;
}
case ForwardLabel::Instructions::InstType::B: {
uint32_t *Instruction = reinterpret_cast<uint32_t*>(Inst.Location);
int64_t Imm = reinterpret_cast<int64_t>(CurrentAddress) - reinterpret_cast<int64_t>(Instruction);
LOGMAN_THROW_A_FMT(Imm >= -134217728 && Imm <= 134217724 && ((Imm & 0b11) == 0), "Unscaled offset too large");
Imm >>= 2;
uint32_t InstMask = 0x3FF'FFFF;
uint32_t Offset = static_cast<uint32_t>(Imm) & InstMask;
uint32_t Inst = *Instruction & ~InstMask;
Inst |= Offset;
*Instruction = Inst;
break;
}
case ForwardLabel::Instructions::InstType::TEST_BRANCH: {
uint32_t *Instruction = reinterpret_cast<uint32_t*>(Inst.Location);
int64_t Imm = reinterpret_cast<int64_t>(CurrentAddress) - reinterpret_cast<int64_t>(Instruction);
LOGMAN_THROW_A_FMT(Imm >= -32768 && Imm <= 32764 && ((Imm & 0b11) == 0), "Unscaled offset too large");
Imm >>= 2;
uint32_t InstMask = 0x3FFF;
uint32_t Offset = static_cast<uint32_t>(Imm) & InstMask;
uint32_t Inst = *Instruction & ~(InstMask << 5);
Inst |= Offset << 5;
*Instruction = Inst;
break;
}
case ForwardLabel::Instructions::InstType::BC:
case ForwardLabel::Instructions::InstType::RELATIVE_LOAD: {
uint32_t *Instruction = reinterpret_cast<uint32_t*>(Inst.Location);
int64_t Imm = reinterpret_cast<int64_t>(CurrentAddress) - reinterpret_cast<int64_t>(Instruction);
LOGMAN_THROW_A_FMT(Imm >= -1048576 && Imm <= 1048575 && ((Imm & 0b11) == 0), "Unscaled offset too large");
Imm >>= 2;
uint32_t InstMask = 0x7'FFFF;
uint32_t Offset = static_cast<uint32_t>(Imm) & InstMask;
uint32_t Inst = *Instruction & ~(InstMask << 5);
Inst |= Offset << 5;
*Instruction = Inst;
break;
}
default: LOGMAN_MSG_A_FMT("Unexpected inst type in label fixup");
}
}
}
// Bind a bidirectional location to a location.
// Binds both forwards and backwards depending on how the label was used.
void Bind(BiDirectionalLabel *Label) {
if (!Label->Backward.Location) {
Bind(&Label->Backward);
}
Bind<false>(&Label->Forward);
}
public:
// TODO: Implement SME when it matters.
#include "Interface/Core/ArchHelpers/CodeEmitter/ALUOps.inl"
#include "Interface/Core/ArchHelpers/CodeEmitter/BranchOps.inl"
#include "Interface/Core/ArchHelpers/CodeEmitter/LoadstoreOps.inl"
#include "Interface/Core/ArchHelpers/CodeEmitter/SystemOps.inl"
#include "Interface/Core/ArchHelpers/CodeEmitter/ScalarOps.inl"
#include "Interface/Core/ArchHelpers/CodeEmitter/ASIMDOps.inl"
#include "Interface/Core/ArchHelpers/CodeEmitter/SVEOps.inl"
private:
template<typename T>
uint32_t Encode_ra(T Reg) const {
return Reg.Idx() << 10;
}
uint32_t Encode_ra(uint32_t Reg) const {
return Reg << 10;
}
template<typename T>
uint32_t Encode_rt2(T Reg) const {
return Reg.Idx() << 10;
}
template<>
uint32_t Encode_rt2(uint32_t Reg) const {
return Reg << 10;
}
template<typename T>
uint32_t Encode_rm(T Reg) const {
return Reg.Idx() << 16;
}
uint32_t Encode_rm(uint32_t Reg) const {
return Reg << 16;
}
template<typename T>
uint32_t Encode_rs(T Reg) const {
return Reg.Idx() << 16;
}
uint32_t Encode_rs(uint32_t Reg) const {
return Reg << 16;
}
template<typename T>
uint32_t Encode_rn(T Reg) const {
return Reg.Idx() << 5;
}
uint32_t Encode_rn(uint32_t Reg) const {
return Reg << 5;
}
template<typename T>
uint32_t Encode_rd(T Reg) const {
return Reg.Idx();
}
uint32_t Encode_rd(uint32_t Reg) const {
return Reg;
}
template<typename T>
uint32_t Encode_rt(T Reg) const {
return Reg.Idx();
}
template<>
uint32_t Encode_rt(Prefetch Reg) const {
return FEXCore::ToUnderlying(Reg);
}
uint32_t Encode_rt(uint32_t Reg) const {
return Reg;
}
template<typename T>
uint32_t Encode_pd(T Reg) const {
return FEXCore::ToUnderlying(Reg);
}
};
}
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,175 @@
/* System instruction emitters.
*
* This is mostly a mashup of various instruction types.
* Nothing follows an explicit pattern since they are mostly different.
*/
public:
// System with result
// TODO: SYSL
// System Instruction
// TODO: AT
// TODO: CFP
// TODO: CPP
void dc(FEXCore::ARMEmitter::DataCacheOperation DCOp, FEXCore::ARMEmitter::Register rt) {
constexpr uint32_t Op = 0b1101'0101'0000'1000'0111 << 12;
SystemInstruction(Op, 0, FEXCore::ToUnderlying(DCOp), rt);
}
// TODO: DVP
// TODO: IC
// TODO: TLBI
// Exception generation
void svc(uint32_t Imm) {
ExceptionGeneration(0b000, 0b000, 0b01, Imm);
}
void hvc(uint32_t Imm) {
ExceptionGeneration(0b000, 0b000, 0b10, Imm);
}
void smc(uint32_t Imm) {
ExceptionGeneration(0b000, 0b000, 0b11, Imm);
}
void brk(uint32_t Imm) {
ExceptionGeneration(0b001, 0b000, 0b00, Imm);
}
void hlt(uint32_t Imm) {
ExceptionGeneration(0b010, 0b000, 0b00, Imm);
}
void tcancel(uint32_t Imm) {
ExceptionGeneration(0b011, 0b000, 0b00, Imm);
}
void dcps1(uint32_t Imm) {
ExceptionGeneration(0b101, 0b000, 0b01, Imm);
}
void dcps2(uint32_t Imm) {
ExceptionGeneration(0b101, 0b000, 0b10, Imm);
}
void dcps3(uint32_t Imm) {
ExceptionGeneration(0b101, 0b000, 0b11, Imm);
}
// System instructions with register argument
void wfet(FEXCore::ARMEmitter::Register rt) {
SystemInstructionWithReg(0b0000, 0b000, rt);
}
void wfit(FEXCore::ARMEmitter::Register rt) {
SystemInstructionWithReg(0b0000, 0b001, rt);
}
// Hints
void nop() {
Hint(FEXCore::ARMEmitter::HintRegister::NOP);
}
void yield() {
Hint(FEXCore::ARMEmitter::HintRegister::YIELD);
}
void wfe() {
Hint(FEXCore::ARMEmitter::HintRegister::WFE);
}
void wfi() {
Hint(FEXCore::ARMEmitter::HintRegister::WFI);
}
void sev() {
Hint(FEXCore::ARMEmitter::HintRegister::SEV);
}
void sevl() {
Hint(FEXCore::ARMEmitter::HintRegister::SEVL);
}
void dgh() {
Hint(FEXCore::ARMEmitter::HintRegister::DGH);
}
void csdb() {
Hint(FEXCore::ARMEmitter::HintRegister::CSDB);
}
// Barriers
void clrex(uint32_t imm = 15) {
LOGMAN_THROW_AA_FMT(imm < 16, "Immediate out of range");
Barrier(FEXCore::ARMEmitter::BarrierRegister::CLREX, imm);
}
void dsb(FEXCore::ARMEmitter::BarrierScope Scope) {
Barrier(FEXCore::ARMEmitter::BarrierRegister::DSB, FEXCore::ToUnderlying(Scope));
}
void dmb(FEXCore::ARMEmitter::BarrierScope Scope) {
Barrier(FEXCore::ARMEmitter::BarrierRegister::DMB, FEXCore::ToUnderlying(Scope));
}
void isb() {
Barrier(FEXCore::ARMEmitter::BarrierRegister::ISB, FEXCore::ToUnderlying(FEXCore::ARMEmitter::BarrierScope::SY));
}
void sb() {
Barrier(FEXCore::ARMEmitter::BarrierRegister::SB, 0);
}
void tcommit() {
Barrier(FEXCore::ARMEmitter::BarrierRegister::TCOMMIT, 0);
}
// System register move
void msr(FEXCore::ARMEmitter::SystemRegister reg, FEXCore::ARMEmitter::Register rt) {
constexpr uint32_t Op = 0b1101'0101'0001 << 20;
SystemRegisterMove(Op, rt, reg);
}
void mrs(FEXCore::ARMEmitter::Register rd, FEXCore::ARMEmitter::SystemRegister reg) {
constexpr uint32_t Op = 0b1101'0101'0011 << 20;
SystemRegisterMove(Op, rd, reg);
}
private:
// Exception Generation
void ExceptionGeneration(uint32_t opc, uint32_t op2, uint32_t LL, uint32_t Imm) {
LOGMAN_THROW_AA_FMT((Imm & 0xFFFF'0000) == 0, "Imm amount too large");
uint32_t Instr = 0b1101'0100 << 24;
Instr |= opc << 21;
Instr |= Imm << 5;
Instr |= op2 << 2;
Instr |= LL;
dc32(Instr);
}
// System instructions with register argument
void SystemInstructionWithReg(uint32_t CRm, uint32_t op2, FEXCore::ARMEmitter::Register rt) {
uint32_t Instr = 0b1101'0101'0000'0011'0001 << 12;
Instr |= CRm << 8;
Instr |= op2 << 5;
Instr |= Encode_rt(rt);
dc32(Instr);
}
// Hints
void Hint(FEXCore::ARMEmitter::HintRegister Reg) {
uint32_t Instr = 0b1101'0101'0000'0011'0010'0000'0001'1111U;
Instr |= FEXCore::ToUnderlying(Reg);
dc32(Instr);
}
// Barriers
void Barrier(FEXCore::ARMEmitter::BarrierRegister Reg, uint32_t CRm) {
uint32_t Instr = 0b1101'0101'0000'0011'0011'0000'0001'1111U;
Instr |= CRm << 8;
Instr |= FEXCore::ToUnderlying(Reg);
dc32(Instr);
}
// System Instruction
void SystemInstruction(uint32_t Op, uint32_t L, uint32_t SubOp, FEXCore::ARMEmitter::Register rt) {
uint32_t Instr = Op;
Instr |= L << 21;
Instr |= SubOp;
Instr |= Encode_rt(rt);
dc32(Instr);
}
// System register move
void SystemRegisterMove(uint32_t Op, FEXCore::ARMEmitter::Register rt, FEXCore::ARMEmitter::SystemRegister reg) {
uint32_t Instr = Op;
Instr |= FEXCore::ToUnderlying(reg);
Instr |= Encode_rt(rt);
dc32(Instr);
}
@@ -3,6 +3,7 @@
#include <FEXCore/Utils/LogManager.h>
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/Core/UContext.h>
#include <FEXCore/Core/X86Enums.h>
#include <signal.h>
#include <string.h>
@@ -10,7 +11,6 @@
#include <stdint.h>
#include <type_traits>
namespace FEXCore::ArchHelpers::Context {
enum ContextFlags : uint32_t {
@@ -76,6 +76,7 @@ static inline mcontext_t* GetMContext(void* ucontext) {
#ifdef _M_ARM_64
constexpr uint32_t FPR_MAGIC = 0x46508001U;
constexpr uint32_t ESR1_MAGIC = 0x45535201U;
struct HostCTXHeader {
uint32_t Magic;
@@ -89,6 +90,11 @@ struct HostFPRState {
__uint128_t FPRs[32];
};
struct HostESRState {
HostCTXHeader Head;
uint64_t ESR;
};
static inline uint64_t GetSp(void* ucontext) {
return GetMContext(ucontext)->sp;
}
@@ -129,6 +135,61 @@ static inline __uint128_t GetArmFPR(void* ucontext, uint32_t id) {
return HostState->FPRs[id];
}
static inline uint64_t GetArmESR(void* ucontext) {
auto MContext = GetMContext(ucontext);
size_t i = 0;
auto HostState = reinterpret_cast<HostCTXHeader*>(&MContext->__reserved[i]);
do {
if (HostState->Magic == ESR1_MAGIC) {
auto ESR = reinterpret_cast<HostESRState*>(HostState);
return ESR->ESR;
}
i += HostState->Size;
HostState = reinterpret_cast<HostCTXHeader*>(&MContext->__reserved[i]);
} while (HostState->Size != 0);
return 0;
}
constexpr static uint64_t ESR1_EC = 0b111111U << 26;
constexpr static uint64_t ESR1_EC_DataAbort = 0b100100U << 26;
// Write-Not-Read flag
// When set - Abort is due to a write
constexpr static uint64_t ESR1_WNR = 1 << 6;
// DFSC - Default Status Code
// Translation fault - No page mapped
// Permissions fault - Page mapped but with incorrect permission from access.
constexpr static uint64_t ESR1_DataAbort_DFSC = 0b111111;
constexpr static uint64_t ESR1_DataAbort_TranslationFault_EL0 = 0b000111;
constexpr static uint64_t ESR1_DataAbort_PermissionFault_EL0 = 0b001111;
constexpr static uint64_t ESR1_DataAbort_Level = 0b11;
constexpr static uint64_t ESR1_DataAbort_Level_EL3 = 0b00;
constexpr static uint64_t ESR1_DataAbort_Level_EL2 = 0b01;
constexpr static uint64_t ESR1_DataAbort_Level_EL1 = 0b10;
constexpr static uint64_t ESR1_DataAbort_Level_EL0 = 0b11;
static inline uint32_t GetProtectFlags(void* ucontext) {
uint64_t ESR = GetArmESR(ucontext);
LOGMAN_THROW_A_FMT((ESR & ESR1_EC) == ESR1_EC_DataAbort, "Unknown ESR1 EC type: 0x{:x} != 0x{:x}", ESR & ESR1_EC, ESR1_EC_DataAbort);
uint32_t ProtectFlags{};
if ((ESR & ESR1_DataAbort_Level) == ESR1_DataAbort_Level_EL0) {
// Always a user error for us.
ProtectFlags |= X86State::X86_PF_USER;
}
if (ESR & ESR1_WNR) {
// Fault was due to a write
ProtectFlags |= X86State::X86_PF_WRITE;
}
// PF_PROT is not returned to user on x86, so don't return the difference between permission fault and translation fault.
return ProtectFlags;
}
using ContextBackup = ArmContextBackup;
template <typename T>
static inline void BackupContext(void* ucontext, T *Backup) {
@@ -222,6 +283,10 @@ static inline __uint128_t GetArmFPR(void* ucontext, uint32_t id) {
ERROR_AND_DIE_FMT("Not implemented for x86 host");
}
static inline uint32_t GetProtectFlags(void* ucontext) {
return GetMContext(ucontext)->gregs[REG_ERR];
}
using ContextBackup = X86ContextBackup;
template <typename T>
static inline void BackupContext(void* ucontext, T *Backup) {
+16 -13
View File
@@ -856,22 +856,25 @@ namespace FEXCore::Context {
Thread->OpDispatcher->_ExitFunction(Thread->OpDispatcher->_EntrypointOffset(Block.Entry - GuestRIP, GPRSize));
}
// If we had a dispatch error then leave early
if (HadDispatchError) {
if (TotalInstructions == 0) {
// Couldn't handle any instruction in op dispatcher
Thread->OpDispatcher->ResetWorkingList();
return { nullptr, nullptr, 0, 0, 0, 0 };
}
else {
const uint8_t GPRSize = GetGPRSize();
const bool NeedsBlockEnd = (HadDispatchError && TotalInstructions > 0) ||
(Thread->OpDispatcher->NeedsBlockEnder() && i + 1 == InstsInBlock);
// We had some instructions. Early exit
Thread->OpDispatcher->_ExitFunction(Thread->OpDispatcher->_EntrypointOffset(Block.Entry + BlockInstructionsLength - GuestRIP, GPRSize));
break;
}
// If we had a dispatch error then leave early
if (HadDispatchError && TotalInstructions == 0) {
// Couldn't handle any instruction in op dispatcher
Thread->OpDispatcher->ResetWorkingList();
return { nullptr, nullptr, 0, 0, 0, 0 };
}
if (NeedsBlockEnd) {
const uint8_t GPRSize = GetGPRSize();
// We had some instructions. Early exit
Thread->OpDispatcher->_ExitFunction(Thread->OpDispatcher->_EntrypointOffset(Block.Entry + BlockInstructionsLength - GuestRIP, GPRSize));
break;
}
if (Thread->OpDispatcher->FinishOp(DecodedInfo->PC + DecodedInfo->InstSize, i + 1 == InstsInBlock)) {
break;
}
@@ -1,3 +1,4 @@
#include "Interface/Core/ArchHelpers/CodeEmitter/Emitter.h"
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/ArchHelpers/MContext.h"
@@ -30,15 +31,9 @@
#include <sys/syscall.h>
#include <unistd.h>
#define STATE_PTR(STATE_TYPE, FIELD) \
MemOperand(STATE, offsetof(FEXCore::Core::STATE_TYPE, FIELD))
namespace FEXCore::CPU {
using namespace vixl;
using namespace vixl::aarch64;
constexpr size_t MAX_DISPATCHER_CODE_SIZE = 8192;
constexpr size_t MAX_DISPATCHER_CODE_SIZE = 4096;
Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, const DispatcherConfig &config)
: FEXCore::CPU::Dispatcher(ctx, config), Arm64Emitter(ctx, MAX_DISPATCHER_CODE_SIZE)
@@ -46,13 +41,18 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, const Dispatche
, Simulator {&Decoder}
#endif
{
#ifdef VIXL_SIMULATOR
// Hardcode a 256-bit vector width if we are running in the simulator.
Simulator.SetVectorLengthInBits(256);
#endif
SetAllowAssembler(true);
EmitDispatcher();
}
void Arm64Dispatcher::EmitDispatcher() {
#ifdef VIXL_DISASSEMBLER
const auto DisasmBegin = GetCursorAddress<const vixl::aarch64::Instruction*>();
#endif
DispatchPtr = GetCursorAddress<AsmDispatch>();
@@ -64,9 +64,9 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, const Dispatche
// Ptr();
// }
Literal l_CTX {reinterpret_cast<uintptr_t>(CTX)};
Literal l_Sleep {reinterpret_cast<uint64_t>(SleepThread)};
Literal l_CompileBlock {GetCompileBlockPtr()};
ARMEmitter::ForwardLabel l_CTX;
ARMEmitter::ForwardLabel l_Sleep;
ARMEmitter::ForwardLabel l_CompileBlock;
// Push all the register we need to save
PushCalleeSavedRegisters();
@@ -74,12 +74,12 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, const Dispatche
// Push our memory base to the correct register
// Move our thread pointer to the correct register
// This is passed in to parameter 0 (x0)
mov(STATE, x0);
mov(STATE, ARMEmitter::XReg::x0);
// Save this stack pointer so we can cleanly shutdown the emulation with a long jump
// regardless of where we were in the stack
add(x0, sp, 0);
str(x0, STATE_PTR(CpuStateFrame, ReturningStackLocation));
add(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r0, ARMEmitter::Reg::rsp, 0);
str(ARMEmitter::XReg::x0, STATE_PTR(CpuStateFrame, ReturningStackLocation));
AbsoluteLoopTopAddressFillSRA = GetCursorAddress<uint64_t>();
@@ -89,91 +89,97 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, const Dispatche
// We want to ensure that we are 16 byte aligned at the top of this loop
Align16B();
aarch64::Label FullLookup{};
aarch64::Label CallBlock{};
aarch64::Label LoopTop{};
aarch64::Label ExitSpillSRA{};
aarch64::Label ThreadPauseHandler{};
ARMEmitter::BiDirectionalLabel FullLookup{};
ARMEmitter::BiDirectionalLabel CallBlock{};
ARMEmitter::BackwardLabel LoopTop{};
bind(&LoopTop);
AbsoluteLoopTopAddress = GetLabelAddress<uint64_t>(&LoopTop);
Bind(&LoopTop);
AbsoluteLoopTopAddress = GetCursorAddress<uint64_t>();
// Load in our RIP
// Don't modify x2 since it contains our RIP once the block doesn't exist
ldr(x2, STATE_PTR(CpuStateFrame, State.rip));
auto RipReg = x2;
auto RipReg = ARMEmitter::XReg::x2;
ldr(RipReg, STATE_PTR(CpuStateFrame, State.rip));
// L1 Cache
ldr(x0, STATE_PTR(CpuStateFrame, Pointers.Common.L1Pointer));
ldr(ARMEmitter::XReg::x0, STATE_PTR(CpuStateFrame, Pointers.Common.L1Pointer));
and_(x3, RipReg, LookupCache::L1_ENTRIES_MASK);
add(x0, x0, Operand(x3, Shift::LSL, 4));
ldp(x3, x0, MemOperand(x0));
cmp(x0, RipReg);
b(&FullLookup, Condition::ne);
and_(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r3, RipReg.R(), LookupCache::L1_ENTRIES_MASK);
add(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r0, ARMEmitter::Reg::r0, ARMEmitter::Reg::r3, ARMEmitter::ShiftType::LSL , 4);
ldp<ARMEmitter::IndexType::OFFSET>(ARMEmitter::XReg::x3, ARMEmitter::XReg::x0, ARMEmitter::Reg::r0, 0);
cmp(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r0, RipReg.R());
b(ARMEmitter::Condition::CC_NE, &FullLookup);
br(x3);
br(ARMEmitter::Reg::r3);
// L1C check failed, do a full lookup
bind(&FullLookup);
Bind(&FullLookup);
// This is the block cache lookup routine
// It matches what is going on it LookupCache.h::FindBlock
ldr(x0, STATE_PTR(CpuStateFrame, Pointers.Common.L2Pointer));
ldr(ARMEmitter::XReg::x0, STATE_PTR(CpuStateFrame, Pointers.Common.L2Pointer));
// Mask the address by the virtual address size so we can check for aliases
uint64_t VirtualMemorySize = CTX->Config.VirtualMemSize;
if (std::popcount(VirtualMemorySize) == 1) {
and_(x3, RipReg, VirtualMemorySize - 1);
and_(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r3, RipReg.R(), VirtualMemorySize - 1);
}
else {
LoadConstant(x3, VirtualMemorySize);
and_(x3, RipReg, x3);
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r3, VirtualMemorySize);
and_(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r3, RipReg.R(), ARMEmitter::Reg::r3);
}
aarch64::Label NoBlock;
#ifdef VIXL_SIMULATOR
// VIXL simulator can't run syscalls.
constexpr bool SignalSafeCompile = false;
#else
constexpr bool SignalSafeCompile = true;
#endif
ARMEmitter::ForwardLabel NoBlock;
{
// Offset the address and add to our page pointer
lsr(x1, x3, 12);
lsr(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r1, ARMEmitter::Reg::r3, 12);
// Load the pointer from the offset
ldr(x0, MemOperand(x0, x1, Shift::LSL, 3));
ldr(ARMEmitter::XReg::x0, ARMEmitter::Reg::r0, ARMEmitter::Reg::r1, ARMEmitter::ExtendedType::LSL_64, 3);
// If page pointer is zero then we have no block
cbz(x0, &NoBlock);
cbz(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r0, &NoBlock);
// Steal the page offset
and_(x1, x3, 0x0FFF);
and_(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r1, ARMEmitter::Reg::r3, 0x0FFF);
// Shift the offset by the size of the block cache entry
add(x0, x0, Operand(x1, Shift::LSL, (int)log2(sizeof(FEXCore::LookupCache::LookupCacheEntry))));
add(ARMEmitter::XReg::x0, ARMEmitter::XReg::x0, ARMEmitter::XReg::x1, ARMEmitter::ShiftType::LSL, (int)log2(sizeof(FEXCore::LookupCache::LookupCacheEntry)));
// Load the guest address first to ensure it maps to the address we are currently at
// This fixes aliasing problems
ldr(x1, MemOperand(x0, offsetof(FEXCore::LookupCache::LookupCacheEntry, GuestCode)));
cmp(x1, RipReg);
b(&NoBlock, Condition::ne);
ldr(ARMEmitter::XReg::x1, ARMEmitter::Reg::r0, offsetof(FEXCore::LookupCache::LookupCacheEntry, GuestCode));
cmp(ARMEmitter::XReg::x1, RipReg);
b(ARMEmitter::Condition::CC_NE, &NoBlock);
// Now load the actual host block to execute if we can
ldr(x3, MemOperand(x0, offsetof(FEXCore::LookupCache::LookupCacheEntry, HostCode)));
cbz(x3, &NoBlock);
ldr(ARMEmitter::XReg::x3, ARMEmitter::Reg::r0, offsetof(FEXCore::LookupCache::LookupCacheEntry, HostCode));
cbz(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r3, &NoBlock);
// If we've made it here then we have a real compiled block
{
// update L1 cache
ldr(x0, STATE_PTR(CpuStateFrame, Pointers.Common.L1Pointer));
ldr(ARMEmitter::XReg::x0, STATE_PTR(CpuStateFrame, Pointers.Common.L1Pointer));
and_(x1, RipReg, LookupCache::L1_ENTRIES_MASK);
add(x0, x0, Operand(x1, Shift::LSL, 4));
stp(x3, x2, MemOperand(x0));
and_(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r1, RipReg.R(), LookupCache::L1_ENTRIES_MASK);
add(ARMEmitter::XReg::x0, ARMEmitter::XReg::x0, ARMEmitter::XReg::x1, ARMEmitter::ShiftType::LSL, 4);
stp<ARMEmitter::IndexType::OFFSET>(ARMEmitter::XReg::x3, ARMEmitter::XReg::x2, ARMEmitter::Reg::r0);
// Jump to the block
br(x3);
br(ARMEmitter::Reg::r3);
}
}
{
bind(&ExitSpillSRA);
ThreadStopHandlerAddressSpillSRA = GetCursorAddress<uint64_t>();
if (config.StaticRegisterAllocation)
SpillStaticRegs();
@@ -187,12 +193,6 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, const Dispatche
ret();
}
#ifdef VIXL_SIMULATOR
// VIXL simulator can't run syscalls.
constexpr bool SignalSafeCompile = false;
#else
constexpr bool SignalSafeCompile = true;
#endif
{
ExitFunctionLinkerAddress = GetCursorAddress<uint64_t>();
if (config.StaticRegisterAllocation)
@@ -207,52 +207,52 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, const Dispatche
// X3: Size of mask, sizeof(uint64_t)
// X8: Syscall
LoadConstant(x0, ~0ULL);
stp(x0, x0, MemOperand(sp, -16, PreIndex));
LoadConstant(x0, SIG_SETMASK);
add(x1, sp, 0);
add(x2, sp, 0);
LoadConstant(x3, 8);
LoadConstant(x8, SYS_rt_sigprocmask);
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r0, ~0ULL);
stp<ARMEmitter::IndexType::PRE>(ARMEmitter::XReg::x0, ARMEmitter::XReg::x0, ARMEmitter::Reg::rsp, -16);
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r0, SIG_SETMASK);
add(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r1, ARMEmitter::Reg::rsp, 0);
add(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r2, ARMEmitter::Reg::rsp, 0);
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r3, 8);
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r8, SYS_rt_sigprocmask);
svc(0);
}
mov(x0, STATE);
mov(x1, lr);
mov(ARMEmitter::XReg::x0, STATE);
mov(ARMEmitter::XReg::x1, ARMEmitter::XReg::lr);
ldr(x2, STATE_PTR(CpuStateFrame, Pointers.Common.ExitFunctionLink));
ldr(ARMEmitter::XReg::x2, STATE_PTR(CpuStateFrame, Pointers.Common.ExitFunctionLink));
#ifdef VIXL_SIMULATOR
GenerateIndirectRuntimeCall<uintptr_t, void *, void *>(x2);
GenerateIndirectRuntimeCall<uintptr_t, void *, void *>(ARMEmitter::Reg::r2);
#else
blr(x2);
blr(ARMEmitter::Reg::r2);
#endif
if (SignalSafeCompile) {
// Now restore the signal mask
// Living in the same location
mov(x4, x0);
LoadConstant(x0, SIG_SETMASK);
add(x1, sp, 0);
LoadConstant(x2, 0);
LoadConstant(x3, 8);
LoadConstant(x8, SYS_rt_sigprocmask);
mov(ARMEmitter::XReg::x4, ARMEmitter::XReg::x0);
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r0, SIG_SETMASK);
add(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r1, ARMEmitter::Reg::rsp, 0);
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r2, 0);
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r3, 8);
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r8, SYS_rt_sigprocmask);
svc(0);
// Bring stack back
add(sp, sp, 16);
mov(x0, x4);
add(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::rsp, ARMEmitter::Reg::rsp, 16);
mov(ARMEmitter::XReg::x0, ARMEmitter::XReg::x4);
}
if (config.StaticRegisterAllocation)
FillStaticRegs();
br(x0);
br(ARMEmitter::Reg::r0);
}
// Need to create the block
{
bind(&NoBlock);
Bind(&NoBlock);
if (config.StaticRegisterAllocation)
SpillStaticRegs();
@@ -266,42 +266,42 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, const Dispatche
// X3: Size of mask, sizeof(uint64_t)
// X8: Syscall
LoadConstant(x0, ~0ULL);
stp(x0, x2, MemOperand(sp, -16, PreIndex));
LoadConstant(x0, SIG_SETMASK);
add(x1, sp, 0);
add(x2, sp, 0);
LoadConstant(x3, 8);
LoadConstant(x8, SYS_rt_sigprocmask);
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r0, ~0ULL);
stp<ARMEmitter::IndexType::PRE>(ARMEmitter::XReg::x0, ARMEmitter::XReg::x2, ARMEmitter::Reg::rsp, -16);
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r0, SIG_SETMASK);
add(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r1, ARMEmitter::Reg::rsp, 0);
add(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r2, ARMEmitter::Reg::rsp, 0);
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r3, 8);
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r8, SYS_rt_sigprocmask);
svc(0);
// Reload x2 to bring back RIP
ldr(x2, MemOperand(sp, 8, Offset));
ldr(ARMEmitter::XReg::x2, ARMEmitter::Reg::rsp, 8);
}
ldr(x0, &l_CTX);
mov(x1, STATE);
ldr(x3, &l_CompileBlock);
ldr(ARMEmitter::XReg::x0, &l_CTX);
mov(ARMEmitter::XReg::x1, STATE);
ldr(ARMEmitter::XReg::x3, &l_CompileBlock);
// X2 contains our guest RIP
#ifdef VIXL_SIMULATOR
GenerateIndirectRuntimeCall<void, void *, uint64_t, void *>(x3);
GenerateIndirectRuntimeCall<void, void *, uint64_t, void *>(ARMEmitter::Reg::r3);
#else
blr(x3); // { CTX, Frame, RIP}
blr(ARMEmitter::Reg::r3); // { CTX, Frame, RIP}
#endif
if (SignalSafeCompile) {
// Now restore the signal mask
// Living in the same location
LoadConstant(x0, SIG_SETMASK);
add(x1, sp, 0);
LoadConstant(x2, 0);
LoadConstant(x3, 8);
LoadConstant(x8, SYS_rt_sigprocmask);
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r0, SIG_SETMASK);
add(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r1, ARMEmitter::Reg::rsp, 0);
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r2, 0);
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r3, 8);
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r8, SYS_rt_sigprocmask);
svc(0);
// Bring stack back
add(sp, sp, 16);
add(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::rsp, ARMEmitter::Reg::rsp, 16);
}
if (config.StaticRegisterAllocation)
@@ -321,7 +321,7 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, const Dispatche
{
// Guest SIGILL handler
// Needs to be distinct from the SignalHandlerReturnAddress
GuestSignal_SIGILL = GetCursorAddress<uint64_t>();
GuestSignal_SIGILL = GetCursorAddress<uint64_t>();
if (config.StaticRegisterAllocation)
SpillStaticRegs();
@@ -352,8 +352,8 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, const Dispatche
// brk = SIGTRAP
// ??? = SIGSEGV
// Force a SIGSEGV by loading zero
LoadConstant(x1, 0);
ldr(x1, MemOperand(x1));
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r1, 0);
ldr(ARMEmitter::XReg::x1, ARMEmitter::Reg::r1);
}
{
@@ -361,19 +361,18 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, const Dispatche
if (config.StaticRegisterAllocation)
SpillStaticRegs();
bind(&ThreadPauseHandler);
ThreadPauseHandlerAddress = GetCursorAddress<uint64_t>();
// We are pausing, this means the frontend should be waiting for this thread to idle
// We will have faulted and jumped to this location at this point
// Call our sleep handler
ldr(x0, &l_CTX);
mov(x1, STATE);
ldr(x2, &l_Sleep);
ldr(ARMEmitter::XReg::x0, &l_CTX);
mov(ARMEmitter::XReg::x1, STATE);
ldr(ARMEmitter::XReg::x2, &l_Sleep);
#ifdef VIXL_SIMULATOR
GenerateIndirectRuntimeCall<void, void *, void *>(x2);
GenerateIndirectRuntimeCall<void, void *, void *>(ARMEmitter::Reg::r2);
#else
blr(x2);
blr(ARMEmitter::Reg::r2);
#endif
PauseReturnInstruction = GetCursorAddress<uint64_t>();
@@ -403,27 +402,27 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, const Dispatche
PushCalleeSavedRegisters();
// First thing we need to move the thread state pointer back in to our register
mov(STATE, x0);
mov(STATE, ARMEmitter::XReg::x0);
// Make sure to adjust the refcounter so we don't clear the cache now
ldr(w2, STATE_PTR(CpuStateFrame, SignalHandlerRefCounter));
add(w2, w2, 1);
str(w2, STATE_PTR(CpuStateFrame, SignalHandlerRefCounter));
ldr(ARMEmitter::WReg::w2, STATE_PTR(CpuStateFrame, SignalHandlerRefCounter));
add(ARMEmitter::Size::i32Bit, ARMEmitter::Reg::r2, ARMEmitter::Reg::r2, 1);
str(ARMEmitter::WReg::w2, STATE_PTR(CpuStateFrame, SignalHandlerRefCounter));
// Now push the callback return trampoline to the guest stack
// Guest will be misaligned because calling a thunk won't correct the guest's stack once we call the callback from the host
LoadConstant(x0, CTX->X86CodeGen.CallbackReturn);
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r0, CTX->X86CodeGen.CallbackReturn);
ldr(x2, STATE_PTR(CpuStateFrame, State.gregs[X86State::REG_RSP]));
sub(x2, x2, 16);
str(x2, STATE_PTR(CpuStateFrame, State.gregs[X86State::REG_RSP]));
ldr(ARMEmitter::XReg::x2, STATE_PTR(CpuStateFrame, State.gregs[X86State::REG_RSP]));
sub(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r2, ARMEmitter::Reg::r2, 16);
str(ARMEmitter::XReg::x2, STATE_PTR(CpuStateFrame, State.gregs[X86State::REG_RSP]));
// Store the trampoline to the guest stack
// Guest stack is now correctly misaligned after a regular call instruction
str(x0, MemOperand(x2));
str(ARMEmitter::XReg::x0, ARMEmitter::Reg::r2, 0);
// Store RIP to the context state
str(x1, STATE_PTR(CpuStateFrame, State.rip));
str(ARMEmitter::XReg::x1, STATE_PTR(CpuStateFrame, State.rip));
// load static regs
if (config.StaticRegisterAllocation)
@@ -436,14 +435,14 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, const Dispatche
{
LUDIVHandlerAddress = GetCursorAddress<uint64_t>();
PushDynamicRegsAndLR();
PushDynamicRegsAndLR(ARMEmitter::Reg::r3);
SpillStaticRegs();
ldr(x3, STATE_PTR(CpuStateFrame, Pointers.AArch64.LUDIV));
ldr(ARMEmitter::XReg::x3, STATE_PTR(CpuStateFrame, Pointers.AArch64.LUDIV));
#ifdef VIXL_SIMULATOR
GenerateIndirectRuntimeCall<uint64_t, uint64_t, uint64_t, uint64_t>(x3);
GenerateIndirectRuntimeCall<uint64_t, uint64_t, uint64_t, uint64_t>(ARMEmitter::Reg::r3);
#else
blr(x3);
blr(ARMEmitter::Reg::r3);
#endif
FillStaticRegs();
@@ -458,14 +457,14 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, const Dispatche
{
LDIVHandlerAddress = GetCursorAddress<uint64_t>();
PushDynamicRegsAndLR();
PushDynamicRegsAndLR(ARMEmitter::Reg::r3);
SpillStaticRegs();
ldr(x3, STATE_PTR(CpuStateFrame, Pointers.AArch64.LDIV));
ldr(ARMEmitter::XReg::x3, STATE_PTR(CpuStateFrame, Pointers.AArch64.LDIV));
#ifdef VIXL_SIMULATOR
GenerateIndirectRuntimeCall<uint64_t, uint64_t, uint64_t, uint64_t>(x3);
GenerateIndirectRuntimeCall<uint64_t, uint64_t, uint64_t, uint64_t>(ARMEmitter::Reg::r3);
#else
blr(x3);
blr(ARMEmitter::Reg::r3);
#endif
FillStaticRegs();
@@ -480,14 +479,14 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, const Dispatche
{
LUREMHandlerAddress = GetCursorAddress<uint64_t>();
PushDynamicRegsAndLR();
PushDynamicRegsAndLR(ARMEmitter::Reg::r3);
SpillStaticRegs();
ldr(x3, STATE_PTR(CpuStateFrame, Pointers.AArch64.LUREM));
ldr(ARMEmitter::XReg::x3, STATE_PTR(CpuStateFrame, Pointers.AArch64.LUREM));
#ifdef VIXL_SIMULATOR
GenerateIndirectRuntimeCall<uint64_t, uint64_t, uint64_t, uint64_t>(x3);
GenerateIndirectRuntimeCall<uint64_t, uint64_t, uint64_t, uint64_t>(ARMEmitter::Reg::r3);
#else
blr(x3);
blr(ARMEmitter::Reg::r3);
#endif
FillStaticRegs();
@@ -502,14 +501,15 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, const Dispatche
{
LREMHandlerAddress = GetCursorAddress<uint64_t>();
PushDynamicRegsAndLR();
PushDynamicRegsAndLR(ARMEmitter::Reg::r3);
SpillStaticRegs();
ldr(x3, STATE_PTR(CpuStateFrame, Pointers.AArch64.LREM));
ldr(ARMEmitter::XReg::x3, STATE_PTR(CpuStateFrame, Pointers.AArch64.LREM));
#ifdef VIXL_SIMULATOR
GenerateIndirectRuntimeCall<uint64_t, uint64_t, uint64_t, uint64_t>(x3);
GenerateIndirectRuntimeCall<uint64_t, uint64_t, uint64_t, uint64_t>(ARMEmitter::Reg::r3);
#else
blr(x3);
blr(ARMEmitter::Reg::r3);
#endif
FillStaticRegs();
@@ -521,16 +521,16 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, const Dispatche
ret();
}
place(&l_CTX);
place(&l_Sleep);
place(&l_CompileBlock);
Bind(&l_CTX);
dc64(reinterpret_cast<uintptr_t>(CTX));
Bind(&l_Sleep);
dc64(reinterpret_cast<uint64_t>(SleepThread));
Bind(&l_CompileBlock);
dc64(GetCompileBlockPtr());
FinalizeCode();
Start = reinterpret_cast<uint64_t>(DispatchPtr);
End = GetCursorAddress<uint64_t>();
vixl::aarch64::CPU::EnsureIAndDCacheCoherency(reinterpret_cast<void*>(DispatchPtr), End - reinterpret_cast<uint64_t>(DispatchPtr));
GetBuffer()->SetExecutable();
ClearICache(reinterpret_cast<void*>(DispatchPtr), End - reinterpret_cast<uint64_t>(DispatchPtr));
if (CTX->Config.BlockJITNaming()) {
std::string Name = "Dispatch_" + std::to_string(FHU::Syscalls::gettid());
@@ -539,106 +539,100 @@ Arm64Dispatcher::Arm64Dispatcher(FEXCore::Context::Context *ctx, const Dispatche
if (CTX->Config.GlobalJITNaming()) {
CTX->Symbols.RegisterJITSpace(reinterpret_cast<void*>(DispatchPtr), End - reinterpret_cast<uint64_t>(DispatchPtr));
}
#ifdef VIXL_DISASSEMBLER
const auto DisasmEnd = GetCursorAddress<const vixl::aarch64::Instruction*>();
Disasm.DisassembleBuffer(DisasmBegin, DisasmEnd);
#endif
}
#ifdef VIXL_SIMULATOR
void Arm64Dispatcher::ExecuteDispatch(FEXCore::Core::CpuStateFrame *Frame) {
Simulator.WriteXRegister(0, reinterpret_cast<int64_t>(Frame));
Simulator.RunFrom(reinterpret_cast<Instruction const*>(DispatchPtr));
Simulator.RunFrom(reinterpret_cast<vixl::aarch64::Instruction const*>(DispatchPtr));
}
void Arm64Dispatcher::ExecuteJITCallback(FEXCore::Core::CpuStateFrame *Frame, uint64_t RIP) {
Simulator.WriteXRegister(0, reinterpret_cast<int64_t>(Frame));
Simulator.WriteXRegister(1, RIP);
Simulator.RunFrom(reinterpret_cast<Instruction const*>(CallbackPtr));
Simulator.RunFrom(reinterpret_cast<vixl::aarch64::Instruction const*>(CallbackPtr));
}
#endif
// Used by GenerateGDBPauseCheck, GenerateInterpreterTrampoline, destination buffer is set before use
static thread_local vixl::aarch64::Assembler emit((uint8_t*)&emit, 1);
size_t Arm64Dispatcher::GenerateGDBPauseCheck(uint8_t *CodeBuffer, uint64_t GuestRIP) {
FEXCore::ARMEmitter::Emitter emit{CodeBuffer, MaxGDBPauseCheckSize};
*emit.GetBuffer() = vixl::CodeBuffer(CodeBuffer, MaxGDBPauseCheckSize);
vixl::CodeBufferCheckScope scope(&emit, MaxGDBPauseCheckSize, vixl::CodeBufferCheckScope::kDontReserveBufferSpace, vixl::CodeBufferCheckScope::kNoAssert);
aarch64::Label RunBlock;
ARMEmitter::ForwardLabel RunBlock;
// If we have a gdb server running then run in a less efficient mode that checks if we need to exit
// This happens when single stepping
static_assert(sizeof(FEXCore::Context::Context::Config.RunningMode) == 4, "This is expected to be size of 4");
emit.ldr(x0, STATE_PTR(CpuStateFrame, Thread)); // Get thread
emit.ldr(x0, MemOperand(x0, offsetof(FEXCore::Core::InternalThreadState, CTX))); // Get Context
emit.ldr(w0, MemOperand(x0, offsetof(FEXCore::Context::Context, Config.RunningMode)));
emit.ldr(ARMEmitter::XReg::x0, STATE_PTR(CpuStateFrame, Thread));
emit.ldr(ARMEmitter::XReg::x0, ARMEmitter::Reg::r0, offsetof(FEXCore::Core::InternalThreadState, CTX)); // Get Context
emit.ldr(ARMEmitter::WReg::w0, ARMEmitter::Reg::r0, offsetof(FEXCore::Context::Context, Config.RunningMode));
// If the value == 0 then we don't need to stop
emit.cbz(w0, &RunBlock);
emit.cbz(ARMEmitter::Size::i32Bit, ARMEmitter::Reg::r0, &RunBlock);
{
Literal l_GuestRIP {GuestRIP};
ARMEmitter::ForwardLabel l_GuestRIP;
// Make sure RIP is syncronized to the context
emit.ldr(x0, &l_GuestRIP);
emit.str(x0, STATE_PTR(CpuStateFrame, State.rip));
emit.ldr(ARMEmitter::XReg::x0, &l_GuestRIP);
emit.str(ARMEmitter::XReg::x0, STATE_PTR(CpuStateFrame, State.rip));
// Stop the thread
emit.ldr(x0, STATE_PTR(CpuStateFrame, Pointers.Common.ThreadPauseHandlerSpillSRA));
emit.br(x0);
emit.place(&l_GuestRIP);
emit.ldr(ARMEmitter::XReg::x0, STATE_PTR(CpuStateFrame, Pointers.Common.ThreadPauseHandlerSpillSRA));
emit.br(ARMEmitter::Reg::r0);
emit.Bind(&l_GuestRIP);
emit.dc64(GuestRIP);
}
emit.bind(&RunBlock);
emit.FinalizeCode();
emit.Bind(&RunBlock);
auto UsedBytes = emit.GetBuffer()->GetCursorOffset();
vixl::aarch64::CPU::EnsureIAndDCacheCoherency(CodeBuffer, UsedBytes);
auto UsedBytes = emit.GetCursorOffset();
emit.ClearICache(CodeBuffer, UsedBytes);
return UsedBytes;
}
size_t Arm64Dispatcher::GenerateInterpreterTrampoline(uint8_t *CodeBuffer) {
LOGMAN_THROW_AA_FMT(!config.StaticRegisterAllocation, "GenerateInterpreterTrampoline dispatcher does not support SRA");
*emit.GetBuffer() = vixl::CodeBuffer(CodeBuffer, MaxInterpreterTrampolineSize);
FEXCore::ARMEmitter::Emitter emit{CodeBuffer, MaxInterpreterTrampolineSize};
ARMEmitter::ForwardLabel InlineIRData;
vixl::CodeBufferCheckScope scope(&emit, MaxInterpreterTrampolineSize, vixl::CodeBufferCheckScope::kDontReserveBufferSpace, vixl::CodeBufferCheckScope::kNoAssert);
emit.mov(ARMEmitter::XReg::x0, STATE);
emit.adr(ARMEmitter::Reg::r1, &InlineIRData);
aarch64::Label InlineIRData;
emit.ldr(ARMEmitter::XReg::x3, STATE_PTR(CpuStateFrame, Pointers.Interpreter.FragmentExecuter));
emit.blr(ARMEmitter::Reg::r3);
emit.mov(x0, STATE);
emit.adr(x1, &InlineIRData);
emit.ldr(ARMEmitter::XReg::x0, STATE_PTR(CpuStateFrame, Pointers.Common.DispatcherLoopTop));
emit.br(ARMEmitter::Reg::r0);
emit.ldr(x3, STATE_PTR(CpuStateFrame, Pointers.Interpreter.FragmentExecuter));
emit.blr(x3);
emit.Bind(&InlineIRData);
emit.ldr(x0, STATE_PTR(CpuStateFrame, Pointers.Common.DispatcherLoopTop));
emit.br(x0);
emit.bind(&InlineIRData);
emit.FinalizeCode();
auto UsedBytes = emit.GetBuffer()->GetCursorOffset();
vixl::aarch64::CPU::EnsureIAndDCacheCoherency(CodeBuffer, UsedBytes);
auto UsedBytes = emit.GetCursorOffset();
emit.ClearICache(CodeBuffer, UsedBytes);
return UsedBytes;
}
void Arm64Dispatcher::SpillSRA(FEXCore::Core::InternalThreadState *Thread, void *ucontext, uint32_t IgnoreMask) {
for (size_t i = 0; i < SRA64.size(); i++) {
if (IgnoreMask & (1U << SRA64[i].GetCode())) {
if (IgnoreMask & (1U << SRA64[i].Idx())) {
// Skip this one, it's already spilled
continue;
}
Thread->CurrentFrame->State.gregs[i] = ArchHelpers::Context::GetArmReg(ucontext, SRA64[i].GetCode());
Thread->CurrentFrame->State.gregs[i] = ArchHelpers::Context::GetArmReg(ucontext, SRA64[i].Idx());
}
if (EmitterCTX->HostFeatures.SupportsAVX) {
for (size_t i = 0; i < SRAFPR.size(); i++) {
auto FPR = ArchHelpers::Context::GetArmFPR(ucontext, SRAFPR[i].GetCode());
auto FPR = ArchHelpers::Context::GetArmFPR(ucontext, SRAFPR[i].Idx());
memcpy(&Thread->CurrentFrame->State.xmm.avx.data[i][0], &FPR, sizeof(__uint128_t));
}
} else {
for (size_t i = 0; i < SRAFPR.size(); i++) {
auto FPR = ArchHelpers::Context::GetArmFPR(ucontext, SRAFPR[i].GetCode());
auto FPR = ArchHelpers::Context::GetArmFPR(ucontext, SRAFPR[i].Idx());
memcpy(&Thread->CurrentFrame->State.xmm.sse.data[i][0], &FPR, sizeof(__uint128_t));
}
}
@@ -15,6 +15,9 @@ namespace FEXCore::Core {
struct InternalThreadState;
}
#define STATE_PTR(STATE_TYPE, FIELD) \
STATE.R(), offsetof(FEXCore::Core::STATE_TYPE, FIELD)
namespace FEXCore::CPU {
class Arm64Dispatcher final : public Dispatcher, public Arm64Emitter {
@@ -29,6 +32,8 @@ class Arm64Dispatcher final : public Dispatcher, public Arm64Emitter {
void ExecuteJITCallback(FEXCore::Core::CpuStateFrame *Frame, uint64_t RIP) override;
#endif
void EmitDispatcher();
protected:
void SpillSRA(FEXCore::Core::InternalThreadState *Thread, void *ucontext, uint32_t IgnoreMask) override;
@@ -288,15 +288,14 @@ static uint32_t ConvertSignalToTrapNo(int Signal, siginfo_t *HostSigInfo) {
return Signal;
}
static uint32_t ConvertSignalToError(int Signal, siginfo_t *HostSigInfo) {
static uint32_t ConvertSignalToError(void *ucontext, int Signal, siginfo_t *HostSigInfo) {
switch (Signal) {
case SIGSEGV:
if (HostSigInfo->si_code == SEGV_MAPERR ||
HostSigInfo->si_code == SEGV_ACCERR) {
// Protection fault
// Always a user fault for us
// XXX: PF_PROT and PF_WRITE
return X86State::X86_PF_USER;
return ArchHelpers::Context::GetProtectFlags(ucontext);
}
break;
}
@@ -477,7 +476,7 @@ bool Dispatcher::HandleGuestSignal(FEXCore::Core::InternalThreadState *Thread, i
}
else {
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_TRAPNO] = ConvertSignalToTrapNo(Signal, HostSigInfo);
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_ERR] = ConvertSignalToError(Signal, HostSigInfo);
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_ERR] = ConvertSignalToError(ucontext, Signal, HostSigInfo);
}
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_OLDMASK] = 0;
guest_uctx->uc_mcontext.gregs[FEXCore::x86_64::FEX_REG_CR2] = 0;
@@ -589,7 +588,7 @@ bool Dispatcher::HandleGuestSignal(FEXCore::Core::InternalThreadState *Thread, i
else {
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_TRAPNO] = ConvertSignalToTrapNo(Signal, HostSigInfo);
guest_siginfo->si_code = HostSigInfo->si_code;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_ERR] = ConvertSignalToError(Signal, HostSigInfo);
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_ERR] = ConvertSignalToError(ucontext, Signal, HostSigInfo);
}
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_EIP] = Frame->State.rip;
guest_uctx->uc_mcontext.gregs[FEXCore::x86::FEX_REG_EFL] = 0;
+41 -10
View File
@@ -459,6 +459,10 @@ bool Decoder::NormalOp(FEXCore::X86Tables::X86InstInfo const *Info, uint16_t Op,
DestSize = 16;
}
}
else if (DstSizeFlag == FEXCore::X86Tables::InstFlags::SIZE_256BIT) {
DecodeInst->Flags |= DecodeFlags::GenSizeDstSize(DecodeFlags::SIZE_256BIT);
DestSize = 32;
}
else if (HasNarrowingDisplacement &&
(DstSizeFlag == FEXCore::X86Tables::InstFlags::SIZE_DEF ||
DstSizeFlag == FEXCore::X86Tables::InstFlags::SIZE_64BITDEF)) {
@@ -1127,6 +1131,35 @@ void Decoder::BranchTargetInMultiblockRange() {
}
}
bool Decoder::BranchTargetCanContinue(bool FinalInstruction) const {
if (FinalInstruction) {
return false;
}
uint64_t TargetRIP = 0;
const uint8_t GPRSize = CTX->GetGPRSize();
if (DecodeInst->OP == 0xE8) { // Call - immediate target
const uint64_t NextRIP = DecodeInst->PC + DecodeInst->InstSize;
LOGMAN_THROW_A_FMT(DecodeInst->Src[0].IsLiteral(), "Had wrong operand type");
TargetRIP = DecodeInst->PC + DecodeInst->InstSize + DecodeInst->Src[0].Data.Literal.Value;
if (GPRSize == 4) {
// If we are running a 32bit guest then wrap around addresses that go above 32bit
TargetRIP &= 0xFFFFFFFFU;
}
if (TargetRIP == NextRIP) {
// Optimize the case that the instruction is jumping just after itself.
// This is a GOT calculation which we can optimize out.
// Optimization occurs inside of the OpDispatcher implementation
return true;
}
}
return false;
}
const uint8_t *Decoder::AdjustAddrForSpecialRegion(uint8_t const* _InstStream, uint64_t EntryPoint, uint64_t RIP) {
constexpr uint64_t VSyscall_Base = 0xFFFF'FFFF'FF60'0000ULL;
constexpr uint64_t VSyscall_End = VSyscall_Base + 0x1000;
@@ -1251,23 +1284,21 @@ void Decoder::DecodeInstructionsAtEntry(uint8_t const* _InstStream, uint64_t PC,
CanContinue = true;
}
bool FinalInstruction = DecodedSize >= CTX->Config.MaxInstPerBlock ||
DecodedSize >= DefaultDecodedBufferSize ||
TotalInstructions >= CTX->Config.MaxInstPerBlock;
if (DecodeInst->TableInfo->Flags & FEXCore::X86Tables::InstFlags::FLAGS_SETS_RIP) {
// If we have multiblock enabled
// If the branch target is within our multiblock range then we can keep going on
// We don't want to short circuit this since we want to calculate our ranges still
BranchTargetInMultiblockRange();
// Bypass branches if we can continue through them in some cases.
CanContinue |= BranchTargetCanContinue(FinalInstruction);
}
if (!CanContinue) {
break;
}
if (DecodedSize >= CTX->Config.MaxInstPerBlock ||
DecodedSize >= DefaultDecodedBufferSize) {
break;
}
if (TotalInstructions >= CTX->Config.MaxInstPerBlock) {
if (FinalInstruction || !CanContinue) {
break;
}
+1
View File
@@ -58,6 +58,7 @@ private:
bool DecodeInstruction(uint64_t PC);
void BranchTargetInMultiblockRange();
bool BranchTargetCanContinue(bool FinalInstruction) const;
uint8_t ReadByte();
uint8_t PeekByte(uint8_t Offset) const;
@@ -195,7 +195,7 @@ DEF_OP(Vector_FToF) {
// Sometimes is used to convert from a 128bit vector register
// in to a 64bit vector register with different sized elements
// eg: %ssa5 i32v2 = Vector_FToF %ssa4 i128, #0x8
uint8_t Elements = (OpSize << 1) / Op->SrcElementSize;
uint8_t Elements = OpSize == 8 ? 2 : OpSize / Op->SrcElementSize;
DO_VECTOR_1SRC_2TYPE_OP_NOSIZE(float, double, Func, 0, 0)
break;
}
@@ -1549,6 +1549,12 @@ DEF_OP(VInsElement) {
Dst_d[Op->DestIdx] = Src2_d[Op->SrcIdx];
break;
}
case 16: {
auto *Dst_d = reinterpret_cast<__uint128_t*>(Tmp);
auto *Src2_d = reinterpret_cast<__uint128_t*>(Src2);
Dst_d[Op->DestIdx] = Src2_d[Op->SrcIdx];
break;
}
default:
LOGMAN_MSG_A_FMT("Unknown Element Size: {}", ElementSize);
break;
@@ -1651,8 +1657,9 @@ DEF_OP(VExtr) {
const auto* VectorsPtr = reinterpret_cast<const uint8_t*>(Vectors.data());
const auto* SrcPtr = VectorsPtr + SanitizedByteIndex;
const auto CopyAmount = std::max(0, int(sizeof(Vectors) - SanitizedByteIndex));
memcpy(GDP, SrcPtr, OpSize);
memcpy(GDP, SrcPtr, CopyAmount);
} else {
uint64_t Offset = Index * ElementSize * 8;
File diff suppressed because it is too large. Load diff
@@ -9,7 +9,7 @@ $end_info$
#include "Interface/HLE/Thunks/Thunks.h"
namespace FEXCore::CPU {
uint64_t Arm64JITCore::GetNamedSymbolLiteral(FEXCore::CPU::RelocNamedSymbolLiteral::NamedSymbol Op) {
switch (Op) {
case FEXCore::CPU::RelocNamedSymbolLiteral::NamedSymbol::SYMBOL_LITERAL_EXITFUNCTION_LINKER:
@@ -22,18 +22,18 @@ uint64_t Arm64JITCore::GetNamedSymbolLiteral(FEXCore::CPU::RelocNamedSymbolLiter
return ~0ULL;
}
void Arm64JITCore::InsertNamedThunkRelocation(vixl::aarch64::Register Reg, const IR::SHA256Sum &Sum) {
void Arm64JITCore::InsertNamedThunkRelocation(ARMEmitter::Register Reg, const IR::SHA256Sum &Sum) {
Relocation MoveABI{};
MoveABI.NamedThunkMove.Header.Type = FEXCore::CPU::RelocationTypes::RELOC_NAMED_THUNK_MOVE;
// Offset is the offset from the entrypoint of the block
auto CurrentCursor = GetCursorAddress<uint8_t *>();
MoveABI.NamedThunkMove.Offset = CurrentCursor - GuestEntry;
MoveABI.NamedThunkMove.Symbol = Sum;
MoveABI.NamedThunkMove.RegisterIndex = Reg.GetCode();
MoveABI.NamedThunkMove.RegisterIndex = Reg.Idx();
uint64_t Pointer = reinterpret_cast<uint64_t>(EmitterCTX->ThunkHandler->LookupThunk(Sum));
LoadConstant(Reg, Pointer, EmitterCTX->Config.CacheObjectCodeCompilation());
LoadConstant(ARMEmitter::Size::i64Bit, Reg, Pointer, EmitterCTX->Config.CacheObjectCodeCompilation());
Relocations.emplace_back(MoveABI);
}
@@ -41,7 +41,7 @@ Arm64JITCore::NamedSymbolLiteralPair Arm64JITCore::InsertNamedSymbolLiteral(FEXC
uint64_t Pointer = GetNamedSymbolLiteral(Op);
Arm64JITCore::NamedSymbolLiteralPair Lit {
.Lit = Literal(Pointer),
.Lit = Pointer,
.MoveABI = {
.NamedSymbolLiteral = {
.Header = {
@@ -60,20 +60,21 @@ void Arm64JITCore::PlaceNamedSymbolLiteral(NamedSymbolLiteralPair &Lit) {
auto CurrentCursor = GetCursorAddress<uint8_t *>();
Lit.MoveABI.NamedSymbolLiteral.Offset = CurrentCursor - GuestEntry;
place(&Lit.Lit);
Bind(&Lit.Loc);
dc64(Lit.Lit);
Relocations.emplace_back(Lit.MoveABI);
}
void Arm64JITCore::InsertGuestRIPMove(vixl::aarch64::Register Reg, uint64_t Constant) {
void Arm64JITCore::InsertGuestRIPMove(ARMEmitter::Register Reg, uint64_t Constant) {
Relocation MoveABI{};
MoveABI.GuestRIPMove.Header.Type = FEXCore::CPU::RelocationTypes::RELOC_GUEST_RIP_MOVE;
// Offset is the offset from the entrypoint of the block
auto CurrentCursor = GetCursorAddress<uint8_t *>();
MoveABI.GuestRIPMove.Offset = CurrentCursor - GuestEntry;
MoveABI.GuestRIPMove.GuestRIP = Constant;
MoveABI.GuestRIPMove.RegisterIndex = Reg.GetCode();
MoveABI.GuestRIPMove.RegisterIndex = Reg.Idx();
LoadConstant(Reg, Constant, EmitterCTX->Config.CacheObjectCodeCompilation());
LoadConstant(ARMEmitter::Size::i64Bit, Reg, Constant, EmitterCTX->Config.CacheObjectCodeCompilation());
Relocations.emplace_back(MoveABI);
}
@@ -87,11 +88,10 @@ bool Arm64JITCore::ApplyRelocations(uint64_t GuestEntry, uint64_t CodeEntry, uin
case FEXCore::CPU::RelocationTypes::RELOC_NAMED_SYMBOL_LITERAL: {
uint64_t Pointer = GetNamedSymbolLiteral(Reloc->NamedSymbolLiteral.Symbol);
// Relocation occurs at the cursorEntry + offset relative to that cursor
GetBuffer()->SetCursorOffset(CursorEntry + Reloc->NamedSymbolLiteral.Offset);
SetCursorOffset(CursorEntry + Reloc->NamedSymbolLiteral.Offset);
// Generate a literal so we can place it
Literal<uint64_t> Lit(Pointer);
place(&Lit);
dc64(Pointer);
DataIndex += sizeof(Reloc->NamedSymbolLiteral);
break;
@@ -103,8 +103,8 @@ bool Arm64JITCore::ApplyRelocations(uint64_t GuestEntry, uint64_t CodeEntry, uin
}
// Relocation occurs at the cursorEntry + offset relative to that cursor.
GetBuffer()->SetCursorOffset(CursorEntry + Reloc->NamedThunkMove.Offset);
LoadConstant(vixl::aarch64::XRegister(Reloc->NamedThunkMove.RegisterIndex), Pointer, true);
SetCursorOffset(CursorEntry + Reloc->NamedThunkMove.Offset);
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Register(Reloc->NamedThunkMove.RegisterIndex), Pointer, true);
DataIndex += sizeof(Reloc->NamedThunkMove);
break;
}
@@ -117,8 +117,8 @@ bool Arm64JITCore::ApplyRelocations(uint64_t GuestEntry, uint64_t CodeEntry, uin
}
// Relocation occurs at the cursorEntry + offset relative to that cursor.
GetBuffer()->SetCursorOffset(CursorEntry + Reloc->GuestRIPMove.Offset);
LoadConstant(vixl::aarch64::XRegister(Reloc->GuestRIPMove.RegisterIndex), Pointer, true);
SetCursorOffset(CursorEntry + Reloc->GuestRIPMove.Offset);
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Register(Reloc->GuestRIPMove.RegisterIndex), Pointer, true);
DataIndex += sizeof(Reloc->GuestRIPMove);
break;
}
File diff suppressed because it is too large. Load diff
+142 -187
View File
@@ -6,6 +6,7 @@ $end_info$
#include "Interface/Context/Context.h"
#include "FEXCore/IR/IR.h"
#include "Interface/Core/ArchHelpers/CodeEmitter/Emitter.h"
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/JIT/Arm64/JITClass.h"
@@ -17,8 +18,6 @@ $end_info$
#include <Interface/HLE/Thunks/Thunks.h>
namespace FEXCore::CPU {
using namespace vixl;
using namespace vixl::aarch64;
#define DEF_OP(x) void Arm64JITCore::Op_##x(IR::IROp_Header const *IROp, IR::NodeID Node)
DEF_OP(SignalReturn) {
@@ -27,12 +26,11 @@ DEF_OP(SignalReturn) {
// Now branch to our signal return helper
// This can't be a direct branch since the code needs to live at a constant location
ldr(x0, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.SignalReturnHandler)));
br(x0);
ldr(ARMEmitter::XReg::x0, STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.SignalReturnHandler));
br(ARMEmitter::Reg::r0);
}
DEF_OP(CallbackReturn) {
// spill back to CTX
SpillStaticRegs();
@@ -41,14 +39,14 @@ DEF_OP(CallbackReturn) {
// We can now lower the ref counter again
ldr(w2, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, SignalHandlerRefCounter)));
sub(w2, w2, 1);
str(w2, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, SignalHandlerRefCounter)));
ldr(ARMEmitter::WReg::w2, STATE, offsetof(FEXCore::Core::CpuStateFrame, SignalHandlerRefCounter));
sub(ARMEmitter::Size::i32Bit, ARMEmitter::Reg::r2, ARMEmitter::Reg::r2, 1);
str(ARMEmitter::WReg::w2, STATE, offsetof(FEXCore::Core::CpuStateFrame, SignalHandlerRefCounter));
// We need to adjust an additional 8 bytes to get back to the original "misaligned" RSP state
ldr(x2, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[X86State::REG_RSP])));
add(x2, x2, 8);
str(x2, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[X86State::REG_RSP])));
ldr(ARMEmitter::XReg::x2, STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[X86State::REG_RSP]));
add(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r2, ARMEmitter::Reg::r2, 8);
str(ARMEmitter::XReg::x2, STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[X86State::REG_RSP]));
PopCalleeSavedRegisters();
@@ -59,39 +57,41 @@ DEF_OP(CallbackReturn) {
DEF_OP(ExitFunction) {
auto Op = IROp->C<IR::IROp_ExitFunction>();
Label FullLookup;
ResetStack();
aarch64::Register RipReg;
uint64_t NewRIP;
if (IsInlineConstant(Op->NewRIP, &NewRIP) || IsInlineEntrypointOffset(Op->NewRIP, &NewRIP)) {
Literal l_BranchHost{ThreadState->CurrentFrame->Pointers.Common.ExitFunctionLinker};
Literal l_BranchGuest{NewRIP};
ARMEmitter::ForwardLabel l_BranchHost;
ARMEmitter::ForwardLabel l_BranchGuest;
ldr(x0, &l_BranchHost);
blr(x0);
ldr(ARMEmitter::XReg::x0, &l_BranchHost);
blr(ARMEmitter::Reg::r0);
Bind(&l_BranchHost);
dc64(ThreadState->CurrentFrame->Pointers.Common.ExitFunctionLinker);
Bind(&l_BranchGuest);
dc64(NewRIP);
place(&l_BranchHost);
place(&l_BranchGuest);
} else {
RipReg = GetReg<RA_64>(Op->NewRIP.ID());
ARMEmitter::ForwardLabel FullLookup;
auto RipReg = GetReg(Op->NewRIP.ID());
// L1 Cache
ldr(x0, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.L1Pointer)));
ldr(ARMEmitter::XReg::x0, STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.L1Pointer));
and_(x3, RipReg, LookupCache::L1_ENTRIES_MASK);
add(x0, x0, Operand(x3, Shift::LSL, 4));
and_(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r3, RipReg, LookupCache::L1_ENTRIES_MASK);
add(ARMEmitter::XReg::x0, ARMEmitter::XReg::x0, ARMEmitter::XReg::x3, ARMEmitter::ShiftType::LSL, 4);
ldp(x1, x0, MemOperand(x0));
cmp(x0, RipReg);
b(&FullLookup, Condition::ne);
br(x1);
ldp<ARMEmitter::IndexType::OFFSET>(ARMEmitter::XReg::x1, ARMEmitter::XReg::x0, ARMEmitter::Reg::r0, 0);
cmp(ARMEmitter::XReg::x0, RipReg.X());
b(ARMEmitter::Condition::CC_NE, &FullLookup);
br(ARMEmitter::Reg::r1);
bind(&FullLookup);
ldr(TMP1, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.DispatcherLoopTop)));
str(RipReg, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.rip)));
Bind(&FullLookup);
ldr(TMP1, STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.DispatcherLoopTop));
str(RipReg.X(), STATE, offsetof(FEXCore::Core::CpuStateFrame, State.rip));
br(TMP1);
}
}
@@ -103,66 +103,65 @@ DEF_OP(Jump) {
PendingTargetLabel = &JumpTargets.try_emplace(Target).first->second;
}
#define GRCMP(Node) (Op->CompareSize == 4 ? GetReg<RA_32>(Node) : GetReg<RA_64>(Node))
#define GRFCMP(Node) (Op->CompareSize == 4 ? GetDst(Node).S() : GetDst(Node).D())
static Condition MapBranchCC(IR::CondClassType Cond) {
static ARMEmitter::Condition MapBranchCC(IR::CondClassType Cond) {
switch (Cond.Val) {
case FEXCore::IR::COND_EQ: return Condition::eq;
case FEXCore::IR::COND_NEQ: return Condition::ne;
case FEXCore::IR::COND_SGE: return Condition::ge;
case FEXCore::IR::COND_SLT: return Condition::lt;
case FEXCore::IR::COND_SGT: return Condition::gt;
case FEXCore::IR::COND_SLE: return Condition::le;
case FEXCore::IR::COND_UGE: return Condition::cs;
case FEXCore::IR::COND_ULT: return Condition::cc;
case FEXCore::IR::COND_UGT: return Condition::hi;
case FEXCore::IR::COND_ULE: return Condition::ls;
case FEXCore::IR::COND_FLU: return Condition::lt;
case FEXCore::IR::COND_FGE: return Condition::ge;
case FEXCore::IR::COND_FLEU:return Condition::le;
case FEXCore::IR::COND_FGT: return Condition::gt;
case FEXCore::IR::COND_FU: return Condition::vs;
case FEXCore::IR::COND_FNU: return Condition::vc;
case FEXCore::IR::COND_EQ: return ARMEmitter::Condition::CC_EQ;
case FEXCore::IR::COND_NEQ: return ARMEmitter::Condition::CC_NE;
case FEXCore::IR::COND_SGE: return ARMEmitter::Condition::CC_GE;
case FEXCore::IR::COND_SLT: return ARMEmitter::Condition::CC_LT;
case FEXCore::IR::COND_SGT: return ARMEmitter::Condition::CC_GT;
case FEXCore::IR::COND_SLE: return ARMEmitter::Condition::CC_LE;
case FEXCore::IR::COND_UGE: return ARMEmitter::Condition::CC_CS;
case FEXCore::IR::COND_ULT: return ARMEmitter::Condition::CC_CC;
case FEXCore::IR::COND_UGT: return ARMEmitter::Condition::CC_HI;
case FEXCore::IR::COND_ULE: return ARMEmitter::Condition::CC_LS;
case FEXCore::IR::COND_FLU: return ARMEmitter::Condition::CC_LT;
case FEXCore::IR::COND_FGE: return ARMEmitter::Condition::CC_GE;
case FEXCore::IR::COND_FLEU:return ARMEmitter::Condition::CC_LE;
case FEXCore::IR::COND_FGT: return ARMEmitter::Condition::CC_GT;
case FEXCore::IR::COND_FU: return ARMEmitter::Condition::CC_VS;
case FEXCore::IR::COND_FNU: return ARMEmitter::Condition::CC_VC;
case FEXCore::IR::COND_VS:
case FEXCore::IR::COND_VC:
case FEXCore::IR::COND_MI:
case FEXCore::IR::COND_PL:
default:
LOGMAN_MSG_A_FMT("Unsupported compare type");
return Condition::nv;
return ARMEmitter::Condition::CC_NV;
}
}
DEF_OP(CondJump) {
auto Op = IROp->C<IR::IROp_CondJump>();
Label *TrueTargetLabel = &JumpTargets.try_emplace(Op->TrueBlock.ID()).first->second;
auto TrueTargetLabel = &JumpTargets.try_emplace(Op->TrueBlock.ID()).first->second;
uint64_t Const;
const bool isConst = IsInlineConstant(Op->Cmp2, &Const);
const auto Size = Op->CompareSize == 4 ? ARMEmitter::Size::i32Bit : ARMEmitter::Size::i64Bit;
const auto SubSize = ARMEmitter::ToVectorSizePair(Op->CompareSize == 4 ? ARMEmitter::SubRegSize::i32Bit : ARMEmitter::SubRegSize::i64Bit);
if (isConst && Const == 0 && Op->Cond.Val == FEXCore::IR::COND_EQ) {
LOGMAN_THROW_A_FMT(IsGPR(Op->Cmp1.ID()), "CondJump: Expected GPR");
cbz(GRCMP(Op->Cmp1.ID()), TrueTargetLabel);
cbz(Size, GetReg(Op->Cmp1.ID()), TrueTargetLabel);
} else if (isConst && Const == 0 && Op->Cond.Val == FEXCore::IR::COND_NEQ) {
LOGMAN_THROW_A_FMT(IsGPR(Op->Cmp1.ID()), "CondJump: Expected GPR");
cbnz(GRCMP(Op->Cmp1.ID()), TrueTargetLabel);
cbnz(Size, GetReg(Op->Cmp1.ID()), TrueTargetLabel);
} else {
if (IsGPR(Op->Cmp1.ID())) {
if (isConst) {
cmp(GRCMP(Op->Cmp1.ID()), Const);
cmp(Size, GetReg(Op->Cmp1.ID()), Const);
} else {
cmp(GRCMP(Op->Cmp1.ID()), GRCMP(Op->Cmp2.ID()));
cmp(Size, GetReg(Op->Cmp1.ID()), GetReg(Op->Cmp2.ID()));
}
} else if (IsFPR(Op->Cmp1.ID())) {
fcmp(GRFCMP(Op->Cmp1.ID()), GRFCMP(Op->Cmp2.ID()));
fcmp(SubSize.Scalar, GetVReg(Op->Cmp1.ID()), GetVReg(Op->Cmp2.ID()));
} else {
LOGMAN_MSG_A_FMT("CondJump: Expected GPR or FPR");
}
b(TrueTargetLabel, MapBranchCC(Op->Cond));
b(MapBranchCC(Op->Cond), TrueTargetLabel);
}
PendingTargetLabel = &JumpTargets.try_emplace(Op->FalseBlock.ID()).first->second;
@@ -176,7 +175,7 @@ DEF_OP(Syscall) {
// X2: Pointer to SyscallArguments
FEXCore::IR::SyscallFlags Flags = Op->Flags;
PushDynamicRegsAndLR();
PushDynamicRegsAndLR(TMP1);
if ((Flags & FEXCore::IR::SyscallFlags::NOSYNCSTATEONENTRY) != FEXCore::IR::SyscallFlags::NOSYNCSTATEONENTRY) {
SpillStaticRegs();
@@ -187,23 +186,25 @@ DEF_OP(Syscall) {
}
uint64_t SPOffset = AlignUp(FEXCore::HLE::SyscallArguments::MAX_ARGS * 8, 16);
sub(sp, sp, SPOffset);
sub(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::rsp, ARMEmitter::Reg::rsp, SPOffset);
for (uint32_t i = 0; i < FEXCore::HLE::SyscallArguments::MAX_ARGS; ++i) {
if (Op->Header.Args[i].IsInvalid()) continue;
str(GetReg<RA_64>(Op->Header.Args[i].ID()), MemOperand(sp, i * 8));
str(GetReg(Op->Header.Args[i].ID()).X(), ARMEmitter::Reg::rsp, i * 8);
}
ldr(x0, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.SyscallHandlerObj)));
ldr(x3, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.SyscallHandlerFunc)));
mov(x1, STATE);
mov(x2, sp);
ldr(ARMEmitter::XReg::x0, STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.SyscallHandlerObj));
ldr(ARMEmitter::XReg::x3, STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.SyscallHandlerFunc));
mov(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r1, STATE.R());
// SP supporting move
add(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r2, ARMEmitter::Reg::rsp, 0);
#ifdef VIXL_SIMULATOR
GenerateIndirectRuntimeCall<uint64_t, void*, void*, void*>(x3);
GenerateIndirectRuntimeCall<uint64_t, void*, void*, void*>(ARMEmitter::Reg::r3);
#else
blr(x3);
blr(ARMEmitter::Reg::r3);
#endif
add(sp, sp, SPOffset);
add(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::rsp, ARMEmitter::Reg::rsp, SPOffset);
if ((Flags & FEXCore::IR::SyscallFlags::NOSYNCSTATEONENTRY) != FEXCore::IR::SyscallFlags::NOSYNCSTATEONENTRY &&
(Flags & FEXCore::IR::SyscallFlags::NORETURN) != FEXCore::IR::SyscallFlags::NORETURN) {
@@ -219,7 +220,7 @@ DEF_OP(Syscall) {
if ((Flags & FEXCore::IR::SyscallFlags::NORETURN) != FEXCore::IR::SyscallFlags::NORETURN) {
// Move result to its destination register
mov(GetReg<RA_64>(Node), x0);
mov(ARMEmitter::Size::i64Bit, GetReg(Node), ARMEmitter::Reg::r0);
}
}
@@ -236,8 +237,8 @@ DEF_OP(InlineSyscall) {
// X6: Arg6 - Doesn't exist in x86-64 land. RA INTERSECT
// One argument is removed from the SyscallArguments::MAX_ARGS since the first argument was syscall number
const static std::array<vixl::aarch64::Register, FEXCore::HLE::SyscallArguments::MAX_ARGS-1> RegArgs = {{
x0, x1, x2, x3, x4, x5
const static std::array<ARMEmitter::XRegister, FEXCore::HLE::SyscallArguments::MAX_ARGS-1> RegArgs = {{
ARMEmitter::XReg::x0, ARMEmitter::XReg::x1, ARMEmitter::XReg::x2, ARMEmitter::XReg::x3, ARMEmitter::XReg::x4, ARMEmitter::XReg::x5
}};
bool Intersects{};
@@ -246,19 +247,15 @@ DEF_OP(InlineSyscall) {
for (uint32_t i = 0; i < FEXCore::HLE::SyscallArguments::MAX_ARGS-1; ++i) {
if (Op->Header.Args[i].IsInvalid()) break;
auto Reg = GetReg<RA_64>(Op->Header.Args[i].ID());
if (Reg.GetCode() == x8.GetCode() ||
Reg.GetCode() == x4.GetCode() ||
Reg.GetCode() == x5.GetCode()) {
auto Reg = GetReg(Op->Header.Args[i].ID());
if (Reg.Idx() == ARMEmitter::Reg::r8.Idx() ||
Reg.Idx() == ARMEmitter::Reg::r4.Idx() ||
Reg.Idx() == ARMEmitter::Reg::r5.Idx()) {
SpillMask |= (1U << Reg.GetCode());
SpillMask |= (1U << Reg.Idx());
Intersects = true;
}
}
// XXX: For some reason spilling only the x4, x5, and x8 registers was causing issues
// Come back to this once investigation reveals why it fails the gvisor ioctl test
// For now override to all GPRs
SpillMask = ~0U;
// Ordering is incredibly important here
// We must spill any overlapping registers first THEN claim we are in a syscall without invalidating state at all
@@ -270,66 +267,31 @@ DEF_OP(InlineSyscall) {
// 16bit LoadConstant to be a single instruction
// We must always spill at least one register (x8) so this value always has a bit set
// This gives the signal handler a value to check to see if we are in a syscall at all
LoadConstant(x0, SpillMask & 0xFFFF);
str(x0, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, InSyscallInfo)));
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r0, SpillMask & 0xFFFF);
str(ARMEmitter::XReg::x0, STATE, offsetof(FEXCore::Core::CpuStateFrame, InSyscallInfo));
// Now that we have claimed to be a syscall we can set up the arguments
const auto EmitSize = CTX->Config.Is64BitMode() ? ARMEmitter::Size::i64Bit : ARMEmitter::Size::i32Bit;
const auto EmitSubSize = CTX->Config.Is64BitMode() ? ARMEmitter::SubRegSize::i64Bit : ARMEmitter::SubRegSize::i32Bit;
if (Intersects) {
for (uint32_t i = 0; i < FEXCore::HLE::SyscallArguments::MAX_ARGS-1; ++i) {
if (Op->Header.Args[i].IsInvalid()) break;
if (CTX->Config.Is64BitMode()) {
auto Reg = GetReg<RA_64>(Op->Header.Args[i].ID());
// In the case of intersection with x4, x5, or x8 then these are currently SRA
// for registers RAX, RBX, and RSI. Which have just been spilled
// Just load back from the context. Could be slightly smarter but this is fairly uncommon
if (Reg.GetCode() == x8.GetCode()) {
ldr(RegArgs[i], MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[X86State::REG_RSI])));
}
else if (Reg.GetCode() == x4.GetCode()) {
ldr(RegArgs[i], MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[X86State::REG_RAX])));
}
else if (Reg.GetCode() == x5.GetCode()) {
ldr(RegArgs[i], MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[X86State::REG_RBX])));
}
auto Reg = GetReg(Op->Header.Args[i].ID());
// In the case of intersection with x4, x5, or x8 then these are currently SRA
// for registers RAX, RBX, and RSI. Which have just been spilled
// Just load back from the context. Could be slightly smarter but this is fairly uncommon
if (Reg.Idx() == FEXCore::ARMEmitter::Reg::r8.Idx()) {
ldr(EmitSubSize, RegArgs[i].R(), STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[X86State::REG_RSI]));
}
}
for (uint32_t i = 0; i < FEXCore::HLE::SyscallArguments::MAX_ARGS-1; ++i) {
if (Op->Header.Args[i].IsInvalid()) break;
if (CTX->Config.Is64BitMode()) {
auto Reg = GetReg<RA_64>(Op->Header.Args[i].ID());
// In the case of intersection with x4, x5, or x8 then these are currently SRA
// for registers RAX, RBX, and RSI. Which have just been spilled
// Just load back from the context. Could be slightly smarter but this is fairly uncommon
if (Reg.GetCode() == x8.GetCode()) {
ldr(RegArgs[i], MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[X86State::REG_RSI])));
}
else if (Reg.GetCode() == x4.GetCode()) {
ldr(RegArgs[i], MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[X86State::REG_RAX])));
}
else if (Reg.GetCode() == x5.GetCode()) {
ldr(RegArgs[i], MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[X86State::REG_RBX])));
}
else {
mov(RegArgs[i], Reg);
}
else if (Reg.Idx() == FEXCore::ARMEmitter::Reg::r4.Idx()) {
ldr(EmitSubSize, RegArgs[i].R(), STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[X86State::REG_RAX]));
}
else if (Reg.Idx() == FEXCore::ARMEmitter::Reg::r5.Idx()) {
ldr(EmitSubSize, RegArgs[i].R(), STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[X86State::REG_RBX]));
}
else {
auto Reg = GetReg<RA_32>(Op->Header.Args[i].ID());
if (Reg.GetCode() == w8.GetCode()) {
ldr(RegArgs[i].W(), MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[X86State::REG_RSI])));
}
else if (Reg.GetCode() == w4.GetCode()) {
ldr(RegArgs[i].W(), MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[X86State::REG_RAX])));
}
else if (Reg.GetCode() == w5.GetCode()) {
ldr(RegArgs[i].W(), MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[X86State::REG_RBX])));
}
else {
uxtw(RegArgs[i].W(), Reg);
}
mov(EmitSize, RegArgs[i].R(), Reg);
}
}
}
@@ -337,16 +299,11 @@ DEF_OP(InlineSyscall) {
for (uint32_t i = 0; i < FEXCore::HLE::SyscallArguments::MAX_ARGS-1; ++i) {
if (Op->Header.Args[i].IsInvalid()) break;
if (CTX->Config.Is64BitMode()) {
mov(RegArgs[i], GetReg<RA_64>(Op->Header.Args[i].ID()));
}
else {
uxtw(RegArgs[i], GetReg<RA_64>(Op->Header.Args[i].ID()));
}
mov(EmitSize, RegArgs[i].R(), GetReg(Op->Header.Args[i].ID()));
}
}
LoadConstant(x8, Op->HostSyscallNumber);
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r8, Op->HostSyscallNumber);
svc(0);
// On updated signal mask we can receive a signal RIGHT HERE
@@ -357,16 +314,11 @@ DEF_OP(InlineSyscall) {
// Now the registers we've spilled are back in their original host registers
// We can safely claim we are no longer in a syscall
str(xzr, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, InSyscallInfo)));
str(ARMEmitter::XReg::zr, STATE, offsetof(FEXCore::Core::CpuStateFrame, InSyscallInfo));
// Result is now in x0
// Move result to its destination register
if (CTX->Config.Is64BitMode()) {
mov(GetReg<RA_64>(Node), x0);
}
else {
uxtw(GetReg<RA_64>(Node), x0);
}
mov(EmitSize, GetReg(Node), ARMEmitter::Reg::r0);
}
}
@@ -378,16 +330,16 @@ DEF_OP(Thunk) {
SpillStaticRegs(); // spill to ctx before ra64 spill
PushDynamicRegsAndLR();
PushDynamicRegsAndLR(TMP1);
mov(x0, GetReg<RA_64>(Op->ArgPtr.ID()));
mov(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r0, GetReg(Op->ArgPtr.ID()));
auto thunkFn = ThreadState->CTX->ThunkHandler->LookupThunk(Op->ThunkNameHash);
LoadConstant(x2, (uintptr_t)thunkFn);
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r2, (uintptr_t)thunkFn);
#ifdef VIXL_SIMULATOR
GenerateIndirectRuntimeCall<void, void*, void*>(x2);
GenerateIndirectRuntimeCall<void, void*, void*>(ARMEmitter::Reg::r2);
#else
blr(x2);
blr(ARMEmitter::Reg::r2);
#endif
PopDynamicRegsAndLR();
@@ -401,43 +353,45 @@ DEF_OP(ValidateCode) {
int len = Op->CodeLength;
int idx = 0;
LoadConstant(GetReg<RA_64>(Node), 0);
LoadConstant(x0, Entry + Op->Offset);
LoadConstant(x1, 1);
LoadConstant(ARMEmitter::Size::i64Bit, GetReg(Node), 0);
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r0, Entry + Op->Offset);
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r1, 1);
const auto Dst = GetReg(Node);
while (len >= 8)
{
ldr(x2, MemOperand(x0, idx));
LoadConstant(x3, *(const uint32_t *)(OldCode + idx));
cmp(x2, x3);
csel(GetReg<RA_64>(Node), GetReg<RA_64>(Node), x1, Condition::eq);
ldr(ARMEmitter::XReg::x2, ARMEmitter::Reg::r0, idx);
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r3, *(const uint32_t *)(OldCode + idx));
cmp(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r2, ARMEmitter::Reg::r3);
csel(ARMEmitter::Size::i64Bit, Dst, Dst, ARMEmitter::Reg::r1, ARMEmitter::Condition::CC_EQ);
len -= 8;
idx += 8;
}
while (len >= 4)
{
ldr(w2, MemOperand(x0, idx));
LoadConstant(w3, *(const uint32_t *)(OldCode + idx));
cmp(w2, w3);
csel(GetReg<RA_64>(Node), GetReg<RA_64>(Node), x1, Condition::eq);
ldr(ARMEmitter::WReg::w2, ARMEmitter::Reg::r0, idx);
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r3, *(const uint32_t *)(OldCode + idx));
cmp(ARMEmitter::Size::i32Bit, ARMEmitter::Reg::r2, ARMEmitter::Reg::r3);
csel(ARMEmitter::Size::i64Bit, Dst, Dst, ARMEmitter::Reg::r1, ARMEmitter::Condition::CC_EQ);
len -= 4;
idx += 4;
}
while (len >= 2)
{
ldrh(w2, MemOperand(x0, idx));
LoadConstant(w3, *(const uint16_t *)(OldCode + idx));
cmp(w2, w3);
csel(GetReg<RA_64>(Node), GetReg<RA_64>(Node), x1, Condition::eq);
ldrh(ARMEmitter::Reg::r2, ARMEmitter::Reg::r0, idx);
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r3, *(const uint16_t *)(OldCode + idx));
cmp(ARMEmitter::Size::i32Bit, ARMEmitter::Reg::r2, ARMEmitter::Reg::r3);
csel(ARMEmitter::Size::i64Bit, Dst, Dst, ARMEmitter::Reg::r1, ARMEmitter::Condition::CC_EQ);
len -= 2;
idx += 2;
}
while (len >= 1)
{
ldrb(w2, MemOperand(x0, idx));
LoadConstant(w3, *(const uint8_t *)(OldCode + idx));
cmp(w2, w3);
csel(GetReg<RA_64>(Node), GetReg<RA_64>(Node), x1, Condition::eq);
ldrb(ARMEmitter::Reg::r2, ARMEmitter::Reg::r0, idx);
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r3, *(const uint8_t *)(OldCode + idx));
cmp(ARMEmitter::Size::i32Bit, ARMEmitter::Reg::r2, ARMEmitter::Reg::r3);
csel(ARMEmitter::Size::i64Bit, Dst, Dst, ARMEmitter::Reg::r1, ARMEmitter::Condition::CC_EQ);
len -= 1;
idx += 1;
}
@@ -448,17 +402,17 @@ DEF_OP(ThreadRemoveCodeEntry) {
// X0: Thread
// X1: RIP
PushDynamicRegsAndLR();
PushDynamicRegsAndLR(TMP1);
mov(x0, STATE);
LoadConstant(x1, Entry);
mov(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r0, STATE.R());
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r1, Entry);
ldr(x2, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.ThreadRemoveCodeEntryFromJIT)));
ldr(ARMEmitter::XReg::x2, STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.ThreadRemoveCodeEntryFromJIT));
SpillStaticRegs();
#ifdef VIXL_SIMULATOR
GenerateIndirectRuntimeCall<void, void*, void*>(x2);
GenerateIndirectRuntimeCall<void, void*, void*>(ARMEmitter::Reg::r2);
#else
blr(x2);
blr(ARMEmitter::Reg::r2);
#endif
FillStaticRegs();
@@ -469,30 +423,31 @@ DEF_OP(ThreadRemoveCodeEntry) {
DEF_OP(CPUID) {
auto Op = IROp->C<IR::IROp_CPUID>();
PushDynamicRegsAndLR();
PushDynamicRegsAndLR(TMP1);
SpillStaticRegs();
// x0 = CPUID Handler
// x1 = CPUID Function
// x2 = CPUID Leaf
ldr(x0, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.CPUIDObj)));
ldr(x3, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.CPUIDFunction)));
mov(x1, GetReg<RA_64>(Op->Function.ID()));
mov(x2, GetReg<RA_64>(Op->Leaf.ID()));
ldr(ARMEmitter::XReg::x0, STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.CPUIDObj));
ldr(ARMEmitter::XReg::x3, STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.CPUIDFunction));
mov(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r1, GetReg(Op->Function.ID()));
mov(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r2, GetReg(Op->Leaf.ID()));
#ifdef VIXL_SIMULATOR
GenerateIndirectRuntimeCall<__uint128_t, void*, uint64_t, uint64_t>(x3);
GenerateIndirectRuntimeCall<__uint128_t, void*, uint64_t, uint64_t>(ARMEmitter::Reg::r3);
#else
blr(x3);
blr(ARMEmitter::Reg::r3);
#endif
FillStaticRegs();
PopDynamicRegsAndLR();
// Results are in x0, x1
// Results want to be in a i64v2 vector
auto Dst = GetSrcPair<RA_64>(Node);
mov(Dst.first, x0);
mov(Dst.second, x1);
auto Dst = GetRegPair(Node);
mov(ARMEmitter::Size::i64Bit, Dst.first, ARMEmitter::Reg::r0);
mov(ARMEmitter::Size::i64Bit, Dst.second, ARMEmitter::Reg::r1);
}
#undef DEF_OP
@@ -4,12 +4,10 @@ tags: backend|arm64
$end_info$
*/
#include "Interface/Core/ArchHelpers/CodeEmitter/Emitter.h"
#include "Interface/Core/JIT/Arm64/JITClass.h"
namespace FEXCore::CPU {
using namespace vixl;
using namespace vixl::aarch64;
#define DEF_OP(x) void Arm64JITCore::Op_##x(IR::IROp_Header const *IROp, IR::NodeID Node)
DEF_OP(VInsGPR) {
const auto Op = IROp->C<IR::IROp_VInsGPR>();
@@ -19,8 +17,16 @@ DEF_OP(VInsGPR) {
const auto ElementSize = Op->Header.ElementSize;
const auto Is256Bit = OpSize == Core::CPUState::XMM_AVX_REG_SIZE;
const auto Dst = GetDst(Node);
const auto DestVector = GetSrc(Op->DestVector.ID());
LOGMAN_THROW_AA_FMT(ElementSize == 8 || ElementSize == 4 || ElementSize == 2 || ElementSize == 1, "Unexpected {} size", __func__);
const auto SubEmitSize = ElementSize == 8 ? ARMEmitter::SubRegSize::i64Bit :
ElementSize == 4 ? ARMEmitter::SubRegSize::i32Bit :
ElementSize == 2 ? ARMEmitter::SubRegSize::i16Bit :
ElementSize == 1 ? ARMEmitter::SubRegSize::i8Bit : ARMEmitter::SubRegSize::i8Bit;
const auto ElementsPer128Bit = 16 / ElementSize;
const auto Dst = GetVReg(Node);
const auto DestVector = GetVReg(Op->DestVector.ID());
const auto Src = GetReg(Op->Src.ID());
if (HostSupportsSVE && Is256Bit) {
const auto ElementSizeBits = ElementSize * 8;
@@ -47,99 +53,56 @@ DEF_OP(VInsGPR) {
if (InUpperLane) {
// Move the upper lane down for the insertion.
const auto CompactPred = p0;
not_(CompactPred.VnB(), PRED_TMP_32B.Zeroing(), PRED_TMP_16B.VnB());
compact(VTMP1.Z().VnD(), CompactPred, DestVector.Z().VnD());
const auto CompactPred = ARMEmitter::PReg::p0;
not_(CompactPred, PRED_TMP_32B.Zeroing(), PRED_TMP_16B);
compact(ARMEmitter::SubRegSize::i64Bit, VTMP1.Z(), CompactPred, DestVector);
}
// Put data in place for destructive SPLICE below.
mov(Dst.Z().VnD(), DestVector.Z().VnD());
mov(Dst.Z(), DestVector.Z());
// Inserts the GPR value into the given V register.
// Also automatically adjusts the index in the case of using the
// moved upper lane.
const auto Insert = [&](const aarch64::VRegister& reg, int index) {
switch (ElementSize) {
case 1:
if (InUpperLane) {
index -= 16;
}
ins(reg.V16B(), index, GetReg<RA_32>(Op->Src.ID()));
break;
case 2:
if (InUpperLane) {
index -= 8;
}
ins(reg.V8H(), index, GetReg<RA_32>(Op->Src.ID()));
break;
case 4:
if (InUpperLane) {
index -= 4;
}
ins(reg.V4S(), index, GetReg<RA_32>(Op->Src.ID()));
break;
case 8:
if (InUpperLane) {
index -= 2;
}
ins(reg.V2D(), index, GetReg<RA_64>(Op->Src.ID()));
break;
default:
LOGMAN_MSG_A_FMT("Unknown Element Size: {}", ElementSize);
break;
const auto Insert = [&](const FEXCore::ARMEmitter::VRegister& reg, int index) {
if (InUpperLane) {
index -= ElementsPer128Bit;
}
ins(SubEmitSize, reg, index, Src);
};
if (InUpperLane) {
Insert(VTMP1, DestIdx);
splice(Dst.Z().VnD(), PRED_TMP_16B, Dst.Z().VnD(), VTMP1.Z().VnD());
splice<ARMEmitter::OpType::Destructive>(ARMEmitter::SubRegSize::i64Bit, Dst.Z(), PRED_TMP_16B, Dst.Z(), VTMP1.Z());
} else {
Insert(Dst, DestIdx);
splice(Dst.Z().VnD(), PRED_TMP_16B, Dst.Z().VnD(), DestVector.Z().VnD());
splice<ARMEmitter::OpType::Destructive>(ARMEmitter::SubRegSize::i64Bit, Dst.Z(), PRED_TMP_16B, Dst.Z(), DestVector.Z());
}
} else {
mov(Dst, DestVector);
switch (ElementSize) {
case 1: {
ins(Dst.V16B(), DestIdx, GetReg<RA_32>(Op->Src.ID()));
break;
}
case 2: {
ins(Dst.V8H(), DestIdx, GetReg<RA_32>(Op->Src.ID()));
break;
}
case 4: {
ins(Dst.V4S(), DestIdx, GetReg<RA_32>(Op->Src.ID()));
break;
}
case 8: {
ins(Dst.V2D(), DestIdx, GetReg<RA_64>(Op->Src.ID()));
break;
}
default:
LOGMAN_MSG_A_FMT("Unknown Element Size: {}", ElementSize);
break;
}
mov(Dst.Q(), DestVector.Q());
ins(SubEmitSize, Dst, DestIdx, Src);
}
}
DEF_OP(VCastFromGPR) {
auto Op = IROp->C<IR::IROp_VCastFromGPR>();
auto Dst = GetVReg(Node);
auto Src = GetReg(Op->Src.ID());
switch (Op->Header.ElementSize) {
case 1:
uxtb(TMP1.W(), GetReg<RA_32>(Op->Src.ID()));
fmov(GetDst(Node).S(), TMP1.W());
uxtb(ARMEmitter::Size::i32Bit, TMP1, Src);
fmov(ARMEmitter::Size::i32Bit, Dst.S(), TMP1);
break;
case 2:
uxth(TMP1.W(), GetReg<RA_32>(Op->Src.ID()));
fmov(GetDst(Node).S(), TMP1.W());
uxth(ARMEmitter::Size::i32Bit, TMP1, Src);
fmov(ARMEmitter::Size::i32Bit, Dst.S(), TMP1);
break;
case 4:
fmov(GetDst(Node).S(), GetReg<RA_32>(Op->Src.ID()).W());
fmov(ARMEmitter::Size::i32Bit, Dst.S(), Src);
break;
case 8:
fmov(GetDst(Node).D(), GetReg<RA_64>(Op->Src.ID()).X());
fmov(ARMEmitter::Size::i64Bit, Dst.D(), Src);
break;
default: LOGMAN_MSG_A_FMT("Unknown castGPR element size: {}", Op->Header.ElementSize);
}
@@ -151,21 +114,24 @@ DEF_OP(Float_FromGPR_S) {
const uint16_t ElementSize = Op->Header.ElementSize;
const uint16_t Conv = (ElementSize << 8) | Op->SrcElementSize;
auto Dst = GetVReg(Node);
auto Src = GetReg(Op->Src.ID());
switch (Conv) {
case 0x0404: { // Float <- int32_t
scvtf(GetDst(Node).S(), GetReg<RA_32>(Op->Src.ID()));
scvtf(ARMEmitter::Size::i32Bit, Dst.S(), Src);
break;
}
case 0x0408: { // Float <- int64_t
scvtf(GetDst(Node).S(), GetReg<RA_64>(Op->Src.ID()));
scvtf(ARMEmitter::Size::i64Bit, Dst.S(), Src);
break;
}
case 0x0804: { // Double <- int32_t
scvtf(GetDst(Node).D(), GetReg<RA_32>(Op->Src.ID()));
scvtf(ARMEmitter::Size::i32Bit, Dst.D(), Src);
break;
}
case 0x0808: { // Double <- int64_t
scvtf(GetDst(Node).D(), GetReg<RA_64>(Op->Src.ID()));
scvtf(ARMEmitter::Size::i64Bit, Dst.D(), Src);
break;
}
default:
@@ -178,13 +144,17 @@ DEF_OP(Float_FromGPR_S) {
DEF_OP(Float_FToF) {
auto Op = IROp->C<IR::IROp_Float_FToF>();
const uint16_t Conv = (Op->Header.ElementSize << 8) | Op->SrcElementSize;
auto Dst = GetVReg(Node);
auto Src = GetVReg(Op->Scalar.ID());
switch (Conv) {
case 0x0804: { // Double <- Float
fcvt(GetDst(Node).D(), GetSrc(Op->Scalar.ID()).S());
fcvt(Dst.D(), Src.S());
break;
}
case 0x0408: { // Float <- Double
fcvt(GetDst(Node).S(), GetSrc(Op->Scalar.ID()).D());
fcvt(Dst.S(), Src.D());
break;
}
default: LOGMAN_MSG_A_FMT("Unknown FCVT sizes: 0x{:x}", Conv);
@@ -198,41 +168,18 @@ DEF_OP(Vector_SToF) {
const auto ElementSize = Op->Header.ElementSize;
const auto Is256Bit = OpSize == Core::CPUState::XMM_AVX_REG_SIZE;
const auto Dst = GetDst(Node);
const auto Vector = GetSrc(Op->Vector.ID());
LOGMAN_THROW_AA_FMT(ElementSize == 8 || ElementSize == 4 || ElementSize == 2, "Unexpected {} size", __func__);
const auto SubEmitSize = ElementSize == 8 ? ARMEmitter::SubRegSize::i64Bit :
ElementSize == 4 ? ARMEmitter::SubRegSize::i32Bit :
ElementSize == 2 ? ARMEmitter::SubRegSize::i16Bit : ARMEmitter::SubRegSize::i16Bit;
const auto Dst = GetVReg(Node);
const auto Vector = GetVReg(Op->Vector.ID());
if (HostSupportsSVE && Is256Bit) {
const auto Mask = PRED_TMP_32B.Merging();
switch (ElementSize) {
case 2:
scvtf(Dst.Z().VnH(), Mask, Vector.Z().VnH());
break;
case 4:
scvtf(Dst.Z().VnS(), Mask, Vector.Z().VnS());
break;
case 8:
scvtf(Dst.Z().VnD(), Mask, Vector.Z().VnD());
break;
default:
LOGMAN_MSG_A_FMT("Unknown Vector_SToF element size: {}", ElementSize);
break;
}
const auto Mask = PRED_TMP_32B;
scvtf(Dst.Z(), SubEmitSize, Mask.Merging(), Vector.Z(), SubEmitSize);
} else {
switch (ElementSize) {
case 2:
scvtf(Dst.V8H(), Vector.V8H());
break;
case 4:
scvtf(Dst.V4S(), Vector.V4S());
break;
case 8:
scvtf(Dst.V2D(), Vector.V2D());
break;
default:
LOGMAN_MSG_A_FMT("Unknown Vector_SToF element size: {}", ElementSize);
break;
}
scvtf(SubEmitSize, Dst.Q(), Vector.Q());
}
}
@@ -243,41 +190,18 @@ DEF_OP(Vector_FToZS) {
const auto ElementSize = Op->Header.ElementSize;
const auto Is256Bit = OpSize == Core::CPUState::XMM_AVX_REG_SIZE;
const auto Dst = GetDst(Node);
const auto Vector = GetSrc(Op->Vector.ID());
LOGMAN_THROW_AA_FMT(ElementSize == 8 || ElementSize == 4 || ElementSize == 2, "Unexpected {} size", __func__);
const auto SubEmitSize = ElementSize == 8 ? ARMEmitter::SubRegSize::i64Bit :
ElementSize == 4 ? ARMEmitter::SubRegSize::i32Bit :
ElementSize == 2 ? ARMEmitter::SubRegSize::i16Bit : ARMEmitter::SubRegSize::i16Bit;
const auto Dst = GetVReg(Node);
const auto Vector = GetVReg(Op->Vector.ID());
if (HostSupportsSVE && Is256Bit) {
const auto Mask = PRED_TMP_32B.Merging();
switch (ElementSize) {
case 2:
fcvtzs(Dst.Z().VnH(), Mask, Vector.Z().VnH());
break;
case 4:
fcvtzs(Dst.Z().VnS(), Mask, Vector.Z().VnS());
break;
case 8:
fcvtzs(Dst.Z().VnD(), Mask, Vector.Z().VnD());
break;
default:
LOGMAN_MSG_A_FMT("Unknown Vector_FToZS element size: {}", ElementSize);
break;
}
const auto Mask = PRED_TMP_32B;
fcvtzs(Dst, SubEmitSize, Mask.Merging(), Vector, SubEmitSize);
} else {
switch (ElementSize) {
case 2:
fcvtzs(Dst.V8H(), Vector.V8H());
break;
case 4:
fcvtzs(Dst.V4S(), Vector.V4S());
break;
case 8:
fcvtzs(Dst.V2D(), Vector.V2D());
break;
default:
LOGMAN_MSG_A_FMT("Unknown Vector_FToZS element size: {}", ElementSize);
break;
}
fcvtzs(SubEmitSize, Dst.Q(), Vector.Q());
}
}
@@ -288,47 +212,23 @@ DEF_OP(Vector_FToS) {
const auto ElementSize = Op->Header.ElementSize;
const auto Is256Bit = OpSize == Core::CPUState::XMM_AVX_REG_SIZE;
const auto Dst = GetDst(Node);
const auto Vector = GetSrc(Op->Vector.ID());
LOGMAN_THROW_AA_FMT(ElementSize == 8 || ElementSize == 4 || ElementSize == 2, "Unexpected {} size", __func__);
const auto SubEmitSize = ElementSize == 8 ? ARMEmitter::SubRegSize::i64Bit :
ElementSize == 4 ? ARMEmitter::SubRegSize::i32Bit :
ElementSize == 2 ? ARMEmitter::SubRegSize::i16Bit : ARMEmitter::SubRegSize::i16Bit;
const auto Dst = GetVReg(Node);
const auto Vector = GetVReg(Op->Vector.ID());
if (HostSupportsSVE && Is256Bit) {
const auto Mask = PRED_TMP_32B.Merging();
switch (ElementSize) {
case 2:
frinti(Dst.Z().VnH(), Mask, Vector.Z().VnH());
fcvtzs(Dst.Z().VnH(), Mask, Dst.Z().VnH());
break;
case 4:
frinti(Dst.Z().VnS(), Mask, Vector.Z().VnS());
fcvtzs(Dst.Z().VnS(), Mask, Dst.Z().VnS());
break;
case 8:
frinti(Dst.Z().VnD(), Mask, Vector.Z().VnD());
fcvtzs(Dst.Z().VnD(), Mask, Dst.Z().VnD());
break;
default:
LOGMAN_MSG_A_FMT("Unknown Vector_FToS element size: {}", ElementSize);
break;
}
const auto Mask = PRED_TMP_32B;
frinti(SubEmitSize, Dst, Mask.Merging(), Vector);
fcvtzs(Dst, SubEmitSize, Mask.Merging(), Dst, SubEmitSize);
} else {
switch (ElementSize) {
case 2:
frinti(Dst.V8H(), Vector.V8H());
fcvtzs(Dst.V8H(), Dst.V8H());
break;
case 4:
frinti(Dst.V4S(), Vector.V4S());
fcvtzs(Dst.V4S(), Dst.V4S());
break;
case 8:
frinti(Dst.V2D(), Vector.V2D());
fcvtzs(Dst.V2D(), Dst.V2D());
break;
default:
LOGMAN_MSG_A_FMT("Unknown Vector_FToS element size: {}", ElementSize);
break;
}
const auto Dst = GetVReg(Node);
const auto Vector = GetVReg(Op->Vector.ID());
frinti(SubEmitSize, Dst.Q(), Vector.Q());
fcvtzs(SubEmitSize, Dst.Q(), Dst.Q());
}
}
@@ -340,8 +240,13 @@ DEF_OP(Vector_FToF) {
const auto Is256Bit = OpSize == Core::CPUState::XMM_AVX_REG_SIZE;
const auto Conv = (ElementSize << 8) | Op->SrcElementSize;
const auto Dst = GetDst(Node);
const auto Vector = GetSrc(Op->Vector.ID());
LOGMAN_THROW_AA_FMT(ElementSize == 8 || ElementSize == 4 || ElementSize == 2, "Unexpected {} size", __func__);
const auto SubEmitSize = ElementSize == 8 ? ARMEmitter::SubRegSize::i64Bit :
ElementSize == 4 ? ARMEmitter::SubRegSize::i32Bit :
ElementSize == 2 ? ARMEmitter::SubRegSize::i16Bit : ARMEmitter::SubRegSize::i16Bit;
const auto Dst = GetVReg(Node);
const auto Vector = GetVReg(Op->Vector.ID());
if (HostSupportsSVE && Is256Bit) {
// Curiously, FCVTLT and FCVTNT have no bottom variants,
@@ -361,23 +266,23 @@ DEF_OP(Vector_FToF) {
switch (Conv) {
case 0x0402: { // Float <- Half
zip1(Dst.Z().VnH(), Vector.Z().VnH(), Vector.Z().VnH());
fcvtlt(Dst.Z().VnS(), Mask, Dst.Z().VnH());
zip1(FEXCore::ARMEmitter::SubRegSize::i16Bit, Dst.Z(), Vector.Z(), Vector.Z());
fcvtlt(FEXCore::ARMEmitter::SubRegSize::i32Bit, Dst.Z(), Mask, Dst.Z());
break;
}
case 0x0804: { // Double <- Float
zip1(Dst.Z().VnS(), Vector.Z().VnS(), Vector.Z().VnS());
fcvtlt(Dst.Z().VnD(), Mask, Dst.Z().VnS());
zip1(FEXCore::ARMEmitter::SubRegSize::i32Bit, Dst.Z(), Vector.Z(), Vector.Z());
fcvtlt(FEXCore::ARMEmitter::SubRegSize::i64Bit, Dst.Z(), Mask, Dst.Z());
break;
}
case 0x0204: { // Half <- Float
fcvtnt(Dst.Z().VnH(), Mask, Vector.Z().VnS());
uzp2(Dst.Z().VnH(), Dst.Z().VnH(), Dst.Z().VnH());
fcvtnt(FEXCore::ARMEmitter::SubRegSize::i16Bit, Dst, Mask, Vector);
uzp2(FEXCore::ARMEmitter::SubRegSize::i16Bit, Dst.Z(), Dst.Z(), Dst.Z());
break;
}
case 0x0408: { // Float <- Double
fcvtnt(Dst.Z().VnS(), Mask, Vector.Z().VnD());
uzp2(Dst.Z().VnS(), Dst.Z().VnS(), Dst.Z().VnS());
fcvtnt(FEXCore::ARMEmitter::SubRegSize::i32Bit, Dst, Mask, Vector);
uzp2(FEXCore::ARMEmitter::SubRegSize::i32Bit, Dst.Z(), Dst.Z(), Dst.Z());
break;
}
default:
@@ -386,20 +291,14 @@ DEF_OP(Vector_FToF) {
}
} else {
switch (Conv) {
case 0x0402: { // Float <- Half
fcvtl(Dst.V4S(), Vector.V4H());
break;
}
case 0x0402: // Float <- Half
case 0x0804: { // Double <- Float
fcvtl(Dst.V2D(), Vector.V2S());
break;
}
case 0x0204: { // Half <- Float
fcvtn(Dst.V4H(), Vector.V4S());
fcvtl(SubEmitSize, Dst.D(), Vector.D());
break;
}
case 0x0204: // Half <- Float
case 0x0408: { // Float <- Double
fcvtn(Dst.V2S(), Vector.V2D());
fcvtn(SubEmitSize, Dst.D(), Vector.D());
break;
}
default:
@@ -415,154 +314,51 @@ DEF_OP(Vector_FToI) {
const auto ElementSize = Op->Header.ElementSize;
const auto Is256Bit = OpSize == Core::CPUState::XMM_AVX_REG_SIZE;
LOGMAN_THROW_AA_FMT(ElementSize == 8 || ElementSize == 4 || ElementSize == 2, "Unexpected {} size", __func__);
const auto Dst = GetDst(Node);
const auto Vector = GetSrc(Op->Vector.ID());
const auto SubEmitSize = ElementSize == 8 ? ARMEmitter::SubRegSize::i64Bit :
ElementSize == 4 ? ARMEmitter::SubRegSize::i32Bit :
ElementSize == 2 ? ARMEmitter::SubRegSize::i16Bit : ARMEmitter::SubRegSize::i16Bit;
const auto Dst = GetVReg(Node);
const auto Vector = GetVReg(Op->Vector.ID());
if (HostSupportsSVE && Is256Bit) {
const auto Mask = PRED_TMP_32B.Merging();
switch (Op->Round) {
case FEXCore::IR::Round_Nearest.Val:
switch (ElementSize) {
case 2:
frintn(Dst.Z().VnH(), Mask, Vector.Z().VnH());
break;
case 4:
frintn(Dst.Z().VnS(), Mask, Vector.Z().VnS());
break;
case 8:
frintn(Dst.Z().VnD(), Mask, Vector.Z().VnD());
break;
}
frintn(SubEmitSize, Dst.Z(), Mask, Vector.Z());
break;
case FEXCore::IR::Round_Negative_Infinity.Val:
switch (ElementSize) {
case 2:
frintm(Dst.Z().VnH(), Mask, Vector.Z().VnH());
break;
case 4:
frintm(Dst.Z().VnS(), Mask, Vector.Z().VnS());
break;
case 8:
frintm(Dst.Z().VnD(), Mask, Vector.Z().VnD());
break;
}
frintm(SubEmitSize, Dst.Z(), Mask, Vector.Z());
break;
case FEXCore::IR::Round_Positive_Infinity.Val:
switch (ElementSize) {
case 2:
frintp(Dst.Z().VnH(), Mask, Vector.Z().VnH());
break;
case 4:
frintp(Dst.Z().VnS(), Mask, Vector.Z().VnS());
break;
case 8:
frintp(Dst.Z().VnD(), Mask, Vector.Z().VnD());
break;
}
frintp(SubEmitSize, Dst.Z(), Mask, Vector.Z());
break;
case FEXCore::IR::Round_Towards_Zero.Val:
switch (ElementSize) {
case 2:
frintz(Dst.Z().VnH(), Mask, Vector.Z().VnH());
break;
case 4:
frintz(Dst.Z().VnS(), Mask, Vector.Z().VnS());
break;
case 8:
frintz(Dst.Z().VnD(), Mask, Vector.Z().VnD());
break;
}
frintz(SubEmitSize, Dst.Z(), Mask, Vector.Z());
break;
case FEXCore::IR::Round_Host.Val:
switch (ElementSize) {
case 2:
frinti(Dst.Z().VnH(), Mask, Vector.Z().VnH());
break;
case 4:
frinti(Dst.Z().VnS(), Mask, Vector.Z().VnS());
break;
case 8:
frinti(Dst.Z().VnD(), Mask, Vector.Z().VnD());
break;
}
frinti(SubEmitSize, Dst.Z(), Mask, Vector.Z());
break;
}
} else {
switch (Op->Round) {
case FEXCore::IR::Round_Nearest.Val:
switch (ElementSize) {
case 2:
frintn(Dst.V8H(), Vector.V8H());
break;
case 4:
frintn(Dst.V4S(), Vector.V4S());
break;
case 8:
frintn(Dst.V2D(), Vector.V2D());
break;
}
frinti(SubEmitSize, Dst.Q(), Vector.Q());
break;
case FEXCore::IR::Round_Negative_Infinity.Val:
switch (ElementSize) {
case 2:
frintm(Dst.V8H(), Vector.V8H());
break;
case 4:
frintm(Dst.V4S(), Vector.V4S());
break;
case 8:
frintm(Dst.V2D(), Vector.V2D());
break;
}
frintm(SubEmitSize, Dst.Q(), Vector.Q());
break;
case FEXCore::IR::Round_Positive_Infinity.Val:
switch (ElementSize) {
case 2:
frintp(Dst.V8H(), Vector.V8H());
break;
case 4:
frintp(Dst.V4S(), Vector.V4S());
break;
case 8:
frintp(Dst.V2D(), Vector.V2D());
break;
}
frintp(SubEmitSize, Dst.Q(), Vector.Q());
break;
case FEXCore::IR::Round_Towards_Zero.Val:
switch (ElementSize) {
case 2:
frintz(Dst.V8H(), Vector.V8H());
break;
case 4:
frintz(Dst.V4S(), Vector.V4S());
break;
case 8:
frintz(Dst.V2D(), Vector.V2D());
break;
}
frintz(SubEmitSize, Dst.Q(), Vector.Q());
break;
case FEXCore::IR::Round_Host.Val:
switch (ElementSize) {
case 2:
frinti(Dst.V8H(), Vector.V8H());
break;
case 4:
frinti(Dst.V4S(), Vector.V4S());
break;
case 8:
frinti(Dst.V2D(), Vector.V2D());
break;
}
frinti(SubEmitSize, Dst.Q(), Vector.Q());
break;
}
}
@@ -4,98 +4,104 @@ tags: backend|arm64
$end_info$
*/
#include "Interface/Core/ArchHelpers/CodeEmitter/Emitter.h"
#include "Interface/Core/JIT/Arm64/JITClass.h"
#include "Interface/IR/Passes/RegisterAllocationPass.h"
namespace FEXCore::CPU {
using namespace vixl;
using namespace vixl::aarch64;
#define DEF_OP(x) void Arm64JITCore::Op_##x(IR::IROp_Header const *IROp, IR::NodeID Node)
DEF_OP(AESImc) {
auto Op = IROp->C<IR::IROp_VAESImc>();
aesimc(GetDst(Node).V16B(), GetSrc(Op->Vector.ID()).V16B());
aesimc(GetVReg(Node), GetVReg(Op->Vector.ID()));
}
DEF_OP(AESEnc) {
auto Op = IROp->C<IR::IROp_VAESEnc>();
eor(VTMP2.V16B(), VTMP2.V16B(), VTMP2.V16B());
mov(VTMP1.V16B(), GetSrc(Op->State.ID()).V16B());
aese(VTMP1.V16B(), VTMP2.V16B());
aesmc(VTMP1.V16B(), VTMP1.V16B());
eor(GetDst(Node).V16B(), VTMP1.V16B(), GetSrc(Op->Key.ID()).V16B());
eor(VTMP2.Q(), VTMP2.Q(), VTMP2.Q());
mov(VTMP1.Q(), GetVReg(Op->State.ID()).Q());
aese(VTMP1, VTMP2);
aesmc(VTMP1, VTMP1);
eor(GetVReg(Node).Q(), VTMP1.Q(), GetVReg(Op->Key.ID()).Q());
}
DEF_OP(AESEncLast) {
auto Op = IROp->C<IR::IROp_VAESEncLast>();
eor(VTMP2.V16B(), VTMP2.V16B(), VTMP2.V16B());
mov(VTMP1.V16B(), GetSrc(Op->State.ID()).V16B());
aese(VTMP1.V16B(), VTMP2.V16B());
eor(GetDst(Node).V16B(), VTMP1.V16B(), GetSrc(Op->Key.ID()).V16B());
eor(VTMP2.Q(), VTMP2.Q(), VTMP2.Q());
mov(VTMP1.Q(), GetVReg(Op->State.ID()).Q());
aese(VTMP1, VTMP2);
eor(GetVReg(Node).Q(), VTMP1.Q(), GetVReg(Op->Key.ID()).Q());
}
DEF_OP(AESDec) {
auto Op = IROp->C<IR::IROp_VAESDec>();
eor(VTMP2.V16B(), VTMP2.V16B(), VTMP2.V16B());
mov(VTMP1.V16B(), GetSrc(Op->State.ID()).V16B());
aesd(VTMP1.V16B(), VTMP2.V16B());
aesimc(VTMP1.V16B(), VTMP1.V16B());
eor(GetDst(Node).V16B(), VTMP1.V16B(), GetSrc(Op->Key.ID()).V16B());
eor(VTMP2.Q(), VTMP2.Q(), VTMP2.Q());
mov(VTMP1.Q(), GetVReg(Op->State.ID()).Q());
aesd(VTMP1, VTMP2);
aesimc(VTMP1, VTMP1);
eor(GetVReg(Node).Q(), VTMP1.Q(), GetVReg(Op->Key.ID()).Q());
}
DEF_OP(AESDecLast) {
auto Op = IROp->C<IR::IROp_VAESDecLast>();
eor(VTMP2.V16B(), VTMP2.V16B(), VTMP2.V16B());
mov(VTMP1.V16B(), GetSrc(Op->State.ID()).V16B());
aesd(VTMP1.V16B(), VTMP2.V16B());
eor(GetDst(Node).V16B(), VTMP1.V16B(), GetSrc(Op->Key.ID()).V16B());
eor(VTMP2.Q(), VTMP2.Q(), VTMP2.Q());
mov(VTMP1.Q(), GetVReg(Op->State.ID()).Q());
aesd(VTMP1, VTMP2);
eor(GetVReg(Node).Q(), VTMP1.Q(), GetVReg(Op->Key.ID()).Q());
}
DEF_OP(AESKeyGenAssist) {
auto Op = IROp->C<IR::IROp_VAESKeyGenAssist>();
aarch64::Literal ConstantLiteral (0x0C030609'0306090CULL, 0x040B0E01'0B0E0104ULL);
aarch64::Label PastConstant;
ARMEmitter::ForwardLabel Constant;
ARMEmitter::ForwardLabel PastConstant;
// Do a "regular" AESE step
eor(VTMP2.V16B(), VTMP2.V16B(), VTMP2.V16B());
mov(VTMP1.V16B(), GetSrc(Op->Src.ID()).V16B());
aese(VTMP1.V16B(), VTMP2.V16B());
eor(VTMP2.Q(), VTMP2.Q(), VTMP2.Q());
mov(VTMP1.Q(), GetVReg(Op->Src.ID()).Q());
aese(VTMP1, VTMP2);
// Do a table shuffle to undo ShiftRows
ldr(VTMP3, &ConstantLiteral);
ldr(VTMP3.Q(), &Constant);
// Now EOR in the RCON
if (Op->RCON) {
tbl(VTMP1.V16B(), VTMP1.V16B(), VTMP3.V16B());
tbl(VTMP1.Q(), VTMP1.Q(), VTMP3.Q());
LoadConstant(TMP1, static_cast<uint64_t>(Op->RCON) << 32);
dup(VTMP2.V2D(), TMP1);
eor(GetDst(Node).V16B(), VTMP1.V16B(), VTMP2.V16B());
LoadConstant(ARMEmitter::Size::i64Bit, TMP1, static_cast<uint64_t>(Op->RCON) << 32);
dup(ARMEmitter::SubRegSize::i64Bit, VTMP2.Q(), TMP1);
eor(GetVReg(Node).Q(), VTMP1.Q(), VTMP2.Q());
}
else {
tbl(GetDst(Node).V16B(), VTMP1.V16B(), VTMP3.V16B());
tbl(GetVReg(Node).Q(), VTMP1.Q(), VTMP3.Q());
}
b(&PastConstant);
place(&ConstantLiteral);
bind(&PastConstant);
Bind(&Constant);
dc64(0x040B0E01'0B0E0104ULL);
dc64(0x0C030609'0306090CULL);
Bind(&PastConstant);
}
DEF_OP(CRC32) {
auto Op = IROp->C<IR::IROp_CRC32>();
const auto Dst = GetReg(Node);
const auto Src1 = GetReg(Op->Src1.ID());
const auto Src2 = GetReg(Op->Src2.ID());
switch (Op->SrcSize) {
case 1:
crc32cb(GetReg<RA_32>(Node), GetReg<RA_32>(Op->Src1.ID()), GetReg<RA_32>(Op->Src2.ID()));
crc32cb(Dst.W(), Src1.W(), Src2.W());
break;
case 2:
crc32ch(GetReg<RA_32>(Node), GetReg<RA_32>(Op->Src1.ID()), GetReg<RA_32>(Op->Src2.ID()));
crc32ch(Dst.W(), Src1.W(), Src2.W());
break;
case 4:
crc32cw(GetReg<RA_32>(Node), GetReg<RA_32>(Op->Src1.ID()), GetReg<RA_32>(Op->Src2.ID()));
crc32cw(Dst.W(), Src1.W(), Src2.W());
break;
case 8:
crc32cx(GetReg<RA_32>(Node), GetReg<RA_32>(Op->Src1.ID()), GetReg<RA_64>(Op->Src2.ID()));
crc32cx(Dst, Src1, Src2);
break;
default: LOGMAN_MSG_A_FMT("Unknown CRC32 size: {}", Op->SrcSize);
}
@@ -104,24 +110,24 @@ DEF_OP(CRC32) {
DEF_OP(PCLMUL) {
auto Op = IROp->C<IR::IROp_PCLMUL>();
auto Dst = GetDst(Node).Q();
auto Src1 = GetSrc(Op->Src1.ID()).V2D();
auto Src2 = GetSrc(Op->Src2.ID()).V2D();
auto Dst = GetVReg(Node);
auto Src1 = GetVReg(Op->Src1.ID());
auto Src2 = GetVReg(Op->Src2.ID());
switch (Op->Selector) {
case 0b00000000:
pmull(Dst, Src1, Src2);
pmull(ARMEmitter::SubRegSize::i128Bit, Dst.D(), Src1.D(), Src2.D());
break;
case 0b00000001:
mov(VTMP1.V1D(), Src1, 1);
pmull(Dst, VTMP1.V2D(), Src2);
dup(ARMEmitter::SubRegSize::i64Bit, VTMP1.Q(), Src1.Q(), 1);
pmull(ARMEmitter::SubRegSize::i128Bit, Dst.D(), VTMP1.D(), Src2.D());
break;
case 0b00010000:
mov(VTMP1.V1D(), Src2, 1);
pmull(Dst, VTMP1.V2D(), Src1);
dup(ARMEmitter::SubRegSize::i64Bit, VTMP1.Q(), Src2.Q(), 1);
pmull(ARMEmitter::SubRegSize::i128Bit, Dst.D(), VTMP1.D(), Src1.D());
break;
case 0b00010001:
pmull2(Dst, Src1, Src2);
pmull2(ARMEmitter::SubRegSize::i128Bit, Dst.Q(), Src1.Q(), Src2.Q());
break;
default:
LOGMAN_MSG_A_FMT("Unknown PCLMUL selector: {}", Op->Selector);
@@ -7,13 +7,10 @@ $end_info$
#include "Interface/Core/JIT/Arm64/JITClass.h"
namespace FEXCore::CPU {
using namespace vixl;
using namespace vixl::aarch64;
#define DEF_OP(x) void Arm64JITCore::Op_##x(IR::IROp_Header const *IROp, IR::NodeID Node)
DEF_OP(GetHostFlag) {
auto Op = IROp->C<IR::IROp_GetHostFlag>();
ubfx(GetReg<RA_64>(Node), GetReg<RA_64>(Op->Value.ID()), Op->Flag, 1);
ubfx(ARMEmitter::Size::i64Bit, GetReg(Node), GetReg(Op->Value.ID()), Op->Flag, 1);
}
#undef DEF_OP
+185 -235
View File
@@ -11,6 +11,7 @@ $end_info$
*/
#include "Interface/Context/Context.h"
#include "Interface/Core/ArchHelpers/CodeEmitter/Emitter.h"
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/ArchHelpers/Arm64.h"
@@ -77,9 +78,6 @@ static void PrintVectorValue(uint64_t Value, uint64_t ValueUpper) {
namespace FEXCore::CPU {
using namespace vixl;
using namespace vixl::aarch64;
void Arm64JITCore::Op_Unhandled(IR::IROp_Header const *IROp, IR::NodeID Node) {
FallbackInfo Info;
if (!InterpreterOps::GetFallbackHandler(IROp, &Info)) {
@@ -91,15 +89,15 @@ void Arm64JITCore::Op_Unhandled(IR::IROp_Header const *IROp, IR::NodeID Node) {
case FABI_VOID_U16:{
SpillStaticRegs();
PushDynamicRegsAndLR();
uxth(w0, GetReg<RA_32>(IROp->Args[0].ID()));
ldr(x1, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.FallbackHandlerPointers[Info.HandlerIndex])));
PushDynamicRegsAndLR(TMP1);
const auto Src1 = GetReg(IROp->Args[0].ID());
uxth(ARMEmitter::Size::i32Bit, ARMEmitter::Reg::r0, Src1);
ldr(ARMEmitter::XReg::x1, STATE_PTR(CpuStateFrame, Pointers.Common.FallbackHandlerPointers[Info.HandlerIndex]));
#ifdef VIXL_SIMULATOR
GenerateIndirectRuntimeCall<void, uint16_t>(x1);
GenerateIndirectRuntimeCall<void, uint16_t>(ARMEmitter::Reg::r1);
#else
blr(x1);
blr(ARMEmitter::Reg::r1);
#endif
PopDynamicRegsAndLR();
@@ -111,46 +109,49 @@ void Arm64JITCore::Op_Unhandled(IR::IROp_Header const *IROp, IR::NodeID Node) {
case FABI_F80_F32:{
SpillStaticRegs();
PushDynamicRegsAndLR();
fmov(v0.S(), GetSrc(IROp->Args[0].ID()).S()) ;
ldr(x0, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.FallbackHandlerPointers[Info.HandlerIndex])));
PushDynamicRegsAndLR(TMP1);
const auto Src1 = GetVReg(IROp->Args[0].ID());
fmov(ARMEmitter::SReg::s0, Src1.S());
ldr(ARMEmitter::XReg::x0, STATE_PTR(CpuStateFrame, Pointers.Common.FallbackHandlerPointers[Info.HandlerIndex]));
#ifdef VIXL_SIMULATOR
GenerateIndirectRuntimeCall<__uint128_t, float>(x0);
GenerateIndirectRuntimeCall<__uint128_t, float>(ARMEmitter::Reg::r0);
#else
blr(x0);
blr(ARMEmitter::Reg::r0);
#endif
PopDynamicRegsAndLR();
FillStaticRegs();
eor(GetDst(Node).V16B(), GetDst(Node).V16B(), GetDst(Node).V16B());
ins(GetDst(Node).V2D(), 0, x0);
ins(GetDst(Node).V8H(), 4, w1);
const auto Dst = GetVReg(Node);
eor(Dst.Q(), Dst.Q(), Dst.Q());
ins(ARMEmitter::SubRegSize::i64Bit, Dst, 0, ARMEmitter::Reg::r0);
ins(ARMEmitter::SubRegSize::i16Bit, Dst, 4, ARMEmitter::Reg::r1);
}
break;
case FABI_F80_F64:{
SpillStaticRegs();
PushDynamicRegsAndLR();
PushDynamicRegsAndLR(TMP1);
mov(v0.D(), GetSrc(IROp->Args[0].ID()).D());
ldr(x0, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.FallbackHandlerPointers[Info.HandlerIndex])));
const auto Src1 = GetVReg(IROp->Args[0].ID());
mov(ARMEmitter::DReg::d0, Src1.D());
ldr(ARMEmitter::XReg::x0, STATE_PTR(CpuStateFrame, Pointers.Common.FallbackHandlerPointers[Info.HandlerIndex]));
#ifdef VIXL_SIMULATOR
GenerateIndirectRuntimeCall<__uint128_t, double>(x0);
GenerateIndirectRuntimeCall<__uint128_t, double>(ARMEmitter::Reg::r0);
#else
blr(x0);
blr(ARMEmitter::Reg::r0);
#endif
PopDynamicRegsAndLR();
FillStaticRegs();
eor(GetDst(Node).V16B(), GetDst(Node).V16B(), GetDst(Node).V16B());
ins(GetDst(Node).V2D(), 0, x0);
ins(GetDst(Node).V8H(), 4, w1);
const auto Dst = GetVReg(Node);
eor(Dst.Q(), Dst.Q(), Dst.Q());
ins(ARMEmitter::SubRegSize::i64Bit, Dst, 0, ARMEmitter::Reg::r0);
ins(ARMEmitter::SubRegSize::i16Bit, Dst, 4, ARMEmitter::Reg::r1);
}
break;
@@ -158,264 +159,296 @@ void Arm64JITCore::Op_Unhandled(IR::IROp_Header const *IROp, IR::NodeID Node) {
case FABI_F80_I32: {
SpillStaticRegs();
PushDynamicRegsAndLR();
PushDynamicRegsAndLR(TMP1);
const auto Src1 = GetReg(IROp->Args[0].ID());
if (Info.ABI == FABI_F80_I16) {
uxth(w0, GetReg<RA_32>(IROp->Args[0].ID()));
uxth(ARMEmitter::Size::i32Bit, ARMEmitter::Reg::r0, Src1);
}
else {
mov(w0, GetReg<RA_32>(IROp->Args[0].ID()));
mov(ARMEmitter::Size::i32Bit, ARMEmitter::Reg::r0, Src1);
}
ldr(x1, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.FallbackHandlerPointers[Info.HandlerIndex])));
ldr(ARMEmitter::XReg::x1, STATE_PTR(CpuStateFrame, Pointers.Common.FallbackHandlerPointers[Info.HandlerIndex]));
#ifdef VIXL_SIMULATOR
GenerateIndirectRuntimeCall<__uint128_t, uint32_t>(x1);
GenerateIndirectRuntimeCall<__uint128_t, uint32_t>(ARMEmitter::Reg::r1);
#else
blr(x1);
blr(ARMEmitter::Reg::r1);
#endif
PopDynamicRegsAndLR();
FillStaticRegs();
eor(GetDst(Node).V16B(), GetDst(Node).V16B(), GetDst(Node).V16B());
ins(GetDst(Node).V2D(), 0, x0);
ins(GetDst(Node).V8H(), 4, w1);
const auto Dst = GetVReg(Node);
eor(Dst.Q(), Dst.Q(), Dst.Q());
ins(ARMEmitter::SubRegSize::i64Bit, Dst, 0, ARMEmitter::Reg::r0);
ins(ARMEmitter::SubRegSize::i16Bit, Dst, 4, ARMEmitter::Reg::r1);
}
break;
case FABI_F32_F80:{
SpillStaticRegs();
PushDynamicRegsAndLR();
PushDynamicRegsAndLR(TMP1);
umov(x0, GetSrc(IROp->Args[0].ID()).V2D(), 0);
umov(w1, GetSrc(IROp->Args[0].ID()).V8H(), 4);
const auto Src1 = GetVReg(IROp->Args[0].ID());
ldr(x2, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.FallbackHandlerPointers[Info.HandlerIndex])));
umov<ARMEmitter::SubRegSize::i64Bit>(ARMEmitter::Reg::r0, Src1, 0);
umov<ARMEmitter::SubRegSize::i16Bit>(ARMEmitter::Reg::r1, Src1, 4);
ldr(ARMEmitter::XReg::x2, STATE_PTR(CpuStateFrame, Pointers.Common.FallbackHandlerPointers[Info.HandlerIndex]));
#ifdef VIXL_SIMULATOR
GenerateIndirectRuntimeCall<float, uint64_t, uint64_t>(x2);
GenerateIndirectRuntimeCall<float, uint64_t, uint64_t>(ARMEmitter::Reg::r2);
#else
blr(x2);
blr(ARMEmitter::Reg::r2);
#endif
PopDynamicRegsAndLR();
FillStaticRegs();
fmov(GetDst(Node).S(), v0.S());
const auto Dst = GetVReg(Node);
fmov(Dst.S(), ARMEmitter::SReg::s0);
}
break;
case FABI_F64_F80:{
SpillStaticRegs();
PushDynamicRegsAndLR();
PushDynamicRegsAndLR(TMP1);
umov(x0, GetSrc(IROp->Args[0].ID()).V2D(), 0);
umov(w1, GetSrc(IROp->Args[0].ID()).V8H(), 4);
const auto Src1 = GetVReg(IROp->Args[0].ID());
ldr(x2, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.FallbackHandlerPointers[Info.HandlerIndex])));
umov<ARMEmitter::SubRegSize::i64Bit>(ARMEmitter::Reg::r0, Src1, 0);
umov<ARMEmitter::SubRegSize::i16Bit>(ARMEmitter::Reg::r1, Src1, 4);
ldr(ARMEmitter::XReg::x2, STATE_PTR(CpuStateFrame, Pointers.Common.FallbackHandlerPointers[Info.HandlerIndex]));
#ifdef VIXL_SIMULATOR
GenerateIndirectRuntimeCall<double, uint64_t, uint64_t>(x2);
GenerateIndirectRuntimeCall<double, uint64_t, uint64_t>(ARMEmitter::Reg::r2);
#else
blr(x2);
blr(ARMEmitter::Reg::r2);
#endif
PopDynamicRegsAndLR();
FillStaticRegs();
mov(GetDst(Node).D(), v0.D());
const auto Dst = GetVReg(Node);
mov(Dst.D(), ARMEmitter::DReg::d0);
}
break;
case FABI_F64_F64: {
SpillStaticRegs();
PushDynamicRegsAndLR();
PushDynamicRegsAndLR(TMP1);
mov(v0.D(), GetSrc(IROp->Args[0].ID()).D());
ldr(x0, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.FallbackHandlerPointers[Info.HandlerIndex])));
const auto Src1 = GetVReg(IROp->Args[0].ID());
mov(ARMEmitter::DReg::d0, Src1.D());
ldr(ARMEmitter::XReg::x0, STATE_PTR(CpuStateFrame, Pointers.Common.FallbackHandlerPointers[Info.HandlerIndex]));
#ifdef VIXL_SIMULATOR
GenerateIndirectRuntimeCall<double, double>(x0);
GenerateIndirectRuntimeCall<double, double>(ARMEmitter::Reg::r0);
#else
blr(x0);
blr(ARMEmitter::Reg::r0);
#endif
PopDynamicRegsAndLR();
FillStaticRegs();
mov(GetDst(Node).D(), v0.D());
const auto Dst = GetVReg(Node);
mov(Dst.D(), ARMEmitter::DReg::d0);
}
break;
case FABI_F64_F64_F64: {
SpillStaticRegs();
PushDynamicRegsAndLR();
PushDynamicRegsAndLR(TMP1);
mov(v0.D(), GetSrc(IROp->Args[0].ID()).D());
mov(v1.D(), GetSrc(IROp->Args[1].ID()).D());
ldr(x0, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.FallbackHandlerPointers[Info.HandlerIndex])));
const auto Src1 = GetVReg(IROp->Args[0].ID());
const auto Src2 = GetVReg(IROp->Args[1].ID());
mov(ARMEmitter::DReg::d0, Src1.D());
mov(ARMEmitter::DReg::d1, Src2.D());
ldr(ARMEmitter::XReg::x0, STATE_PTR(CpuStateFrame, Pointers.Common.FallbackHandlerPointers[Info.HandlerIndex]));
#ifdef VIXL_SIMULATOR
GenerateIndirectRuntimeCall<double, double, double>(x0);
GenerateIndirectRuntimeCall<double, double, double>(ARMEmitter::Reg::r0);
#else
blr(x0);
blr(ARMEmitter::Reg::r0);
#endif
PopDynamicRegsAndLR();
FillStaticRegs();
mov(GetDst(Node).D(), v0.D());
const auto Dst = GetVReg(Node);
mov(Dst.D(), ARMEmitter::DReg::d0);
}
break;
case FABI_I16_F80:{
SpillStaticRegs();
PushDynamicRegsAndLR();
PushDynamicRegsAndLR(TMP1);
umov(x0, GetSrc(IROp->Args[0].ID()).V2D(), 0);
umov(w1, GetSrc(IROp->Args[0].ID()).V8H(), 4);
const auto Src1 = GetVReg(IROp->Args[0].ID());
ldr(x2, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.FallbackHandlerPointers[Info.HandlerIndex])));
umov<ARMEmitter::SubRegSize::i64Bit>(ARMEmitter::Reg::r0, Src1, 0);
umov<ARMEmitter::SubRegSize::i16Bit>(ARMEmitter::Reg::r1, Src1, 4);
ldr(ARMEmitter::XReg::x2, STATE_PTR(CpuStateFrame, Pointers.Common.FallbackHandlerPointers[Info.HandlerIndex]));
#ifdef VIXL_SIMULATOR
GenerateIndirectRuntimeCall<uint32_t, uint64_t, uint64_t>(x2);
GenerateIndirectRuntimeCall<uint32_t, uint64_t, uint64_t>(ARMEmitter::Reg::r2);
#else
blr(x2);
blr(ARMEmitter::Reg::r2);
#endif
PopDynamicRegsAndLR();
FillStaticRegs();
uxth(GetReg<RA_64>(Node), x0);
const auto Dst = GetReg(Node);
uxth(ARMEmitter::Size::i64Bit, Dst, ARMEmitter::Reg::r0);
}
break;
case FABI_I32_F80:{
SpillStaticRegs();
PushDynamicRegsAndLR();
PushDynamicRegsAndLR(TMP1);
umov(x0, GetSrc(IROp->Args[0].ID()).V2D(), 0);
umov(w1, GetSrc(IROp->Args[0].ID()).V8H(), 4);
const auto Src1 = GetVReg(IROp->Args[0].ID());
ldr(x2, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.FallbackHandlerPointers[Info.HandlerIndex])));
umov<ARMEmitter::SubRegSize::i64Bit>(ARMEmitter::Reg::r0, Src1, 0);
umov<ARMEmitter::SubRegSize::i16Bit>(ARMEmitter::Reg::r1, Src1, 4);
ldr(ARMEmitter::XReg::x2, STATE_PTR(CpuStateFrame, Pointers.Common.FallbackHandlerPointers[Info.HandlerIndex]));
#ifdef VIXL_SIMULATOR
GenerateIndirectRuntimeCall<uint32_t, uint64_t, uint64_t>(x2);
GenerateIndirectRuntimeCall<uint32_t, uint64_t, uint64_t>(ARMEmitter::Reg::r2);
#else
blr(x2);
blr(ARMEmitter::Reg::r2);
#endif
PopDynamicRegsAndLR();
FillStaticRegs();
mov(GetReg<RA_32>(Node), w0);
const auto Dst = GetReg(Node);
mov(ARMEmitter::Size::i32Bit, Dst, ARMEmitter::Reg::r0);
}
break;
case FABI_I64_F80:{
SpillStaticRegs();
PushDynamicRegsAndLR();
PushDynamicRegsAndLR(TMP1);
umov(x0, GetSrc(IROp->Args[0].ID()).V2D(), 0);
umov(w1, GetSrc(IROp->Args[0].ID()).V8H(), 4);
const auto Src1 = GetVReg(IROp->Args[0].ID());
ldr(x2, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.FallbackHandlerPointers[Info.HandlerIndex])));
umov<ARMEmitter::SubRegSize::i64Bit>(ARMEmitter::Reg::r0, Src1, 0);
umov<ARMEmitter::SubRegSize::i16Bit>(ARMEmitter::Reg::r1, Src1, 4);
ldr(ARMEmitter::XReg::x2, STATE_PTR(CpuStateFrame, Pointers.Common.FallbackHandlerPointers[Info.HandlerIndex]));
#ifdef VIXL_SIMULATOR
GenerateIndirectRuntimeCall<uint64_t, uint64_t, uint64_t>(x2);
GenerateIndirectRuntimeCall<uint64_t, uint64_t, uint64_t>(ARMEmitter::Reg::r2);
#else
blr(x2);
blr(ARMEmitter::Reg::r2);
#endif
PopDynamicRegsAndLR();
FillStaticRegs();
mov(GetReg<RA_64>(Node), x0);
const auto Dst = GetReg(Node);
mov(ARMEmitter::Size::i64Bit, Dst, ARMEmitter::Reg::r0);
}
break;
case FABI_I64_F80_F80:{
SpillStaticRegs();
PushDynamicRegsAndLR();
PushDynamicRegsAndLR(TMP1);
umov(x0, GetSrc(IROp->Args[0].ID()).V2D(), 0);
umov(w1, GetSrc(IROp->Args[0].ID()).V8H(), 4);
const auto Src1 = GetVReg(IROp->Args[0].ID());
const auto Src2 = GetVReg(IROp->Args[1].ID());
umov(x2, GetSrc(IROp->Args[1].ID()).V2D(), 0);
umov(w3, GetSrc(IROp->Args[1].ID()).V8H(), 4);
umov<ARMEmitter::SubRegSize::i64Bit>(ARMEmitter::Reg::r0, Src1, 0);
umov<ARMEmitter::SubRegSize::i16Bit>(ARMEmitter::Reg::r1, Src1, 4);
ldr(x4, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.FallbackHandlerPointers[Info.HandlerIndex])));
umov<ARMEmitter::SubRegSize::i64Bit>(ARMEmitter::Reg::r2, Src2, 0);
umov<ARMEmitter::SubRegSize::i16Bit>(ARMEmitter::Reg::r3, Src2, 4);
ldr(ARMEmitter::XReg::x4, STATE_PTR(CpuStateFrame, Pointers.Common.FallbackHandlerPointers[Info.HandlerIndex]));
#ifdef VIXL_SIMULATOR
GenerateIndirectRuntimeCall<uint64_t, uint64_t, uint64_t, uint64_t, uint64_t>(x4);
GenerateIndirectRuntimeCall<uint64_t, uint64_t, uint64_t, uint64_t, uint64_t>(ARMEmitter::Reg::r4);
#else
blr(x4);
blr(ARMEmitter::Reg::r4);
#endif
PopDynamicRegsAndLR();
FillStaticRegs();
mov(GetReg<RA_64>(Node), x0);
const auto Dst = GetReg(Node);
mov(ARMEmitter::Size::i64Bit, Dst, ARMEmitter::Reg::r0);
}
break;
case FABI_F80_F80:{
SpillStaticRegs();
PushDynamicRegsAndLR();
PushDynamicRegsAndLR(TMP1);
umov(x0, GetSrc(IROp->Args[0].ID()).V2D(), 0);
umov(w1, GetSrc(IROp->Args[0].ID()).V8H(), 4);
const auto Src1 = GetVReg(IROp->Args[0].ID());
ldr(x2, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.FallbackHandlerPointers[Info.HandlerIndex])));
umov<ARMEmitter::SubRegSize::i64Bit>(ARMEmitter::Reg::r0, Src1, 0);
umov<ARMEmitter::SubRegSize::i16Bit>(ARMEmitter::Reg::r1, Src1, 4);
ldr(ARMEmitter::XReg::x2, STATE_PTR(CpuStateFrame, Pointers.Common.FallbackHandlerPointers[Info.HandlerIndex]));
#ifdef VIXL_SIMULATOR
GenerateIndirectRuntimeCall<__uint128_t, uint64_t, uint64_t>(x2);
GenerateIndirectRuntimeCall<__uint128_t, uint64_t, uint64_t>(ARMEmitter::Reg::r2);
#else
blr(x2);
blr(ARMEmitter::Reg::r2);
#endif
PopDynamicRegsAndLR();
FillStaticRegs();
eor(GetDst(Node).V16B(), GetDst(Node).V16B(), GetDst(Node).V16B());
ins(GetDst(Node).V2D(), 0, x0);
ins(GetDst(Node).V8H(), 4, w1);
const auto Dst = GetVReg(Node);
eor(Dst.Q(), Dst.Q(), Dst.Q());
ins(ARMEmitter::SubRegSize::i64Bit, Dst, 0, ARMEmitter::Reg::r0);
ins(ARMEmitter::SubRegSize::i16Bit, Dst, 4, ARMEmitter::Reg::r1);
}
break;
case FABI_F80_F80_F80:{
SpillStaticRegs();
PushDynamicRegsAndLR();
PushDynamicRegsAndLR(TMP1);
umov(x0, GetSrc(IROp->Args[0].ID()).V2D(), 0);
umov(w1, GetSrc(IROp->Args[0].ID()).V8H(), 4);
const auto Src1 = GetVReg(IROp->Args[0].ID());
const auto Src2 = GetVReg(IROp->Args[1].ID());
umov(x2, GetSrc(IROp->Args[1].ID()).V2D(), 0);
umov(w3, GetSrc(IROp->Args[1].ID()).V8H(), 4);
umov<ARMEmitter::SubRegSize::i64Bit>(ARMEmitter::Reg::r0, Src1, 0);
umov<ARMEmitter::SubRegSize::i16Bit>(ARMEmitter::Reg::r1, Src1, 4);
ldr(x4, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.FallbackHandlerPointers[Info.HandlerIndex])));
umov<ARMEmitter::SubRegSize::i64Bit>(ARMEmitter::Reg::r2, Src2, 0);
umov<ARMEmitter::SubRegSize::i16Bit>(ARMEmitter::Reg::r3, Src2, 4);
ldr(ARMEmitter::XReg::x4, STATE_PTR(CpuStateFrame, Pointers.Common.FallbackHandlerPointers[Info.HandlerIndex]));
#ifdef VIXL_SIMULATOR
GenerateIndirectRuntimeCall<__uint128_t, uint64_t, uint64_t, uint64_t, uint64_t>(x4);
GenerateIndirectRuntimeCall<__uint128_t, uint64_t, uint64_t, uint64_t, uint64_t>(ARMEmitter::Reg::r4);
#else
blr(x4);
blr(ARMEmitter::Reg::r4);
#endif
PopDynamicRegsAndLR();
FillStaticRegs();
eor(GetDst(Node).V16B(), GetDst(Node).V16B(), GetDst(Node).V16B());
ins(GetDst(Node).V2D(), 0, x0);
ins(GetDst(Node).V8H(), 4, w1);
const auto Dst = GetVReg(Node);
eor(Dst.Q(), Dst.Q(), Dst.Q());
ins(ARMEmitter::SubRegSize::i64Bit, Dst, 0, ARMEmitter::Reg::r0);
ins(ARMEmitter::SubRegSize::i16Bit, Dst, 4, ARMEmitter::Reg::r1);
}
break;
case FABI_UNKNOWN:
default:
#if defined(ASSERTIONS_ENABLED) && ASSERTIONS_ENABLED
@@ -435,7 +468,6 @@ static uint64_t Arm64JITCore_ExitFunctionLink(FEXCore::Core::CpuStateFrame *Fram
auto HostCode = Thread->LookupCache->FindBlock(GuestRip);
if (!HostCode) {
//fmt::print("ExitFunctionLink: Aborting, {:X} not in cache\n", GuestRip);
Frame->State.rip = GuestRip;
return Frame->Pointers.Common.DispatcherLoopTop;
}
@@ -444,25 +476,22 @@ static uint64_t Arm64JITCore_ExitFunctionLink(FEXCore::Core::CpuStateFrame *Fram
auto LinkerAddress = Frame->Pointers.Common.ExitFunctionLinker;
auto offset = HostCode/4 - branch/4;
if (IsInt26(offset)) {
if (vixl::IsInt26(offset)) {
// optimal case - can branch directly
// patch the code
vixl::aarch64::Assembler emit((uint8_t*)(branch), 24);
vixl::CodeBufferCheckScope scope(&emit, 24, vixl::CodeBufferCheckScope::kDontReserveBufferSpace, vixl::CodeBufferCheckScope::kNoAssert);
FEXCore::ARMEmitter::Emitter emit((uint8_t*)(branch), 24);
emit.b(offset);
emit.FinalizeCode();
vixl::aarch64::CPU::EnsureIAndDCacheCoherency((void*)branch, 24);
FEXCore::ARMEmitter::Emitter::ClearICache((void*)branch, 24);
// Add de-linking handler
Context::Context::ThreadAddBlockLink(Thread, GuestRip, (uintptr_t)record, [branch, LinkerAddress]{
vixl::aarch64::Assembler emit((uint8_t*)(branch), 24);
vixl::CodeBufferCheckScope scope(&emit, 24, vixl::CodeBufferCheckScope::kDontReserveBufferSpace, vixl::CodeBufferCheckScope::kNoAssert);
Literal l_BranchHost{LinkerAddress};
emit.ldr(x0, &l_BranchHost);
emit.blr(x0);
emit.place(&l_BranchHost);
emit.FinalizeCode();
vixl::aarch64::CPU::EnsureIAndDCacheCoherency((void*)branch, 24);
FEXCore::ARMEmitter::Emitter emit((uint8_t*)(branch), 24);
FEXCore::ARMEmitter::ForwardLabel l_BranchHost;
emit.ldr(FEXCore::ARMEmitter::XReg::x0, &l_BranchHost);
emit.blr(FEXCore::ARMEmitter::Reg::r0);
emit.Bind(&l_BranchHost);
emit.dc64(LinkerAddress);
FEXCore::ARMEmitter::Emitter::ClearICache((void*)branch, 24);
});
} else {
// fallback case - do a soft-er link by patching the pointer
@@ -488,10 +517,6 @@ Arm64JITCore::Arm64JITCore(FEXCore::Context::Context *ctx, FEXCore::Core::Intern
RAPass = Thread->PassManager->GetPass<IR::RegisterAllocationPass>("RA");
#if DEBUG
Decoder.AppendVisitor(&Disasm)
#endif
uint32_t NumUsedGPRs = NumGPRs;
uint32_t NumUsedGPRPairs = NumGPRPairs;
uint32_t UsedRegisterCount = RegisterCount;
@@ -559,7 +584,6 @@ Arm64JITCore::Arm64JITCore(FEXCore::Context::Context *ctx, FEXCore::Core::Intern
}
// Must be done after Dispatcher init
SetAllowAssembler(true);
ClearCache();
}
@@ -582,16 +606,15 @@ void Arm64JITCore::InitializeSignalHandlers(FEXCore::Context::Context *CTX) {
void Arm64JITCore::EmitDetectionString() {
const char JITString[] = "FEXJIT::Arm64JITCore::";
auto Buffer = GetBuffer();
Buffer->EmitString(JITString);
Buffer->Align();
EmitString(JITString);
Align();
}
void Arm64JITCore::ClearCache() {
// Get the backing code buffer
auto CodeBuffer = GetEmptyCodeBuffer();
*GetBuffer() = vixl::CodeBuffer(CodeBuffer->Ptr, CodeBuffer->Size);
SetBuffer(CodeBuffer->Ptr, CodeBuffer->Size);
EmitDetectionString();
}
@@ -599,84 +622,6 @@ Arm64JITCore::~Arm64JITCore() {
}
IR::PhysicalRegister Arm64JITCore::GetPhys(IR::NodeID Node) const {
auto PhyReg = RAData->GetNodeRegister(Node);
LOGMAN_THROW_A_FMT(!PhyReg.IsInvalid(), "Couldn't Allocate register for node: ssa{}. Class: {}", Node, PhyReg.Class);
return PhyReg;
}
template<>
aarch64::Register Arm64JITCore::GetReg<Arm64JITCore::RA_32>(IR::NodeID Node) const {
auto Reg = GetPhys(Node);
LOGMAN_THROW_AA_FMT(Reg.Class == IR::GPRFixedClass.Val || Reg.Class == IR::GPRClass.Val, "Unexpected Class: {}", Reg.Class);
if (Reg.Class == IR::GPRFixedClass.Val) {
return SRA64[Reg.Reg].W();
} else if (Reg.Class == IR::GPRClass.Val) {
return RA64[Reg.Reg].W();
}
FEX_UNREACHABLE;
}
template<>
aarch64::Register Arm64JITCore::GetReg<Arm64JITCore::RA_64>(IR::NodeID Node) const {
auto Reg = GetPhys(Node);
LOGMAN_THROW_AA_FMT(Reg.Class == IR::GPRFixedClass.Val || Reg.Class == IR::GPRClass.Val, "Unexpected Class: {}", Reg.Class);
if (Reg.Class == IR::GPRFixedClass.Val) {
return SRA64[Reg.Reg];
} else if (Reg.Class == IR::GPRClass.Val) {
return RA64[Reg.Reg];
}
FEX_UNREACHABLE;
}
template<>
std::pair<aarch64::Register, aarch64::Register> Arm64JITCore::GetSrcPair<Arm64JITCore::RA_32>(IR::NodeID Node) const {
uint32_t Reg = GetPhys(Node).Reg;
return RA32Pair[Reg];
}
template<>
std::pair<aarch64::Register, aarch64::Register> Arm64JITCore::GetSrcPair<Arm64JITCore::RA_64>(IR::NodeID Node) const {
uint32_t Reg = GetPhys(Node).Reg;
return RA64Pair[Reg];
}
aarch64::VRegister Arm64JITCore::GetSrc(IR::NodeID Node) const {
auto Reg = GetPhys(Node);
LOGMAN_THROW_AA_FMT(Reg.Class == IR::FPRFixedClass.Val || Reg.Class == IR::FPRClass.Val, "Unexpected Class: {}", Reg.Class);
if (Reg.Class == IR::FPRFixedClass.Val) {
return SRAFPR[Reg.Reg];
} else if (Reg.Class == IR::FPRClass.Val) {
return RAFPR[Reg.Reg];
}
FEX_UNREACHABLE;
}
aarch64::VRegister Arm64JITCore::GetDst(IR::NodeID Node) const {
auto Reg = GetPhys(Node);
LOGMAN_THROW_AA_FMT(Reg.Class == IR::FPRFixedClass.Val || Reg.Class == IR::FPRClass.Val, "Unexpected Class: {}", Reg.Class);
if (Reg.Class == IR::FPRFixedClass.Val) {
return SRAFPR[Reg.Reg];
} else if (Reg.Class == IR::FPRClass.Val) {
return RAFPR[Reg.Reg];
}
FEX_UNREACHABLE;
}
bool Arm64JITCore::IsInlineConstant(const IR::OrderedNodeWrapper& WNode, uint64_t* Value) const {
auto OpHeader = IR->GetOp<IR::IROp_Header>(WNode);
@@ -714,7 +659,6 @@ FEXCore::IR::RegisterClassType Arm64JITCore::GetRegClass(IR::NodeID Node) const
return FEXCore::IR::RegisterClassType {GetPhys(Node).Class};
}
bool Arm64JITCore::IsFPR(IR::NodeID Node) const {
auto Class = GetRegClass(Node);
@@ -734,7 +678,6 @@ void *Arm64JITCore::CompileCode(uint64_t Entry,
bool GDBEnabled) {
FEXCORE_PROFILE_SCOPED("Arm64::CompileCode");
using namespace aarch64;
JumpTargets.clear();
uint32_t SSACount = IR->GetSSACount();
@@ -742,9 +685,13 @@ void *Arm64JITCore::CompileCode(uint64_t Entry,
this->RAData = RAData;
this->DebugData = DebugData;
#ifndef NDEBUG
LoadConstant(x0, Entry);
#endif
#ifdef VIXL_DISASSEMBLER
const auto DisasmBegin = GetCursorAddress<const vixl::aarch64::Instruction*>();
#endif
#ifndef NDEBUG
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r0, Entry);
#endif
this->IR = IR;
@@ -778,7 +725,7 @@ void *Arm64JITCore::CompileCode(uint64_t Entry,
if (GDBEnabled) {
auto GDBSize = CTX->Dispatcher->GenerateGDBPauseCheck(GuestEntry, Entry);
GetBuffer()->CursorForward(GDBSize);
CursorIncrement(GDBSize);
}
//LOGMAN_THROW_A_FMT(RAData->HasFullRA(), "Arm64 JIT only works with RA");
@@ -788,11 +735,11 @@ void *Arm64JITCore::CompileCode(uint64_t Entry,
if (SpillSlots) {
const auto TotalSpillSlotsSize = SpillSlots * MaxSpillSlotSize;
if (IsImmAddSub(TotalSpillSlotsSize)) {
sub(sp, sp, TotalSpillSlotsSize);
if (vixl::aarch64::Assembler::IsImmAddSub(TotalSpillSlotsSize)) {
sub(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::rsp, ARMEmitter::Reg::rsp, TotalSpillSlotsSize);
} else {
LoadConstant(x0, TotalSpillSlotsSize);
sub(sp, sp, x0);
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r0, TotalSpillSlotsSize);
sub(ARMEmitter::Size::i64Bit, ARMEmitter::XReg::rsp, ARMEmitter::XReg::rsp, ARMEmitter::XReg::x0, ARMEmitter::ExtendedType::LSL_64, 0);
}
}
@@ -817,7 +764,7 @@ void *Arm64JITCore::CompileCode(uint64_t Entry,
}
PendingTargetLabel = nullptr;
bind(&IsTarget->second);
Bind(&IsTarget->second);
}
for (auto [CodeNode, IROp] : IR->GetCode(BlockNode)) {
@@ -843,10 +790,13 @@ void *Arm64JITCore::CompileCode(uint64_t Entry,
}
PendingTargetLabel = nullptr;
FinalizeCode();
auto CodeEnd = GetCursorAddress<uint8_t *>();
CPU.EnsureIAndDCacheCoherency(GuestEntry, CodeEnd - GuestEntry);
ClearICache(GuestEntry, CodeEnd - GuestEntry);
#ifdef VIXL_DISASSEMBLER
const auto DisasmEnd = GetCursorAddress<const vixl::aarch64::Instruction*>();
Disasm.DisassembleBuffer(DisasmBegin, DisasmEnd);
#endif
if (DebugData) {
DebugData->HostCodeSize = CodeEnd - GuestEntry;
@@ -865,12 +815,12 @@ void Arm64JITCore::ResetStack() {
const auto TotalSpillSlotsSize = SpillSlots * MaxSpillSlotSize;
if (IsImmAddSub(TotalSpillSlotsSize)) {
add(sp, sp, TotalSpillSlotsSize);
if (vixl::aarch64::Assembler::IsImmAddSub(TotalSpillSlotsSize)) {
add(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::rsp, ARMEmitter::Reg::rsp, TotalSpillSlotsSize);
} else {
// Too big to fit in a 12bit immediate
LoadConstant(x0, TotalSpillSlotsSize);
add(sp, sp, x0);
// Too big to fit in a 12bit immediate
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r0, TotalSpillSlotsSize);
add(ARMEmitter::Size::i64Bit, ARMEmitter::XReg::rsp, ARMEmitter::XReg::rsp, ARMEmitter::XReg::x0, ARMEmitter::ExtendedType::LSL_64, 0);
}
}
+49 -27
View File
@@ -8,6 +8,7 @@ $end_info$
#include <FEXCore/IR/RegisterAllocationData.h>
#include "Interface/Core/ArchHelpers/Arm64Emitter.h"
#include "Interface/Core/ArchHelpers/CodeEmitter/Emitter.h"
#include "Interface/Core/Dispatcher/Dispatcher.h"
#include <aarch64/assembler-aarch64.h>
@@ -28,9 +29,6 @@ namespace FEXCore::Core {
}
namespace FEXCore::CPU {
using namespace vixl;
using namespace vixl::aarch64;
class Arm64JITCore final : public CPUBackend, public Arm64Emitter {
public:
explicit Arm64JITCore(FEXCore::Context::Context *ctx,
@@ -58,12 +56,12 @@ private:
FEX_CONFIG_OPT(ParanoidTSO, PARANOIDTSO);
const bool HostSupportsSVE{};
Label *PendingTargetLabel;
ARMEmitter::BiDirectionalLabel *PendingTargetLabel;
FEXCore::Context::Context *CTX;
FEXCore::IR::IRListView const *IR;
uint64_t Entry;
std::map<IR::NodeID, aarch64::Label> JumpTargets;
std::map<IR::NodeID, ARMEmitter::BiDirectionalLabel> JumpTargets;
/**
* @name Register Allocation
@@ -86,34 +84,57 @@ private:
constexpr static uint8_t RA_64 = 1;
constexpr static uint8_t RA_FPR = 2;
template<uint8_t RAType>
[[nodiscard]] aarch64::Register GetReg(IR::NodeID Node) const;
[[nodiscard]] FEXCore::ARMEmitter::Register GetReg(IR::NodeID Node) const {
const auto Reg = GetPhys(Node);
template<>
[[nodiscard]] aarch64::Register GetReg<RA_32>(IR::NodeID Node) const;
template<>
[[nodiscard]] aarch64::Register GetReg<RA_64>(IR::NodeID Node) const;
LOGMAN_THROW_AA_FMT(Reg.Class == IR::GPRFixedClass.Val || Reg.Class == IR::GPRClass.Val, "Unexpected Class: {}", Reg.Class);
template<uint8_t RAType>
[[nodiscard]] std::pair<aarch64::Register, aarch64::Register> GetSrcPair(IR::NodeID Node) const;
if (Reg.Class == IR::GPRFixedClass.Val) {
return SRA64[Reg.Reg];
} else if (Reg.Class == IR::GPRClass.Val) {
return RA64[Reg.Reg];
}
template<>
[[nodiscard]] std::pair<aarch64::Register, aarch64::Register> GetSrcPair<RA_32>(IR::NodeID Node) const;
template<>
[[nodiscard]] std::pair<aarch64::Register, aarch64::Register> GetSrcPair<RA_64>(IR::NodeID Node) const;
FEX_UNREACHABLE;
}
[[nodiscard]] aarch64::VRegister GetSrc(IR::NodeID Node) const;
[[nodiscard]] aarch64::VRegister GetDst(IR::NodeID Node) const;
[[nodiscard]] FEXCore::ARMEmitter::VRegister GetVReg(IR::NodeID Node) const {
const auto Reg = GetPhys(Node);
LOGMAN_THROW_AA_FMT(Reg.Class == IR::FPRFixedClass.Val || Reg.Class == IR::FPRClass.Val, "Unexpected Class: {}", Reg.Class);
if (Reg.Class == IR::FPRFixedClass.Val) {
return SRAFPR[Reg.Reg];
} else if (Reg.Class == IR::FPRClass.Val) {
return RAFPR[Reg.Reg];
}
FEX_UNREACHABLE;
}
[[nodiscard]] std::pair<FEXCore::ARMEmitter::Register, FEXCore::ARMEmitter::Register> GetRegPair(IR::NodeID Node) const {
const auto Reg = GetPhys(Node);
LOGMAN_THROW_AA_FMT(Reg.Class == IR::GPRPairClass.Val, "Unexpected Class: {}", Reg.Class);
return RA64Pair[Reg.Reg];
}
[[nodiscard]] FEXCore::IR::RegisterClassType GetRegClass(IR::NodeID Node) const;
[[nodiscard]] IR::PhysicalRegister GetPhys(IR::NodeID Node) const;
[[nodiscard]] IR::PhysicalRegister GetPhys(IR::NodeID Node) const {
auto PhyReg = RAData->GetNodeRegister(Node);
LOGMAN_THROW_A_FMT(!PhyReg.IsInvalid(), "Couldn't Allocate register for node: ssa{}. Class: {}", Node, PhyReg.Class);
return PhyReg;
}
[[nodiscard]] bool IsFPR(IR::NodeID Node) const;
[[nodiscard]] bool IsGPR(IR::NodeID Node) const;
[[nodiscard]] MemOperand GenerateMemOperand(uint8_t AccessSize,
aarch64::Register Base,
[[nodiscard]] FEXCore::ARMEmitter::ExtendedMemOperand GenerateMemOperand(uint8_t AccessSize,
FEXCore::ARMEmitter::Register Base,
IR::OrderedNodeWrapper Offset,
IR::MemOffsetType OffsetType,
uint8_t OffsetScale);
@@ -123,8 +144,8 @@ private:
//
// TMP1 is safe to use again once this memory operand is used with its
// equivalent loads or stores that this was called for.
[[nodiscard]] SVEMemOperand GenerateSVEMemOperand(uint8_t AccessSize,
aarch64::Register Base,
[[nodiscard]] FEXCore::ARMEmitter::SVEMemOperand GenerateSVEMemOperand(uint8_t AccessSize,
FEXCore::ARMEmitter::Register Base,
IR::OrderedNodeWrapper Offset,
IR::MemOffsetType OffsetType,
uint8_t OffsetScale);
@@ -162,7 +183,8 @@ private:
* @brief A literal pair relocation object for named symbol literals
*/
struct NamedSymbolLiteralPair {
Literal<uint64_t> Lit;
ARMEmitter::ForwardLabel Loc;
uint64_t Lit;
Relocation MoveABI{};
};
@@ -172,7 +194,7 @@ private:
* @param Reg - The GPR to move the thunk handler in to
* @param Sum - The hash of the thunk
*/
void InsertNamedThunkRelocation(vixl::aarch64::Register Reg, const IR::SHA256Sum &Sum);
void InsertNamedThunkRelocation(ARMEmitter::Register Reg, const IR::SHA256Sum &Sum);
/**
* @brief Inserts a guest GPR move relocation
@@ -180,7 +202,7 @@ private:
* @param Reg - The GPR to move the guest RIP in to
* @param Constant - The guest RIP that will be relocated
*/
void InsertGuestRIPMove(vixl::aarch64::Register Reg, uint64_t Constant);
void InsertGuestRIPMove(ARMEmitter::Register Reg, uint64_t Constant);
/**
* @brief Inserts a named symbol as a literal in memory
File diff suppressed because it is too large. Load diff
+63 -66
View File
@@ -5,12 +5,11 @@ $end_info$
*/
#include <syscall.h>
#include "Interface/Core/ArchHelpers/CodeEmitter/Emitter.h"
#include "Interface/Core/JIT/Arm64/JITClass.h"
#include "FEXCore/Debug/InternalThreadState.h"
namespace FEXCore::CPU {
using namespace vixl;
using namespace vixl::aarch64;
#define DEF_OP(x) void Arm64JITCore::Op_##x(IR::IROp_Header const *IROp, IR::NodeID Node)
DEF_OP(GuestOpcode) {
@@ -23,13 +22,13 @@ DEF_OP(Fence) {
auto Op = IROp->C<IR::IROp_Fence>();
switch (Op->Fence) {
case IR::Fence_Load.Val:
dmb(FullSystem, BarrierReads);
dmb(FEXCore::ARMEmitter::BarrierScope::LD);
break;
case IR::Fence_LoadStore.Val:
dmb(FullSystem, BarrierAll);
dmb(FEXCore::ARMEmitter::BarrierScope::SY);
break;
case IR::Fence_Store.Val:
dmb(FullSystem, BarrierWrites);
dmb(FEXCore::ARMEmitter::BarrierScope::ST);
break;
default: LOGMAN_MSG_A_FMT("Unknown Fence: {}", Op->Fence); break;
}
@@ -52,108 +51,107 @@ DEF_OP(Break) {
uint64_t Constant{};
memcpy(&Constant, &State, sizeof(State));
LoadConstant(x1, Constant);
str(x1, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, SynchronousFaultData)));
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r1, Constant);
str(ARMEmitter::XReg::x1, STATE, offsetof(FEXCore::Core::CpuStateFrame, SynchronousFaultData));
switch (Op->Reason.Signal) {
case SIGILL:
ldr(TMP1, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.GuestSignal_SIGILL)));
ldr(TMP1, STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.GuestSignal_SIGILL));
br(TMP1);
break;
case SIGTRAP:
ldr(TMP1, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.GuestSignal_SIGTRAP)));
ldr(TMP1, STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.GuestSignal_SIGTRAP));
br(TMP1);
break;
case SIGSEGV:
ldr(TMP1, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.GuestSignal_SIGSEGV)));
ldr(TMP1, STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.GuestSignal_SIGSEGV));
br(TMP1);
break;
default:
ldr(TMP1, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.GuestSignal_SIGTRAP)));
ldr(TMP1, STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.GuestSignal_SIGTRAP));
br(TMP1);
break;
}
}
DEF_OP(GetRoundingMode) {
auto Dst = GetReg<RA_64>(Node);
mrs(Dst, FPCR);
lsr(Dst, Dst, 22);
auto Dst = GetReg(Node);
mrs(Dst, ARMEmitter::SystemRegister::FPCR);
lsr(ARMEmitter::Size::i64Bit, Dst, Dst, 22);
// FTZ is already in the correct location
// Rounding mode is different
and_(TMP1, Dst, 0b11);
and_(ARMEmitter::Size::i64Bit, TMP1, Dst, 0b11);
cmp(TMP1, 1);
LoadConstant(TMP3, IR::ROUND_MODE_POSITIVE_INFINITY);
csel(TMP2, TMP3, xzr, vixl::aarch64::Condition::eq);
cmp(ARMEmitter::Size::i64Bit, TMP1, 1);
LoadConstant(ARMEmitter::Size::i64Bit, TMP3, IR::ROUND_MODE_POSITIVE_INFINITY);
csel(ARMEmitter::Size::i64Bit, TMP2, TMP3, ARMEmitter::Reg::zr, ARMEmitter::Condition::CC_EQ);
cmp(TMP1, 2);
LoadConstant(TMP3, IR::ROUND_MODE_NEGATIVE_INFINITY);
csel(TMP2, TMP3, TMP2, vixl::aarch64::Condition::eq);
cmp(ARMEmitter::Size::i64Bit, TMP1, 2);
LoadConstant(ARMEmitter::Size::i64Bit, TMP3, IR::ROUND_MODE_NEGATIVE_INFINITY);
csel(ARMEmitter::Size::i64Bit, TMP2, TMP3, TMP2, ARMEmitter::Condition::CC_EQ);
cmp(TMP1, 3);
LoadConstant(TMP3, IR::ROUND_MODE_TOWARDS_ZERO);
csel(TMP2, TMP3, TMP2, vixl::aarch64::Condition::eq);
cmp(ARMEmitter::Size::i64Bit, TMP1, 3);
LoadConstant(ARMEmitter::Size::i64Bit, TMP3, IR::ROUND_MODE_TOWARDS_ZERO);
csel(ARMEmitter::Size::i64Bit, TMP2, TMP3, TMP2, ARMEmitter::Condition::CC_EQ);
orr(Dst, Dst, TMP2);
orr(ARMEmitter::Size::i64Bit, Dst, Dst, TMP2.R());
bfi(Dst, TMP2, 0, 2);
bfi(ARMEmitter::Size::i64Bit, Dst, TMP2, 0, 2);
}
DEF_OP(SetRoundingMode) {
auto Op = IROp->C<IR::IROp_SetRoundingMode>();
auto Src = GetReg<RA_64>(Op->RoundMode.ID());
auto Src = GetReg(Op->RoundMode.ID());
// Setup the rounding flags correctly
and_(TMP1, Src, 0b11);
and_(ARMEmitter::Size::i64Bit, TMP1, Src, 0b11);
cmp(TMP1, IR::ROUND_MODE_POSITIVE_INFINITY);
LoadConstant(TMP3, 1);
csel(TMP2, TMP3, xzr, vixl::aarch64::Condition::eq);
cmp(ARMEmitter::Size::i64Bit, TMP1, IR::ROUND_MODE_POSITIVE_INFINITY);
LoadConstant(ARMEmitter::Size::i64Bit, TMP3, 1);
csel(ARMEmitter::Size::i64Bit, TMP2, TMP3, ARMEmitter::Reg::zr, ARMEmitter::Condition::CC_EQ);
cmp(TMP1, IR::ROUND_MODE_NEGATIVE_INFINITY);
LoadConstant(TMP3, 2);
csel(TMP2, TMP3, TMP2, vixl::aarch64::Condition::eq);
cmp(ARMEmitter::Size::i64Bit, TMP1, IR::ROUND_MODE_NEGATIVE_INFINITY);
LoadConstant(ARMEmitter::Size::i64Bit, TMP3, 2);
csel(ARMEmitter::Size::i64Bit, TMP2, TMP3, TMP2, ARMEmitter::Condition::CC_EQ);
cmp(TMP1, IR::ROUND_MODE_TOWARDS_ZERO);
LoadConstant(TMP3, 3);
csel(TMP2, TMP3, TMP2, vixl::aarch64::Condition::eq);
cmp(ARMEmitter::Size::i64Bit, TMP1, IR::ROUND_MODE_TOWARDS_ZERO);
LoadConstant(ARMEmitter::Size::i64Bit, TMP3, 3);
csel(ARMEmitter::Size::i64Bit, TMP2, TMP3, TMP2, ARMEmitter::Condition::CC_EQ);
mrs(TMP1, FPCR);
mrs(TMP1, ARMEmitter::SystemRegister::FPCR);
// vixl simulator doesn't support anything beyond ties-to-even rounding
#ifndef VIXL_SIMULATOR
// Insert the rounding flags
bfi(TMP1, TMP2, 22, 2);
bfi(ARMEmitter::Size::i64Bit, TMP1, TMP2, 22, 2);
#endif
// Insert the FTZ flag
lsr(TMP2, Src, 2);
bfi(TMP1, TMP2, 24, 1);
lsr(ARMEmitter::Size::i64Bit, TMP2, Src, 2);
bfi(ARMEmitter::Size::i64Bit, TMP1, TMP2, 24, 1);
// Now save the new FPCR
msr(FPCR, TMP1);
msr(ARMEmitter::SystemRegister::FPCR, TMP1);
}
DEF_OP(Print) {
auto Op = IROp->C<IR::IROp_Print>();
PushDynamicRegsAndLR();
PushDynamicRegsAndLR(TMP1);
SpillStaticRegs();
if (IsGPR(Op->Value.ID())) {
mov(x0, GetReg<RA_64>(Op->Value.ID()));
ldr(x3, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.PrintValue)));
mov(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r0, GetReg(Op->Value.ID()));
ldr(ARMEmitter::XReg::x3, STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.PrintValue));
}
else {
fmov(x0, GetSrc(Op->Value.ID()).V1D());
// Bug in vixl that source vector needs to b V1D rather than V2D?
fmov(x1, GetSrc(Op->Value.ID()).V1D(), 1);
ldr(x3, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.PrintVectorValue)));
fmov(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r0, GetVReg(Op->Value.ID()), false);
fmov(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r1, GetVReg(Op->Value.ID()), true);
ldr(ARMEmitter::XReg::x3, STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.PrintVectorValue));
}
blr(x3);
blr(ARMEmitter::Reg::r3);
FillStaticRegs();
PopDynamicRegsAndLR();
@@ -173,27 +171,27 @@ DEF_OP(ProcessorID) {
// 16bit LoadConstant to be a single instruction
// We must always spill at least one register (x8) so this value always has a bit set
// This gives the signal handler a value to check to see if we are in a syscall at all
LoadConstant(x0, SpillMask & 0xFFFF);
str(x0, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, InSyscallInfo)));
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r0, SpillMask & 0xFFFF);
str(ARMEmitter::XReg::x0, STATE, offsetof(FEXCore::Core::CpuStateFrame, InSyscallInfo));
// Allocate some temporary space for storing the uint32_t CPU and Node IDs
sub(sp, sp, 16);
sub(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::rsp, ARMEmitter::Reg::rsp, 16);
// Load the getcpu syscall number
LoadConstant(x8, SYS_getcpu);
LoadConstant(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r8, SYS_getcpu);
// CPU pointer in x0
add(x0, sp, 0);
add(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r0, ARMEmitter::Reg::rsp, 0);
// Node in x1
add(x1, sp, 4);
add(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::r1, ARMEmitter::Reg::rsp, 4);
svc(0);
// On updated signal mask we can receive a signal RIGHT HERE
// Load the values returned by the kernel
ldp(w0, w1, MemOperand(sp));
ldp<ARMEmitter::IndexType::OFFSET>(ARMEmitter::WReg::w0, ARMEmitter::WReg::w1, ARMEmitter::Reg::rsp);
// Deallocate stack space
sub(sp, sp, 16);
sub(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::rsp, ARMEmitter::Reg::rsp, 16);
// Now that we are done in the syscall we need to carefully peel back the state
// First unspill the registers from before
@@ -201,14 +199,13 @@ DEF_OP(ProcessorID) {
// Now the registers we've spilled are back in their original host registers
// We can safely claim we are no longer in a syscall
str(xzr, MemOperand(STATE, offsetof(FEXCore::Core::CpuStateFrame, InSyscallInfo)));
str(ARMEmitter::XReg::zr, STATE, offsetof(FEXCore::Core::CpuStateFrame, InSyscallInfo));
// Now store the result in the destination in the expected format
// uint32_t Res = (node << 12) | cpu;
// CPU is in w0
// Node is in w1
orr(GetReg<RA_64>(Node), x0, Operand(x1, LSL, 12));
orr(ARMEmitter::Size::i64Bit, GetReg(Node), ARMEmitter::Reg::r0, ARMEmitter::Reg::r1, ARMEmitter::ShiftType::LSL, 12);
}
DEF_OP(RDRAND) {
@@ -216,21 +213,21 @@ DEF_OP(RDRAND) {
// Results are in x0, x1
// Results want to be in a i64v2 vector
auto Dst = GetSrcPair<RA_64>(Node);
auto Dst = GetRegPair(Node);
if (Op->GetReseeded) {
mrs(Dst.first, RNDRRS);
mrs(Dst.first, ARMEmitter::SystemRegister::RNDRRS);
}
else {
mrs(Dst.first, RNDR);
mrs(Dst.first, ARMEmitter::SystemRegister::RNDR);
}
// If the rng number is valid then NZCV is 0b0000, otherwise NZCV is 0b0100
cset(Dst.second, Condition::ne);
cset(ARMEmitter::Size::i64Bit, Dst.second, ARMEmitter::Condition::CC_NE);
}
DEF_OP(Yield) {
hint(SystemHint::YIELD);
yield();
}
#undef DEF_OP
+21 -48
View File
@@ -7,64 +7,37 @@ $end_info$
#include "Interface/Core/JIT/Arm64/JITClass.h"
namespace FEXCore::CPU {
using namespace vixl;
using namespace vixl::aarch64;
#define DEF_OP(x) void Arm64JITCore::Op_##x(IR::IROp_Header const *IROp, IR::NodeID Node)
DEF_OP(ExtractElementPair) {
auto Op = IROp->C<IR::IROp_ExtractElementPair>();
switch (Op->Header.Size) {
case 4: {
auto Src = GetSrcPair<RA_32>(Op->Pair.ID());
std::array<aarch64::Register, 2> Regs = {Src.first, Src.second};
mov (GetReg<RA_32>(Node), Regs[Op->Element]);
break;
}
case 8: {
auto Src = GetSrcPair<RA_64>(Op->Pair.ID());
std::array<aarch64::Register, 2> Regs = {Src.first, Src.second};
mov (GetReg<RA_64>(Node), Regs[Op->Element]);
break;
}
default: LOGMAN_MSG_A_FMT("Unknown Size"); break;
}
LOGMAN_THROW_AA_FMT(Op->Header.Size == 4 || Op->Header.Size == 8, "Invalid size");
const auto EmitSize = Op->Header.Size == 8 ? ARMEmitter::Size::i64Bit : ARMEmitter::Size::i32Bit;
const auto Src = GetRegPair(Op->Pair.ID());
const std::array<ARMEmitter::Register, 2> Regs = {Src.first, Src.second};
mov(EmitSize, GetReg(Node), Regs[Op->Element]);
}
DEF_OP(CreateElementPair) {
auto Op = IROp->C<IR::IROp_CreateElementPair>();
std::pair<aarch64::Register, aarch64::Register> Dst;
aarch64::Register RegFirst;
aarch64::Register RegSecond;
aarch64::Register RegTmp;
LOGMAN_THROW_AA_FMT(IROp->ElementSize == 4 || IROp->ElementSize == 8, "Invalid size");
std::pair<ARMEmitter::Register, ARMEmitter::Register> Dst = GetRegPair(Node);
ARMEmitter::Register RegFirst = GetReg(Op->Lower.ID());
ARMEmitter::Register RegSecond = GetReg(Op->Upper.ID());
ARMEmitter::Register RegTmp = TMP1.R();
switch (IROp->ElementSize) {
case 4: {
Dst = GetSrcPair<RA_32>(Node);
RegFirst = GetReg<RA_32>(Op->Lower.ID());
RegSecond = GetReg<RA_32>(Op->Upper.ID());
RegTmp = w0;
break;
}
case 8: {
Dst = GetSrcPair<RA_64>(Node);
RegFirst = GetReg<RA_64>(Op->Lower.ID());
RegSecond = GetReg<RA_64>(Op->Upper.ID());
RegTmp = x0;
break;
}
default: LOGMAN_MSG_A_FMT("Unknown Size"); break;
}
const auto EmitSize = IROp->ElementSize == 8 ? ARMEmitter::Size::i64Bit : ARMEmitter::Size::i32Bit;
if (Dst.first.GetCode() != RegSecond.GetCode()) {
mov(Dst.first, RegFirst);
mov(Dst.second, RegSecond);
} else if (Dst.second.GetCode() != RegFirst.GetCode()) {
mov(Dst.second, RegSecond);
mov(Dst.first, RegFirst);
if (Dst.first.Idx() != RegSecond.Idx()) {
mov(EmitSize, Dst.first, RegFirst);
mov(EmitSize, Dst.second, RegSecond);
} else if (Dst.second.Idx() != RegFirst.Idx()) {
mov(EmitSize, Dst.second, RegSecond);
mov(EmitSize, Dst.first, RegFirst);
} else {
mov(RegTmp, RegFirst);
mov(Dst.second, RegSecond);
mov(Dst.first, RegTmp);
mov(EmitSize, RegTmp, RegFirst);
mov(EmitSize, Dst.second, RegSecond);
mov(EmitSize, Dst.first, RegTmp);
}
}
File diff suppressed because it is too large. Load diff
@@ -186,8 +186,8 @@ DEF_OP(Vector_SToF) {
pextrq(rcx, Vector, 0);
cvtsi2sd(Dst, rcx);
cvtsi2sd(xmm15, rax);
vmovlhps(Dst, Dst, xmm15);
if (Is256Bit) {
movlhps(Dst, xmm15);
vextracti128(xmm15, ToYMM(Vector), 1);
pextrq(rax, xmm15, 1);
@@ -197,6 +197,8 @@ DEF_OP(Vector_SToF) {
movlhps(xmm15, xmm14);
vinserti128(ToYMM(Dst), ToYMM(Dst), xmm15, 1);
} else {
vmovlhps(Dst, Dst, xmm15);
}
break;
default:
@@ -433,6 +433,7 @@ DEF_OP(VAddP) {
const auto Op = IROp->C<IR::IROp_VAddP>();
const auto OpSize = IROp->Size;
const auto Is256Bit = OpSize == Core::CPUState::XMM_AVX_REG_SIZE;
const auto ElementSize = Op->Header.ElementSize;
const auto Dst = GetDst(Node);
@@ -478,30 +479,64 @@ DEF_OP(VAddP) {
const auto VectorLowerYMM = ToYMM(VectorLower);
const auto VectorUpperYMM = ToYMM(VectorUpper);
// To behave like ADDP, we need to swap the second and third elements around
// in the 256-bit case. ADDP operates as if both vectors are concatenated
// together and runs down the length of it adding pairs as it goes, whereas
// VPHADDW/D operates on both individual halves of the entire register.
switch (ElementSize) {
case 1:
vmovdqu(ymm15, VectorLowerYMM);
vmovdqu(ymm14, VectorUpperYMM);
if (Is256Bit) {
vmovdqu(ymm15, VectorLowerYMM);
vmovdqu(ymm14, VectorUpperYMM);
vpunpcklbw(ymm0, ymm15, ymm14);
vpunpckhbw(ymm12, ymm15, ymm14);
vpunpcklbw(ymm0, ymm15, ymm14);
vpunpckhbw(ymm12, ymm15, ymm14);
vpunpcklbw(ymm15, ymm0, ymm12);
vpunpckhbw(ymm14, ymm0, ymm12);
vpunpcklbw(ymm15, ymm0, ymm12);
vpunpckhbw(ymm14, ymm0, ymm12);
vpunpcklbw(ymm0, ymm15, ymm14);
vpunpckhbw(ymm12, ymm15, ymm14);
vpunpcklbw(ymm0, ymm15, ymm14);
vpunpckhbw(ymm12, ymm15, ymm14);
vpunpcklbw(ymm15, ymm0, ymm12);
vpunpckhbw(ymm14, ymm0, ymm12);
vpunpcklbw(ymm15, ymm0, ymm12);
vpunpckhbw(ymm14, ymm0, ymm12);
vpaddb(DstYMM, ymm15, ymm14);
vpaddb(DstYMM, ymm15, ymm14);
vpermq(DstYMM, DstYMM, 0b11'01'10'00);
} else {
vmovdqu(xmm15, VectorLower);
vmovdqu(xmm14, VectorUpper);
vpunpcklbw(xmm0, xmm15, xmm14);
vpunpckhbw(xmm12, xmm15, xmm14);
vpunpcklbw(xmm15, xmm0, xmm12);
vpunpckhbw(xmm14, xmm0, xmm12);
vpunpcklbw(xmm0, xmm15, xmm14);
vpunpckhbw(xmm12, xmm15, xmm14);
vpunpcklbw(xmm15, xmm0, xmm12);
vpunpckhbw(xmm14, xmm0, xmm12);
vpaddb(Dst, xmm15, xmm14);
}
break;
case 2:
vphaddw(DstYMM, VectorLowerYMM, VectorUpperYMM);
if (Is256Bit) {
vphaddw(DstYMM, VectorLowerYMM, VectorUpperYMM);
vpermq(DstYMM, DstYMM, 0b11'01'10'00);
} else {
vphaddw(Dst, VectorLower, VectorUpper);
}
break;
case 4:
vphaddd(DstYMM, VectorLowerYMM, VectorUpperYMM);
if (Is256Bit) {
vphaddd(DstYMM, VectorLowerYMM, VectorUpperYMM);
vpermq(DstYMM, DstYMM, 0b11'01'10'00);
} else {
vphaddd(Dst, VectorLower, VectorUpper);
}
break;
default:
LOGMAN_MSG_A_FMT("Unknown Element Size: {}", ElementSize);
@@ -811,10 +846,15 @@ DEF_OP(VFAddP) {
const auto VectorLower = GetSrc(Op->VectorLower.ID());
const auto VectorUpper = GetSrc(Op->VectorUpper.ID());
// To behave like FADDP, we need to swap the second and third elements around
// in the 256-bit case. FADDP operates as if both vectors are concatenated
// together and runs down the length of it adding pairs as it goes, whereas
// VHADDPS operates on both individual halves of the entire register.
switch (ElementSize) {
case 4:
if (Is256Bit) {
vhaddpd(ToYMM(Dst), ToYMM(VectorLower), ToYMM(VectorUpper));
vhaddps(ToYMM(Dst), ToYMM(VectorLower), ToYMM(VectorUpper));
vpermq(ToYMM(Dst), ToYMM(Dst), 0b11'01'10'00);
} else {
vhaddps(Dst, VectorLower, VectorUpper);
}
@@ -822,6 +862,7 @@ DEF_OP(VFAddP) {
case 8:
if (Is256Bit) {
vhaddpd(ToYMM(Dst), ToYMM(VectorLower), ToYMM(VectorUpper));
vpermq(ToYMM(Dst), ToYMM(Dst), 0b11'01'10'00);
} else {
vhaddpd(Dst, VectorLower, VectorUpper);
}
@@ -1709,7 +1750,36 @@ DEF_OP(VUnZip) {
const auto VectorUpper = GetSrc(Op->VectorUpper.ID());
if (OpSize == 8) {
LOGMAN_MSG_A_FMT("Unsupported register size on VUnZip");
switch (ElementSize) {
case 1: {
mov(rax, 0x80'80'80'80'06'04'02'00); // Lower
mov(rcx, 0x80'80'80'80'80'80'80'80); // Upper
vmovq(xmm15, rax);
pinsrq(xmm15, rcx, 1);
vpshufb(xmm14, VectorLower, xmm15);
vpshufb(xmm13, VectorUpper, xmm15);
vpunpckldq(Dst, xmm14, xmm13);
break;
}
case 2: {
mov(rax, 0x80'80'80'80'05'04'01'00); // Lower
mov(rcx, 0x80'80'80'80'80'80'80'80); // Upper
vmovq(xmm15, rax);
pinsrq(xmm15, rcx, 1);
vpshufb(xmm14, VectorLower, xmm15);
vpshufb(xmm13, VectorUpper, xmm15);
vpunpckldq(Dst, xmm14, xmm13);
break;
}
case 4: {
vshufps(Dst, VectorLower, VectorUpper, 0b10'00'10'00);
vmovq(Dst, Dst);
break;
}
default:
LOGMAN_MSG_A_FMT("Unknown Element Size: {}", ElementSize);
break;
}
}
else {
switch (ElementSize) {
@@ -1724,6 +1794,7 @@ DEF_OP(VUnZip) {
vpshufb(ymm14, ToYMM(VectorLower), ymm15);
vpshufb(ymm13, ToYMM(VectorUpper), ymm15);
vpunpcklqdq(ToYMM(Dst), ymm14, ymm13);
vpermq(ToYMM(Dst), ToYMM(Dst), 0b11'01'10'00);
} else {
vpshufb(xmm14, VectorLower, xmm15);
vpshufb(xmm13, VectorUpper, xmm15);
@@ -1743,6 +1814,7 @@ DEF_OP(VUnZip) {
vpshufb(ymm14, ToYMM(VectorLower), ymm15);
vpshufb(ymm13, ToYMM(VectorUpper), ymm15);
vpunpcklqdq(ToYMM(Dst), ymm14, ymm13);
vpermq(ToYMM(Dst), ToYMM(Dst), 0b11'01'10'00);
} else {
vpshufb(xmm14, VectorLower, xmm15);
vpshufb(xmm13, VectorUpper, xmm15);
@@ -1754,6 +1826,7 @@ DEF_OP(VUnZip) {
case 4: {
if (Is256Bit) {
vshufps(ToYMM(Dst), ToYMM(VectorLower), ToYMM(VectorUpper), 0b10'00'10'00);
vpermq(ToYMM(Dst), ToYMM(Dst), 0b11'01'10'00);
} else {
vshufps(Dst, VectorLower, VectorUpper, 0b10'00'10'00);
}
@@ -1762,6 +1835,7 @@ DEF_OP(VUnZip) {
case 8: {
if (Is256Bit) {
vshufpd(ToYMM(Dst), ToYMM(VectorLower), ToYMM(VectorUpper), 0b0'0);
vpermq(ToYMM(Dst), ToYMM(Dst), 0b11'01'10'00);
} else {
vshufpd(Dst, VectorLower, VectorUpper, 0b0'0);
}
@@ -1786,7 +1860,36 @@ DEF_OP(VUnZip2) {
const auto VectorUpper = GetSrc(Op->VectorUpper.ID());
if (OpSize == 8) {
LOGMAN_MSG_A_FMT("Unsupported register size on VUnZip2");
switch (ElementSize) {
case 1: {
mov(rax, 0x80'80'80'80'07'05'03'01); // Lower
mov(rcx, 0x80'80'80'80'80'80'80'80); // Upper
vmovq(xmm15, rax);
pinsrq(xmm15, rcx, 1);
vpshufb(xmm14, VectorLower, xmm15);
vpshufb(xmm13, VectorUpper, xmm15);
vpunpckldq(Dst, xmm14, xmm13);
break;
}
case 2: {
mov(rax, 0x80'80'80'80'07'06'03'02); // Lower
mov(rcx, 0x80'80'80'80'80'80'80'80); // Upper
vmovq(xmm15, rax);
pinsrq(xmm15, rcx, 1);
vpshufb(xmm14, VectorLower, xmm15);
vpshufb(xmm13, VectorUpper, xmm15);
vpunpckldq(Dst, xmm14, xmm13);
break;
}
case 4: {
vshufps(Dst, VectorLower, VectorUpper, 0b11'01'11'01);
vmovq(Dst, Dst);
break;
}
default:
LOGMAN_MSG_A_FMT("Unknown Element Size: {}", ElementSize);
break;
}
}
else {
switch (ElementSize) {
@@ -1801,6 +1904,7 @@ DEF_OP(VUnZip2) {
vpshufb(ymm14, ToYMM(VectorLower), ymm15);
vpshufb(ymm13, ToYMM(VectorUpper), ymm15);
vpunpcklqdq(ToYMM(Dst), ymm14, ymm13);
vpermq(ToYMM(Dst), ToYMM(Dst), 0b11'01'10'00);
} else {
vpshufb(xmm14, VectorLower, xmm15);
vpshufb(xmm13, VectorUpper, xmm15);
@@ -1820,6 +1924,7 @@ DEF_OP(VUnZip2) {
vpshufb(ymm14, ToYMM(VectorLower), ymm15);
vpshufb(ymm13, ToYMM(VectorUpper), ymm15);
vpunpcklqdq(ToYMM(Dst), ymm14, ymm13);
vpermq(ToYMM(Dst), ToYMM(Dst), 0b11'01'10'00);
} else {
vpshufb(xmm14, VectorLower, xmm15);
vpshufb(xmm13, VectorUpper, xmm15);
@@ -1831,6 +1936,7 @@ DEF_OP(VUnZip2) {
case 4: {
if (Is256Bit) {
vshufps(ToYMM(Dst), ToYMM(VectorLower), ToYMM(VectorUpper), 0b11'01'11'01);
vpermq(ToYMM(Dst), ToYMM(Dst), 0b11'01'10'00);
} else {
vshufps(Dst, VectorLower, VectorUpper, 0b11'01'11'01);
}
@@ -1839,6 +1945,7 @@ DEF_OP(VUnZip2) {
case 8: {
if (Is256Bit) {
vshufpd(ToYMM(Dst), ToYMM(VectorLower), ToYMM(VectorUpper), 0b1'1);
vpermq(ToYMM(Dst), ToYMM(Dst), 0b11'01'10'00);
} else {
vshufpd(Dst, VectorLower, VectorUpper, 0b1'1);
}
@@ -2679,19 +2786,26 @@ DEF_OP(VInsElement) {
}
};
const auto SrcReg = GetSrcVector(xmm14);
const auto DstReg = GetDstVector(xmm15);
const auto SanitizedDstIdx = SanitizeIndex(DestIdx, DstIsUpper);
const auto SanitizedSrcIdx = SanitizeIndex(SrcIdx, SrcIsUpper);
PerformInsertion(SrcReg, SanitizedSrcIdx, DstReg, SanitizedDstIdx);
vmovapd(ToYMM(Dst), ToYMM(DestVector));
if (DstIsUpper) {
vinserti128(ToYMM(Dst), ToYMM(Dst), DstReg, 1);
const auto Is128BitElement = ElementSize == Core::CPUState::XMM_SSE_REG_SIZE;
if (Is128BitElement) {
vextracti128(xmm14, ToYMM(SrcVector), SrcIdx);
vmovapd(ToYMM(Dst), ToYMM(DestVector));
vinserti128(ToYMM(Dst), ToYMM(Dst), xmm14, DestIdx);
} else {
vinserti128(ToYMM(Dst), ToYMM(Dst), DstReg, 0);
const auto SrcReg = GetSrcVector(xmm14);
const auto DstReg = GetDstVector(xmm15);
const auto SanitizedDstIdx = SanitizeIndex(DestIdx, DstIsUpper);
const auto SanitizedSrcIdx = SanitizeIndex(SrcIdx, SrcIsUpper);
PerformInsertion(SrcReg, SanitizedSrcIdx, DstReg, SanitizedDstIdx);
vmovapd(ToYMM(Dst), ToYMM(DestVector));
if (DstIsUpper) {
vinserti128(ToYMM(Dst), ToYMM(Dst), DstReg, 1);
} else {
vinserti128(ToYMM(Dst), ToYMM(Dst), DstReg, 0);
}
}
} else {
vmovapd(xmm15, DestVector);
+20 -14
View File
@@ -17,6 +17,10 @@ namespace FEXCore {
LookupCache::LookupCache(FEXCore::Context::Context *CTX)
: ctx {CTX} {
TotalCacheSize = ctx->Config.VirtualMemSize / 4096 * 8 + CODE_SIZE + L1_SIZE;
// Setup our PMR map.
BlockLinks = BlockLinks_pma.new_object<BlockLinksMapType>();
// Block cache ends up looking like this
// PageMemoryMap[VirtualMemoryRegion >> 12]
// |
@@ -29,46 +33,48 @@ LookupCache::LookupCache(FEXCore::Context::Context *CTX)
// Allocate a region of memory that we can use to back our block pointers
// We need one pointer per page of virtual memory
// At 64GB of virtual memory this will allocate 128MB of virtual memory space
PagePointer = reinterpret_cast<uintptr_t>(FEXCore::Allocator::mmap(nullptr, ctx->Config.VirtualMemSize / 4096 * 8, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0));
PagePointer = reinterpret_cast<uintptr_t>(FEXCore::Allocator::mmap(nullptr, TotalCacheSize, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0));
// Allocate our memory backing our pages
// We need 32KB per guest page (One pointer per byte)
// XXX: We can drop down to 16KB if we store 4byte offsets from the code base
// We currently limit to 128MB of real memory for caching for the total cache size.
// Can end up being inefficient if we compile a small number of blocks per page
PageMemory = reinterpret_cast<uintptr_t>(FEXCore::Allocator::mmap(nullptr, CODE_SIZE, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0));
PageMemory = PagePointer + ctx->Config.VirtualMemSize / 4096 * 8;
LOGMAN_THROW_AA_FMT(PageMemory != -1ULL, "Failed to allocate page memory");
// L1 Cache
L1Pointer = reinterpret_cast<uintptr_t>(FEXCore::Allocator::mmap(nullptr, L1_SIZE, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0));
L1Pointer = PageMemory + CODE_SIZE;
LOGMAN_THROW_AA_FMT(L1Pointer != -1ULL, "Failed to allocate L1Pointer");
VirtualMemSize = ctx->Config.VirtualMemSize;
}
LookupCache::~LookupCache() {
FEXCore::Allocator::munmap(reinterpret_cast<void*>(PagePointer), ctx->Config.VirtualMemSize / 4096 * 8);
FEXCore::Allocator::munmap(reinterpret_cast<void*>(PageMemory), CODE_SIZE);
FEXCore::Allocator::munmap(reinterpret_cast<void*>(L1Pointer), L1_SIZE);
const size_t TotalCacheSize = ctx->Config.VirtualMemSize / 4096 * 8 + CODE_SIZE + L1_SIZE;
FEXCore::Allocator::munmap(reinterpret_cast<void*>(PagePointer), TotalCacheSize);
// No need to free BlockLinks map.
// These will get freed when their memory allocators are deallocated.
}
void LookupCache::ClearL2Cache() {
std::lock_guard<std::recursive_mutex> lk(WriteLock);
// Clear out the page memory
madvise(reinterpret_cast<void*>(PagePointer), ctx->Config.VirtualMemSize / 4096 * 8, MADV_DONTNEED);
madvise(reinterpret_cast<void*>(PageMemory), CODE_SIZE, MADV_DONTNEED);
// PagePointer and PageMemory are sequential with each other. Clear both at once.
madvise(reinterpret_cast<void*>(PagePointer), ctx->Config.VirtualMemSize / 4096 * 8 + CODE_SIZE, MADV_DONTNEED);
AllocateOffset = 0;
}
void LookupCache::ClearCache() {
std::lock_guard<std::recursive_mutex> lk(WriteLock);
// Clear L1
madvise(reinterpret_cast<void*>(L1Pointer), L1_SIZE, MADV_DONTNEED);
// Clear L2
ClearL2Cache();
// All code is gone, remove links
BlockLinks.clear();
// Clear L1 and L2 by clearing the full cache.
madvise(reinterpret_cast<void*>(PagePointer), TotalCacheSize, MADV_DONTNEED);
// Clear the BlockLinks allocator which frees the BlockLinks map implicitly.
BlockLinks_mbr.release();
// Allocate a new pointer from the BlockLinks pma again.
BlockLinks = BlockLinks_pma.new_object<BlockLinksMapType>();
// All code is gone, clear the block list
BlockList.clear();
}
+17 -5
View File
@@ -4,6 +4,7 @@
#include <cstdint>
#include <functional>
#include <map>
#include <memory_resource>
#include <stddef.h>
#include <utility>
#include <vector>
@@ -105,9 +106,9 @@ public:
std::lock_guard<std::recursive_mutex> lk(WriteLock);
// Sever any links to this block
auto lower = BlockLinks.lower_bound({Address, 0});
auto upper = BlockLinks.upper_bound({Address, UINTPTR_MAX});
for (auto it = lower; it != upper; it = BlockLinks.erase(it)) {
auto lower = BlockLinks->lower_bound({Address, 0});
auto upper = BlockLinks->upper_bound({Address, UINTPTR_MAX});
for (auto it = lower; it != upper; it = BlockLinks->erase(it)) {
it->second();
}
@@ -145,7 +146,7 @@ public:
void AddBlockLink(uint64_t GuestDestination, uintptr_t HostLink, const std::function<void()> &delinker) {
std::lock_guard<std::recursive_mutex> lk(WriteLock);
BlockLinks.insert({{GuestDestination, HostLink}, delinker});
BlockLinks->insert({{GuestDestination, HostLink}, delinker});
}
void ClearCache();
@@ -238,10 +239,21 @@ private:
}
};
// Use a monotonic buffer resource to allocate both the std::pmr::map and its members.
// This allows us to quickly clear the block link map by clearing the monotonic allocator.
// If we had allocated the block link map without the MBR, then clearing the map would require slowly
// walking each block member and destructing objects.
//
// This makes `BlockLinks` look like a raw pointer that could memory leak, but since it is backed by the MBR, it won't.
std::pmr::monotonic_buffer_resource BlockLinks_mbr;
using BlockLinksMapType = std::pmr::map<BlockLinkTag, std::function<void()>>;
std::pmr::polymorphic_allocator<std::byte> BlockLinks_pma {&BlockLinks_mbr};
BlockLinksMapType *BlockLinks;
std::map<BlockLinkTag, std::function<void()>> BlockLinks;
tsl::robin_map<uint64_t, uint64_t> BlockList;
size_t TotalCacheSize;
constexpr static size_t CODE_SIZE = 128 * 1024 * 1024;
constexpr static size_t SIZE_PER_PAGE = 4096 * sizeof(LookupCacheEntry);
constexpr static size_t L1_SIZE = L1_ENTRIES * sizeof(LookupCacheEntry);
+290 -54
View File
@@ -783,8 +783,17 @@ void OpDispatchBuilder::CALLOp(OpcodeArgs) {
_StoreMem(GPRClass, GPRSize, NewSP, ConstantPCReturn, GPRSize);
// Store the RIP
_ExitFunction(NewRIP); // If we get here then leave the function now
const uint64_t NextRIP = Op->PC + Op->InstSize;
LOGMAN_THROW_A_FMT(Op->Src[0].IsLiteral(), "Had wrong operand type");
const uint64_t TargetRIP = Op->PC + Op->InstSize + Op->Src[0].Data.Literal.Value;
if (NextRIP != TargetRIP) {
// Store the RIP
_ExitFunction(NewRIP); // If we get here then leave the function now
}
else {
NeedsBlockEnd = true;
}
}
void OpDispatchBuilder::CALLAbsoluteOp(OpcodeArgs) {
@@ -4971,12 +4980,6 @@ OrderedNode *OpDispatchBuilder::LoadSource_WithOpSize(FEXCore::IR::RegisterClass
if (OpSize < Core::CPUState::XMM_SSE_REG_SIZE) {
Src = _VMov(OpSize, Src);
}
// OpSize of 16 is special in that it is expected to zero the upper bits of the 256-bit operation.
// TODO: Longer term we should enforce the difference between zero and insert.
if (regSize == Core::CPUState::XMM_AVX_REG_SIZE && OpSize == Core::CPUState::XMM_SSE_REG_SIZE) {
Src = _VMov(OpSize, Src);
}
}
else {
Src = _LoadRegister(false, offsetof(FEXCore::Core::CPUState, gregs[gpr]) + (highIndex ? 1 : 0), GPRClass, GPRFixedClass, OpSize);
@@ -5813,6 +5816,12 @@ void OpDispatchBuilder::InstallHostSpecificOpcodeHandlers() {
{OPD(1, 0b00, 0x13), 1, &OpDispatchBuilder::VMOVLPOp},
{OPD(1, 0b01, 0x13), 1, &OpDispatchBuilder::VMOVLPOp},
{OPD(1, 0b00, 0x14), 1, &OpDispatchBuilder::VPUNPCKLOp<4>},
{OPD(1, 0b01, 0x14), 1, &OpDispatchBuilder::VPUNPCKLOp<8>},
{OPD(1, 0b00, 0x15), 1, &OpDispatchBuilder::VPUNPCKHOp<4>},
{OPD(1, 0b01, 0x15), 1, &OpDispatchBuilder::VPUNPCKHOp<8>},
{OPD(1, 0b00, 0x16), 1, &OpDispatchBuilder::VMOVHPOp},
{OPD(1, 0b01, 0x16), 1, &OpDispatchBuilder::VMOVHPOp},
{OPD(1, 0b10, 0x16), 1, &OpDispatchBuilder::VMOVSHDUPOp},
@@ -5827,6 +5836,31 @@ void OpDispatchBuilder::InstallHostSpecificOpcodeHandlers() {
{OPD(1, 0b00, 0x2B), 1, &OpDispatchBuilder::VMOVVectorNTOp},
{OPD(1, 0b01, 0x2B), 1, &OpDispatchBuilder::VMOVVectorNTOp},
{OPD(1, 0b10, 0x2C), 1, &OpDispatchBuilder::CVTFPR_To_GPR<4, false>},
{OPD(1, 0b11, 0x2C), 1, &OpDispatchBuilder::CVTFPR_To_GPR<8, false>},
{OPD(1, 0b10, 0x2D), 1, &OpDispatchBuilder::CVTFPR_To_GPR<4, true>},
{OPD(1, 0b11, 0x2D), 1, &OpDispatchBuilder::CVTFPR_To_GPR<8, true>},
{OPD(1, 0b00, 0x2E), 1, &OpDispatchBuilder::UCOMISxOp<4>},
{OPD(1, 0b01, 0x2E), 1, &OpDispatchBuilder::UCOMISxOp<8>},
{OPD(1, 0b00, 0x2F), 1, &OpDispatchBuilder::UCOMISxOp<4>},
{OPD(1, 0b01, 0x2F), 1, &OpDispatchBuilder::UCOMISxOp<8>},
{OPD(1, 0b00, 0x50), 1, &OpDispatchBuilder::MOVMSKOp<4>},
{OPD(1, 0b01, 0x50), 1, &OpDispatchBuilder::MOVMSKOp<8>},
{OPD(1, 0b00, 0x51), 1, &OpDispatchBuilder::AVXVectorUnaryOp<IR::OP_VFSQRT, 4, false>},
{OPD(1, 0b01, 0x51), 1, &OpDispatchBuilder::AVXVectorUnaryOp<IR::OP_VFSQRT, 8, false>},
{OPD(1, 0b10, 0x51), 1, &OpDispatchBuilder::AVXVectorUnaryOp<IR::OP_VFSQRT, 4, true>},
{OPD(1, 0b11, 0x51), 1, &OpDispatchBuilder::AVXVectorUnaryOp<IR::OP_VFSQRT, 8, true>},
{OPD(1, 0b00, 0x52), 1, &OpDispatchBuilder::AVXVectorUnaryOp<IR::OP_VFRSQRT, 4, false>},
{OPD(1, 0b10, 0x52), 1, &OpDispatchBuilder::AVXVectorUnaryOp<IR::OP_VFRSQRT, 4, true>},
{OPD(1, 0b00, 0x53), 1, &OpDispatchBuilder::AVXVectorUnaryOp<IR::OP_VFRECP, 4, false>},
{OPD(1, 0b10, 0x53), 1, &OpDispatchBuilder::AVXVectorUnaryOp<IR::OP_VFRECP, 4, true>},
{OPD(1, 0b00, 0x54), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VAND, 16>},
{OPD(1, 0b01, 0x54), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VAND, 16>},
@@ -5841,45 +5875,246 @@ void OpDispatchBuilder::InstallHostSpecificOpcodeHandlers() {
{OPD(1, 0b00, 0x58), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VFADD, 4>},
{OPD(1, 0b01, 0x58), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VFADD, 8>},
{OPD(1, 0b10, 0x58), 1, &OpDispatchBuilder::AVXVectorScalarALUOp<IR::OP_VFADD, 4>},
{OPD(1, 0b11, 0x58), 1, &OpDispatchBuilder::AVXVectorScalarALUOp<IR::OP_VFADD, 8>},
{OPD(1, 0b00, 0x59), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VFMUL, 4>},
{OPD(1, 0b01, 0x59), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VFMUL, 8>},
{OPD(1, 0b10, 0x59), 1, &OpDispatchBuilder::AVXVectorScalarALUOp<IR::OP_VFMUL, 4>},
{OPD(1, 0b11, 0x59), 1, &OpDispatchBuilder::AVXVectorScalarALUOp<IR::OP_VFMUL, 8>},
{OPD(1, 0b00, 0x5B), 1, &OpDispatchBuilder::AVXVector_CVT_Int_To_Float<4, false>},
{OPD(1, 0b01, 0x5B), 1, &OpDispatchBuilder::AVXVector_CVT_Float_To_Int<4, false, true>},
{OPD(1, 0b10, 0x5B), 1, &OpDispatchBuilder::AVXVector_CVT_Float_To_Int<4, false, false>},
{OPD(1, 0b00, 0x5C), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VFSUB, 4>},
{OPD(1, 0b01, 0x5C), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VFSUB, 8>},
{OPD(1, 0b10, 0x5C), 1, &OpDispatchBuilder::AVXVectorScalarALUOp<IR::OP_VFSUB, 4>},
{OPD(1, 0b11, 0x5C), 1, &OpDispatchBuilder::AVXVectorScalarALUOp<IR::OP_VFSUB, 8>},
{OPD(1, 0b00, 0x5D), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VFMIN, 4>},
{OPD(1, 0b01, 0x5D), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VFMIN, 8>},
{OPD(1, 0b10, 0x5D), 1, &OpDispatchBuilder::AVXVectorScalarALUOp<IR::OP_VFMIN, 4>},
{OPD(1, 0b11, 0x5D), 1, &OpDispatchBuilder::AVXVectorScalarALUOp<IR::OP_VFMIN, 8>},
{OPD(1, 0b00, 0x5E), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VFDIV, 4>},
{OPD(1, 0b01, 0x5E), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VFDIV, 8>},
{OPD(1, 0b10, 0x5E), 1, &OpDispatchBuilder::AVXVectorScalarALUOp<IR::OP_VFDIV, 4>},
{OPD(1, 0b11, 0x5E), 1, &OpDispatchBuilder::AVXVectorScalarALUOp<IR::OP_VFDIV, 8>},
{OPD(1, 0b00, 0x5F), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VFMAX, 4>},
{OPD(1, 0b01, 0x5F), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VFMAX, 8>},
{OPD(1, 0b10, 0x5F), 1, &OpDispatchBuilder::AVXVectorScalarALUOp<IR::OP_VFMAX, 4>},
{OPD(1, 0b11, 0x5F), 1, &OpDispatchBuilder::AVXVectorScalarALUOp<IR::OP_VFMAX, 8>},
{OPD(1, 0b01, 0x60), 1, &OpDispatchBuilder::VPUNPCKLOp<1>},
{OPD(1, 0b01, 0x61), 1, &OpDispatchBuilder::VPUNPCKLOp<2>},
{OPD(1, 0b01, 0x62), 1, &OpDispatchBuilder::VPUNPCKLOp<4>},
{OPD(1, 0b01, 0x63), 1, &OpDispatchBuilder::VPACKSSOp<2>},
{OPD(1, 0b01, 0x64), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VCMPGT, 1>},
{OPD(1, 0b01, 0x65), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VCMPGT, 2>},
{OPD(1, 0b01, 0x66), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VCMPGT, 4>},
{OPD(1, 0b01, 0x67), 1, &OpDispatchBuilder::VPACKUSOp<2>},
{OPD(1, 0b01, 0x68), 1, &OpDispatchBuilder::VPUNPCKHOp<1>},
{OPD(1, 0b01, 0x69), 1, &OpDispatchBuilder::VPUNPCKHOp<2>},
{OPD(1, 0b01, 0x6A), 1, &OpDispatchBuilder::VPUNPCKHOp<4>},
{OPD(1, 0b01, 0x6B), 1, &OpDispatchBuilder::VPACKSSOp<4>},
{OPD(1, 0b01, 0x6C), 1, &OpDispatchBuilder::VPUNPCKLOp<8>},
{OPD(1, 0b01, 0x6D), 1, &OpDispatchBuilder::VPUNPCKHOp<8>},
{OPD(1, 0b01, 0x6E), 1, &OpDispatchBuilder::MOVBetweenGPR_FPR},
{OPD(1, 0b01, 0x6F), 1, &OpDispatchBuilder::VMOVAPS_VMOVAPD_Op},
{OPD(1, 0b10, 0x6F), 1, &OpDispatchBuilder::VMOVUPS_VMOVUPD_Op},
{OPD(1, 0b01, 0x74), 3, &OpDispatchBuilder::UnimplementedOp},
{OPD(1, 0b01, 0x74), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VCMPEQ, 1>},
{OPD(1, 0b01, 0x75), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VCMPEQ, 2>},
{OPD(1, 0b01, 0x76), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VCMPEQ, 4>},
{OPD(1, 0b00, 0x77), 1, &OpDispatchBuilder::VZEROOp},
{OPD(1, 0b01, 0x7C), 1, &OpDispatchBuilder::VHADDPOp<IR::OP_VFADDP, 8>},
{OPD(1, 0b11, 0x7C), 1, &OpDispatchBuilder::VHADDPOp<IR::OP_VFADDP, 4>},
{OPD(1, 0b01, 0x7E), 1, &OpDispatchBuilder::MOVBetweenGPR_FPR},
{OPD(1, 0b10, 0x7E), 1, &OpDispatchBuilder::MOVQOp},
{OPD(1, 0b01, 0x7F), 1, &OpDispatchBuilder::VMOVAPS_VMOVAPD_Op},
{OPD(1, 0b10, 0x7F), 1, &OpDispatchBuilder::VMOVUPS_VMOVUPD_Op},
{OPD(1, 0b00, 0xC2), 1, &OpDispatchBuilder::AVXVFCMPOp<4, false>},
{OPD(1, 0b01, 0xC2), 1, &OpDispatchBuilder::AVXVFCMPOp<8, false>},
{OPD(1, 0b10, 0xC2), 1, &OpDispatchBuilder::AVXVFCMPOp<4, true>},
{OPD(1, 0b11, 0xC2), 1, &OpDispatchBuilder::AVXVFCMPOp<8, true>},
{OPD(1, 0b01, 0xC5), 1, &OpDispatchBuilder::PExtrOp<2>},
{OPD(1, 0b01, 0xD0), 1, &OpDispatchBuilder::VADDSUBPOp<8>},
{OPD(1, 0b11, 0xD0), 1, &OpDispatchBuilder::VADDSUBPOp<4>},
{OPD(1, 0b01, 0xD1), 1, &OpDispatchBuilder::VPSRLDOp<2>},
{OPD(1, 0b01, 0xD2), 1, &OpDispatchBuilder::VPSRLDOp<4>},
{OPD(1, 0b01, 0xD3), 1, &OpDispatchBuilder::VPSRLDOp<8>},
{OPD(1, 0b01, 0xD4), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VADD, 8>},
{OPD(1, 0b01, 0xD5), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VSMUL, 2>},
{OPD(1, 0b01, 0xD6), 1, &OpDispatchBuilder::MOVQOp},
{OPD(1, 0b01, 0xD7), 1, &OpDispatchBuilder::UnimplementedOp},
{OPD(1, 0b01, 0xD8), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VUQSUB, 1>},
{OPD(1, 0b01, 0xD9), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VUQSUB, 2>},
{OPD(1, 0b01, 0xDA), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VUMIN, 1>},
{OPD(1, 0b01, 0xDB), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VAND, 16>},
{OPD(1, 0b01, 0xDC), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VUQADD, 1>},
{OPD(1, 0b01, 0xDD), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VUQADD, 2>},
{OPD(1, 0b01, 0xDE), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VUMAX, 1>},
{OPD(1, 0b01, 0xDF), 1, &OpDispatchBuilder::VANDNOp},
{OPD(1, 0b01, 0xE0), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VURAVG, 1>},
{OPD(1, 0b01, 0xE1), 1, &OpDispatchBuilder::VPSRAOp<2>},
{OPD(1, 0b01, 0xE2), 1, &OpDispatchBuilder::VPSRAOp<4>},
{OPD(1, 0b01, 0xE3), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VURAVG, 2>},
{OPD(1, 0b01, 0xE4), 1, &OpDispatchBuilder::VPMULHWOp<false>},
{OPD(1, 0b01, 0xE5), 1, &OpDispatchBuilder::VPMULHWOp<true>},
{OPD(1, 0b01, 0xE6), 1, &OpDispatchBuilder::AVXVector_CVT_Float_To_Int<8, true, false>},
{OPD(1, 0b10, 0xE6), 1, &OpDispatchBuilder::AVXVector_CVT_Int_To_Float<4, true>},
{OPD(1, 0b11, 0xE6), 1, &OpDispatchBuilder::AVXVector_CVT_Float_To_Int<8, true, true>},
{OPD(1, 0b01, 0xE7), 1, &OpDispatchBuilder::VMOVVectorNTOp},
{OPD(1, 0b01, 0xE8), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VSQSUB, 1>},
{OPD(1, 0b01, 0xE9), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VSQSUB, 2>},
{OPD(1, 0b01, 0xEA), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VSMIN, 2>},
{OPD(1, 0b01, 0xEB), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VOR, 16>},
{OPD(1, 0b01, 0xEC), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VSQADD, 1>},
{OPD(1, 0b01, 0xED), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VSQADD, 2>},
{OPD(1, 0b01, 0xEE), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VSMAX, 2>},
{OPD(1, 0b01, 0xEF), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VXOR, 16>},
{OPD(1, 0b11, 0xF0), 1, &OpDispatchBuilder::VMOVUPS_VMOVUPD_Op},
{OPD(1, 0b01, 0xF1), 1, &OpDispatchBuilder::VPSLLOp<2>},
{OPD(1, 0b01, 0xF2), 1, &OpDispatchBuilder::VPSLLOp<4>},
{OPD(1, 0b01, 0xF3), 1, &OpDispatchBuilder::VPSLLOp<8>},
{OPD(1, 0b01, 0xF4), 1, &OpDispatchBuilder::VPMULLOp<4, false>},
{OPD(1, 0b01, 0xF7), 1, &OpDispatchBuilder::MASKMOVOp},
{OPD(1, 0b01, 0xF8), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VSUB, 1>},
{OPD(1, 0b01, 0xF9), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VSUB, 2>},
{OPD(1, 0b01, 0xFA), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VSUB, 4>},
{OPD(1, 0b01, 0xFB), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VSUB, 8>},
{OPD(1, 0b01, 0xFC), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VADD, 1>},
{OPD(1, 0b01, 0xFD), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VADD, 2>},
{OPD(1, 0b01, 0xFE), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VADD, 4>},
{OPD(2, 0b01, 0x01), 1, &OpDispatchBuilder::VHADDPOp<IR::OP_VADDP, 2>},
{OPD(2, 0b01, 0x02), 1, &OpDispatchBuilder::VHADDPOp<IR::OP_VADDP, 4>},
{OPD(2, 0b01, 0x05), 1, &OpDispatchBuilder::VPHSUBOp<2>},
{OPD(2, 0b01, 0x06), 1, &OpDispatchBuilder::VPHSUBOp<4>},
{OPD(2, 0b01, 0x08), 1, &OpDispatchBuilder::VPSIGN<1>},
{OPD(2, 0b01, 0x09), 1, &OpDispatchBuilder::VPSIGN<2>},
{OPD(2, 0b01, 0x0A), 1, &OpDispatchBuilder::VPSIGN<4>},
{OPD(2, 0b01, 0x0B), 1, &OpDispatchBuilder::VPMULHRSWOp},
{OPD(2, 0b01, 0x18), 1, &OpDispatchBuilder::VBROADCASTOp<4>},
{OPD(2, 0b01, 0x19), 1, &OpDispatchBuilder::VBROADCASTOp<8>},
{OPD(2, 0b01, 0x1A), 1, &OpDispatchBuilder::VBROADCASTOp<16>},
{OPD(2, 0b01, 0x1C), 1, &OpDispatchBuilder::AVXVectorUnaryOp<IR::OP_VABS, 1, false>},
{OPD(2, 0b01, 0x1D), 1, &OpDispatchBuilder::AVXVectorUnaryOp<IR::OP_VABS, 2, false>},
{OPD(2, 0b01, 0x1E), 1, &OpDispatchBuilder::AVXVectorUnaryOp<IR::OP_VABS, 4, false>},
{OPD(2, 0b01, 0x20), 1, &OpDispatchBuilder::AVXExtendVectorElements<1, 2, true>},
{OPD(2, 0b01, 0x21), 1, &OpDispatchBuilder::AVXExtendVectorElements<1, 4, true>},
{OPD(2, 0b01, 0x22), 1, &OpDispatchBuilder::AVXExtendVectorElements<1, 8, true>},
{OPD(2, 0b01, 0x23), 1, &OpDispatchBuilder::AVXExtendVectorElements<2, 4, true>},
{OPD(2, 0b01, 0x24), 1, &OpDispatchBuilder::AVXExtendVectorElements<2, 8, true>},
{OPD(2, 0b01, 0x25), 1, &OpDispatchBuilder::AVXExtendVectorElements<4, 8, true>},
{OPD(2, 0b01, 0x28), 1, &OpDispatchBuilder::VPMULLOp<4, true>},
{OPD(2, 0b01, 0x29), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VCMPEQ, 8>},
{OPD(2, 0b01, 0x2A), 1, &OpDispatchBuilder::VMOVVectorNTOp},
{OPD(2, 0b01, 0x3B), 1, &OpDispatchBuilder::UnimplementedOp},
{OPD(2, 0b01, 0x2B), 1, &OpDispatchBuilder::VPACKUSOp<4>},
{OPD(2, 0b01, 0x58), 3, &OpDispatchBuilder::UnimplementedOp},
{OPD(2, 0b01, 0x30), 1, &OpDispatchBuilder::AVXExtendVectorElements<1, 2, false>},
{OPD(2, 0b01, 0x31), 1, &OpDispatchBuilder::AVXExtendVectorElements<1, 4, false>},
{OPD(2, 0b01, 0x32), 1, &OpDispatchBuilder::AVXExtendVectorElements<1, 8, false>},
{OPD(2, 0b01, 0x33), 1, &OpDispatchBuilder::AVXExtendVectorElements<2, 4, false>},
{OPD(2, 0b01, 0x34), 1, &OpDispatchBuilder::AVXExtendVectorElements<2, 8, false>},
{OPD(2, 0b01, 0x35), 1, &OpDispatchBuilder::AVXExtendVectorElements<4, 8, false>},
{OPD(2, 0b01, 0x78), 1, &OpDispatchBuilder::UnimplementedOp},
{OPD(2, 0b01, 0x79), 1, &OpDispatchBuilder::UnimplementedOp},
{OPD(2, 0b01, 0x37), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VCMPGT, 8>},
{OPD(2, 0b01, 0x38), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VSMIN, 1>},
{OPD(2, 0b01, 0x39), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VSMIN, 4>},
{OPD(2, 0b01, 0x3A), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VUMIN, 2>},
{OPD(2, 0b01, 0x3B), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VUMIN, 4>},
{OPD(2, 0b01, 0x3C), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VSMAX, 1>},
{OPD(2, 0b01, 0x3D), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VSMAX, 4>},
{OPD(2, 0b01, 0x3E), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VUMAX, 2>},
{OPD(2, 0b01, 0x3F), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VUMAX, 4>},
{OPD(2, 0b01, 0x40), 1, &OpDispatchBuilder::AVXVectorALUOp<IR::OP_VSMUL, 4>},
{OPD(2, 0b01, 0x41), 1, &OpDispatchBuilder::VPHMINPOSUWOp},
{OPD(2, 0b01, 0x58), 1, &OpDispatchBuilder::VBROADCASTOp<4>},
{OPD(2, 0b01, 0x59), 1, &OpDispatchBuilder::VBROADCASTOp<8>},
{OPD(2, 0b01, 0x5A), 1, &OpDispatchBuilder::VBROADCASTOp<16>},
{OPD(2, 0b01, 0x78), 1, &OpDispatchBuilder::VBROADCASTOp<1>},
{OPD(2, 0b01, 0x79), 1, &OpDispatchBuilder::VBROADCASTOp<2>},
{OPD(2, 0b01, 0xDB), 1, &OpDispatchBuilder::VAESIMCOp},
{OPD(2, 0b01, 0xDC), 1, &OpDispatchBuilder::VAESEncOp},
{OPD(2, 0b01, 0xDD), 1, &OpDispatchBuilder::VAESEncLastOp},
{OPD(2, 0b01, 0xDE), 1, &OpDispatchBuilder::VAESDecOp},
{OPD(2, 0b01, 0xDF), 1, &OpDispatchBuilder::VAESDecLastOp},
{OPD(3, 0b01, 0x00), 1, &OpDispatchBuilder::VPERMQOp},
{OPD(3, 0b01, 0x01), 1, &OpDispatchBuilder::VPERMQOp},
{OPD(3, 0b01, 0x04), 1, &OpDispatchBuilder::VPERMILImmOp<4>},
{OPD(3, 0b01, 0x05), 1, &OpDispatchBuilder::VPERMILImmOp<8>},
{OPD(3, 0b01, 0x06), 1, &OpDispatchBuilder::VPERM2Op},
{OPD(3, 0b01, 0x08), 1, &OpDispatchBuilder::AVXVectorRound<4, false>},
{OPD(3, 0b01, 0x09), 1, &OpDispatchBuilder::AVXVectorRound<8, false>},
{OPD(3, 0b01, 0x0A), 1, &OpDispatchBuilder::AVXVectorRound<4, true>},
{OPD(3, 0b01, 0x0B), 1, &OpDispatchBuilder::AVXVectorRound<8, true>},
{OPD(3, 0b01, 0x14), 1, &OpDispatchBuilder::PExtrOp<1>},
{OPD(3, 0b01, 0x15), 1, &OpDispatchBuilder::PExtrOp<2>},
{OPD(3, 0b01, 0x16), 1, &OpDispatchBuilder::PExtrOp<4>},
{OPD(3, 0b01, 0x17), 1, &OpDispatchBuilder::PExtrOp<4>},
{OPD(3, 0b01, 0x18), 1, &OpDispatchBuilder::VINSERTOp},
{OPD(3, 0b01, 0x21), 1, &OpDispatchBuilder::VINSERTPSOp},
{OPD(3, 0b01, 0x38), 1, &OpDispatchBuilder::VINSERTOp},
{OPD(3, 0b01, 0x40), 1, &OpDispatchBuilder::VDPPOp<4>},
{OPD(3, 0b01, 0x41), 1, &OpDispatchBuilder::VDPPOp<8>},
{OPD(3, 0b01, 0x46), 1, &OpDispatchBuilder::VPERM2Op},
{OPD(3, 0b01, 0xDF), 1, &OpDispatchBuilder::VAESKeyGenAssistOp},
};
#undef OPD
#define OPD(group, pp, opcode) (((group - X86Tables::TYPE_VEX_GROUP_12) << 4) | (pp << 3) | (opcode))
static constexpr std::tuple<uint8_t, uint8_t, X86Tables::OpDispatchPtr> VEXTableGroupOps[] {
{OPD(X86Tables::TYPE_VEX_GROUP_12, 1, 0b010), 1, &OpDispatchBuilder::VPSRLIOp<2>},
{OPD(X86Tables::TYPE_VEX_GROUP_12, 1, 0b110), 1, &OpDispatchBuilder::VPSLLIOp<2>},
{OPD(X86Tables::TYPE_VEX_GROUP_12, 1, 0b100), 1, &OpDispatchBuilder::VPSRAIOp<2>},
{OPD(X86Tables::TYPE_VEX_GROUP_13, 1, 0b010), 1, &OpDispatchBuilder::VPSRLIOp<4>},
{OPD(X86Tables::TYPE_VEX_GROUP_13, 1, 0b110), 1, &OpDispatchBuilder::VPSLLIOp<4>},
{OPD(X86Tables::TYPE_VEX_GROUP_13, 1, 0b100), 1, &OpDispatchBuilder::VPSRAIOp<4>},
{OPD(X86Tables::TYPE_VEX_GROUP_14, 1, 0b010), 1, &OpDispatchBuilder::VPSRLIOp<8>},
{OPD(X86Tables::TYPE_VEX_GROUP_14, 1, 0b011), 1, &OpDispatchBuilder::VPSRLDQOp},
{OPD(X86Tables::TYPE_VEX_GROUP_14, 1, 0b110), 1, &OpDispatchBuilder::VPSLLIOp<8>},
{OPD(X86Tables::TYPE_VEX_GROUP_14, 1, 0b111), 1, &OpDispatchBuilder::VPSLLDQOp},
{OPD(X86Tables::TYPE_VEX_GROUP_15, 0, 0b010), 1, &OpDispatchBuilder::LDMXCSR},
{OPD(X86Tables::TYPE_VEX_GROUP_15, 0, 0b011), 1, &OpDispatchBuilder::STMXCSR},
};
#undef OPD
@@ -5915,6 +6150,7 @@ void OpDispatchBuilder::InstallHostSpecificOpcodeHandlers() {
if (CTX->HostFeatures.SupportsAVX) {
InstallToTable(FEXCore::X86Tables::VEXTableOps, AVXTable);
InstallToTable(FEXCore::X86Tables::VEXTableGroupOps, VEXTableGroupOps);
}
if (CTX->HostFeatures.SupportsPMULL_128Bit) {
@@ -6081,7 +6317,7 @@ void InstallOpcodeHandlers(Context::OperatingMode Mode) {
{0x52, 1, &OpDispatchBuilder::VectorUnaryOp<IR::OP_VFRSQRT, 4, false>},
{0x53, 1, &OpDispatchBuilder::VectorUnaryOp<IR::OP_VFRECP, 4, false>},
{0x54, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VAND, 16>},
{0x55, 1, &OpDispatchBuilder::ANDNOp},
{0x55, 1, &OpDispatchBuilder::VectorALUROp<IR::OP_VBIC, 8>},
{0x56, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VOR, 16>},
{0x57, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VXOR, 16>},
{0x58, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VFADD, 4>},
@@ -6117,30 +6353,30 @@ void InstallOpcodeHandlers(Context::OperatingMode Mode) {
{0xD1, 1, &OpDispatchBuilder::PSRLDOp<2>},
{0xD2, 1, &OpDispatchBuilder::PSRLDOp<4>},
{0xD3, 1, &OpDispatchBuilder::PSRLDOp<8>},
{0xD4, 1, &OpDispatchBuilder::PADDQOp<8>},
{0xD4, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VADD, 8>},
{0xD5, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VSMUL, 2>},
{0xD7, 1, &OpDispatchBuilder::MOVMSKOpOne}, // PMOVMSKB
{0xD8, 1, &OpDispatchBuilder::PSUBSOp<1, false>},
{0xD9, 1, &OpDispatchBuilder::PSUBSOp<2, false>},
{0xD8, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VUQSUB, 1>},
{0xD9, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VUQSUB, 2>},
{0xDA, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VUMIN, 1>},
{0xDB, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VAND, 8>},
{0xDC, 1, &OpDispatchBuilder::PADDSOp<1, false>},
{0xDD, 1, &OpDispatchBuilder::PADDSOp<2, false>},
{0xDC, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VUQADD, 1>},
{0xDD, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VUQADD, 2>},
{0xDE, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VUMAX, 1>},
{0xDF, 1, &OpDispatchBuilder::ANDNOp},
{0xE0, 1, &OpDispatchBuilder::PAVGOp<1>},
{0xDF, 1, &OpDispatchBuilder::VectorALUROp<IR::OP_VBIC, 8>},
{0xE0, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VURAVG, 1>},
{0xE1, 1, &OpDispatchBuilder::PSRAOp<2>},
{0xE2, 1, &OpDispatchBuilder::PSRAOp<4>},
{0xE3, 1, &OpDispatchBuilder::PAVGOp<2>},
{0xE3, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VURAVG, 2>},
{0xE4, 1, &OpDispatchBuilder::PMULHW<false>},
{0xE5, 1, &OpDispatchBuilder::PMULHW<true>},
{0xE7, 1, &OpDispatchBuilder::MOVVectorNTOp},
{0xE8, 1, &OpDispatchBuilder::PSUBSOp<1, true>},
{0xE9, 1, &OpDispatchBuilder::PSUBSOp<2, true>},
{0xE8, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VSQSUB, 1>},
{0xE9, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VSQSUB, 2>},
{0xEA, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VSMIN, 2>},
{0xEB, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VOR, 8>},
{0xEC, 1, &OpDispatchBuilder::PADDSOp<1, true>},
{0xED, 1, &OpDispatchBuilder::PADDSOp<2, true>},
{0xEC, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VSQADD, 1>},
{0xED, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VSQADD, 2>},
{0xEE, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VSMAX, 2>},
{0xEF, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VXOR, 8>},
@@ -6151,13 +6387,13 @@ void InstallOpcodeHandlers(Context::OperatingMode Mode) {
{0xF5, 1, &OpDispatchBuilder::PMADDWD},
{0xF6, 1, &OpDispatchBuilder::PSADBW},
{0xF7, 1, &OpDispatchBuilder::MASKMOVOp},
{0xF8, 1, &OpDispatchBuilder::PSUBQOp<1>},
{0xF9, 1, &OpDispatchBuilder::PSUBQOp<2>},
{0xFA, 1, &OpDispatchBuilder::PSUBQOp<4>},
{0xFB, 1, &OpDispatchBuilder::PSUBQOp<8>},
{0xFC, 1, &OpDispatchBuilder::PADDQOp<1>},
{0xFD, 1, &OpDispatchBuilder::PADDQOp<2>},
{0xFE, 1, &OpDispatchBuilder::PADDQOp<4>},
{0xF8, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VSUB, 1>},
{0xF9, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VSUB, 2>},
{0xFA, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VSUB, 4>},
{0xFB, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VSUB, 8>},
{0xFC, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VADD, 1>},
{0xFD, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VADD, 2>},
{0xFE, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VADD, 4>},
// FEX reserved instructions
{0x36, 1, &OpDispatchBuilder::SIGRETOp},
@@ -6344,7 +6580,7 @@ void InstallOpcodeHandlers(Context::OperatingMode Mode) {
{0x5E, 1, &OpDispatchBuilder::VectorScalarALUOp<IR::OP_VFDIV, 8>},
{0x5F, 1, &OpDispatchBuilder::VectorScalarALUOp<IR::OP_VFMAX, 8>},
{0x70, 1, &OpDispatchBuilder::PSHUFDOp<2, true, true>},
{0x7C, 1, &OpDispatchBuilder::HADDP<4>},
{0x7C, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VFADDP, 4>},
{0x7D, 1, &OpDispatchBuilder::HSUBP<4>},
{0xD0, 1, &OpDispatchBuilder::ADDSUBPOp<4>},
{0xD6, 1, &OpDispatchBuilder::MOVQ2DQ<false>},
@@ -6371,7 +6607,7 @@ void InstallOpcodeHandlers(Context::OperatingMode Mode) {
{0x50, 1, &OpDispatchBuilder::MOVMSKOp<8>},
{0x51, 1, &OpDispatchBuilder::VectorUnaryOp<IR::OP_VFSQRT, 8, false>},
{0x54, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VAND, 16>},
{0x55, 1, &OpDispatchBuilder::ANDNOp},
{0x55, 1, &OpDispatchBuilder::VectorALUROp<IR::OP_VBIC, 8>},
{0x56, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VOR, 16>},
{0x57, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VXOR, 16>},
{0x58, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VFADD, 8>},
@@ -6404,7 +6640,7 @@ void InstallOpcodeHandlers(Context::OperatingMode Mode) {
{0x75, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VCMPEQ, 2>},
{0x76, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VCMPEQ, 4>},
{0x78, 1, nullptr}, // GROUP 17
{0x7C, 1, &OpDispatchBuilder::HADDP<8>},
{0x7C, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VFADDP, 8>},
{0x7D, 1, &OpDispatchBuilder::HSUBP<8>},
{0x7E, 1, &OpDispatchBuilder::MOVBetweenGPR_FPR},
{0x7F, 1, &OpDispatchBuilder::MOVUPSOp},
@@ -6417,7 +6653,7 @@ void InstallOpcodeHandlers(Context::OperatingMode Mode) {
{0xD1, 1, &OpDispatchBuilder::PSRLDOp<2>},
{0xD2, 1, &OpDispatchBuilder::PSRLDOp<4>},
{0xD3, 1, &OpDispatchBuilder::PSRLDOp<8>},
{0xD4, 1, &OpDispatchBuilder::PADDQOp<8>},
{0xD4, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VADD, 8>},
{0xD5, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VSMUL, 2>},
{0xD6, 1, &OpDispatchBuilder::MOVQOp},
{0xD7, 1, &OpDispatchBuilder::MOVMSKOpOne}, // PMOVMSKB
@@ -6428,11 +6664,11 @@ void InstallOpcodeHandlers(Context::OperatingMode Mode) {
{0xDC, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VUQADD, 1>},
{0xDD, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VUQADD, 2>},
{0xDE, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VUMAX, 1>},
{0xDF, 1, &OpDispatchBuilder::ANDNOp},
{0xE0, 1, &OpDispatchBuilder::PAVGOp<1>},
{0xDF, 1, &OpDispatchBuilder::VectorALUROp<IR::OP_VBIC, 8>},
{0xE0, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VURAVG, 1>},
{0xE1, 1, &OpDispatchBuilder::PSRAOp<2>},
{0xE2, 1, &OpDispatchBuilder::PSRAOp<4>},
{0xE3, 1, &OpDispatchBuilder::PAVGOp<2>},
{0xE3, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VURAVG, 2>},
{0xE4, 1, &OpDispatchBuilder::PMULHW<false>},
{0xE5, 1, &OpDispatchBuilder::PMULHW<true>},
{0xE6, 1, &OpDispatchBuilder::Vector_CVT_Float_To_Int<8, true, false>},
@@ -6453,13 +6689,13 @@ void InstallOpcodeHandlers(Context::OperatingMode Mode) {
{0xF5, 1, &OpDispatchBuilder::PMADDWD},
{0xF6, 1, &OpDispatchBuilder::PSADBW},
{0xF7, 1, &OpDispatchBuilder::MASKMOVOp},
{0xF8, 1, &OpDispatchBuilder::PSUBQOp<1>},
{0xF9, 1, &OpDispatchBuilder::PSUBQOp<2>},
{0xFA, 1, &OpDispatchBuilder::PSUBQOp<4>},
{0xFB, 1, &OpDispatchBuilder::PSUBQOp<8>},
{0xFC, 1, &OpDispatchBuilder::PADDQOp<1>},
{0xFD, 1, &OpDispatchBuilder::PADDQOp<2>},
{0xFE, 1, &OpDispatchBuilder::PADDQOp<4>},
{0xF8, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VSUB, 1>},
{0xF9, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VSUB, 2>},
{0xFA, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VSUB, 4>},
{0xFB, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VSUB, 8>},
{0xFC, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VADD, 1>},
{0xFD, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VADD, 2>},
{0xFE, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VADD, 4>},
};
constexpr uint16_t PF_NONE = 0;
@@ -7045,10 +7281,10 @@ constexpr uint16_t PF_F2 = 3;
constexpr std::tuple<uint16_t, uint8_t, FEXCore::X86Tables::OpDispatchPtr> H0F38Table[] = {
{OPD(PF_38_NONE, 0x00), 1, &OpDispatchBuilder::PSHUFBOp},
{OPD(PF_38_66, 0x00), 1, &OpDispatchBuilder::PSHUFBOp},
{OPD(PF_38_NONE, 0x01), 1, &OpDispatchBuilder::PHADD<2>},
{OPD(PF_38_66, 0x01), 1, &OpDispatchBuilder::PHADD<2>},
{OPD(PF_38_NONE, 0x02), 1, &OpDispatchBuilder::PHADD<4>},
{OPD(PF_38_66, 0x02), 1, &OpDispatchBuilder::PHADD<4>},
{OPD(PF_38_NONE, 0x01), 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VADDP, 2>},
{OPD(PF_38_66, 0x01), 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VADDP, 2>},
{OPD(PF_38_NONE, 0x02), 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VADDP, 4>},
{OPD(PF_38_66, 0x02), 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VADDP, 4>},
{OPD(PF_38_NONE, 0x03), 1, &OpDispatchBuilder::PHADDS},
{OPD(PF_38_66, 0x03), 1, &OpDispatchBuilder::PHADDS},
{OPD(PF_38_NONE, 0x04), 1, &OpDispatchBuilder::PMADDUBSW},
@@ -7178,7 +7414,7 @@ constexpr uint16_t PF_F2 = 3;
{0xA7, 1, &OpDispatchBuilder::MOVVectorOp},
{0xAA, 1, &OpDispatchBuilder::VectorALUROp<IR::OP_VFSUB, 4>},
{0xAE, 1, &OpDispatchBuilder::HADDP<4>},
{0xAE, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VFADDP, 4>},
{0xB0, 1, &OpDispatchBuilder::VPFCMPOp<0>},
{0xB4, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VFMUL, 4>},
@@ -7187,7 +7423,7 @@ constexpr uint16_t PF_F2 = 3;
{0xB7, 1, &OpDispatchBuilder::PMULHRWOp},
{0xBB, 1, &OpDispatchBuilder::PSWAPDOp},
{0xBF, 1, &OpDispatchBuilder::PAVGOp<1>},
{0xBF, 1, &OpDispatchBuilder::VectorALUOp<IR::OP_VURAVG, 1>},
};
#define OPD(map_select, pp, opcode) (((map_select - 1) << 10) | (pp << 8) | (opcode))
+172 -22
View File
@@ -153,8 +153,9 @@ public:
OpDispatchBuilder(FEXCore::Utils::IntrusivePooledAllocator &Allocator);
void ResetWorkingList();
void ResetDecodeFailure() { DecodeFailure = false; }
void ResetDecodeFailure() { NeedsBlockEnd = DecodeFailure = false; }
bool HadDecodeFailure() const { return DecodeFailure; }
bool NeedsBlockEnder() const { return NeedsBlockEnd; }
void BeginFunction(uint64_t RIP, std::vector<FEXCore::Frontend::Decoder::DecodedBlocks> const *Blocks);
void Finalize();
@@ -322,10 +323,6 @@ public:
void MOVQOp(OpcodeArgs);
template<size_t ElementSize>
void PADDQOp(OpcodeArgs);
template<size_t ElementSize>
void PSUBQOp(OpcodeArgs);
template<size_t ElementSize>
void MOVMSKOp(OpcodeArgs);
void MOVMSKOpOne(OpcodeArgs);
template<size_t ElementSize>
@@ -350,8 +347,6 @@ public:
void PSLLDQ(OpcodeArgs);
template<size_t ElementSize>
void PSRAIOp(OpcodeArgs);
template<size_t ElementSize>
void PAVGOp(OpcodeArgs);
void MOVDDUPOp(OpcodeArgs);
template<size_t DstElementSize>
void CVTGPR_To_FPR(OpcodeArgs);
@@ -378,15 +373,16 @@ public:
void VFCMPOp(OpcodeArgs);
template<size_t ElementSize>
void SHUFOp(OpcodeArgs);
void ANDNOp(OpcodeArgs);
template<size_t ElementSize>
void PINSROp(OpcodeArgs);
void InsertPSOp(OpcodeArgs);
template<size_t ElementSize>
void PExtrOp(OpcodeArgs);
template<size_t ElementSize>
template <size_t ElementSize>
void PSIGN(OpcodeArgs);
template <size_t ElementSize>
void VPSIGN(OpcodeArgs);
// BMI1 Ops
void ANDNBMIOp(OpcodeArgs);
@@ -409,9 +405,50 @@ public:
// AVX Ops
template <IROps IROp, size_t ElementSize>
void AVXVectorALUOp(OpcodeArgs);
template <IROps IROp, size_t ElementSize>
void AVXVectorScalarALUOp(OpcodeArgs);
template <IROps IROp, size_t ElementSize, bool Scalar>
void AVXVectorUnaryOp(OpcodeArgs);
template <size_t ElementSize, size_t DstElementSize, bool Signed>
void AVXExtendVectorElements(OpcodeArgs);
template <size_t ElementSize, bool Scalar>
void AVXVectorRound(OpcodeArgs);
template <size_t SrcElementSize, bool Narrow, bool HostRoundingMode>
void AVXVector_CVT_Float_To_Int(OpcodeArgs);
template <size_t SrcElementSize, bool Widen>
void AVXVector_CVT_Int_To_Float(OpcodeArgs);
template <size_t ElementSize, bool Scalar>
void AVXVFCMPOp(OpcodeArgs);
template <size_t ElementSize>
void VADDSUBPOp(OpcodeArgs);
void VAESDecOp(OpcodeArgs);
void VAESDecLastOp(OpcodeArgs);
void VAESEncOp(OpcodeArgs);
void VAESEncLastOp(OpcodeArgs);
void VAESIMCOp(OpcodeArgs);
void VAESKeyGenAssistOp(OpcodeArgs);
void VANDNOp(OpcodeArgs);
template <size_t ElementSize>
void VBROADCASTOp(OpcodeArgs);
template <size_t ElementSize>
void VDPPOp(OpcodeArgs);
template <IROps IROp, size_t ElementSize>
void VHADDPOp(OpcodeArgs);
void VINSERTOp(OpcodeArgs);
void VINSERTPSOp(OpcodeArgs);
void VMOVAPS_VMOVAPD_Op(OpcodeArgs);
void VMOVUPS_VMOVUPD_Op(OpcodeArgs);
@@ -424,6 +461,56 @@ public:
void VMOVVectorNTOp(OpcodeArgs);
template <size_t ElementSize>
void VPACKSSOp(OpcodeArgs);
template <size_t ElementSize>
void VPACKUSOp(OpcodeArgs);
void VPERM2Op(OpcodeArgs);
void VPERMQOp(OpcodeArgs);
template <size_t ElementSize>
void VPERMILImmOp(OpcodeArgs);
void VPHMINPOSUWOp(OpcodeArgs);
template <size_t ElementSize>
void VPHSUBOp(OpcodeArgs);
void VPMULHRSWOp(OpcodeArgs);
template <bool Signed>
void VPMULHWOp(OpcodeArgs);
template <size_t ElementSize, bool Signed>
void VPMULLOp(OpcodeArgs);
template <size_t ElementSize>
void VPSLLOp(OpcodeArgs);
void VPSLLDQOp(OpcodeArgs);
template <size_t ElementSize>
void VPSLLIOp(OpcodeArgs);
template <size_t ElementSize>
void VPSRAOp(OpcodeArgs);
template <size_t ElementSize>
void VPSRAIOp(OpcodeArgs);
template <size_t ElementSize>
void VPSRLDOp(OpcodeArgs);
void VPSRLDQOp(OpcodeArgs);
template <size_t ElementSize>
void VPUNPCKHOp(OpcodeArgs);
template <size_t ElementSize>
void VPUNPCKLOp(OpcodeArgs);
template <size_t ElementSize>
void VPSRLIOp(OpcodeArgs);
void VZEROOp(OpcodeArgs);
// X87 Ops
@@ -568,12 +655,6 @@ public:
template<bool ToXMM>
void MOVQ2DQ(OpcodeArgs);
template<size_t ElementSize, bool Signed>
void PADDSOp(OpcodeArgs);
template<size_t ElementSize, bool Signed>
void PSUBSOp(OpcodeArgs);
template<size_t ElementSize>
void ADDSUBPOp(OpcodeArgs);
@@ -598,12 +679,7 @@ public:
void MOVBEOp(OpcodeArgs);
template<size_t ElementSize>
void HADDP(OpcodeArgs);
template<size_t ElementSize>
void HSUBP(OpcodeArgs);
template<size_t ElementSize>
void PHADD(OpcodeArgs);
template<size_t ElementSize>
void PHSUB(OpcodeArgs);
@@ -660,8 +736,6 @@ public:
void InvalidOp(OpcodeArgs);
#undef OpcodeArgs
void SetPackedRFLAG(bool Lower8, OrderedNode *Src);
OrderedNode *GetPackedRFLAG(bool Lower8);
@@ -670,9 +744,85 @@ public:
bool HandledLock = false;
private:
bool DecodeFailure{false};
bool NeedsBlockEnd{false};
FEXCore::IR::IROp_IRHeader *Current_Header{};
OrderedNode *Current_HeaderNode{};
// Opcode helpers for generalizing behavior across VEX and non-VEX variants.
OrderedNode* ADDSUBPOpImpl(OpcodeArgs, size_t ElementSize,
OrderedNode *Src1, OrderedNode *Src2);
void AVXVectorALUOpImpl(OpcodeArgs, IROps IROp, size_t ElementSize);
void AVXVectorScalarALUOpImpl(OpcodeArgs, IROps IROp, size_t ElementSize);
void AVXVectorUnaryOpImpl(OpcodeArgs, IROps IROp, size_t ElementSize, bool Scalar);
OrderedNode* AESKeyGenAssistImpl(OpcodeArgs);
OrderedNode* AESIMCImpl(OpcodeArgs);
OrderedNode* DPPOpImpl(OpcodeArgs, const X86Tables::DecodedOperand& Src1,
const X86Tables::DecodedOperand& Src2,
const X86Tables::DecodedOperand& Imm, size_t ElementSize);
OrderedNode* ExtendVectorElementsImpl(OpcodeArgs, size_t ElementSize,
size_t DstElementSize, bool Signed);
OrderedNode* InsertPSOpImpl(OpcodeArgs, const X86Tables::DecodedOperand& Src1,
const X86Tables::DecodedOperand& Src2,
const X86Tables::DecodedOperand& Imm);
OrderedNode* PACKSSOpImpl(OpcodeArgs, size_t ElementSize,
OrderedNode *Src1, OrderedNode *Src2);
OrderedNode* PACKUSOpImpl(OpcodeArgs, size_t ElementSize,
OrderedNode *Src1, OrderedNode *Src2);
OrderedNode* PHMINPOSUWOpImpl(OpcodeArgs);
OrderedNode* PHSUBOpImpl(OpcodeArgs, const X86Tables::DecodedOperand& Src1,
const X86Tables::DecodedOperand& Src2, size_t ElementSize);
OrderedNode* PMULHRSWOpImpl(OpcodeArgs, OrderedNode *Src1, OrderedNode *Src2);
OrderedNode* PMULHWOpImpl(OpcodeArgs, bool Signed,
OrderedNode *Src1, OrderedNode *Src2);
OrderedNode* PMULLOpImpl(OpcodeArgs, size_t ElementSize, bool Signed,
OrderedNode *Src1, OrderedNode *Src2);
OrderedNode* PSIGNImpl(OpcodeArgs, size_t ElementSize,
OrderedNode *Src1, OrderedNode *Src2);
OrderedNode* PSLLIImpl(OpcodeArgs, size_t ElementSize,
OrderedNode *Src, uint64_t Shift);
OrderedNode* PSLLImpl(OpcodeArgs, size_t ElementSize,
OrderedNode *Src, OrderedNode *ShiftVec);
OrderedNode* PSRAOpImpl(OpcodeArgs, size_t ElementSize,
OrderedNode *Src, OrderedNode *ShiftVec);
OrderedNode* PSRLDOpImpl(OpcodeArgs, size_t ElementSize,
OrderedNode *Src, OrderedNode *ShiftVec);
OrderedNode* VFCMPOpImpl(OpcodeArgs, size_t ElementSize, bool Scalar,
OrderedNode *Src1, OrderedNode *Src2, uint8_t CompType);
void VectorALUOpImpl(OpcodeArgs, IROps IROp, size_t ElementSize);
void VectorALUROpImpl(OpcodeArgs, IROps IROp, size_t ElementSize);
void VectorScalarALUOpImpl(OpcodeArgs, IROps IROp, size_t ElementSize);
void VectorUnaryOpImpl(OpcodeArgs, IROps IROp, size_t ElementSize, bool Scalar);
void VectorUnaryDuplicateOpImpl(OpcodeArgs, IROps IROp, size_t ElementSize);
OrderedNode* VectorRoundImpl(OpcodeArgs, size_t ElementSize,
OrderedNode *Src, uint64_t Mode);
OrderedNode* Vector_CVT_Float_To_IntImpl(OpcodeArgs, size_t SrcElementSize, bool Narrow, bool HostRoundingMode);
OrderedNode* Vector_CVT_Int_To_FloatImpl(OpcodeArgs, size_t SrcElementSize, bool Widen);
#undef OpcodeArgs
OrderedNode *AppendSegmentOffset(OrderedNode *Value, uint32_t Flags, uint32_t DefaultPrefix = 0, bool Override = false);
void UpdatePrefixFromSegment(OrderedNode *Segment, uint32_t SegmentReg);
@@ -35,19 +35,16 @@ void OpDispatchBuilder::SHA1MSG1Op(OpcodeArgs) {
OrderedNode *Dest = LoadSource(FPRClass, Op, Op->Dest, Op->Flags, -1);
OrderedNode *Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
auto W0 = _VExtractToGPR(16, 4, Dest, 3);
auto W1 = _VExtractToGPR(16, 4, Dest, 2);
auto W2 = _VExtractToGPR(16, 4, Dest, 1);
auto W3 = _VExtractToGPR(16, 4, Dest, 0);
auto W4 = _VExtractToGPR(16, 4, Src, 3);
auto W5 = _VExtractToGPR(16, 4, Src, 2);
OrderedNode *NewVec{};
NewVec = _VInsElement(16, 4, 3, 1, Dest, Dest);
NewVec = _VInsElement(16, 4, 2, 0, NewVec, Dest);
NewVec = _VInsElement(16, 4, 1, 3, NewVec, Src);
NewVec = _VInsElement(16, 4, 0, 2, NewVec, Src);
auto D3 = _VInsGPR(16, 4, 3, Dest, _Xor(W2, W0));
auto D2 = _VInsGPR(16, 4, 2, D3, _Xor(W3, W1));
auto D1 = _VInsGPR(16, 4, 1, D2, _Xor(W4, W2));
auto D0 = _VInsGPR(16, 4, 0, D1, _Xor(W5, W3));
// [W0, W1, W2, W3] ^ [W2, W3, W4, W5]
OrderedNode *Result = _VXor(16, 1, Dest, NewVec);
StoreResult(FPRClass, Op, D0, -1);
StoreResult(FPRClass, Op, Result, -1);
}
void OpDispatchBuilder::SHA1MSG2Op(OpcodeArgs) {
@@ -264,47 +261,135 @@ void OpDispatchBuilder::SHA256RNDS2Op(OpcodeArgs) {
StoreResult(FPRClass, Op, Res0, -1);
}
void OpDispatchBuilder::AESImcOp(OpcodeArgs) {
OrderedNode* OpDispatchBuilder::AESIMCImpl(OpcodeArgs) {
OrderedNode *Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
auto Res = _VAESImc(Src);
StoreResult(FPRClass, Op, Res, -1);
return _VAESImc(Src);
}
void OpDispatchBuilder::AESImcOp(OpcodeArgs) {
OrderedNode *Result = AESIMCImpl(Op);
StoreResult(FPRClass, Op, Result, -1);
}
void OpDispatchBuilder::VAESIMCOp(OpcodeArgs) {
OrderedNode *Mixed = AESIMCImpl(Op);
OrderedNode *Result = _VMov(16, Mixed);
StoreResult(FPRClass, Op, Result, -1);
}
void OpDispatchBuilder::AESEncOp(OpcodeArgs) {
OrderedNode *Dest = LoadSource(FPRClass, Op, Op->Dest, Op->Flags, -1);
OrderedNode *Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
auto Res = _VAESEnc(Dest, Src);
StoreResult(FPRClass, Op, Res, -1);
OrderedNode *Result = _VAESEnc(Dest, Src);
StoreResult(FPRClass, Op, Result, -1);
}
void OpDispatchBuilder::VAESEncOp(OpcodeArgs) {
const auto DstSize = GetDstSize(Op);
const auto Is128Bit = DstSize == Core::CPUState::XMM_SSE_REG_SIZE;
// TODO: Handle 256-bit VAESENC.
LOGMAN_THROW_A_FMT(Is128Bit, "256-bit VAESENC unimplemented");
OrderedNode *State = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
OrderedNode *Key = LoadSource(FPRClass, Op, Op->Src[1], Op->Flags, -1);
OrderedNode *Result = _VAESEnc(State, Key);
if (Is128Bit) {
Result = _VMov(16, Result);
}
StoreResult(FPRClass, Op, Result, -1);
}
void OpDispatchBuilder::AESEncLastOp(OpcodeArgs) {
OrderedNode *Dest = LoadSource(FPRClass, Op, Op->Dest, Op->Flags, -1);
OrderedNode *Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
auto Res = _VAESEncLast(Dest, Src);
StoreResult(FPRClass, Op, Res, -1);
OrderedNode *Result = _VAESEncLast(Dest, Src);
StoreResult(FPRClass, Op, Result, -1);
}
void OpDispatchBuilder::VAESEncLastOp(OpcodeArgs) {
const auto DstSize = GetDstSize(Op);
const auto Is128Bit = DstSize == Core::CPUState::XMM_SSE_REG_SIZE;
// TODO: Handle 256-bit VAESENCLAST.
LOGMAN_THROW_A_FMT(Is128Bit, "256-bit VAESENCLAST unimplemented");
OrderedNode *State = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
OrderedNode *Key = LoadSource(FPRClass, Op, Op->Src[1], Op->Flags, -1);
OrderedNode *Result = _VAESEncLast(State, Key);
if (Is128Bit) {
Result = _VMov(16, Result);
}
StoreResult(FPRClass, Op, Result, -1);
}
void OpDispatchBuilder::AESDecOp(OpcodeArgs) {
OrderedNode *Dest = LoadSource(FPRClass, Op, Op->Dest, Op->Flags, -1);
OrderedNode *Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
auto Res = _VAESDec(Dest, Src);
StoreResult(FPRClass, Op, Res, -1);
OrderedNode *Result = _VAESDec(Dest, Src);
StoreResult(FPRClass, Op, Result, -1);
}
void OpDispatchBuilder::VAESDecOp(OpcodeArgs) {
const auto DstSize = GetDstSize(Op);
const auto Is128Bit = DstSize == Core::CPUState::XMM_SSE_REG_SIZE;
// TODO: Handle 256-bit VAESDEC.
LOGMAN_THROW_A_FMT(Is128Bit, "256-bit VAESDEC unimplemented");
OrderedNode *State = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
OrderedNode *Key = LoadSource(FPRClass, Op, Op->Src[1], Op->Flags, -1);
OrderedNode *Result = _VAESDec(State, Key);
if (Is128Bit) {
Result = _VMov(16, Result);
}
StoreResult(FPRClass, Op, Result, -1);
}
void OpDispatchBuilder::AESDecLastOp(OpcodeArgs) {
OrderedNode *Dest = LoadSource(FPRClass, Op, Op->Dest, Op->Flags, -1);
OrderedNode *Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
auto Res = _VAESDecLast(Dest, Src);
StoreResult(FPRClass, Op, Res, -1);
OrderedNode *Result = _VAESDecLast(Dest, Src);
StoreResult(FPRClass, Op, Result, -1);
}
void OpDispatchBuilder::VAESDecLastOp(OpcodeArgs) {
const auto DstSize = GetDstSize(Op);
const auto Is128Bit = DstSize == Core::CPUState::XMM_SSE_REG_SIZE;
// TODO: Handle 256-bit VAESDECLAST.
LOGMAN_THROW_A_FMT(Is128Bit, "256-bit VAESDECLAST unimplemented");
OrderedNode *State = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
OrderedNode *Key = LoadSource(FPRClass, Op, Op->Src[1], Op->Flags, -1);
OrderedNode *Result = _VAESDecLast(State, Key);
if (Is128Bit) {
Result = _VMov(16, Result);
}
StoreResult(FPRClass, Op, Result, -1);
}
OrderedNode* OpDispatchBuilder::AESKeyGenAssistImpl(OpcodeArgs) {
OrderedNode *Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
LOGMAN_THROW_A_FMT(Op->Src[1].IsLiteral(), "Src1 needs to be literal here");
const uint64_t RCON = Op->Src[1].Data.Literal.Value;
return _VAESKeyGenAssist(Src, RCON);
}
void OpDispatchBuilder::AESKeyGenAssist(OpcodeArgs) {
OrderedNode *Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
LOGMAN_THROW_A_FMT(Op->Src[1].IsLiteral(), "Src1 needs to be literal here");
uint64_t RCON = Op->Src[1].Data.Literal.Value;
OrderedNode *Result = AESKeyGenAssistImpl(Op);
StoreResult(FPRClass, Op, Result, -1);
}
auto Res = _VAESKeyGenAssist(Src, RCON);
StoreResult(FPRClass, Op, Res, -1);
void OpDispatchBuilder::VAESKeyGenAssistOp(OpcodeArgs) {
OrderedNode *Assist = AESKeyGenAssistImpl(Op);
OrderedNode *Result = _VMov(16, Assist);
StoreResult(FPRClass, Op, Result, -1);
}
void OpDispatchBuilder::PCLMULQDQOp(OpcodeArgs) {
@@ -321,11 +406,18 @@ void OpDispatchBuilder::PCLMULQDQOp(OpcodeArgs) {
void OpDispatchBuilder::VPCLMULQDQOp(OpcodeArgs) {
LOGMAN_THROW_A_FMT(Op->Src[2].IsLiteral(), "Selector needs to be literal here");
const auto DstSize = GetDstSize(Op);
const auto Is128Bit = DstSize == Core::CPUState::XMM_SSE_REG_SIZE;
OrderedNode *Src1 = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags, -1);
OrderedNode *Src2 = LoadSource(FPRClass, Op, Op->Src[1], Op->Flags, -1);
const auto Selector = static_cast<uint8_t>(Op->Src[2].Data.Literal.Value);
auto Res = _PCLMUL(Src1, Src2, Selector);
OrderedNode *Res = _PCLMUL(Src1, Src2, Selector);
if (Is128Bit) {
Res = _VMov(16, Res);
}
StoreResult(FPRClass, Op, Res, -1);
}
File diff suppressed because it is too large. Load diff
@@ -24,8 +24,8 @@ void InitializeH0F3ATables(Context::OperatingMode Mode) {
{OPD(0, PF_3A_NONE, 0x0F), 1, X86InstInfo{"PALIGNR", TYPE_INST, GenFlagsSameSize(SIZE_64BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS | FLAGS_SF_MMX, 1, nullptr}},
{OPD(0, PF_3A_66, 0x08), 1, X86InstInfo{"ROUNDPS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(0, PF_3A_66, 0x09), 1, X86InstInfo{"ROUNDPD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(0, PF_3A_66, 0x0A), 1, X86InstInfo{"ROUNDSS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(0, PF_3A_66, 0x0B), 1, X86InstInfo{"ROUNDSD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(0, PF_3A_66, 0x0A), 1, X86InstInfo{"ROUNDSS", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_32BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(0, PF_3A_66, 0x0B), 1, X86InstInfo{"ROUNDSD", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_64BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(0, PF_3A_66, 0x0C), 1, X86InstInfo{"BLENDPS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(0, PF_3A_66, 0x0D), 1, X86InstInfo{"BLENDPD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(0, PF_3A_66, 0x0E), 1, X86InstInfo{"PBLENDW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 1, nullptr}},
@@ -353,7 +353,7 @@ void InitializeSecondaryTables(Context::OperatingMode Mode) {
{0xD8, 8, X86InstInfo{"", TYPE_INVALID, FLAGS_NONE, 0, nullptr}},
{0xE0, 6, X86InstInfo{"", TYPE_INVALID, FLAGS_NONE, 0, nullptr}},
{0xE6, 1, X86InstInfo{"CVTDQ2PD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{0xE6, 1, X86InstInfo{"CVTDQ2PD", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_64BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{0xE7, 1, X86InstInfo{"", TYPE_INVALID, FLAGS_NONE, 0, nullptr}},
{0xE8, 8, X86InstInfo{"", TYPE_INVALID, FLAGS_NONE, 0, nullptr}},
@@ -375,7 +375,7 @@ void InitializeSecondaryTables(Context::OperatingMode Mode) {
{0x24, 6, X86InstInfo{"", TYPE_INVALID, FLAGS_NONE, 0, nullptr}},
{0x2A, 1, X86InstInfo{"CVTSI2SD", TYPE_INST, GenFlagsDstSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS | FLAGS_SF_SRC_GPR, 0, nullptr}},
{0x2B, 1, X86InstInfo{"MOVNTSD", TYPE_INST, GenFlagsSameSize(SIZE_64BIT) | FLAGS_MODRM | FLAGS_SF_MOD_MEM_ONLY | FLAGS_SF_MOD_DST | FLAGS_XMM_FLAGS, 0, nullptr}},
{0x2C, 1, X86InstInfo{"CVTTSD2SI", TYPE_INST, GenFlagsSrcSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS | FLAGS_SF_DST_GPR, 0, nullptr}},
{0x2C, 1, X86InstInfo{"CVTTSD2SI", TYPE_INST, GenFlagsSrcSize(SIZE_64BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS | FLAGS_SF_DST_GPR, 0, nullptr}},
{0x2D, 1, X86InstInfo{"CVTSD2SI", TYPE_INST, GenFlagsSameSize(SIZE_64BITDEF) | FLAGS_MODRM | FLAGS_XMM_FLAGS | FLAGS_SF_DST_GPR, 0, nullptr}},
{0x2E, 2, X86InstInfo{"", TYPE_INVALID, FLAGS_NONE, 0, nullptr}},
+192 -193
View File
@@ -35,11 +35,11 @@ void InitializeVEXTables() {
{OPD(1, 0b00, 0x13), 1, X86InstInfo{"VMOVLPS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_SF_MOD_MEM_ONLY | FLAGS_SF_MOD_DST | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x13), 1, X86InstInfo{"VMOVLPD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_SF_MOD_MEM_ONLY | FLAGS_SF_MOD_DST | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b00, 0x14), 1, X86InstInfo{"VUNPCKLPS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0x14), 1, X86InstInfo{"VUNPCKLPD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b00, 0x14), 1, X86InstInfo{"VUNPCKLPS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x14), 1, X86InstInfo{"VUNPCKLPD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b00, 0x15), 1, X86InstInfo{"VUNPCKHPS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0x15), 1, X86InstInfo{"VUNPCKHPD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b00, 0x15), 1, X86InstInfo{"VUNPCKHPS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x15), 1, X86InstInfo{"VUNPCKHPD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b00, 0x16), 1, X86InstInfo{"VMOVHPS", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_64BIT) | FLAGS_MODRM | FLAGS_SF_MOD_MEM_ONLY | FLAGS_XMM_FLAGS | FLAGS_VEX_1ST_SRC, 0, nullptr}},
{OPD(1, 0b01, 0x16), 1, X86InstInfo{"VMOVHPD", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_64BIT) | FLAGS_MODRM | FLAGS_SF_MOD_MEM_ONLY | FLAGS_XMM_FLAGS | FLAGS_VEX_1ST_SRC, 0, nullptr}},
@@ -48,19 +48,19 @@ void InitializeVEXTables() {
{OPD(1, 0b00, 0x17), 1, X86InstInfo{"VMOVHPS", TYPE_INST, GenFlagsSizes(SIZE_64BIT, SIZE_128BIT) | FLAGS_MODRM | FLAGS_SF_MOD_MEM_ONLY | FLAGS_SF_MOD_DST | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x17), 1, X86InstInfo{"VMOVHPD", TYPE_INST, GenFlagsSizes(SIZE_64BIT, SIZE_128BIT) | FLAGS_MODRM | FLAGS_SF_MOD_MEM_ONLY | FLAGS_SF_MOD_DST | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b00, 0x50), 1, X86InstInfo{"VMOVMSKPS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0x50), 1, X86InstInfo{"VMOVMSKPD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b00, 0x50), 1, X86InstInfo{"VMOVMSKPS", TYPE_INST, GenFlagsSizes(SIZE_32BIT, SIZE_128BIT) | FLAGS_MODRM | FLAGS_SF_MOD_REG_ONLY | FLAGS_XMM_FLAGS | FLAGS_SF_DST_GPR, 0, nullptr}},
{OPD(1, 0b01, 0x50), 1, X86InstInfo{"VMOVMSKPD", TYPE_INST, GenFlagsSizes(SIZE_32BIT, SIZE_128BIT) | FLAGS_MODRM | FLAGS_SF_MOD_REG_ONLY | FLAGS_XMM_FLAGS | FLAGS_SF_DST_GPR, 0, nullptr}},
{OPD(1, 0b00, 0x51), 1, X86InstInfo{"VSQRTPS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0x51), 1, X86InstInfo{"VSQRTPD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b10, 0x51), 1, X86InstInfo{"VSQRTSS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b11, 0x51), 1, X86InstInfo{"VSQRTSD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b00, 0x51), 1, X86InstInfo{"VSQRTPS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x51), 1, X86InstInfo{"VSQRTPD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b10, 0x51), 1, X86InstInfo{"VSQRTSS", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_32BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b11, 0x51), 1, X86InstInfo{"VSQRTSD", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_64BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b00, 0x52), 1, X86InstInfo{"VRSQRTPS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b10, 0x52), 1, X86InstInfo{"VRSQRTSS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b00, 0x52), 1, X86InstInfo{"VRSQRTPS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b10, 0x52), 1, X86InstInfo{"VRSQRTSS", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_32BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b00, 0x53), 1, X86InstInfo{"VRCPPS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b10, 0x53), 1, X86InstInfo{"VRCPSS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b00, 0x53), 1, X86InstInfo{"VRCPPS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b10, 0x53), 1, X86InstInfo{"VRCPSS", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_32BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b00, 0x54), 1, X86InstInfo{"VANDPS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x54), 1, X86InstInfo{"VANDPD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
@@ -74,14 +74,14 @@ void InitializeVEXTables() {
{OPD(1, 0b00, 0x57), 1, X86InstInfo{"VXORPS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x57), 1, X86InstInfo{"VXORPD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x60), 1, X86InstInfo{"VPUNPCKLBW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0x61), 1, X86InstInfo{"VPUNPCKLWD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0x62), 1, X86InstInfo{"VPUNPCKLDQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0x63), 1, X86InstInfo{"VPACKSSWB", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0x64), 1, X86InstInfo{"VPCMPGTB", TYPE_INST, FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x65), 1, X86InstInfo{"VPVMPGTW", TYPE_INST, FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x66), 1, X86InstInfo{"VPVMPGTD", TYPE_INST, FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x67), 1, X86InstInfo{"VPACKUSWB", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0x60), 1, X86InstInfo{"VPUNPCKLBW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x61), 1, X86InstInfo{"VPUNPCKLWD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x62), 1, X86InstInfo{"VPUNPCKLDQ", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x63), 1, X86InstInfo{"VPACKSSWB", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x64), 1, X86InstInfo{"VPCMPGTB", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x65), 1, X86InstInfo{"VPCMPGTW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x66), 1, X86InstInfo{"VPCMPGTD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x67), 1, X86InstInfo{"VPACKUSWB", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x70), 1, X86InstInfo{"VPSHUFD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b10, 0x70), 1, X86InstInfo{"VPSHUFHW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
@@ -91,19 +91,19 @@ void InitializeVEXTables() {
{OPD(1, 0b01, 0x72), 1, X86InstInfo{"", TYPE_VEX_GROUP_13, FLAGS_NONE, 0, nullptr}}, // VEX Group 13
{OPD(1, 0b01, 0x73), 1, X86InstInfo{"", TYPE_VEX_GROUP_14, FLAGS_NONE, 0, nullptr}}, // VEX Group 14
{OPD(1, 0b01, 0x74), 1, X86InstInfo{"VPCMPEQB", TYPE_INST, FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x75), 1, X86InstInfo{"VPCMPEQW", TYPE_INST, FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x76), 1, X86InstInfo{"VPCMPEQD", TYPE_INST, FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x74), 1, X86InstInfo{"VPCMPEQB", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x75), 1, X86InstInfo{"VPCMPEQW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x76), 1, X86InstInfo{"VPCMPEQD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b00, 0x77), 1, X86InstInfo{"VZERO*", TYPE_INST, GenFlagsDstSize(SIZE_128BIT), 0, nullptr}},
{OPD(1, 0b00, 0xC2), 1, X86InstInfo{"VCMPccPS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0xC2), 1, X86InstInfo{"VCMPccPD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b10, 0xC2), 1, X86InstInfo{"VCMPccSS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b11, 0xC2), 1, X86InstInfo{"VCMPccSD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b00, 0xC2), 1, X86InstInfo{"VCMPccPS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(1, 0b01, 0xC2), 1, X86InstInfo{"VCMPccPD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(1, 0b10, 0xC2), 1, X86InstInfo{"VCMPccSS", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_32BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(1, 0b11, 0xC2), 1, X86InstInfo{"VCMPccSD", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_64BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(1, 0b01, 0xC4), 1, X86InstInfo{"VPINSRW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0xC5), 1, X86InstInfo{"VPEXTRW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0xC5), 1, X86InstInfo{"VPEXTRW", TYPE_INST, GenFlagsSizes(SIZE_32BIT, SIZE_128BIT) | FLAGS_MODRM | FLAGS_SF_MOD_REG_ONLY | FLAGS_SF_DST_GPR | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(1, 0b00, 0xC6), 1, X86InstInfo{"VSHUFPS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0xC6), 1, X86InstInfo{"VSHUFPD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
@@ -124,66 +124,65 @@ void InitializeVEXTables() {
{OPD(1, 0b00, 0x2B), 1, X86InstInfo{"VMOVNTPS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_SF_MOD_MEM_ONLY | FLAGS_SF_MOD_DST | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x2B), 1, X86InstInfo{"VMOVNTPD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_SF_MOD_MEM_ONLY | FLAGS_SF_MOD_DST | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b10, 0x2C), 1, X86InstInfo{"VCVTTSS2SI", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b11, 0x2C), 1, X86InstInfo{"VCVTTSD2SI", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b10, 0x2C), 1, X86InstInfo{"VCVTTSS2SI", TYPE_INST, FLAGS_MODRM | FLAGS_XMM_FLAGS | FLAGS_SF_DST_GPR, 0, nullptr}},
{OPD(1, 0b11, 0x2C), 1, X86InstInfo{"VCVTTSD2SI", TYPE_INST, GenFlagsSrcSize(SIZE_64BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS | FLAGS_SF_DST_GPR, 0, nullptr}},
{OPD(1, 0b10, 0x2D), 1, X86InstInfo{"VCVTSS2SI", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b11, 0x2D), 1, X86InstInfo{"VCVTSD2SI", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b10, 0x2D), 1, X86InstInfo{"VCVTSS2SI", TYPE_INST, FLAGS_MODRM | FLAGS_XMM_FLAGS | FLAGS_SF_DST_GPR, 0, nullptr}},
{OPD(1, 0b11, 0x2D), 1, X86InstInfo{"VCVTSD2SI", TYPE_INST, GenFlagsSameSize(SIZE_64BITDEF) | FLAGS_MODRM | FLAGS_XMM_FLAGS | FLAGS_SF_DST_GPR, 0, nullptr}},
{OPD(1, 0b00, 0x2E), 1, X86InstInfo{"VUCOMISS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0x2E), 1, X86InstInfo{"VUCOMISD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b00, 0x2E), 1, X86InstInfo{"VUCOMISS", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_32BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x2E), 1, X86InstInfo{"VUCOMISD", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_64BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b00, 0x2F), 1, X86InstInfo{"VUCOMISS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0x2F), 1, X86InstInfo{"VUCOMISD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b00, 0x2F), 1, X86InstInfo{"VCOMISS", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_32BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x2F), 1, X86InstInfo{"VCOMISD", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_64BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b00, 0x58), 1, X86InstInfo{"VADDPS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x58), 1, X86InstInfo{"VADDPD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b10, 0x58), 1, X86InstInfo{"VADDSS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b11, 0x58), 1, X86InstInfo{"VADDSD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b10, 0x58), 1, X86InstInfo{"VADDSS", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_32BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b11, 0x58), 1, X86InstInfo{"VADDSD", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_64BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b00, 0x59), 1, X86InstInfo{"VMULPS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0x59), 1, X86InstInfo{"VMULPD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b10, 0x59), 1, X86InstInfo{"VMULSS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b11, 0x59), 1, X86InstInfo{"VMULSD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b00, 0x59), 1, X86InstInfo{"VMULPS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x59), 1, X86InstInfo{"VMULPD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b10, 0x59), 1, X86InstInfo{"VMULSS", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_32BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b11, 0x59), 1, X86InstInfo{"VMULSD", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_64BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b00, 0x5B), 1, X86InstInfo{"VCVTDQ2PS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0x5B), 1, X86InstInfo{"VCVTPS2DQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b10, 0x5B), 1, X86InstInfo{"VCVTPS2DQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b00, 0x5B), 1, X86InstInfo{"VCVTDQ2PS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x5B), 1, X86InstInfo{"VCVTPS2DQ", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b10, 0x5B), 1, X86InstInfo{"VCVTTPS2DQ", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b00, 0x5C), 1, X86InstInfo{"VSUBPS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0x5C), 1, X86InstInfo{"VSUBPD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b10, 0x5C), 1, X86InstInfo{"VSUBSS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b11, 0x5C), 1, X86InstInfo{"VSUBSD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b00, 0x5C), 1, X86InstInfo{"VSUBPS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x5C), 1, X86InstInfo{"VSUBPD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b10, 0x5C), 1, X86InstInfo{"VSUBSS", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_32BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b11, 0x5C), 1, X86InstInfo{"VSUBSD", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_64BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b00, 0x5D), 1, X86InstInfo{"VMINPS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0x5D), 1, X86InstInfo{"VMINPD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b10, 0x5D), 1, X86InstInfo{"VMINSS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b11, 0x5D), 1, X86InstInfo{"VMINSD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b00, 0x5D), 1, X86InstInfo{"VMINPS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x5D), 1, X86InstInfo{"VMINPD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b10, 0x5D), 1, X86InstInfo{"VMINSS", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_32BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b11, 0x5D), 1, X86InstInfo{"VMINSD", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_64BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b00, 0x5E), 1, X86InstInfo{"VDIVPS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0x5E), 1, X86InstInfo{"VDIVPD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b10, 0x5E), 1, X86InstInfo{"VDIVSS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b11, 0x5E), 1, X86InstInfo{"VDIVSD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b00, 0x5E), 1, X86InstInfo{"VDIVPS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x5E), 1, X86InstInfo{"VDIVPD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b10, 0x5E), 1, X86InstInfo{"VDIVSS", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_32BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b11, 0x5E), 1, X86InstInfo{"VDIVSD", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_64BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b00, 0x5F), 1, X86InstInfo{"VMAXPS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0x5F), 1, X86InstInfo{"VMAXPD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b10, 0x5F), 1, X86InstInfo{"VMAXSS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b11, 0x5F), 1, X86InstInfo{"VMAXSD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b00, 0x5F), 1, X86InstInfo{"VMAXPS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x5F), 1, X86InstInfo{"VMAXPD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b10, 0x5F), 1, X86InstInfo{"VMAXSS", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_32BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b11, 0x5F), 1, X86InstInfo{"VMAXSD", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_64BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x68), 1, X86InstInfo{"VPUNPCKHBW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0x69), 1, X86InstInfo{"VPUNPCKHWD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0x6A), 1, X86InstInfo{"VPUNPCKHDQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0x6B), 1, X86InstInfo{"VPACKSSDW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0x6C), 1, X86InstInfo{"VPUNPCKLQDQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0x6D), 1, X86InstInfo{"VPUNPCKHQDQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0x68), 1, X86InstInfo{"VPUNPCKHBW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x69), 1, X86InstInfo{"VPUNPCKHWD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x6A), 1, X86InstInfo{"VPUNPCKHDQ", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x6B), 1, X86InstInfo{"VPACKSSDW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x6C), 1, X86InstInfo{"VPUNPCKLQDQ", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x6D), 1, X86InstInfo{"VPUNPCKHQDQ", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x6E), 1, X86InstInfo{"VMOV*", TYPE_INST, GenFlagsDstSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS | FLAGS_SF_SRC_GPR, 0, nullptr}},
{OPD(1, 0b01, 0x6F), 1, X86InstInfo{"VMOVDQA", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b10, 0x6F), 1, X86InstInfo{"VMOVDQU", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x7C), 1, X86InstInfo{"VHADDPD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b11, 0x7C), 1, X86InstInfo{"VHADDPS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0x7C), 1, X86InstInfo{"VHADDPD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b11, 0x7C), 1, X86InstInfo{"VHADDPS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0x7D), 1, X86InstInfo{"VHSUBPD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b11, 0x7D), 1, X86InstInfo{"VHSUBPS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
@@ -199,80 +198,80 @@ void InitializeVEXTables() {
{OPD(1, 0b10, 0xAE), 1, X86InstInfo{"", TYPE_VEX_GROUP_15, FLAGS_NONE, 0, nullptr}}, // VEX Group 15
{OPD(1, 0b11, 0xAE), 1, X86InstInfo{"", TYPE_VEX_GROUP_15, FLAGS_NONE, 0, nullptr}}, // VEX Group 15
{OPD(1, 0b01, 0xD0), 1, X86InstInfo{"VADDSUBPD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b11, 0xD0), 1, X86InstInfo{"VADDSUBPS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0xD0), 1, X86InstInfo{"VADDSUBPD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b11, 0xD0), 1, X86InstInfo{"VADDSUBPS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xD1), 1, X86InstInfo{"VPSRLW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0xD2), 1, X86InstInfo{"VPSRLD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0xD3), 1, X86InstInfo{"VPSRLQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0xD1), 1, X86InstInfo{"VPSRLW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xD2), 1, X86InstInfo{"VPSRLD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xD3), 1, X86InstInfo{"VPSRLQ", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xD4), 1, X86InstInfo{"VPADDQ", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xD5), 1, X86InstInfo{"VPMULLW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0xD5), 1, X86InstInfo{"VPMULLW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xD6), 1, X86InstInfo{"VMOVQ", TYPE_INST, GenFlagsSameSize(SIZE_64BIT) | FLAGS_MODRM | FLAGS_SF_MOD_DST | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xD7), 1, X86InstInfo{"VPMOVMSKB", TYPE_INST, FLAGS_MODRM | FLAGS_XMM_FLAGS | FLAGS_SF_DST_GPR | FLAGS_SF_MOD_REG_ONLY, 0, nullptr}},
{OPD(1, 0b01, 0xD7), 1, X86InstInfo{"VPMOVMSKB", TYPE_UNDEC, FLAGS_MODRM | FLAGS_XMM_FLAGS | FLAGS_SF_DST_GPR | FLAGS_SF_MOD_REG_ONLY, 0, nullptr}},
{OPD(1, 0b01, 0xD8), 1, X86InstInfo{"VPSUBUSB", TYPE_INST, FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xD9), 1, X86InstInfo{"VPSUBUSW", TYPE_INST, FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xDA), 1, X86InstInfo{"VPMINUB", TYPE_INST, FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xD8), 1, X86InstInfo{"VPSUBUSB", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xD9), 1, X86InstInfo{"VPSUBUSW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xDA), 1, X86InstInfo{"VPMINUB", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xDB), 1, X86InstInfo{"VPAND", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xDC), 1, X86InstInfo{"VPADDUSB", TYPE_INST, FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xDD), 1, X86InstInfo{"VPADDUSW", TYPE_INST, FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xDE), 1, X86InstInfo{"VPMAXUB", TYPE_INST, FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xDC), 1, X86InstInfo{"VPADDUSB", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xDD), 1, X86InstInfo{"VPADDUSW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xDE), 1, X86InstInfo{"VPMAXUB", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xDF), 1, X86InstInfo{"VPANDN", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xE0), 1, X86InstInfo{"VPAVGB", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0xE1), 1, X86InstInfo{"VPSRAW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0xE2), 1, X86InstInfo{"VPSRAD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0xE3), 1, X86InstInfo{"VPAVGW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0xE4), 1, X86InstInfo{"VPMULHUW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0xE5), 1, X86InstInfo{"VPMULHW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0xE0), 1, X86InstInfo{"VPAVGB", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xE1), 1, X86InstInfo{"VPSRAW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xE2), 1, X86InstInfo{"VPSRAD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xE3), 1, X86InstInfo{"VPAVGW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xE4), 1, X86InstInfo{"VPMULHUW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xE5), 1, X86InstInfo{"VPMULHW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xE6), 1, X86InstInfo{"VCVTTPD2DQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b10, 0xE6), 1, X86InstInfo{"VCVTDQ2PD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b11, 0xE6), 1, X86InstInfo{"VCVTPD2DQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0xE6), 1, X86InstInfo{"VCVTTPD2DQ", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b10, 0xE6), 1, X86InstInfo{"VCVTDQ2PD", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_64BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b11, 0xE6), 1, X86InstInfo{"VCVTPD2DQ", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xE7), 1, X86InstInfo{"VMOVNTDQ", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_SF_MOD_DST | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xE8), 1, X86InstInfo{"VPSUBSB", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0xE9), 1, X86InstInfo{"VPSUBSW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0xEA), 1, X86InstInfo{"VPMINSW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0xE8), 1, X86InstInfo{"VPSUBSB", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xE9), 1, X86InstInfo{"VPSUBSW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xEA), 1, X86InstInfo{"VPMINSW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xEB), 1, X86InstInfo{"VPOR", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xEC), 1, X86InstInfo{"VPADDSB", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0xED), 1, X86InstInfo{"VPADDSW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0xEE), 1, X86InstInfo{"VPMAXSW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0xEC), 1, X86InstInfo{"VPADDSB", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xED), 1, X86InstInfo{"VPADDSW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xEE), 1, X86InstInfo{"VPMAXSW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xEF), 1, X86InstInfo{"VPXOR", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b11, 0xF0), 1, X86InstInfo{"VLDDQU", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b11, 0xF0), 1, X86InstInfo{"VLDDQU", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xF1), 1, X86InstInfo{"VPSLLW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0xF2), 1, X86InstInfo{"VPSLLD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0xF3), 1, X86InstInfo{"VPSLLQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0xF4), 1, X86InstInfo{"VPMULUDQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0xF1), 1, X86InstInfo{"VPSLLW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xF2), 1, X86InstInfo{"VPSLLD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xF3), 1, X86InstInfo{"VPSLLQ", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xF4), 1, X86InstInfo{"VPMULUDQ", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xF5), 1, X86InstInfo{"VPMADDWD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0xF6), 1, X86InstInfo{"VPSADBW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0xF7), 1, X86InstInfo{"VMASKMOVDQU", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 0b01, 0xF7), 1, X86InstInfo{"VMASKMOVDQU", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_SF_MOD_REG_ONLY | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xF8), 1, X86InstInfo{"VPSUBB", TYPE_INST, GenFlagsSameSize(SIZE_256BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xF9), 1, X86InstInfo{"VPSUBW", TYPE_INST, GenFlagsSameSize(SIZE_256BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xFA), 1, X86InstInfo{"VPSUBD", TYPE_INST, GenFlagsSameSize(SIZE_256BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xFB), 1, X86InstInfo{"VPSUBQ", TYPE_INST, GenFlagsSameSize(SIZE_256BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xF8), 1, X86InstInfo{"VPSUBB", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xF9), 1, X86InstInfo{"VPSUBW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xFA), 1, X86InstInfo{"VPSUBD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xFB), 1, X86InstInfo{"VPSUBQ", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xFC), 1, X86InstInfo{"VPADDB", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xFD), 1, X86InstInfo{"VPADDW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(1, 0b01, 0xFE), 1, X86InstInfo{"VPADDD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
// VEX Map 2
{OPD(2, 0b01, 0x00), 1, X86InstInfo{"VPSHUFB", TYPE_INST, FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x01), 1, X86InstInfo{"VPADDW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x02), 1, X86InstInfo{"VPHADDD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x00), 1, X86InstInfo{"VPSHUFB", TYPE_UNDEC, FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x01), 1, X86InstInfo{"VPHADDW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x02), 1, X86InstInfo{"VPHADDD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x03), 1, X86InstInfo{"VPHADDSW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x04), 1, X86InstInfo{"VPMADDUBSW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x05), 1, X86InstInfo{"VPHSUBW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x06), 1, X86InstInfo{"VPHSUBD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x05), 1, X86InstInfo{"VPHSUBW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x06), 1, X86InstInfo{"VPHSUBD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x07), 1, X86InstInfo{"VPHSUBSW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x08), 1, X86InstInfo{"VPSIGNB", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x09), 1, X86InstInfo{"VPSIGNW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x0A), 1, X86InstInfo{"VPSIGND", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x0B), 1, X86InstInfo{"VPMULHRSW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x08), 1, X86InstInfo{"VPSIGNB", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x09), 1, X86InstInfo{"VPSIGNW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x0A), 1, X86InstInfo{"VPSIGND", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x0B), 1, X86InstInfo{"VPMULHRSW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x0C), 1, X86InstInfo{"VPERMILPS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x0D), 1, X86InstInfo{"VPERMILPD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x0E), 1, X86InstInfo{"VTESTPS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
@@ -282,59 +281,59 @@ void InitializeVEXTables() {
{OPD(2, 0b01, 0x16), 1, X86InstInfo{"VPERMPS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x17), 1, X86InstInfo{"VPTEST", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x18), 1, X86InstInfo{"VBROADCASTSS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x19), 1, X86InstInfo{"VBROADCASTSD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x1A), 1, X86InstInfo{"VBROADCASTF128", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x1C), 1, X86InstInfo{"VPABSB", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x1D), 1, X86InstInfo{"VPABSW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x1E), 1, X86InstInfo{"VPABSD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x18), 1, X86InstInfo{"VBROADCASTSS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x19), 1, X86InstInfo{"VBROADCASTSD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x1A), 1, X86InstInfo{"VBROADCASTF128", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x1C), 1, X86InstInfo{"VPABSB", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x1D), 1, X86InstInfo{"VPABSW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x1E), 1, X86InstInfo{"VPABSD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x20), 1, X86InstInfo{"VPMOVSXBW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x21), 1, X86InstInfo{"VPMOVSXBD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x22), 1, X86InstInfo{"VPMOVSXBQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x23), 1, X86InstInfo{"VPMOVSXWD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x24), 1, X86InstInfo{"VPMOVSXWQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x25), 1, X86InstInfo{"VPMOVSXDQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x20), 1, X86InstInfo{"VPMOVSXBW", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_64BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x21), 1, X86InstInfo{"VPMOVSXBD", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_32BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x22), 1, X86InstInfo{"VPMOVSXBQ", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_16BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x23), 1, X86InstInfo{"VPMOVSXWD", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_64BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x24), 1, X86InstInfo{"VPMOVSXWQ", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_32BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x25), 1, X86InstInfo{"VPMOVSXDQ", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_64BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x28), 1, X86InstInfo{"VPMULDQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x29), 1, X86InstInfo{"VPCMPEQQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x28), 1, X86InstInfo{"VPMULDQ", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x29), 1, X86InstInfo{"VPCMPEQQ", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x2A), 1, X86InstInfo{"VMOVNTDQA", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_SF_MOD_MEM_ONLY | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x2B), 1, X86InstInfo{"VPACKUSDW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x2B), 1, X86InstInfo{"VPACKUSDW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x2C), 1, X86InstInfo{"VMASKMOVPS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x2D), 1, X86InstInfo{"VMASKMOVPD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x2E), 1, X86InstInfo{"VMASKMOVPS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x2F), 1, X86InstInfo{"VMASKMOVPD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x30), 1, X86InstInfo{"VPMOVZXBW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x31), 1, X86InstInfo{"VPMOVZXBD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x32), 1, X86InstInfo{"VPMOVZXBQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x33), 1, X86InstInfo{"VPMOVZXWD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x34), 1, X86InstInfo{"VPMOVZXWQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x35), 1, X86InstInfo{"VPMOVZXDQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x30), 1, X86InstInfo{"VPMOVZXBW", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_64BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x31), 1, X86InstInfo{"VPMOVZXBD", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_32BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x32), 1, X86InstInfo{"VPMOVZXBQ", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_16BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x33), 1, X86InstInfo{"VPMOVZXWD", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_64BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x34), 1, X86InstInfo{"VPMOVZXWQ", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_32BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x35), 1, X86InstInfo{"VPMOVZXDQ", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_64BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x36), 1, X86InstInfo{"VPERMD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x37), 1, X86InstInfo{"VPVMPGTQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x37), 1, X86InstInfo{"VPCMPGTQ", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x38), 1, X86InstInfo{"VPMINSB", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x39), 1, X86InstInfo{"VPMINSD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x3A), 1, X86InstInfo{"VPMINUW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x3B), 1, X86InstInfo{"VPMINUD", TYPE_INST, FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x3C), 1, X86InstInfo{"VPMAXSB", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x3D), 1, X86InstInfo{"VPMAXSD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x3E), 1, X86InstInfo{"VPMAXUW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x3F), 1, X86InstInfo{"VPMAXUD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x38), 1, X86InstInfo{"VPMINSB", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x39), 1, X86InstInfo{"VPMINSD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x3A), 1, X86InstInfo{"VPMINUW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x3B), 1, X86InstInfo{"VPMINUD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x3C), 1, X86InstInfo{"VPMAXSB", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x3D), 1, X86InstInfo{"VPMAXSD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x3E), 1, X86InstInfo{"VPMAXUW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x3F), 1, X86InstInfo{"VPMAXUD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x40), 1, X86InstInfo{"VPMULLD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x41), 1, X86InstInfo{"VPHMINPOSUW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x40), 1, X86InstInfo{"VPMULLD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x41), 1, X86InstInfo{"VPHMINPOSUW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x45), 1, X86InstInfo{"VPSRLV", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x46), 1, X86InstInfo{"VPSRAVD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x47), 1, X86InstInfo{"VPSLLV", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x58), 1, X86InstInfo{"VPBROADCASTD", TYPE_INST, FLAGS_MODRM, 0, nullptr}},
{OPD(2, 0b01, 0x59), 1, X86InstInfo{"VPBROADCASTQ", TYPE_INST, FLAGS_MODRM, 0, nullptr}},
{OPD(2, 0b01, 0x5A), 1, X86InstInfo{"VBBROADCASTI128", TYPE_INST, FLAGS_MODRM, 0, nullptr}},
{OPD(2, 0b01, 0x58), 1, X86InstInfo{"VPBROADCASTD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x59), 1, X86InstInfo{"VPBROADCASTQ", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x5A), 1, X86InstInfo{"VBROADCASTI128", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x78), 1, X86InstInfo{"VPBROADCASTB", TYPE_INST, FLAGS_MODRM, 0, nullptr}},
{OPD(2, 0b01, 0x79), 1, X86InstInfo{"VPBROADCASTW", TYPE_INST, FLAGS_MODRM, 0, nullptr}},
{OPD(2, 0b01, 0x78), 1, X86InstInfo{"VPBROADCASTB", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x79), 1, X86InstInfo{"VPBROADCASTW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0x8C), 1, X86InstInfo{"VPMASKMOV", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0x8E), 1, X86InstInfo{"VPMASKMOV", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
@@ -380,11 +379,11 @@ void InitializeVEXTables() {
{OPD(2, 0b01, 0xB6), 1, X86InstInfo{"VFMADDSUB231", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0xB7), 1, X86InstInfo{"VFMSUBADD231", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0xDB), 1, X86InstInfo{"VAESIMC", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0xDC), 1, X86InstInfo{"VAESENC", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0xDD), 1, X86InstInfo{"VAESENCLAST", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0xDE), 1, X86InstInfo{"VAESDEC", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0xDF), 1, X86InstInfo{"VAESDECLAST", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 0b01, 0xDB), 1, X86InstInfo{"VAESIMC", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0xDC), 1, X86InstInfo{"VAESENC", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0xDD), 1, X86InstInfo{"VAESENCLAST", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0xDE), 1, X86InstInfo{"VAESDEC", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b01, 0xDF), 1, X86InstInfo{"VAESDECLAST", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 0, nullptr}},
{OPD(2, 0b00, 0xF2), 1, X86InstInfo{"ANDN", TYPE_INST, FLAGS_MODRM | FLAGS_VEX_1ST_SRC, 0, nullptr}},
@@ -406,43 +405,43 @@ void InitializeVEXTables() {
{OPD(2, 0b11, 0xF7), 1, X86InstInfo{"SHRX", TYPE_INST, FLAGS_MODRM | FLAGS_VEX_2ND_SRC, 0, nullptr}},
// VEX Map 3
{OPD(3, 0b01, 0x00), 1, X86InstInfo{"VPERMQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(3, 0b01, 0x01), 1, X86InstInfo{"VPERMPD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(3, 0b01, 0x00), 1, X86InstInfo{"VPERMQ", TYPE_INST, GenFlagsSameSize(SIZE_256BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(3, 0b01, 0x01), 1, X86InstInfo{"VPERMPD", TYPE_INST, GenFlagsSameSize(SIZE_256BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(3, 0b01, 0x02), 1, X86InstInfo{"VPBLENDD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(3, 0b01, 0x04), 1, X86InstInfo{"VPERMILPS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(3, 0b01, 0x05), 1, X86InstInfo{"VPERMILPD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(3, 0b01, 0x06), 1, X86InstInfo{"VPERM2F128", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(3, 0b01, 0x04), 1, X86InstInfo{"VPERMILPS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(3, 0b01, 0x05), 1, X86InstInfo{"VPERMILPD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(3, 0b01, 0x06), 1, X86InstInfo{"VPERM2F128", TYPE_INST, GenFlagsSameSize(SIZE_256BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(3, 0b01, 0x08), 1, X86InstInfo{"VROUNDPS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(3, 0b01, 0x09), 1, X86InstInfo{"VROUNDPD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(3, 0b01, 0x0A), 1, X86InstInfo{"VROUNDSS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(3, 0b01, 0x0B), 1, X86InstInfo{"VROUNDSD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(3, 0b01, 0x08), 1, X86InstInfo{"VROUNDPS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(3, 0b01, 0x09), 1, X86InstInfo{"VROUNDPD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(3, 0b01, 0x0A), 1, X86InstInfo{"VROUNDSS", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_32BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(3, 0b01, 0x0B), 1, X86InstInfo{"VROUNDSD", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_64BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(3, 0b01, 0x0C), 1, X86InstInfo{"VBLENDPS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(3, 0b01, 0x0D), 1, X86InstInfo{"VBLENDPD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(3, 0b01, 0x0E), 1, X86InstInfo{"VBLENDW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(3, 0b01, 0x0F), 1, X86InstInfo{"VPALIGNR", TYPE_INST, FLAGS_MODRM | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(3, 0b01, 0x14), 1, X86InstInfo{"VPEXTRB", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(3, 0b01, 0x15), 1, X86InstInfo{"VPEXTRW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(3, 0b01, 0x16), 1, X86InstInfo{"VPEXTRD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(3, 0b01, 0x17), 1, X86InstInfo{"VEXTRACTPS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(3, 0b01, 0x14), 1, X86InstInfo{"VPEXTRB", TYPE_INST, GenFlagsSizes(SIZE_32BIT, SIZE_128BIT) | FLAGS_MODRM | FLAGS_SF_MOD_DST | FLAGS_SF_DST_GPR | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(3, 0b01, 0x15), 1, X86InstInfo{"VPEXTRW", TYPE_INST, GenFlagsSizes(SIZE_16BIT, SIZE_128BIT) | FLAGS_MODRM | FLAGS_SF_MOD_DST | FLAGS_SF_DST_GPR | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(3, 0b01, 0x16), 1, X86InstInfo{"VPEXTRD", TYPE_INST, GenFlagsSizes(SIZE_32BIT, SIZE_128BIT) | FLAGS_MODRM | FLAGS_SF_MOD_DST | FLAGS_SF_DST_GPR | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(3, 0b01, 0x17), 1, X86InstInfo{"VEXTRACTPS", TYPE_INST, GenFlagsSizes(SIZE_32BIT, SIZE_128BIT) | FLAGS_MODRM | FLAGS_SF_MOD_DST | FLAGS_SF_DST_GPR | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(3, 0b01, 0x18), 1, X86InstInfo{"VINSERTF128", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(3, 0b01, 0x18), 1, X86InstInfo{"VINSERTF128", TYPE_INST, GenFlagsSameSize(SIZE_256BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(3, 0b01, 0x19), 1, X86InstInfo{"VEXTRACTF128", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(3, 0b01, 0x1D), 1, X86InstInfo{"VCVTPS2PH", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(3, 0b01, 0x20), 1, X86InstInfo{"VPINSRB", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(3, 0b01, 0x21), 1, X86InstInfo{"VINSERTPS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(3, 0b01, 0x21), 1, X86InstInfo{"VINSERTPS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(3, 0b01, 0x22), 1, X86InstInfo{"VPINSRD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(3, 0b01, 0x38), 1, X86InstInfo{"VINSERTI128", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(3, 0b01, 0x38), 1, X86InstInfo{"VINSERTI128", TYPE_INST, GenFlagsSameSize(SIZE_256BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(3, 0b01, 0x39), 1, X86InstInfo{"VEXTRACTI128", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(3, 0b01, 0x40), 1, X86InstInfo{"VDPPS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(3, 0b01, 0x41), 1, X86InstInfo{"VDPPD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(3, 0b01, 0x40), 1, X86InstInfo{"VDPPS", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(3, 0b01, 0x41), 1, X86InstInfo{"VDPPD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(3, 0b01, 0x42), 1, X86InstInfo{"VMPSADBW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(3, 0b01, 0x44), 1, X86InstInfo{"VPCLMULQDQ", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(3, 0b01, 0x46), 1, X86InstInfo{"VPERM2I128", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(3, 0b01, 0x46), 1, X86InstInfo{"VPERM2I128", TYPE_INST, GenFlagsSameSize(SIZE_256BIT) | FLAGS_MODRM | FLAGS_VEX_1ST_SRC | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(3, 0b01, 0x48), 1, X86InstInfo{"VPERMILzz2PS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(3, 0b01, 0x49), 1, X86InstInfo{"VPERMILzz2PD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
@@ -478,7 +477,7 @@ void InitializeVEXTables() {
{OPD(3, 0b01, 0x7E), 1, X86InstInfo{"VFNMSUBSS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(3, 0b01, 0x7F), 1, X86InstInfo{"VFNMSUBSD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(3, 0b01, 0xDF), 1, X86InstInfo{"VAESKEYGENASSIST", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(3, 0b01, 0xDF), 1, X86InstInfo{"VAESKEYGENASSIST", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(3, 0b11, 0xF0), 1, X86InstInfo{"RORX", TYPE_INST, FLAGS_MODRM, 1, nullptr}},
@@ -488,21 +487,21 @@ void InitializeVEXTables() {
#define OPD(group, pp, opcode) (((group - TYPE_VEX_GROUP_12) << 4) | (pp << 3) | (opcode))
static constexpr U8U8InfoStruct VEXGroupTable[] = {
{OPD(TYPE_VEX_GROUP_12, 1, 0b010), 1, X86InstInfo{"VPSRLW", TYPE_UNDEC, FLAGS_MODRM, 0, nullptr}},
{OPD(TYPE_VEX_GROUP_12, 1, 0b100), 1, X86InstInfo{"VPSRAW", TYPE_UNDEC, FLAGS_MODRM, 0, nullptr}},
{OPD(TYPE_VEX_GROUP_12, 1, 0b110), 1, X86InstInfo{"VPSLLW", TYPE_UNDEC, FLAGS_MODRM, 0, nullptr}},
{OPD(TYPE_VEX_GROUP_12, 1, 0b010), 1, X86InstInfo{"VPSRLW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_DST | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(TYPE_VEX_GROUP_12, 1, 0b100), 1, X86InstInfo{"VPSRAW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_DST | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(TYPE_VEX_GROUP_12, 1, 0b110), 1, X86InstInfo{"VPSLLW", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_DST | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(TYPE_VEX_GROUP_13, 1, 0b010), 1, X86InstInfo{"VPSRLD", TYPE_UNDEC, FLAGS_MODRM, 0, nullptr}},
{OPD(TYPE_VEX_GROUP_13, 1, 0b100), 1, X86InstInfo{"VPSRAD", TYPE_UNDEC, FLAGS_MODRM, 0, nullptr}},
{OPD(TYPE_VEX_GROUP_13, 1, 0b110), 1, X86InstInfo{"VPSLLD", TYPE_UNDEC, FLAGS_MODRM, 0, nullptr}},
{OPD(TYPE_VEX_GROUP_13, 1, 0b010), 1, X86InstInfo{"VPSRLD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_DST | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(TYPE_VEX_GROUP_13, 1, 0b100), 1, X86InstInfo{"VPSRAD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_DST | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(TYPE_VEX_GROUP_13, 1, 0b110), 1, X86InstInfo{"VPSLLD", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_DST | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(TYPE_VEX_GROUP_14, 1, 0b010), 1, X86InstInfo{"VPSRLQ", TYPE_INST, FLAGS_MODRM | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(TYPE_VEX_GROUP_14, 1, 0b011), 1, X86InstInfo{"VPSRLDQ", TYPE_INST, FLAGS_MODRM | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(TYPE_VEX_GROUP_14, 1, 0b110), 1, X86InstInfo{"VPSLLQ", TYPE_INST, FLAGS_MODRM | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(TYPE_VEX_GROUP_14, 1, 0b111), 1, X86InstInfo{"VPSLLDQ", TYPE_INST, FLAGS_MODRM | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(TYPE_VEX_GROUP_14, 1, 0b010), 1, X86InstInfo{"VPSRLQ", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_DST | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(TYPE_VEX_GROUP_14, 1, 0b011), 1, X86InstInfo{"VPSRLDQ", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_DST | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(TYPE_VEX_GROUP_14, 1, 0b110), 1, X86InstInfo{"VPSLLQ", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_DST | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(TYPE_VEX_GROUP_14, 1, 0b111), 1, X86InstInfo{"VPSLLDQ", TYPE_INST, GenFlagsSameSize(SIZE_128BIT) | FLAGS_MODRM | FLAGS_VEX_DST | FLAGS_XMM_FLAGS, 1, nullptr}},
{OPD(TYPE_VEX_GROUP_15, 1, 0b010), 1, X86InstInfo{"VLDMXCSR", TYPE_UNDEC, FLAGS_MODRM, 0, nullptr}},
{OPD(TYPE_VEX_GROUP_15, 1, 0b011), 1, X86InstInfo{"VSTMXCSR", TYPE_UNDEC, FLAGS_MODRM, 0, nullptr}},
{OPD(TYPE_VEX_GROUP_15, 0, 0b010), 1, X86InstInfo{"VLDMXCSR", TYPE_INST, GenFlagsSameSize(SIZE_32BIT) | FLAGS_MODRM | FLAGS_SF_MOD_DST | FLAGS_SF_MOD_MEM_ONLY, 0, nullptr}},
{OPD(TYPE_VEX_GROUP_15, 0, 0b011), 1, X86InstInfo{"VSTMXCSR", TYPE_INST, GenFlagsSameSize(SIZE_32BIT) | FLAGS_MODRM | FLAGS_SF_MOD_DST | FLAGS_SF_MOD_MEM_ONLY, 0, nullptr}},
{OPD(TYPE_VEX_GROUP_17, 0, 0b001), 1, X86InstInfo{"BLSR", TYPE_INST, FLAGS_MODRM | FLAGS_VEX_DST, 0, nullptr}},
{OPD(TYPE_VEX_GROUP_17, 0, 0b010), 1, X86InstInfo{"BLSMSK", TYPE_INST, FLAGS_MODRM | FLAGS_VEX_DST, 0, nullptr}},
+12 -2
View File
@@ -167,6 +167,8 @@ namespace FEXCore {
* address to another function. The original callee address is passed
* to the target function through an implicit argument stored in r11.
*
* For 32-bit the implicit argument is stored in the lower 32-bits of mm0.
*
* The primary use case of this is ensuring that host function pointers
* returned from thunked APIs can safely be called by the guest.
*/
@@ -199,7 +201,12 @@ namespace FEXCore {
const uint8_t GPRSize = CTX->GetGPRSize();
emit->_StoreRegister(emit->_Constant(Entrypoint), false, offsetof(Core::CPUState, gregs[X86State::REG_R11]), IR::GPRClass, IR::GPRFixedClass, GPRSize);
if (GPRSize == 8) {
emit->_StoreRegister(emit->_Constant(Entrypoint), false, offsetof(Core::CPUState, gregs[X86State::REG_R11]), IR::GPRClass, IR::GPRFixedClass, GPRSize);
}
else {
emit->_StoreRegister(emit->_Constant(Entrypoint), false, offsetof(Core::CPUState, mm[0][0]), IR::GPRClass, IR::GPRFixedClass, GPRSize);
}
emit->_ExitFunction(emit->_Constant(GuestThunkEntrypoint));
}, CTX->ThunkHandler.get(), (void*)args->target_addr);
@@ -263,7 +270,10 @@ namespace FEXCore {
auto Name = Args->Name;
auto SOName = CTX->Config.ThunkHostLibsPath() + "/" + (const char*)Name + "-host.so";
auto SOName = (CTX->Config.Is64BitMode() ?
CTX->Config.ThunkHostLibsPath() :
CTX->Config.ThunkHostLibsPath32())
+ "/" + (const char*)Name + "-host.so";
LogMan::Msg::DFmt("LoadLib: {} -> {}", Name, SOName);
@@ -77,7 +77,9 @@ struct RegisterAllocationDataDeleter {
inline auto RegisterAllocationData::Create(uint32_t NodeCount) -> UniquePtr {
auto Ret = (RegisterAllocationData*)FEXCore::Allocator::malloc(Size(NodeCount));
memset(&Ret->Map[0], PhysicalRegister::Invalid().Raw, NodeCount);
Ret->SpillSlotCount = 0;
Ret->MapCount = NodeCount;
Ret->IsShared = false;
return UniquePtr { Ret };
}
+1
View File
@@ -0,0 +1 @@
add_subdirectory(Emitter/)
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+450
View File
@@ -0,0 +1,450 @@
#include "TestDisassembler.h"
#include <catch2/catch.hpp>
#include <fcntl.h>
using namespace FEXCore::ARMEmitter;
TEST_CASE_METHOD(TestDisassembler, "Emitter: Branch: Conditional branch immediate") {
{
BackwardLabel Label;
Bind(&Label);
dc32(0);
b(Condition::CC_PL, &Label);
CHECK(DisassembleEncoding(1) == 0x54ffffe5);
}
{
ForwardLabel Label;
b(Condition::CC_PL, &Label);
Bind(&Label);
dc32(0);
CHECK(DisassembleEncoding(0) == 0x54000025);
}
{
BiDirectionalLabel Label;
Bind(&Label);
dc32(0);
b(Condition::CC_PL, &Label);
CHECK(DisassembleEncoding(1) == 0x54ffffe5);
}
{
BiDirectionalLabel Label;
b(Condition::CC_PL, &Label);
Bind(&Label);
dc32(0);
CHECK(DisassembleEncoding(0) == 0x54000025);
}
}
TEST_CASE_METHOD(TestDisassembler, "Emitter: Branch: Branch consistent conditional") {
{
BackwardLabel Label;
Bind(&Label);
dc32(0);
bc(Condition::CC_PL, &Label);
CHECK(DisassembleEncoding(1) == 0x54fffff5);
}
{
ForwardLabel Label;
bc(Condition::CC_PL, &Label);
Bind(&Label);
dc32(0);
CHECK(DisassembleEncoding(0) == 0x54000035);
}
{
BiDirectionalLabel Label;
Bind(&Label);
dc32(0);
bc(Condition::CC_PL, &Label);
CHECK(DisassembleEncoding(1) == 0x54fffff5);
}
{
BiDirectionalLabel Label;
bc(Condition::CC_PL, &Label);
Bind(&Label);
dc32(0);
CHECK(DisassembleEncoding(0) == 0x54000035);
}
}
TEST_CASE_METHOD(TestDisassembler, "Emitter: Branch: Unconditional branch register") {
TEST_SINGLE(br(Reg::r29), "br x29");
TEST_SINGLE(blr(Reg::r29), "blr x29");
TEST_SINGLE(ret(), "ret");
TEST_SINGLE(ret(Reg::r29), "ret x29");
}
TEST_CASE_METHOD(TestDisassembler, "Emitter: Branch: Unconditional branch immediate") {
{
BackwardLabel Label;
Bind(&Label);
dc32(0);
b(&Label);
CHECK(DisassembleEncoding(1) == 0x17ffffff);
}
{
ForwardLabel Label;
b(&Label);
Bind(&Label);
dc32(0);
CHECK(DisassembleEncoding(0) == 0x14000001);
}
{
BiDirectionalLabel Label;
Bind(&Label);
dc32(0);
b(&Label);
CHECK(DisassembleEncoding(1) == 0x17ffffff);
}
{
BiDirectionalLabel Label;
b(&Label);
Bind(&Label);
dc32(0);
CHECK(DisassembleEncoding(0) == 0x14000001);
}
{
BackwardLabel Label;
Bind(&Label);
dc32(0);
bl(&Label);
CHECK(DisassembleEncoding(1) == 0x97ffffff);
}
{
ForwardLabel Label;
bl(&Label);
Bind(&Label);
dc32(0);
CHECK(DisassembleEncoding(0) == 0x94000001);
}
{
BiDirectionalLabel Label;
Bind(&Label);
dc32(0);
bl(&Label);
CHECK(DisassembleEncoding(1) == 0x97ffffff);
}
{
BiDirectionalLabel Label;
bl(&Label);
Bind(&Label);
dc32(0);
CHECK(DisassembleEncoding(0) == 0x94000001);
}
}
TEST_CASE_METHOD(TestDisassembler, "Emitter: Branch: Compare and branch") {
{
BackwardLabel Label;
Bind(&Label);
dc32(0);
cbz(Size::i32Bit, Reg::r29, &Label);
CHECK(DisassembleEncoding(1) == 0x34fffffd);
}
{
ForwardLabel Label;
cbz(Size::i32Bit, Reg::r29, &Label);
Bind(&Label);
dc32(0);
CHECK(DisassembleEncoding(0) == 0x3400003d);
}
{
BiDirectionalLabel Label;
Bind(&Label);
dc32(0);
cbz(Size::i32Bit, Reg::r29, &Label);
CHECK(DisassembleEncoding(1) == 0x34fffffd);
}
{
BiDirectionalLabel Label;
cbz(Size::i32Bit, Reg::r29, &Label);
Bind(&Label);
dc32(0);
CHECK(DisassembleEncoding(0) == 0x3400003d);
}
{
BackwardLabel Label;
Bind(&Label);
dc32(0);
cbz(Size::i64Bit, Reg::r29, &Label);
CHECK(DisassembleEncoding(1) == 0xb4fffffd);
}
{
ForwardLabel Label;
cbz(Size::i64Bit, Reg::r29, &Label);
Bind(&Label);
dc32(0);
CHECK(DisassembleEncoding(0) == 0xb400003d);
}
{
BiDirectionalLabel Label;
Bind(&Label);
dc32(0);
cbz(Size::i64Bit, Reg::r29, &Label);
CHECK(DisassembleEncoding(1) == 0xb4fffffd);
}
{
BiDirectionalLabel Label;
cbz(Size::i64Bit, Reg::r29, &Label);
Bind(&Label);
dc32(0);
CHECK(DisassembleEncoding(0) == 0xb400003d);
}
{
BackwardLabel Label;
Bind(&Label);
dc32(0);
cbnz(Size::i32Bit, Reg::r29, &Label);
CHECK(DisassembleEncoding(1) == 0x35fffffd);
}
{
ForwardLabel Label;
cbnz(Size::i32Bit, Reg::r29, &Label);
Bind(&Label);
dc32(0);
CHECK(DisassembleEncoding(0) == 0x3500003d);
}
{
BiDirectionalLabel Label;
Bind(&Label);
dc32(0);
cbnz(Size::i32Bit, Reg::r29, &Label);
CHECK(DisassembleEncoding(1) == 0x35fffffd);
}
{
BiDirectionalLabel Label;
cbnz(Size::i32Bit, Reg::r29, &Label);
Bind(&Label);
dc32(0);
CHECK(DisassembleEncoding(0) == 0x3500003d);
}
{
BackwardLabel Label;
Bind(&Label);
dc32(0);
cbnz(Size::i64Bit, Reg::r29, &Label);
CHECK(DisassembleEncoding(1) == 0xb5fffffd);
}
{
ForwardLabel Label;
cbnz(Size::i64Bit, Reg::r29, &Label);
Bind(&Label);
dc32(0);
CHECK(DisassembleEncoding(0) == 0xb500003d);
}
{
BiDirectionalLabel Label;
Bind(&Label);
dc32(0);
cbnz(Size::i64Bit, Reg::r29, &Label);
CHECK(DisassembleEncoding(1) == 0xb5fffffd);
}
{
BiDirectionalLabel Label;
cbnz(Size::i64Bit, Reg::r29, &Label);
Bind(&Label);
dc32(0);
CHECK(DisassembleEncoding(0) == 0xb500003d);
}
}
TEST_CASE_METHOD(TestDisassembler, "Emitter: Branch: Test and branch immediate") {
{
BackwardLabel Label;
Bind(&Label);
dc32(0);
tbz(Reg::r29, 0, &Label);
CHECK(DisassembleEncoding(1) == 0x3607fffd);
}
{
ForwardLabel Label;
tbz(Reg::r29, 0, &Label);
Bind(&Label);
dc32(0);
CHECK(DisassembleEncoding(0) == 0x3600003d);
}
{
BiDirectionalLabel Label;
Bind(&Label);
dc32(0);
tbz(Reg::r29, 0, &Label);
CHECK(DisassembleEncoding(1) == 0x3607fffd);
}
{
BiDirectionalLabel Label;
tbz(Reg::r29, 0, &Label);
Bind(&Label);
dc32(0);
CHECK(DisassembleEncoding(0) == 0x3600003d);
}
{
BackwardLabel Label;
Bind(&Label);
dc32(0);
tbz(Reg::r29, 63, &Label);
CHECK(DisassembleEncoding(1) == 0xb6fffffd);
}
{
ForwardLabel Label;
tbz(Reg::r29, 63, &Label);
Bind(&Label);
dc32(0);
CHECK(DisassembleEncoding(0) == 0xb6f8003d);
}
{
BiDirectionalLabel Label;
Bind(&Label);
dc32(0);
tbz(Reg::r29, 63, &Label);
CHECK(DisassembleEncoding(1) == 0xb6fffffd);
}
{
BiDirectionalLabel Label;
tbz(Reg::r29, 63, &Label);
Bind(&Label);
dc32(0);
CHECK(DisassembleEncoding(0) == 0xb6f8003d);
}
{
BackwardLabel Label;
Bind(&Label);
dc32(0);
tbnz(Reg::r29, 0, &Label);
CHECK(DisassembleEncoding(1) == 0x3707fffd);
}
{
ForwardLabel Label;
tbnz(Reg::r29, 0, &Label);
Bind(&Label);
dc32(0);
CHECK(DisassembleEncoding(0) == 0x3700003d);
}
{
BiDirectionalLabel Label;
Bind(&Label);
dc32(0);
tbnz(Reg::r29, 0, &Label);
CHECK(DisassembleEncoding(1) == 0x3707fffd);
}
{
BiDirectionalLabel Label;
tbnz(Reg::r29, 0, &Label);
Bind(&Label);
dc32(0);
CHECK(DisassembleEncoding(0) == 0x3700003d);
}
{
BackwardLabel Label;
Bind(&Label);
dc32(0);
tbnz(Reg::r29, 63, &Label);
CHECK(DisassembleEncoding(1) == 0xb7fffffd);
}
{
ForwardLabel Label;
tbnz(Reg::r29, 63, &Label);
Bind(&Label);
dc32(0);
CHECK(DisassembleEncoding(0) == 0xb7f8003d);
}
{
BiDirectionalLabel Label;
Bind(&Label);
dc32(0);
tbnz(Reg::r29, 63, &Label);
CHECK(DisassembleEncoding(1) == 0xb7fffffd);
}
{
BiDirectionalLabel Label;
tbnz(Reg::r29, 63, &Label);
Bind(&Label);
dc32(0);
CHECK(DisassembleEncoding(0) == 0xb7f8003d);
}
}
+24
View File
@@ -0,0 +1,24 @@
if (ENABLE_VIXL_DISASSEMBLER)
file(GLOB_RECURSE TESTS CONFIGURE_DEPENDS *.cpp)
set (LIBS fmt::fmt vixl Catch2::Catch2WithMain FEXCore_Base)
foreach(TEST ${TESTS})
get_filename_component(TEST_NAME ${TEST} NAME_WLE)
add_executable(Emitter_${TEST_NAME} ${TEST})
target_link_libraries(Emitter_${TEST_NAME} PRIVATE ${LIBS})
target_include_directories(Emitter_${TEST_NAME} PUBLIC "${CMAKE_CURRENT_SOURCE_DIR}/../../Source/")
set_target_properties(Emitter_${TEST_NAME} PROPERTIES RUNTIME_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/EmitterTests")
catch_discover_tests(Emitter_${TEST_NAME} TEST_SUFFIX ".${TEST_NAME}.Emitter")
endforeach()
execute_process(COMMAND "nproc" OUTPUT_VARIABLE CORES)
string(STRIP ${CORES} CORES)
add_custom_target(
emitter_tests
WORKING_DIRECTORY "${CMAKE_BINARY_DIR}/"
USES_TERMINAL
COMMAND "ctest" "--timeout" "302" "-j${CORES}" "-R" "\.*.Emitter$$")
else()
message(AUTHOR_WARNING "Tests are enabled but vixl disassembler is not. Emitter tests won't be built.")
endif()
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+852
View File
@@ -0,0 +1,852 @@
#include "TestDisassembler.h"
#include <catch2/catch.hpp>
#include <fcntl.h>
using namespace FEXCore::ARMEmitter;
TEST_CASE_METHOD(TestDisassembler, "Emitter: Scalar: Advanced SIMD scalar copy") {
TEST_SINGLE(dup(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 0), "mov b30, v29.b[0]");
TEST_SINGLE(dup(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 15), "mov b30, v29.b[15]");
TEST_SINGLE(mov(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 0), "mov b30, v29.b[0]");
TEST_SINGLE(mov(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 15), "mov b30, v29.b[15]");
TEST_SINGLE(dup(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 0), "mov h30, v29.h[0]");
TEST_SINGLE(dup(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 7), "mov h30, v29.h[7]");
TEST_SINGLE(mov(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 0), "mov h30, v29.h[0]");
TEST_SINGLE(mov(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 7), "mov h30, v29.h[7]");
TEST_SINGLE(dup(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 0), "mov s30, v29.s[0]");
TEST_SINGLE(dup(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 3), "mov s30, v29.s[3]");
TEST_SINGLE(mov(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 0), "mov s30, v29.s[0]");
TEST_SINGLE(mov(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 3), "mov s30, v29.s[3]");
TEST_SINGLE(dup(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 0), "mov d30, v29.d[0]");
TEST_SINGLE(dup(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 1), "mov d30, v29.d[1]");
TEST_SINGLE(mov(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 0), "mov d30, v29.d[0]");
TEST_SINGLE(mov(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 1), "mov d30, v29.d[1]");
}
TEST_CASE_METHOD(TestDisassembler, "Emitter: Scalar: Advanced SIMD scalar three same FP16") {
TEST_SINGLE(fmulx(HReg::h30, HReg::h29, HReg::h28), "fmulx h30, h29, h28");
TEST_SINGLE(fcmeq(HReg::h30, HReg::h29, HReg::h28), "fcmeq h30, h29, h28");
TEST_SINGLE(frecps(HReg::h30, HReg::h29, HReg::h28), "frecps h30, h29, h28");
TEST_SINGLE(frsqrts(HReg::h30, HReg::h29, HReg::h28), "frsqrts h30, h29, h28");
TEST_SINGLE(fcmge(HReg::h30, HReg::h29, HReg::h28), "fcmge h30, h29, h28");
TEST_SINGLE(facge(HReg::h30, HReg::h29, HReg::h28), "facge h30, h29, h28");
TEST_SINGLE(fabd(HReg::h30, HReg::h29, HReg::h28), "fabd h30, h29, h28");
TEST_SINGLE(fcmgt(HReg::h30, HReg::h29, HReg::h28), "fcmgt h30, h29, h28");
TEST_SINGLE(facgt(HReg::h30, HReg::h29, HReg::h28), "facgt h30, h29, h28");
}
TEST_CASE_METHOD(TestDisassembler, "Emitter: Scalar: Advanced SIMD scalar two-register miscellaneous FP16") {
TEST_SINGLE(fcvtns(HReg::h30, HReg::h29), "fcvtns h30, h29");
TEST_SINGLE(fcvtms(HReg::h30, HReg::h29), "fcvtms h30, h29");
TEST_SINGLE(fcvtas(HReg::h30, HReg::h29), "fcvtas h30, h29");
TEST_SINGLE(scvtf(HReg::h30, HReg::h29), "scvtf h30, h29");
TEST_SINGLE(fcmgt(HReg::h30, HReg::h29), "fcmgt h30, h29, #0.0");
TEST_SINGLE(fcmeq(HReg::h30, HReg::h29), "fcmeq h30, h29, #0.0");
TEST_SINGLE(fcmlt(HReg::h30, HReg::h29), "fcmlt h30, h29, #0.0");
TEST_SINGLE(fcvtps(HReg::h30, HReg::h29), "fcvtps h30, h29");
TEST_SINGLE(fcvtzs(HReg::h30, HReg::h29), "fcvtzs h30, h29");
TEST_SINGLE(frecpe(HReg::h30, HReg::h29), "frecpe h30, h29");
TEST_SINGLE(frecpx(HReg::h30, HReg::h29), "frecpx h30, h29");
TEST_SINGLE(fcvtnu(HReg::h30, HReg::h29), "fcvtnu h30, h29");
TEST_SINGLE(fcvtmu(HReg::h30, HReg::h29), "fcvtmu h30, h29");
TEST_SINGLE(fcvtau(HReg::h30, HReg::h29), "fcvtau h30, h29");
TEST_SINGLE(ucvtf(HReg::h30, HReg::h29), "ucvtf h30, h29");
TEST_SINGLE(fcmge(HReg::h30, HReg::h29), "fcmge h30, h29, #0.0");
TEST_SINGLE(fcmle(HReg::h30, HReg::h29), "fcmle h30, h29, #0.0");
TEST_SINGLE(fcvtpu(HReg::h30, HReg::h29), "fcvtpu h30, h29");
TEST_SINGLE(fcvtzu(HReg::h30, HReg::h29), "fcvtzu h30, h29");
TEST_SINGLE(frsqrte(HReg::h30, HReg::h29), "frsqrte h30, h29");
}
TEST_CASE_METHOD(TestDisassembler, "Emitter: Scalar: Advanced SIMD scalar three same extra") {
// TODO: Implement in emitter
}
TEST_CASE_METHOD(TestDisassembler, "Emitter: Scalar: Advanced SIMD scalar two-register miscellaneous") {
// Commented out lines showcase unallocated encodings.
TEST_SINGLE(suqadd(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "suqadd b30, b29");
TEST_SINGLE(suqadd(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "suqadd h30, h29");
TEST_SINGLE(suqadd(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "suqadd s30, s29");
TEST_SINGLE(suqadd(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "suqadd d30, d29");
TEST_SINGLE(sqabs(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "sqabs b30, b29");
TEST_SINGLE(sqabs(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "sqabs h30, h29");
TEST_SINGLE(sqabs(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "sqabs s30, s29");
TEST_SINGLE(sqabs(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "sqabs d30, d29");
//TEST_SINGLE(cmgt(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "cmgt b30, b29, #0");
//TEST_SINGLE(cmgt(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "cmgt h30, h29, #0");
//TEST_SINGLE(cmgt(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "cmgt s30, s29, #0");
TEST_SINGLE(cmgt(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "cmgt d30, d29, #0");
//TEST_SINGLE(cmeq(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "cmeq b30, b29, #0");
//TEST_SINGLE(cmeq(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "cmeq h30, h29, #0");
//TEST_SINGLE(cmeq(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "cmeq s30, s29, #0");
TEST_SINGLE(cmeq(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "cmeq d30, d29, #0");
//TEST_SINGLE(cmlt(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "cmlt b30, b29, #0");
//TEST_SINGLE(cmlt(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "cmlt h30, h29, #0");
//TEST_SINGLE(cmlt(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "cmlt s30, s29, #0");
TEST_SINGLE(cmlt(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "cmlt d30, d29, #0");
//TEST_SINGLE(abs(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "abs b30, b29");
//TEST_SINGLE(abs(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "abs h30, h29");
//TEST_SINGLE(abs(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "abs s30, s29");
TEST_SINGLE(abs(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "abs d30, d29");
TEST_SINGLE(sqxtn(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "sqxtn b30, h29");
TEST_SINGLE(sqxtn(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "sqxtn h30, s29");
TEST_SINGLE(sqxtn(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "sqxtn s30, d29");
//TEST_SINGLE(sqxtn(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "sqxtn d30, d29");
//TEST_SINGLE(fcvtns(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "fcvtns b30, b29");
//TEST_SINGLE(fcvtns(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "fcvtns h30, h29");
TEST_SINGLE(fcvtns(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "fcvtns s30, s29");
TEST_SINGLE(fcvtns(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "fcvtns d30, d29");
//TEST_SINGLE(fcvtms(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "fcvtms b30, b29");
//TEST_SINGLE(fcvtms(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "fcvtms h30, h29");
TEST_SINGLE(fcvtms(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "fcvtms s30, s29");
TEST_SINGLE(fcvtms(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "fcvtms d30, d29");
//TEST_SINGLE(fcvtas(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "fcvtas b30, b29");
//TEST_SINGLE(fcvtas(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "fcvtas h30, h29");
TEST_SINGLE(fcvtas(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "fcvtas s30, s29");
TEST_SINGLE(fcvtas(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "fcvtas d30, d29");
//TEST_SINGLE(scvtf(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "scvtf b30, b29");
//TEST_SINGLE(scvtf(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "scvtf h30, h29");
TEST_SINGLE(scvtf(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "scvtf s30, s29");
TEST_SINGLE(scvtf(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "scvtf d30, d29");
//TEST_SINGLE(fcmeq(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "fcmeq b30, b29");
//TEST_SINGLE(fcmeq(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "fcmeq h30, h29");
TEST_SINGLE(fcmeq(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "fcmeq s30, s29, #0.0");
TEST_SINGLE(fcmeq(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "fcmeq d30, d29, #0.0");
//TEST_SINGLE(fcmlt(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "fcmlt b30, b29");
//TEST_SINGLE(fcmlt(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "fcmlt h30, h29");
TEST_SINGLE(fcmlt(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "fcmlt s30, s29, #0.0");
TEST_SINGLE(fcmlt(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "fcmlt d30, d29, #0.0");
//TEST_SINGLE(fcvtps(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "fcvtps b30, b29");
//TEST_SINGLE(fcvtps(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "fcvtps h30, h29");
TEST_SINGLE(fcvtps(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "fcvtps s30, s29");
TEST_SINGLE(fcvtps(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "fcvtps d30, d29");
//TEST_SINGLE(fcvtzs(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "fcvtzs b30, b29");
//TEST_SINGLE(fcvtzs(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "fcvtzs h30, h29");
TEST_SINGLE(fcvtzs(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "fcvtzs s30, s29");
TEST_SINGLE(fcvtzs(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "fcvtzs d30, d29");
//TEST_SINGLE(frecpe(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "frecpe b30, b29");
//TEST_SINGLE(frecpe(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "frecpe h30, h29");
TEST_SINGLE(frecpe(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "frecpe s30, s29");
TEST_SINGLE(frecpe(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "frecpe d30, d29");
//TEST_SINGLE(frecpx(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "frecpx b30, b29");
//TEST_SINGLE(frecpx(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "frecpx h30, h29");
TEST_SINGLE(frecpx(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "frecpx s30, s29");
TEST_SINGLE(frecpx(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "frecpx d30, d29");
TEST_SINGLE(usqadd(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "usqadd b30, b29");
TEST_SINGLE(usqadd(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "usqadd h30, h29");
TEST_SINGLE(usqadd(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "usqadd s30, s29");
TEST_SINGLE(usqadd(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "usqadd d30, d29");
TEST_SINGLE(sqneg(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "sqneg b30, b29");
TEST_SINGLE(sqneg(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "sqneg h30, h29");
TEST_SINGLE(sqneg(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "sqneg s30, s29");
TEST_SINGLE(sqneg(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "sqneg d30, d29");
//TEST_SINGLE(cmge(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "cmge b30, b29");
//TEST_SINGLE(cmge(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "cmge h30, h29");
//TEST_SINGLE(cmge(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "cmge s30, s29");
TEST_SINGLE(cmge(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "cmge d30, d29, #0");
//TEST_SINGLE(cmle(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "cmle b30, b29");
//TEST_SINGLE(cmle(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "cmle h30, h29");
//TEST_SINGLE(cmle(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "cmle s30, s29");
TEST_SINGLE(cmle(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "cmle d30, d29, #0");
//TEST_SINGLE(neg(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "neg b30, b29");
//TEST_SINGLE(neg(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "neg h30, h29");
//TEST_SINGLE(neg(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "neg s30, s29");
TEST_SINGLE(neg(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "neg d30, d29");
TEST_SINGLE(sqxtun(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "sqxtun b30, h29");
TEST_SINGLE(sqxtun(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "sqxtun h30, s29");
TEST_SINGLE(sqxtun(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "sqxtun s30, d29");
//TEST_SINGLE(sqxtun(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "sqxtun d30, d29");
TEST_SINGLE(uqxtn(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "uqxtn b30, h29");
TEST_SINGLE(uqxtn(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "uqxtn h30, s29");
TEST_SINGLE(uqxtn(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "uqxtn s30, d29");
//TEST_SINGLE(uqxtn(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "uqxtn d30, d29");
//TEST_SINGLE(fcvtxn(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "fcvtxn b30, b29");
//TEST_SINGLE(fcvtxn(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "fcvtxn h30, h29");
TEST_SINGLE(fcvtxn(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "fcvtxn s30, d29");
//TEST_SINGLE(fcvtxn(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "fcvtxn d30, d29");
//TEST_SINGLE(fcvtnu(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "fcvtnu b30, b29");
//TEST_SINGLE(fcvtnu(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "fcvtnu h30, h29");
TEST_SINGLE(fcvtnu(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "fcvtnu s30, s29");
TEST_SINGLE(fcvtnu(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "fcvtnu d30, d29");
//TEST_SINGLE(fcvtmu(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "fcvtmu b30, b29");
//TEST_SINGLE(fcvtmu(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "fcvtmu h30, h29");
TEST_SINGLE(fcvtmu(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "fcvtmu s30, s29");
TEST_SINGLE(fcvtmu(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "fcvtmu d30, d29");
//TEST_SINGLE(fcvtau(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "fcvtau b30, b29");
//TEST_SINGLE(fcvtau(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "fcvtau h30, h29");
TEST_SINGLE(fcvtau(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "fcvtau s30, s29");
TEST_SINGLE(fcvtau(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "fcvtau d30, d29");
//TEST_SINGLE(ucvtf(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "ucvtf b30, b29");
//TEST_SINGLE(ucvtf(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "ucvtf h30, h29");
TEST_SINGLE(ucvtf(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "ucvtf s30, s29");
TEST_SINGLE(ucvtf(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "ucvtf d30, d29");
//TEST_SINGLE(fcmge(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "fcmge b30, b29");
//TEST_SINGLE(fcmge(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "fcmge h30, h29");
TEST_SINGLE(fcmge(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "fcmge s30, s29, #0.0");
TEST_SINGLE(fcmge(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "fcmge d30, d29, #0.0");
//TEST_SINGLE(fcmle(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "fcmle b30, b29");
//TEST_SINGLE(fcmle(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "fcmle h30, h29");
TEST_SINGLE(fcmle(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "fcmle s30, s29, #0.0");
TEST_SINGLE(fcmle(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "fcmle d30, d29, #0.0");
//TEST_SINGLE(fcvtpu(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "fcvtpu b30, b29");
//TEST_SINGLE(fcvtpu(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "fcvtpu h30, h29");
TEST_SINGLE(fcvtpu(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "fcvtpu s30, s29");
TEST_SINGLE(fcvtpu(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "fcvtpu d30, d29");
//TEST_SINGLE(fcvtzu(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "fcvtzu b30, b29");
//TEST_SINGLE(fcvtzu(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "fcvtzu h30, h29");
TEST_SINGLE(fcvtzu(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "fcvtzu s30, s29");
TEST_SINGLE(fcvtzu(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "fcvtzu d30, d29");
//TEST_SINGLE(frsqrte(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "frsqrte b30, b29");
//TEST_SINGLE(frsqrte(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "frsqrte h30, h29");
TEST_SINGLE(frsqrte(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "frsqrte s30, s29");
TEST_SINGLE(frsqrte(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "frsqrte d30, d29");
}
TEST_CASE_METHOD(TestDisassembler, "Emitter: Scalar: Advanced SIMD scalar pairwise") {
// Commented out lines showcase unallocated encodings.
//TEST_SINGLE(addp(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "addp b30, b29");
//TEST_SINGLE(addp(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "addp h30, h29");
//TEST_SINGLE(addp(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "addp s30, s29");
TEST_SINGLE(addp(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "addp d30, v29.2d");
TEST_SINGLE(fmaxnmp(HReg::h30, HReg::h29), "fmaxnmp h30, v29.2h");
//TEST_SINGLE(fmaxnmp(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "fmaxnmp b30, b29");
//TEST_SINGLE(fmaxnmp(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "fmaxnmp h30, h29");
TEST_SINGLE(fmaxnmp(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "fmaxnmp s30, v29.2s");
TEST_SINGLE(fmaxnmp(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "fmaxnmp d30, v29.2d");
TEST_SINGLE(faddp(HReg::h30, HReg::h29), "faddp h30, v29.2h");
//TEST_SINGLE(faddp(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "faddp b30, b29");
//TEST_SINGLE(faddp(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "faddp h30, h29");
TEST_SINGLE(faddp(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "faddp s30, v29.2s");
TEST_SINGLE(faddp(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "faddp d30, v29.2d");
TEST_SINGLE(fmaxp(HReg::h30, HReg::h29), "fmaxp h30, v29.2h");
//TEST_SINGLE(fmaxp(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "fmaxp b30, b29");
//TEST_SINGLE(fmaxp(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "fmaxp h30, h29");
TEST_SINGLE(fmaxp(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "fmaxp s30, v29.2s");
TEST_SINGLE(fmaxp(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "fmaxp d30, v29.2d");
TEST_SINGLE(fminnmp(HReg::h30, HReg::h29), "fminnmp h30, v29.2h");
//TEST_SINGLE(fminnmp(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "fminnmp b30, b29");
//TEST_SINGLE(fminnmp(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "fminnmp h30, h29");
TEST_SINGLE(fminnmp(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "fminnmp s30, v29.2s");
TEST_SINGLE(fminnmp(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "fminnmp d30, v29.2d");
TEST_SINGLE(fminp(HReg::h30, HReg::h29), "fminp h30, v29.2h");
//TEST_SINGLE(fminp(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "fminp b30, b29");
//TEST_SINGLE(fminp(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "fminp h30, h29");
TEST_SINGLE(fminp(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "fminp s30, v29.2s");
TEST_SINGLE(fminp(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "fminp d30, v29.2d");
}
TEST_CASE_METHOD(TestDisassembler, "Emitter: Scalar: Advanced SIMD scalar three different") {
// Commented out lines showcase unallocated encodings.
//TEST_SINGLE(sqdmlal(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, VReg::v28), "sqdmlal v30.16b, v29.16b, v28.v16b");
//TEST_SINGLE(sqdmlal(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, VReg::v28), "sqdmlal v30.16b, v29.16b, v28.v16b");
TEST_SINGLE(sqdmlal(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, VReg::v28), "sqdmlal s30, h29, h28");
TEST_SINGLE(sqdmlal(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, VReg::v28), "sqdmlal d30, s29, s28");
//TEST_SINGLE(sqdmlsl(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, VReg::v28), "sqdmlsl v30.16b, v29.16b, v28.v16b");
//TEST_SINGLE(sqdmlsl(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, VReg::v28), "sqdmlsl v30.16b, v29.16b, v28.v16b");
TEST_SINGLE(sqdmlsl(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, VReg::v28), "sqdmlsl s30, h29, h28");
TEST_SINGLE(sqdmlsl(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, VReg::v28), "sqdmlsl d30, s29, s28");
//TEST_SINGLE(sqdmull(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, VReg::v28), "sqdmull v30.16b, v29.16b, v28.v16b");
//TEST_SINGLE(sqdmull(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, VReg::v28), "sqdmull v30.16b, v29.16b, v28.v16b");
TEST_SINGLE(sqdmull(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, VReg::v28), "sqdmull s30, h29, h28");
TEST_SINGLE(sqdmull(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, VReg::v28), "sqdmull d30, s29, s28");
}
TEST_CASE_METHOD(TestDisassembler, "Emitter: Scalar: Advanced SIMD scalar three same") {
TEST_SINGLE(sqadd(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, VReg::v28), "sqadd b30, b29, b28");
TEST_SINGLE(sqadd(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, VReg::v28), "sqadd h30, h29, h28");
TEST_SINGLE(sqadd(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, VReg::v28), "sqadd s30, s29, s28");
TEST_SINGLE(sqadd(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, VReg::v28), "sqadd d30, d29, d28");
TEST_SINGLE(sqsub(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, VReg::v28), "sqsub b30, b29, b28");
TEST_SINGLE(sqsub(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, VReg::v28), "sqsub h30, h29, h28");
TEST_SINGLE(sqsub(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, VReg::v28), "sqsub s30, s29, s28");
TEST_SINGLE(sqsub(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, VReg::v28), "sqsub d30, d29, d28");
//TEST_SINGLE(cmgt(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, VReg::v28), "cmgt b30, b29, b28");
//TEST_SINGLE(cmgt(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, VReg::v28), "cmgt h30, h29, h28");
//TEST_SINGLE(cmgt(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, VReg::v28), "cmgt s30, s29, s28");
TEST_SINGLE(cmgt(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, VReg::v28), "cmgt d30, d29, d28");
//TEST_SINGLE(cmge(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, VReg::v28), "cmge b30, b29, b28");
//TEST_SINGLE(cmge(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, VReg::v28), "cmge h30, h29, h28");
//TEST_SINGLE(cmge(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, VReg::v28), "cmge s30, s29, s28");
TEST_SINGLE(cmge(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, VReg::v28), "cmge d30, d29, d28");
//TEST_SINGLE(sshl(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, VReg::v28), "sshl b30, b29, b28");
//TEST_SINGLE(sshl(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, VReg::v28), "sshl h30, h29, h28");
//TEST_SINGLE(sshl(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, VReg::v28), "sshl s30, s29, s28");
TEST_SINGLE(sshl(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, VReg::v28), "sshl d30, d29, d28");
TEST_SINGLE(sqshl(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, VReg::v28), "sqshl b30, b29, b28");
TEST_SINGLE(sqshl(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, VReg::v28), "sqshl h30, h29, h28");
TEST_SINGLE(sqshl(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, VReg::v28), "sqshl s30, s29, s28");
TEST_SINGLE(sqshl(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, VReg::v28), "sqshl d30, d29, d28");
//TEST_SINGLE(srshl(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, VReg::v28), "srshl b30, b29, b28");
//TEST_SINGLE(srshl(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, VReg::v28), "srshl h30, h29, h28");
//TEST_SINGLE(srshl(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, VReg::v28), "srshl s30, s29, s28");
TEST_SINGLE(srshl(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, VReg::v28), "srshl d30, d29, d28");
TEST_SINGLE(sqrshl(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, VReg::v28), "sqrshl b30, b29, b28");
TEST_SINGLE(sqrshl(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, VReg::v28), "sqrshl h30, h29, h28");
TEST_SINGLE(sqrshl(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, VReg::v28), "sqrshl s30, s29, s28");
TEST_SINGLE(sqrshl(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, VReg::v28), "sqrshl d30, d29, d28");
//TEST_SINGLE(add(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, VReg::v28), "add b30, b29, b28");
//TEST_SINGLE(add(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, VReg::v28), "add h30, h29, h28");
//TEST_SINGLE(add(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, VReg::v28), "add s30, s29, s28");
TEST_SINGLE(add(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, VReg::v28), "add d30, d29, d28");
//TEST_SINGLE(cmtst(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, VReg::v28), "cmtst b30, b29, b28");
//TEST_SINGLE(cmtst(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, VReg::v28), "cmtst h30, h29, h28");
//TEST_SINGLE(cmtst(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, VReg::v28), "cmtst s30, s29, s28");
TEST_SINGLE(cmtst(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, VReg::v28), "cmtst d30, d29, d28");
//TEST_SINGLE(sqdmulh(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, VReg::v28), "sqdmulh b30, b29, b28");
TEST_SINGLE(sqdmulh(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, VReg::v28), "sqdmulh h30, h29, h28");
TEST_SINGLE(sqdmulh(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, VReg::v28), "sqdmulh s30, s29, s28");
//TEST_SINGLE(sqdmulh(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, VReg::v28), "sqdmulh d30, d29, d28");
//TEST_SINGLE(fmulx(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, VReg::v28), "fmulx b30, b29, b28");
//TEST_SINGLE(fmulx(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, VReg::v28), "fmulx h30, h29, h28");
TEST_SINGLE(fmulx(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, VReg::v28), "fmulx s30, s29, s28");
TEST_SINGLE(fmulx(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, VReg::v28), "fmulx d30, d29, d28");
//TEST_SINGLE(fcmeq(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, VReg::v28), "fcmeq b30, b29, b28");
//TEST_SINGLE(fcmeq(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, VReg::v28), "fcmeq h30, h29, h28");
TEST_SINGLE(fcmeq(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, VReg::v28), "fcmeq s30, s29, s28");
TEST_SINGLE(fcmeq(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, VReg::v28), "fcmeq d30, d29, d28");
//TEST_SINGLE(frecps(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, VReg::v28), "frecps b30, b29, b28");
//TEST_SINGLE(frecps(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, VReg::v28), "frecps h30, h29, h28");
TEST_SINGLE(frecps(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, VReg::v28), "frecps s30, s29, s28");
TEST_SINGLE(frecps(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, VReg::v28), "frecps d30, d29, d28");
//TEST_SINGLE(frsqrts(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, VReg::v28), "frsqrts b30, b29, b28");
//TEST_SINGLE(frsqrts(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, VReg::v28), "frsqrts h30, h29, h28");
TEST_SINGLE(frsqrts(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, VReg::v28), "frsqrts s30, s29, s28");
TEST_SINGLE(frsqrts(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, VReg::v28), "frsqrts d30, d29, d28");
TEST_SINGLE(uqadd(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, VReg::v28), "uqadd b30, b29, b28");
TEST_SINGLE(uqadd(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, VReg::v28), "uqadd h30, h29, h28");
TEST_SINGLE(uqadd(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, VReg::v28), "uqadd s30, s29, s28");
TEST_SINGLE(uqadd(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, VReg::v28), "uqadd d30, d29, d28");
TEST_SINGLE(uqsub(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, VReg::v28), "uqsub b30, b29, b28");
TEST_SINGLE(uqsub(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, VReg::v28), "uqsub h30, h29, h28");
TEST_SINGLE(uqsub(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, VReg::v28), "uqsub s30, s29, s28");
TEST_SINGLE(uqsub(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, VReg::v28), "uqsub d30, d29, d28");
//TEST_SINGLE(cmhi(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, VReg::v28), "cmhi b30, b29, b28");
//TEST_SINGLE(cmhi(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, VReg::v28), "cmhi h30, h29, h28");
//TEST_SINGLE(cmhi(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, VReg::v28), "cmhi s30, s29, s28");
TEST_SINGLE(cmhi(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, VReg::v28), "cmhi d30, d29, d28");
//TEST_SINGLE(cmhs(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, VReg::v28), "cmhs b30, b29, b28");
//TEST_SINGLE(cmhs(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, VReg::v28), "cmhs h30, h29, h28");
//TEST_SINGLE(cmhs(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, VReg::v28), "cmhs s30, s29, s28");
TEST_SINGLE(cmhs(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, VReg::v28), "cmhs d30, d29, d28");
//TEST_SINGLE(ushl(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, VReg::v28), "ushl b30, b29, b28");
//TEST_SINGLE(ushl(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, VReg::v28), "ushl h30, h29, h28");
//TEST_SINGLE(ushl(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, VReg::v28), "ushl s30, s29, s28");
TEST_SINGLE(ushl(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, VReg::v28), "ushl d30, d29, d28");
TEST_SINGLE(uqshl(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, VReg::v28), "uqshl b30, b29, b28");
TEST_SINGLE(uqshl(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, VReg::v28), "uqshl h30, h29, h28");
TEST_SINGLE(uqshl(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, VReg::v28), "uqshl s30, s29, s28");
TEST_SINGLE(uqshl(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, VReg::v28), "uqshl d30, d29, d28");
//TEST_SINGLE(urshl(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, VReg::v28), "urshl b30, b29, b28");
//TEST_SINGLE(urshl(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, VReg::v28), "urshl h30, h29, h28");
//TEST_SINGLE(urshl(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, VReg::v28), "urshl s30, s29, s28");
TEST_SINGLE(urshl(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, VReg::v28), "urshl d30, d29, d28");
TEST_SINGLE(uqrshl(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, VReg::v28), "uqrshl b30, b29, b28");
TEST_SINGLE(uqrshl(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, VReg::v28), "uqrshl h30, h29, h28");
TEST_SINGLE(uqrshl(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, VReg::v28), "uqrshl s30, s29, s28");
TEST_SINGLE(uqrshl(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, VReg::v28), "uqrshl d30, d29, d28");
//TEST_SINGLE(sub(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, VReg::v28), "sub b30, b29, b28");
//TEST_SINGLE(sub(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, VReg::v28), "sub h30, h29, h28");
//TEST_SINGLE(sub(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, VReg::v28), "sub s30, s29, s28");
TEST_SINGLE(sub(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, VReg::v28), "sub d30, d29, d28");
//TEST_SINGLE(cmeq(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, VReg::v28), "cmeq b30, b29, b28");
//TEST_SINGLE(cmeq(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, VReg::v28), "cmeq h30, h29, h28");
//TEST_SINGLE(cmeq(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, VReg::v28), "cmeq s30, s29, s28");
TEST_SINGLE(cmeq(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, VReg::v28), "cmeq d30, d29, d28");
//TEST_SINGLE(sqrdmulh(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, VReg::v28), "sqrdmulh b30, b29, b28");
TEST_SINGLE(sqrdmulh(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, VReg::v28), "sqrdmulh h30, h29, h28");
TEST_SINGLE(sqrdmulh(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, VReg::v28), "sqrdmulh s30, s29, s28");
//TEST_SINGLE(sqrdmulh(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, VReg::v28), "sqrdmulh d30, d29, d28");
//TEST_SINGLE(fcmge(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, VReg::v28), "fcmge b30, b29, b28");
//TEST_SINGLE(fcmge(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, VReg::v28), "fcmge h30, h29, h28");
TEST_SINGLE(fcmge(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, VReg::v28), "fcmge s30, s29, s28");
TEST_SINGLE(fcmge(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, VReg::v28), "fcmge d30, d29, d28");
//TEST_SINGLE(facge(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, VReg::v28), "facge b30, b29, b28");
//TEST_SINGLE(facge(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, VReg::v28), "facge h30, h29, h28");
TEST_SINGLE(facge(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, VReg::v28), "facge s30, s29, s28");
TEST_SINGLE(facge(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, VReg::v28), "facge d30, d29, d28");
//TEST_SINGLE(fabd(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, VReg::v28), "fabd b30, b29, b28");
//TEST_SINGLE(fabd(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, VReg::v28), "fabd h30, h29, h28");
TEST_SINGLE(fabd(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, VReg::v28), "fabd s30, s29, s28");
TEST_SINGLE(fabd(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, VReg::v28), "fabd d30, d29, d28");
//TEST_SINGLE(fcmgt(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, VReg::v28), "fcmgt b30, b29, b28");
//TEST_SINGLE(fcmgt(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, VReg::v28), "fcmgt h30, h29, h28");
TEST_SINGLE(fcmgt(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, VReg::v28), "fcmgt s30, s29, s28");
TEST_SINGLE(fcmgt(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, VReg::v28), "fcmgt d30, d29, d28");
//TEST_SINGLE(facgt(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, VReg::v28), "facgt b30, b29, b28");
//TEST_SINGLE(facgt(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, VReg::v28), "facgt h30, h29, h28");
TEST_SINGLE(facgt(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, VReg::v28), "facgt s30, s29, s28");
TEST_SINGLE(facgt(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, VReg::v28), "facgt d30, d29, d28");
}
TEST_CASE_METHOD(TestDisassembler, "Emitter: Scalar: Advanced SIMD scalar shift by immediate") {
// TODO: Implement `UCVTF, FCVTZU' in emitter
// TEST_SINGLE(sshr(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 1), "sshr b30, b29, #1");
// TEST_SINGLE(sshr(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 7), "sshr b30, b29, #7");
// TEST_SINGLE(sshr(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 1), "sshr h30, h29, #1");
// TEST_SINGLE(sshr(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 15), "sshr h30, h29, #15");
// TEST_SINGLE(sshr(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 1), "sshr s30, s29, #1");
// TEST_SINGLE(sshr(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 31), "sshr s30, s29, #31");
TEST_SINGLE(sshr(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 1), "sshr d30, d29, #1");
TEST_SINGLE(sshr(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 63), "sshr d30, d29, #63");
//TEST_SINGLE(ssra(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 1), "ssra b30, b29, #1");
//TEST_SINGLE(ssra(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 7), "ssra b30, b29, #7");
//TEST_SINGLE(ssra(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 1), "ssra h30, h29, #1");
//TEST_SINGLE(ssra(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 15), "ssra h30, h29, #15");
//TEST_SINGLE(ssra(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 1), "ssra s30, s29, #1");
//TEST_SINGLE(ssra(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 31), "ssra s30, s29, #31");
TEST_SINGLE(ssra(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 1), "ssra d30, d29, #1");
TEST_SINGLE(ssra(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 63), "ssra d30, d29, #63");
//TEST_SINGLE(srshr(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 1), "srshr b30, b29, #1");
//TEST_SINGLE(srshr(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 7), "srshr b30, b29, #7");
//TEST_SINGLE(srshr(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 1), "srshr h30, h29, #1");
//TEST_SINGLE(srshr(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 15), "srshr h30, h29, #15");
//TEST_SINGLE(srshr(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 1), "srshr s30, s29, #1");
//TEST_SINGLE(srshr(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 31), "srshr s30, s29, #31");
TEST_SINGLE(srshr(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 1), "srshr d30, d29, #1");
TEST_SINGLE(srshr(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 63), "srshr d30, d29, #63");
//TEST_SINGLE(srsra(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 1), "srsra b30, b29, #1");
//TEST_SINGLE(srsra(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 7), "srsra b30, b29, #7");
//TEST_SINGLE(srsra(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 1), "srsra h30, h29, #1");
//TEST_SINGLE(srsra(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 15), "srsra h30, h29, #15");
//TEST_SINGLE(srsra(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 1), "srsra s30, s29, #1");
//TEST_SINGLE(srsra(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 31), "srsra s30, s29, #31");
TEST_SINGLE(srsra(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 1), "srsra d30, d29, #1");
TEST_SINGLE(srsra(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 63), "srsra d30, d29, #63");
//TEST_SINGLE(shl(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 1), "shl b30, b29, #1");
//TEST_SINGLE(shl(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 7), "shl b30, b29, #7");
//TEST_SINGLE(shl(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 1), "shl h30, h29, #1");
//TEST_SINGLE(shl(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 15), "shl h30, h29, #15");
//TEST_SINGLE(shl(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 1), "shl s30, s29, #1");
//TEST_SINGLE(shl(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 31), "shl s30, s29, #31");
TEST_SINGLE(shl(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 1), "shl d30, d29, #1");
TEST_SINGLE(shl(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 63), "shl d30, d29, #63");
TEST_SINGLE(sqshl(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 1), "sqshl b30, b29, #1");
TEST_SINGLE(sqshl(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 7), "sqshl b30, b29, #7");
TEST_SINGLE(sqshl(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 1), "sqshl h30, h29, #1");
TEST_SINGLE(sqshl(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 15), "sqshl h30, h29, #15");
TEST_SINGLE(sqshl(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 1), "sqshl s30, s29, #1");
TEST_SINGLE(sqshl(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 31), "sqshl s30, s29, #31");
TEST_SINGLE(sqshl(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 1), "sqshl d30, d29, #1");
TEST_SINGLE(sqshl(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 63), "sqshl d30, d29, #63");
TEST_SINGLE(sqshrn(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 1), "sqshrn b30, h29, #1");
TEST_SINGLE(sqshrn(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 7), "sqshrn b30, h29, #7");
TEST_SINGLE(sqshrn(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 1), "sqshrn h30, s29, #1");
TEST_SINGLE(sqshrn(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 15), "sqshrn h30, s29, #15");
TEST_SINGLE(sqshrn(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 1), "sqshrn s30, d29, #1");
TEST_SINGLE(sqshrn(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 31), "sqshrn s30, d29, #31");
//TEST_SINGLE(sqshrn(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 1), "sqshrn d30, d29, #1");
//TEST_SINGLE(sqshrn(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 63), "sqshrn d30, d29, #63");
TEST_SINGLE(sqrshrn(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 1), "sqrshrn b30, h29, #1");
TEST_SINGLE(sqrshrn(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 7), "sqrshrn b30, h29, #7");
TEST_SINGLE(sqrshrn(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 1), "sqrshrn h30, s29, #1");
TEST_SINGLE(sqrshrn(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 15), "sqrshrn h30, s29, #15");
TEST_SINGLE(sqrshrn(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 1), "sqrshrn s30, d29, #1");
TEST_SINGLE(sqrshrn(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 31), "sqrshrn s30, d29, #31");
//TEST_SINGLE(sqrshrn(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 1), "sqrshrn d30, d29, #1");
//TEST_SINGLE(sqrshrn(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 63), "sqrshrn d30, d29, #63");
// TODO: Implement `SCVTF, FCVTZS` in emitter
//TEST_SINGLE(ushr(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 1), "ushr b30, b29, #1");
//TEST_SINGLE(ushr(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 7), "ushr b30, b29, #7");
//TEST_SINGLE(ushr(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 1), "ushr h30, h29, #1");
//TEST_SINGLE(ushr(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 15), "ushr h30, h29, #15");
//TEST_SINGLE(ushr(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 1), "ushr s30, s29, #1");
//TEST_SINGLE(ushr(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 31), "ushr s30, s29, #31");
TEST_SINGLE(ushr(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 1), "ushr d30, d29, #1");
TEST_SINGLE(ushr(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 63), "ushr d30, d29, #63");
//TEST_SINGLE(usra(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 1), "usra b30, b29, #1");
//TEST_SINGLE(usra(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 7), "usra b30, b29, #7");
//TEST_SINGLE(usra(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 1), "usra h30, h29, #1");
//TEST_SINGLE(usra(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 15), "usra h30, h29, #15");
//TEST_SINGLE(usra(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 1), "usra s30, s29, #1");
//TEST_SINGLE(usra(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 31), "usra s30, s29, #31");
TEST_SINGLE(usra(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 1), "usra d30, d29, #1");
TEST_SINGLE(usra(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 63), "usra d30, d29, #63");
//TEST_SINGLE(urshr(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 1), "urshr b30, b29, #1");
//TEST_SINGLE(urshr(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 7), "urshr b30, b29, #7");
//TEST_SINGLE(urshr(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 1), "urshr h30, h29, #1");
//TEST_SINGLE(urshr(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 15), "urshr h30, h29, #15");
//TEST_SINGLE(urshr(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 1), "urshr s30, s29, #1");
//TEST_SINGLE(urshr(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 31), "urshr s30, s29, #31");
TEST_SINGLE(urshr(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 1), "urshr d30, d29, #1");
TEST_SINGLE(urshr(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 63), "urshr d30, d29, #63");
//TEST_SINGLE(ursra(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 1), "ursra b30, b29, #1");
//TEST_SINGLE(ursra(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 7), "ursra b30, b29, #7");
//TEST_SINGLE(ursra(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 1), "ursra h30, h29, #1");
//TEST_SINGLE(ursra(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 15), "ursra h30, h29, #15");
//TEST_SINGLE(ursra(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 1), "ursra s30, s29, #1");
//TEST_SINGLE(ursra(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 31), "ursra s30, s29, #31");
TEST_SINGLE(ursra(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 1), "ursra d30, d29, #1");
TEST_SINGLE(ursra(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 63), "ursra d30, d29, #63");
//TEST_SINGLE(sri(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 1), "sri b30, b29, #1");
//TEST_SINGLE(sri(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 7), "sri b30, b29, #7");
//TEST_SINGLE(sri(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 1), "sri h30, h29, #1");
//TEST_SINGLE(sri(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 15), "sri h30, h29, #15");
//TEST_SINGLE(sri(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 1), "sri s30, s29, #1");
//TEST_SINGLE(sri(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 31), "sri s30, s29, #31");
TEST_SINGLE(sri(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 1), "sri d30, d29, #1");
TEST_SINGLE(sri(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 63), "sri d30, d29, #63");
//TEST_SINGLE(sli(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 1), "sli b30, b29, #1");
//TEST_SINGLE(sli(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 7), "sli b30, b29, #7");
//TEST_SINGLE(sli(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 1), "sli h30, h29, #1");
//TEST_SINGLE(sli(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 15), "sli h30, h29, #15");
//TEST_SINGLE(sli(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 1), "sli s30, s29, #1");
//TEST_SINGLE(sli(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 31), "sli s30, s29, #31");
TEST_SINGLE(sli(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 1), "sli d30, d29, #1");
TEST_SINGLE(sli(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 63), "sli d30, d29, #63");
TEST_SINGLE(sqshlu(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 1), "sqshlu b30, b29, #1");
TEST_SINGLE(sqshlu(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 7), "sqshlu b30, b29, #7");
TEST_SINGLE(sqshlu(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 1), "sqshlu h30, h29, #1");
TEST_SINGLE(sqshlu(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 15), "sqshlu h30, h29, #15");
TEST_SINGLE(sqshlu(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 1), "sqshlu s30, s29, #1");
TEST_SINGLE(sqshlu(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 31), "sqshlu s30, s29, #31");
TEST_SINGLE(sqshlu(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 1), "sqshlu d30, d29, #1");
TEST_SINGLE(sqshlu(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 63), "sqshlu d30, d29, #63");
TEST_SINGLE(uqshl(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 1), "uqshl b30, b29, #1");
TEST_SINGLE(uqshl(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 7), "uqshl b30, b29, #7");
TEST_SINGLE(uqshl(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 1), "uqshl h30, h29, #1");
TEST_SINGLE(uqshl(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 15), "uqshl h30, h29, #15");
TEST_SINGLE(uqshl(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 1), "uqshl s30, s29, #1");
TEST_SINGLE(uqshl(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 31), "uqshl s30, s29, #31");
TEST_SINGLE(uqshl(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 1), "uqshl d30, d29, #1");
TEST_SINGLE(uqshl(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 63), "uqshl d30, d29, #63");
TEST_SINGLE(sqshrun(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 1), "sqshrun b30, h29, #1");
TEST_SINGLE(sqshrun(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 7), "sqshrun b30, h29, #7");
TEST_SINGLE(sqshrun(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 1), "sqshrun h30, s29, #1");
TEST_SINGLE(sqshrun(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 15), "sqshrun h30, s29, #15");
TEST_SINGLE(sqshrun(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 1), "sqshrun s30, d29, #1");
TEST_SINGLE(sqshrun(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 31), "sqshrun s30, d29, #31");
//TEST_SINGLE(sqshrun(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 1), "sqshrun d30, d29, #1");
//TEST_SINGLE(sqshrun(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 63), "sqshrun d30, d29, #63");
TEST_SINGLE(sqrshrun(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 1), "sqrshrun b30, h29, #1");
TEST_SINGLE(sqrshrun(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 7), "sqrshrun b30, h29, #7");
TEST_SINGLE(sqrshrun(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 1), "sqrshrun h30, s29, #1");
TEST_SINGLE(sqrshrun(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 15), "sqrshrun h30, s29, #15");
TEST_SINGLE(sqrshrun(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 1), "sqrshrun s30, d29, #1");
TEST_SINGLE(sqrshrun(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 31), "sqrshrun s30, d29, #31");
//TEST_SINGLE(sqrshrun(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 1), "sqrshrun d30, d29, #1");
//TEST_SINGLE(sqrshrun(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 63), "sqrshrun d30, d29, #63");
TEST_SINGLE(uqshrn(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 1), "uqshrn b30, h29, #1");
TEST_SINGLE(uqshrn(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 7), "uqshrn b30, h29, #7");
TEST_SINGLE(uqshrn(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 1), "uqshrn h30, s29, #1");
TEST_SINGLE(uqshrn(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 15), "uqshrn h30, s29, #15");
TEST_SINGLE(uqshrn(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 1), "uqshrn s30, d29, #1");
TEST_SINGLE(uqshrn(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 31), "uqshrn s30, d29, #31");
//TEST_SINGLE(uqshrn(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 1), "uqshrn d30, d29, #1");
//TEST_SINGLE(uqshrn(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 63), "uqshrn d30, d29, #63");
TEST_SINGLE(uqrshrn(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 1), "uqrshrn b30, h29, #1");
TEST_SINGLE(uqrshrn(ScalarRegSize::i8Bit, VReg::v30, VReg::v29, 7), "uqrshrn b30, h29, #7");
TEST_SINGLE(uqrshrn(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 1), "uqrshrn h30, s29, #1");
TEST_SINGLE(uqrshrn(ScalarRegSize::i16Bit, VReg::v30, VReg::v29, 15), "uqrshrn h30, s29, #15");
TEST_SINGLE(uqrshrn(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 1), "uqrshrn s30, d29, #1");
TEST_SINGLE(uqrshrn(ScalarRegSize::i32Bit, VReg::v30, VReg::v29, 31), "uqrshrn s30, d29, #31");
//TEST_SINGLE(uqrshrn(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 1), "uqrshrn d30, d29, #1");
//TEST_SINGLE(uqrshrn(ScalarRegSize::i64Bit, VReg::v30, VReg::v29, 63), "uqrshrn d30, d29, #63");
// TODO: Implement `UCVTF, FCVTZU' in emitter
}
TEST_CASE_METHOD(TestDisassembler, "Emitter: Scalar: Advanced SIMD scalar x indexed element") {
// TODO: Implement in emitter
}
TEST_CASE_METHOD(TestDisassembler, "Emitter: Scalar: Floating-point data-processing (1 source)") {
TEST_SINGLE(fmov(SReg::s30, SReg::s29), "fmov s30, s29");
TEST_SINGLE(fabs(SReg::s30, SReg::s29), "fabs s30, s29");
TEST_SINGLE(fneg(SReg::s30, SReg::s29), "fneg s30, s29");
TEST_SINGLE(fsqrt(SReg::s30, SReg::s29), "fsqrt s30, s29");
TEST_SINGLE(fcvt(DReg::d30, SReg::s29), "fcvt d30, s29");
TEST_SINGLE(fcvt(HReg::h30, SReg::s29), "fcvt h30, s29");
TEST_SINGLE(frintn(SReg::s30, SReg::s29), "frintn s30, s29");
TEST_SINGLE(frintp(SReg::s30, SReg::s29), "frintp s30, s29");
TEST_SINGLE(frintm(SReg::s30, SReg::s29), "frintm s30, s29");
TEST_SINGLE(frintz(SReg::s30, SReg::s29), "frintz s30, s29");
TEST_SINGLE(frinta(SReg::s30, SReg::s29), "frinta s30, s29");
TEST_SINGLE(frintx(SReg::s30, SReg::s29), "frintx s30, s29");
TEST_SINGLE(frinti(SReg::s30, SReg::s29), "frinti s30, s29");
TEST_SINGLE(frint32z(SReg::s30, SReg::s29), "frint32z s30, s29");
TEST_SINGLE(frint32x(SReg::s30, SReg::s29), "frint32x s30, s29");
TEST_SINGLE(frint64z(SReg::s30, SReg::s29), "frint64z s30, s29");
TEST_SINGLE(frint64x(SReg::s30, SReg::s29), "frint64x s30, s29");
TEST_SINGLE(fmov(DReg::d30, DReg::d29), "fmov d30, d29");
TEST_SINGLE(fabs(DReg::d30, DReg::d29), "fabs d30, d29");
TEST_SINGLE(fneg(DReg::d30, DReg::d29), "fneg d30, d29");
TEST_SINGLE(fsqrt(DReg::d30, DReg::d29), "fsqrt d30, d29");
TEST_SINGLE(fcvt(SReg::s30, DReg::d29), "fcvt s30, d29");
if (false) {
// vixl doesn't support this instruction.
TEST_SINGLE(bfcvt(HReg::h30, DReg::d29), "bfcvt h30, d29");
}
TEST_SINGLE(fcvt(HReg::h30, DReg::d29), "fcvt h30, d29");
TEST_SINGLE(frintn(DReg::d30, DReg::d29), "frintn d30, d29");
TEST_SINGLE(frintp(DReg::d30, DReg::d29), "frintp d30, d29");
TEST_SINGLE(frintm(DReg::d30, DReg::d29), "frintm d30, d29");
TEST_SINGLE(frintz(DReg::d30, DReg::d29), "frintz d30, d29");
TEST_SINGLE(frinta(DReg::d30, DReg::d29), "frinta d30, d29");
TEST_SINGLE(frintx(DReg::d30, DReg::d29), "frintx d30, d29");
TEST_SINGLE(frinti(DReg::d30, DReg::d29), "frinti d30, d29");
TEST_SINGLE(frint32z(DReg::d30, DReg::d29), "frint32z d30, d29");
TEST_SINGLE(frint32x(DReg::d30, DReg::d29), "frint32x d30, d29");
TEST_SINGLE(frint64z(DReg::d30, DReg::d29), "frint64z d30, d29");
TEST_SINGLE(frint64x(DReg::d30, DReg::d29), "frint64x d30, d29");
TEST_SINGLE(fmov(HReg::h30, HReg::h29), "fmov h30, h29");
TEST_SINGLE(fabs(HReg::h30, HReg::h29), "fabs h30, h29");
TEST_SINGLE(fneg(HReg::h30, HReg::h29), "fneg h30, h29");
TEST_SINGLE(fsqrt(HReg::h30, HReg::h29), "fsqrt h30, h29");
TEST_SINGLE(fcvt(SReg::s30, HReg::h29), "fcvt s30, h29");
TEST_SINGLE(fcvt(DReg::d30, HReg::h29), "fcvt d30, h29");
TEST_SINGLE(frintn(HReg::h30, HReg::h29), "frintn h30, h29");
TEST_SINGLE(frintp(HReg::h30, HReg::h29), "frintp h30, h29");
TEST_SINGLE(frintm(HReg::h30, HReg::h29), "frintm h30, h29");
TEST_SINGLE(frintz(HReg::h30, HReg::h29), "frintz h30, h29");
TEST_SINGLE(frinta(HReg::h30, HReg::h29), "frinta h30, h29");
TEST_SINGLE(frintx(HReg::h30, HReg::h29), "frintx h30, h29");
TEST_SINGLE(frinti(HReg::h30, HReg::h29), "frinti h30, h29");
}
TEST_CASE_METHOD(TestDisassembler, "Emitter: Scalar: Floating-point compare") {
// Commented out lines showcase unallocated encodings.
//TEST_SINGLE(fcmp(ScalarRegSize::i8Bit, VReg::v30, VReg::v29), "fcmp b30, b29");
TEST_SINGLE(fcmp(ScalarRegSize::i16Bit, VReg::v30, VReg::v29), "fcmp h30, h29");
TEST_SINGLE(fcmp(ScalarRegSize::i32Bit, VReg::v30, VReg::v29), "fcmp s30, s29");
TEST_SINGLE(fcmp(ScalarRegSize::i64Bit, VReg::v30, VReg::v29), "fcmp d30, d29");
TEST_SINGLE(fcmp(SReg::s30, SReg::s29), "fcmp s30, s29");
TEST_SINGLE(fcmp(SReg::s30), "fcmp s30, #0.0");
TEST_SINGLE(fcmpe(SReg::s30, SReg::s29), "fcmpe s30, s29");
TEST_SINGLE(fcmpe(SReg::s30), "fcmpe s30, #0.0");
TEST_SINGLE(fcmp(DReg::d30, DReg::d29), "fcmp d30, d29");
TEST_SINGLE(fcmp(DReg::d30), "fcmp d30, #0.0");
TEST_SINGLE(fcmpe(DReg::d30, DReg::d29), "fcmpe d30, d29");
TEST_SINGLE(fcmpe(DReg::d30), "fcmpe d30, #0.0");
TEST_SINGLE(fcmp(HReg::h30, HReg::h29), "fcmp h30, h29");
TEST_SINGLE(fcmp(HReg::h30), "fcmp h30, #0.0");
TEST_SINGLE(fcmpe(HReg::h30, HReg::h29), "fcmpe h30, h29");
TEST_SINGLE(fcmpe(HReg::h30), "fcmpe h30, #0.0");
}
TEST_CASE_METHOD(TestDisassembler, "Emitter: Scalar: Floating-point immediate") {
TEST_SINGLE(fmov(ScalarRegSize::i16Bit, VReg::v30, 1.0), "fmov h30, #0x70 (1.0000)");
TEST_SINGLE(fmov(ScalarRegSize::i32Bit, VReg::v30, 1.0), "fmov s30, #0x70 (1.0000)");
TEST_SINGLE(fmov(ScalarRegSize::i64Bit, VReg::v30, 1.0), "fmov d30, #0x70 (1.0000)");
}
TEST_CASE_METHOD(TestDisassembler, "Emitter: Scalar: Floating-point conditional compare") {
TEST_SINGLE(fccmp(SReg::s30, SReg::s29, StatusFlags::None, Condition::CC_AL), "fccmp s30, s29, #nzcv, al");
TEST_SINGLE(fccmp(SReg::s30, SReg::s29, StatusFlags::Flag_N, Condition::CC_AL), "fccmp s30, s29, #Nzcv, al");
TEST_SINGLE(fccmp(SReg::s30, SReg::s29, StatusFlags::Flag_Z, Condition::CC_AL), "fccmp s30, s29, #nZcv, al");
TEST_SINGLE(fccmp(SReg::s30, SReg::s29, StatusFlags::Flag_C, Condition::CC_AL), "fccmp s30, s29, #nzCv, al");
TEST_SINGLE(fccmp(SReg::s30, SReg::s29, StatusFlags::Flag_V, Condition::CC_AL), "fccmp s30, s29, #nzcV, al");
TEST_SINGLE(fccmp(SReg::s30, SReg::s29, StatusFlags::Flag_NZCV, Condition::CC_AL), "fccmp s30, s29, #NZCV, al");
TEST_SINGLE(fccmp(SReg::s30, SReg::s29, StatusFlags::None, Condition::CC_EQ), "fccmp s30, s29, #nzcv, eq");
TEST_SINGLE(fccmp(SReg::s30, SReg::s29, StatusFlags::Flag_N, Condition::CC_EQ), "fccmp s30, s29, #Nzcv, eq");
TEST_SINGLE(fccmp(SReg::s30, SReg::s29, StatusFlags::Flag_Z, Condition::CC_EQ), "fccmp s30, s29, #nZcv, eq");
TEST_SINGLE(fccmp(SReg::s30, SReg::s29, StatusFlags::Flag_C, Condition::CC_EQ), "fccmp s30, s29, #nzCv, eq");
TEST_SINGLE(fccmp(SReg::s30, SReg::s29, StatusFlags::Flag_V, Condition::CC_EQ), "fccmp s30, s29, #nzcV, eq");
TEST_SINGLE(fccmp(SReg::s30, SReg::s29, StatusFlags::Flag_NZCV, Condition::CC_EQ), "fccmp s30, s29, #NZCV, eq");
TEST_SINGLE(fccmpe(SReg::s30, SReg::s29, StatusFlags::None, Condition::CC_AL), "fccmpe s30, s29, #nzcv, al");
TEST_SINGLE(fccmpe(SReg::s30, SReg::s29, StatusFlags::Flag_N, Condition::CC_AL), "fccmpe s30, s29, #Nzcv, al");
TEST_SINGLE(fccmpe(SReg::s30, SReg::s29, StatusFlags::Flag_Z, Condition::CC_AL), "fccmpe s30, s29, #nZcv, al");
TEST_SINGLE(fccmpe(SReg::s30, SReg::s29, StatusFlags::Flag_C, Condition::CC_AL), "fccmpe s30, s29, #nzCv, al");
TEST_SINGLE(fccmpe(SReg::s30, SReg::s29, StatusFlags::Flag_V, Condition::CC_AL), "fccmpe s30, s29, #nzcV, al");
TEST_SINGLE(fccmpe(SReg::s30, SReg::s29, StatusFlags::Flag_NZCV, Condition::CC_AL), "fccmpe s30, s29, #NZCV, al");
TEST_SINGLE(fccmpe(SReg::s30, SReg::s29, StatusFlags::None, Condition::CC_EQ), "fccmpe s30, s29, #nzcv, eq");
TEST_SINGLE(fccmpe(SReg::s30, SReg::s29, StatusFlags::Flag_N, Condition::CC_EQ), "fccmpe s30, s29, #Nzcv, eq");
TEST_SINGLE(fccmpe(SReg::s30, SReg::s29, StatusFlags::Flag_Z, Condition::CC_EQ), "fccmpe s30, s29, #nZcv, eq");
TEST_SINGLE(fccmpe(SReg::s30, SReg::s29, StatusFlags::Flag_C, Condition::CC_EQ), "fccmpe s30, s29, #nzCv, eq");
TEST_SINGLE(fccmpe(SReg::s30, SReg::s29, StatusFlags::Flag_V, Condition::CC_EQ), "fccmpe s30, s29, #nzcV, eq");
TEST_SINGLE(fccmpe(SReg::s30, SReg::s29, StatusFlags::Flag_NZCV, Condition::CC_EQ), "fccmpe s30, s29, #NZCV, eq");
TEST_SINGLE(fccmp(DReg::d30, DReg::d29, StatusFlags::None, Condition::CC_AL), "fccmp d30, d29, #nzcv, al");
TEST_SINGLE(fccmp(DReg::d30, DReg::d29, StatusFlags::Flag_N, Condition::CC_AL), "fccmp d30, d29, #Nzcv, al");
TEST_SINGLE(fccmp(DReg::d30, DReg::d29, StatusFlags::Flag_Z, Condition::CC_AL), "fccmp d30, d29, #nZcv, al");
TEST_SINGLE(fccmp(DReg::d30, DReg::d29, StatusFlags::Flag_C, Condition::CC_AL), "fccmp d30, d29, #nzCv, al");
TEST_SINGLE(fccmp(DReg::d30, DReg::d29, StatusFlags::Flag_V, Condition::CC_AL), "fccmp d30, d29, #nzcV, al");
TEST_SINGLE(fccmp(DReg::d30, DReg::d29, StatusFlags::Flag_NZCV, Condition::CC_AL), "fccmp d30, d29, #NZCV, al");
TEST_SINGLE(fccmp(DReg::d30, DReg::d29, StatusFlags::None, Condition::CC_EQ), "fccmp d30, d29, #nzcv, eq");
TEST_SINGLE(fccmp(DReg::d30, DReg::d29, StatusFlags::Flag_N, Condition::CC_EQ), "fccmp d30, d29, #Nzcv, eq");
TEST_SINGLE(fccmp(DReg::d30, DReg::d29, StatusFlags::Flag_Z, Condition::CC_EQ), "fccmp d30, d29, #nZcv, eq");
TEST_SINGLE(fccmp(DReg::d30, DReg::d29, StatusFlags::Flag_C, Condition::CC_EQ), "fccmp d30, d29, #nzCv, eq");
TEST_SINGLE(fccmp(DReg::d30, DReg::d29, StatusFlags::Flag_V, Condition::CC_EQ), "fccmp d30, d29, #nzcV, eq");
TEST_SINGLE(fccmp(DReg::d30, DReg::d29, StatusFlags::Flag_NZCV, Condition::CC_EQ), "fccmp d30, d29, #NZCV, eq");
TEST_SINGLE(fccmpe(DReg::d30, DReg::d29, StatusFlags::None, Condition::CC_AL), "fccmpe d30, d29, #nzcv, al");
TEST_SINGLE(fccmpe(DReg::d30, DReg::d29, StatusFlags::Flag_N, Condition::CC_AL), "fccmpe d30, d29, #Nzcv, al");
TEST_SINGLE(fccmpe(DReg::d30, DReg::d29, StatusFlags::Flag_Z, Condition::CC_AL), "fccmpe d30, d29, #nZcv, al");
TEST_SINGLE(fccmpe(DReg::d30, DReg::d29, StatusFlags::Flag_C, Condition::CC_AL), "fccmpe d30, d29, #nzCv, al");
TEST_SINGLE(fccmpe(DReg::d30, DReg::d29, StatusFlags::Flag_V, Condition::CC_AL), "fccmpe d30, d29, #nzcV, al");
TEST_SINGLE(fccmpe(DReg::d30, DReg::d29, StatusFlags::Flag_NZCV, Condition::CC_AL), "fccmpe d30, d29, #NZCV, al");
TEST_SINGLE(fccmpe(DReg::d30, DReg::d29, StatusFlags::None, Condition::CC_EQ), "fccmpe d30, d29, #nzcv, eq");
TEST_SINGLE(fccmpe(DReg::d30, DReg::d29, StatusFlags::Flag_N, Condition::CC_EQ), "fccmpe d30, d29, #Nzcv, eq");
TEST_SINGLE(fccmpe(DReg::d30, DReg::d29, StatusFlags::Flag_Z, Condition::CC_EQ), "fccmpe d30, d29, #nZcv, eq");
TEST_SINGLE(fccmpe(DReg::d30, DReg::d29, StatusFlags::Flag_C, Condition::CC_EQ), "fccmpe d30, d29, #nzCv, eq");
TEST_SINGLE(fccmpe(DReg::d30, DReg::d29, StatusFlags::Flag_V, Condition::CC_EQ), "fccmpe d30, d29, #nzcV, eq");
TEST_SINGLE(fccmpe(DReg::d30, DReg::d29, StatusFlags::Flag_NZCV, Condition::CC_EQ), "fccmpe d30, d29, #NZCV, eq");
TEST_SINGLE(fccmp(HReg::h30, HReg::h29, StatusFlags::None, Condition::CC_AL), "fccmp h30, h29, #nzcv, al");
TEST_SINGLE(fccmp(HReg::h30, HReg::h29, StatusFlags::Flag_N, Condition::CC_AL), "fccmp h30, h29, #Nzcv, al");
TEST_SINGLE(fccmp(HReg::h30, HReg::h29, StatusFlags::Flag_Z, Condition::CC_AL), "fccmp h30, h29, #nZcv, al");
TEST_SINGLE(fccmp(HReg::h30, HReg::h29, StatusFlags::Flag_C, Condition::CC_AL), "fccmp h30, h29, #nzCv, al");
TEST_SINGLE(fccmp(HReg::h30, HReg::h29, StatusFlags::Flag_V, Condition::CC_AL), "fccmp h30, h29, #nzcV, al");
TEST_SINGLE(fccmp(HReg::h30, HReg::h29, StatusFlags::Flag_NZCV, Condition::CC_AL), "fccmp h30, h29, #NZCV, al");
TEST_SINGLE(fccmp(HReg::h30, HReg::h29, StatusFlags::None, Condition::CC_EQ), "fccmp h30, h29, #nzcv, eq");
TEST_SINGLE(fccmp(HReg::h30, HReg::h29, StatusFlags::Flag_N, Condition::CC_EQ), "fccmp h30, h29, #Nzcv, eq");
TEST_SINGLE(fccmp(HReg::h30, HReg::h29, StatusFlags::Flag_Z, Condition::CC_EQ), "fccmp h30, h29, #nZcv, eq");
TEST_SINGLE(fccmp(HReg::h30, HReg::h29, StatusFlags::Flag_C, Condition::CC_EQ), "fccmp h30, h29, #nzCv, eq");
TEST_SINGLE(fccmp(HReg::h30, HReg::h29, StatusFlags::Flag_V, Condition::CC_EQ), "fccmp h30, h29, #nzcV, eq");
TEST_SINGLE(fccmp(HReg::h30, HReg::h29, StatusFlags::Flag_NZCV, Condition::CC_EQ), "fccmp h30, h29, #NZCV, eq");
TEST_SINGLE(fccmpe(HReg::h30, HReg::h29, StatusFlags::None, Condition::CC_AL), "fccmpe h30, h29, #nzcv, al");
TEST_SINGLE(fccmpe(HReg::h30, HReg::h29, StatusFlags::Flag_N, Condition::CC_AL), "fccmpe h30, h29, #Nzcv, al");
TEST_SINGLE(fccmpe(HReg::h30, HReg::h29, StatusFlags::Flag_Z, Condition::CC_AL), "fccmpe h30, h29, #nZcv, al");
TEST_SINGLE(fccmpe(HReg::h30, HReg::h29, StatusFlags::Flag_C, Condition::CC_AL), "fccmpe h30, h29, #nzCv, al");
TEST_SINGLE(fccmpe(HReg::h30, HReg::h29, StatusFlags::Flag_V, Condition::CC_AL), "fccmpe h30, h29, #nzcV, al");
TEST_SINGLE(fccmpe(HReg::h30, HReg::h29, StatusFlags::Flag_NZCV, Condition::CC_AL), "fccmpe h30, h29, #NZCV, al");
TEST_SINGLE(fccmpe(HReg::h30, HReg::h29, StatusFlags::None, Condition::CC_EQ), "fccmpe h30, h29, #nzcv, eq");
TEST_SINGLE(fccmpe(HReg::h30, HReg::h29, StatusFlags::Flag_N, Condition::CC_EQ), "fccmpe h30, h29, #Nzcv, eq");
TEST_SINGLE(fccmpe(HReg::h30, HReg::h29, StatusFlags::Flag_Z, Condition::CC_EQ), "fccmpe h30, h29, #nZcv, eq");
TEST_SINGLE(fccmpe(HReg::h30, HReg::h29, StatusFlags::Flag_C, Condition::CC_EQ), "fccmpe h30, h29, #nzCv, eq");
TEST_SINGLE(fccmpe(HReg::h30, HReg::h29, StatusFlags::Flag_V, Condition::CC_EQ), "fccmpe h30, h29, #nzcV, eq");
TEST_SINGLE(fccmpe(HReg::h30, HReg::h29, StatusFlags::Flag_NZCV, Condition::CC_EQ), "fccmpe h30, h29, #NZCV, eq");
}
TEST_CASE_METHOD(TestDisassembler, "Emitter: Scalar: Floating-point data-processing (2 source)") {
TEST_SINGLE(fmul(SReg::s30, SReg::s29, SReg::s28), "fmul s30, s29, s28");
TEST_SINGLE(fdiv(SReg::s30, SReg::s29, SReg::s28), "fdiv s30, s29, s28");
TEST_SINGLE(fadd(SReg::s30, SReg::s29, SReg::s28), "fadd s30, s29, s28");
TEST_SINGLE(fsub(SReg::s30, SReg::s29, SReg::s28), "fsub s30, s29, s28");
TEST_SINGLE(fmax(SReg::s30, SReg::s29, SReg::s28), "fmax s30, s29, s28");
TEST_SINGLE(fmin(SReg::s30, SReg::s29, SReg::s28), "fmin s30, s29, s28");
TEST_SINGLE(fmaxnm(SReg::s30, SReg::s29, SReg::s28), "fmaxnm s30, s29, s28");
TEST_SINGLE(fminnm(SReg::s30, SReg::s29, SReg::s28), "fminnm s30, s29, s28");
TEST_SINGLE(fnmul(SReg::s30, SReg::s29, SReg::s28), "fnmul s30, s29, s28");
TEST_SINGLE(fmul(DReg::d30, DReg::d29, DReg::d28), "fmul d30, d29, d28");
TEST_SINGLE(fdiv(DReg::d30, DReg::d29, DReg::d28), "fdiv d30, d29, d28");
TEST_SINGLE(fadd(DReg::d30, DReg::d29, DReg::d28), "fadd d30, d29, d28");
TEST_SINGLE(fsub(DReg::d30, DReg::d29, DReg::d28), "fsub d30, d29, d28");
TEST_SINGLE(fmax(DReg::d30, DReg::d29, DReg::d28), "fmax d30, d29, d28");
TEST_SINGLE(fmin(DReg::d30, DReg::d29, DReg::d28), "fmin d30, d29, d28");
TEST_SINGLE(fmaxnm(DReg::d30, DReg::d29, DReg::d28), "fmaxnm d30, d29, d28");
TEST_SINGLE(fminnm(DReg::d30, DReg::d29, DReg::d28), "fminnm d30, d29, d28");
TEST_SINGLE(fnmul(DReg::d30, DReg::d29, DReg::d28), "fnmul d30, d29, d28");
TEST_SINGLE(fmul(HReg::h30, HReg::h29, HReg::h28), "fmul h30, h29, h28");
TEST_SINGLE(fdiv(HReg::h30, HReg::h29, HReg::h28), "fdiv h30, h29, h28");
TEST_SINGLE(fadd(HReg::h30, HReg::h29, HReg::h28), "fadd h30, h29, h28");
TEST_SINGLE(fsub(HReg::h30, HReg::h29, HReg::h28), "fsub h30, h29, h28");
TEST_SINGLE(fmax(HReg::h30, HReg::h29, HReg::h28), "fmax h30, h29, h28");
TEST_SINGLE(fmin(HReg::h30, HReg::h29, HReg::h28), "fmin h30, h29, h28");
TEST_SINGLE(fmaxnm(HReg::h30, HReg::h29, HReg::h28), "fmaxnm h30, h29, h28");
TEST_SINGLE(fminnm(HReg::h30, HReg::h29, HReg::h28), "fminnm h30, h29, h28");
TEST_SINGLE(fnmul(HReg::h30, HReg::h29, HReg::h28), "fnmul h30, h29, h28");
}
TEST_CASE_METHOD(TestDisassembler, "Emitter: Scalar: Floating-point conditional select") {
TEST_SINGLE(fcsel(SReg::s30, SReg::s29, SReg::s28, Condition::CC_AL), "fcsel s30, s29, s28, al");
TEST_SINGLE(fcsel(SReg::s30, SReg::s29, SReg::s28, Condition::CC_EQ), "fcsel s30, s29, s28, eq");
TEST_SINGLE(fcsel(DReg::d30, DReg::d29, DReg::d28, Condition::CC_AL), "fcsel d30, d29, d28, al");
TEST_SINGLE(fcsel(DReg::d30, DReg::d29, DReg::d28, Condition::CC_EQ), "fcsel d30, d29, d28, eq");
TEST_SINGLE(fcsel(HReg::h30, HReg::h29, HReg::h28, Condition::CC_AL), "fcsel h30, h29, h28, al");
TEST_SINGLE(fcsel(HReg::h30, HReg::h29, HReg::h28, Condition::CC_EQ), "fcsel h30, h29, h28, eq");
}
TEST_CASE_METHOD(TestDisassembler, "Emitter: Scalar: Floating-point data-processing (3 source)") {
TEST_SINGLE(fmadd(SReg::s30, SReg::s29, SReg::s28, SReg::s27), "fmadd s30, s29, s28, s27");
TEST_SINGLE(fmsub(SReg::s30, SReg::s29, SReg::s28, SReg::s27), "fmsub s30, s29, s28, s27");
TEST_SINGLE(fnmadd(SReg::s30, SReg::s29, SReg::s28, SReg::s27), "fnmadd s30, s29, s28, s27");
TEST_SINGLE(fnmsub(SReg::s30, SReg::s29, SReg::s28, SReg::s27), "fnmsub s30, s29, s28, s27");
TEST_SINGLE(fmadd(DReg::d30, DReg::d29, DReg::d28, DReg::d27), "fmadd d30, d29, d28, d27");
TEST_SINGLE(fmsub(DReg::d30, DReg::d29, DReg::d28, DReg::d27), "fmsub d30, d29, d28, d27");
TEST_SINGLE(fnmadd(DReg::d30, DReg::d29, DReg::d28, DReg::d27), "fnmadd d30, d29, d28, d27");
TEST_SINGLE(fnmsub(DReg::d30, DReg::d29, DReg::d28, DReg::d27), "fnmsub d30, d29, d28, d27");
TEST_SINGLE(fmadd(HReg::h30, HReg::h29, HReg::h28, HReg::h27), "fmadd h30, h29, h28, h27");
TEST_SINGLE(fmsub(HReg::h30, HReg::h29, HReg::h28, HReg::h27), "fmsub h30, h29, h28, h27");
TEST_SINGLE(fnmadd(HReg::h30, HReg::h29, HReg::h28, HReg::h27), "fnmadd h30, h29, h28, h27");
TEST_SINGLE(fnmsub(HReg::h30, HReg::h29, HReg::h28, HReg::h27), "fnmsub h30, h29, h28, h27");
}
+139
View File
@@ -0,0 +1,139 @@
#include "TestDisassembler.h"
#include <catch2/catch.hpp>
#include <fcntl.h>
using namespace FEXCore::ARMEmitter;
TEST_CASE_METHOD(TestDisassembler, "Emitter: System: System with result") {
// TODO: Implement in emitter.
}
TEST_CASE_METHOD(TestDisassembler, "Emitter: System: System Instruction") {
// TODO: AT
// TODO: CFP
// TODO: CPP
// vixl doesn't understand a bunch of data cache operation names.
TEST_SINGLE(dc(DataCacheOperation::IVAC, Reg::r30), "sys #0, C7, C6, #1, x30");
TEST_SINGLE(dc(DataCacheOperation::ISW, Reg::r30), "sys #0, C7, C6, #2, x30");
TEST_SINGLE(dc(DataCacheOperation::CSW, Reg::r30), "sys #0, C7, C10, #2, x30");
TEST_SINGLE(dc(DataCacheOperation::CISW, Reg::r30), "sys #0, C7, C14, #2, x30");
TEST_SINGLE(dc(DataCacheOperation::ZVA, Reg::r30), "dc zva, x30");
TEST_SINGLE(dc(DataCacheOperation::CVAC, Reg::r30), "dc cvac, x30");
TEST_SINGLE(dc(DataCacheOperation::CVAU, Reg::r30), "dc cvau, x30");
TEST_SINGLE(dc(DataCacheOperation::CIVAC, Reg::r30), "dc civac, x30");
TEST_SINGLE(dc(DataCacheOperation::IGVAC, Reg::r30), "sys #0, C7, C6, #3, x30");
TEST_SINGLE(dc(DataCacheOperation::IGSW, Reg::r30), "sys #0, C7, C6, #4, x30");
TEST_SINGLE(dc(DataCacheOperation::IGDVAC, Reg::r30), "sys #0, C7, C6, #5, x30");
TEST_SINGLE(dc(DataCacheOperation::IGDSW, Reg::r30), "sys #0, C7, C6, #6, x30");
TEST_SINGLE(dc(DataCacheOperation::CGSW, Reg::r30), "sys #0, C7, C10, #4, x30");
TEST_SINGLE(dc(DataCacheOperation::CGDSW, Reg::r30), "sys #0, C7, C10, #6, x30");
TEST_SINGLE(dc(DataCacheOperation::CIGSW, Reg::r30), "sys #0, C7, C14, #4, x30");
TEST_SINGLE(dc(DataCacheOperation::CIGDSW, Reg::r30), "sys #0, C7, C14, #6, x30");
TEST_SINGLE(dc(DataCacheOperation::GVA, Reg::r30), "sys #3, C7, C4, #3, x30");
TEST_SINGLE(dc(DataCacheOperation::GZVA, Reg::r30), "sys #3, C7, C4, #4, x30");
TEST_SINGLE(dc(DataCacheOperation::CGVAC, Reg::r30), "sys #3, C7, C10, #3, x30");
TEST_SINGLE(dc(DataCacheOperation::CGDVAC, Reg::r30), "sys #3, C7, C10, #5, x30");
TEST_SINGLE(dc(DataCacheOperation::CGVAP, Reg::r30), "sys #3, C7, C12, #3, x30");
TEST_SINGLE(dc(DataCacheOperation::CGDVAP, Reg::r30), "sys #3, C7, C12, #5, x30");
TEST_SINGLE(dc(DataCacheOperation::CGVADP, Reg::r30), "sys #3, C7, C13, #3, x30");
TEST_SINGLE(dc(DataCacheOperation::CGDVADP, Reg::r30), "sys #3, C7, C13, #5, x30");
TEST_SINGLE(dc(DataCacheOperation::CIGVAC, Reg::r30), "sys #3, C7, C14, #3, x30");
TEST_SINGLE(dc(DataCacheOperation::CIGDVAC, Reg::r30), "sys #3, C7, C14, #5, x30");
TEST_SINGLE(dc(DataCacheOperation::CVAP, Reg::r30), "dc cvap, x30");
TEST_SINGLE(dc(DataCacheOperation::CVADP, Reg::r30), "dc cvadp, x30");
// TODO: DVP
// TODO: IC
// TODO: TLBI
}
TEST_CASE_METHOD(TestDisassembler, "Emitter: System: Exception generation") {
TEST_SINGLE(svc(65535), "svc #0xffff");
TEST_SINGLE(hvc(65535), "hvc #0xffff");
TEST_SINGLE(smc(65535), "smc #0xffff");
TEST_SINGLE(brk(65535), "brk #0xffff");
TEST_SINGLE(hlt(65535), "hlt #0xffff");
TEST_SINGLE(tcancel(65535), "unimplemented (Unimplemented)");
TEST_SINGLE(dcps1(65535), "dcps1 {#0xffff}");
TEST_SINGLE(dcps2(65535), "dcps2 {#0xffff}");
TEST_SINGLE(dcps3(65535), "dcps3 {#0xffff}");
}
TEST_CASE_METHOD(TestDisassembler, "Emitter: System: System instructions with register argument") {
if (false) {
// Unsupported in vixl.
TEST_SINGLE(wfet(Reg::r30), "wfet x30");
TEST_SINGLE(wfit(Reg::r30), "wfit x30");
}
}
TEST_CASE_METHOD(TestDisassembler, "Emitter: System: Hints") {
TEST_SINGLE(nop(), "nop");
TEST_SINGLE(yield(), "yield");
TEST_SINGLE(wfe(), "wfe");
TEST_SINGLE(wfi(), "wfi");
TEST_SINGLE(sev(), "sev");
TEST_SINGLE(sevl(), "sevl");
TEST_SINGLE(dgh(), "dgh");
TEST_SINGLE(csdb(), "csdb");
}
TEST_CASE_METHOD(TestDisassembler, "Emitter: System: Barriers") {
TEST_SINGLE(clrex(0), "clrex #0x0");
TEST_SINGLE(clrex(15), "clrex");
TEST_SINGLE(dsb(BarrierScope::OSHLD), "dsb oshld");
TEST_SINGLE(dsb(BarrierScope::OSHST), "dsb oshst");
TEST_SINGLE(dsb(BarrierScope::OSH), "dsb osh");
TEST_SINGLE(dsb(BarrierScope::NSHLD), "dsb nshld");
TEST_SINGLE(dsb(BarrierScope::NSHST), "dsb nshst");
TEST_SINGLE(dsb(BarrierScope::NSH), "dsb nsh");
TEST_SINGLE(dsb(BarrierScope::ISHLD), "dsb ishld");
TEST_SINGLE(dsb(BarrierScope::ISHST), "dsb ishst");
TEST_SINGLE(dsb(BarrierScope::ISH), "dsb ish");
TEST_SINGLE(dsb(BarrierScope::LD), "dsb ld");
TEST_SINGLE(dsb(BarrierScope::ST), "dsb st");
TEST_SINGLE(dsb(BarrierScope::SY), "dsb sy");
TEST_SINGLE(dmb(BarrierScope::OSHLD), "dmb oshld");
TEST_SINGLE(dmb(BarrierScope::OSHST), "dmb oshst");
TEST_SINGLE(dmb(BarrierScope::OSH), "dmb osh");
TEST_SINGLE(dmb(BarrierScope::NSHLD), "dmb nshld");
TEST_SINGLE(dmb(BarrierScope::NSHST), "dmb nshst");
TEST_SINGLE(dmb(BarrierScope::NSH), "dmb nsh");
TEST_SINGLE(dmb(BarrierScope::ISHLD), "dmb ishld");
TEST_SINGLE(dmb(BarrierScope::ISHST), "dmb ishst");
TEST_SINGLE(dmb(BarrierScope::ISH), "dmb ish");
TEST_SINGLE(dmb(BarrierScope::LD), "dmb ld");
TEST_SINGLE(dmb(BarrierScope::ST), "dmb st");
TEST_SINGLE(dmb(BarrierScope::SY), "dmb sy");
TEST_SINGLE(isb(), "isb");
// vixl has a decoding bug claiming these are system level instructions.
TEST_SINGLE(sb(), "sb (System)");
TEST_SINGLE(tcommit(), "tcommit (System)");
}
TEST_CASE_METHOD(TestDisassembler, "Emitter: System: System register move") {
// vixl doesn't have decoding for a bunch of these.
// Also most of these aren't writeable from el0, just testing the encoding.
TEST_SINGLE(msr(SystemRegister::CTR_EL0, Reg::r30), "msr S3_3_c0_c0_1, x30");
TEST_SINGLE(msr(SystemRegister::DCZID_EL0, Reg::r30), "msr S3_3_c0_c0_7, x30");
TEST_SINGLE(msr(SystemRegister::TPIDR_EL0, Reg::r30), "msr S3_3_c13_c0_2, x30");
TEST_SINGLE(msr(SystemRegister::RNDR, Reg::r30), "msr rndr, x30");
TEST_SINGLE(msr(SystemRegister::RNDRRS, Reg::r30), "msr rndrrs, x30");
TEST_SINGLE(msr(SystemRegister::NZCV, Reg::r30), "msr nzcv, x30");
TEST_SINGLE(msr(SystemRegister::FPCR, Reg::r30), "msr fpcr, x30");
TEST_SINGLE(msr(SystemRegister::CNTFRQ_EL0, Reg::r30), "msr S3_3_c14_c0_0, x30");
TEST_SINGLE(msr(SystemRegister::CNTVCT_EL0, Reg::r30), "msr S3_3_c14_c0_2, x30");
TEST_SINGLE(mrs(Reg::r30, SystemRegister::CTR_EL0), "mrs x30, S3_3_c0_c0_1");
TEST_SINGLE(mrs(Reg::r30, SystemRegister::DCZID_EL0), "mrs x30, S3_3_c0_c0_7");
TEST_SINGLE(mrs(Reg::r30, SystemRegister::TPIDR_EL0), "mrs x30, S3_3_c13_c0_2");
TEST_SINGLE(mrs(Reg::r30, SystemRegister::RNDR), "mrs x30, rndr");
TEST_SINGLE(mrs(Reg::r30, SystemRegister::RNDRRS), "mrs x30, rndrrs");
TEST_SINGLE(mrs(Reg::r30, SystemRegister::NZCV), "mrs x30, nzcv");
TEST_SINGLE(mrs(Reg::r30, SystemRegister::FPCR), "mrs x30, fpcr");
TEST_SINGLE(mrs(Reg::r30, SystemRegister::CNTFRQ_EL0), "mrs x30, S3_3_c14_c0_0");
TEST_SINGLE(mrs(Reg::r30, SystemRegister::CNTVCT_EL0), "mrs x30, S3_3_c14_c0_2");
}
+75
View File
@@ -0,0 +1,75 @@
#pragma once
#include "Interface/Core/ArchHelpers/CodeEmitter/Emitter.h"
#include <aarch64/cpu-aarch64.h>
#include <aarch64/instructions-aarch64.h>
#include <aarch64/disasm-aarch64.h>
#include <sys/mman.h>
class TestDisassembler : public FEXCore::ARMEmitter::Emitter {
public:
TestDisassembler() {
fp = tmpfile();
Disasm = std::make_unique<vixl::aarch64::PrintDisassembler>(fp);
SetBuffer(reinterpret_cast<uint8_t*>(mmap(nullptr, 4096, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0)), 4096);
BufferBegin = GetCursorAddress<const vixl::aarch64::Instruction*>();
}
~TestDisassembler() {
fclose(fp);
}
std::string DisassembleSingle() {
HandleDisasm();
char Tmp[512];
uint64_t Addr;
uint32_t Encoding;
int Num = fscanf(fp, "0x%lx %x %[^\n]\n", &Addr, &Encoding, Tmp);
if (Num != 3) {
return "<Invalid>";
}
ResetFP();
return Tmp;
}
uint32_t DisassembleEncoding(size_t Offset = 0) {
const uint32_t *Values = reinterpret_cast<const uint32_t*>(GetBufferBase());
SetCursorOffset(0);
ResetFP();
return Values[Offset];
}
std::string DisassembleString() {
HandleDisasm();
std::string Decoded{};
char Tmp[512];
uint64_t Addr;
uint32_t Encoding;
while (fscanf(fp, "0x%lx %x %[^\n]\n", &Addr, &Encoding, Tmp) == 3) {
Decoded += std::string_view(Tmp);
Decoded += "\n";
}
ResetFP();
return Decoded;
}
private:
void HandleDisasm() {
const auto BufferEnd = GetCursorAddress<const vixl::aarch64::Instruction*>();
Disasm->DisassembleBuffer(BufferBegin, BufferEnd);
SetCursorOffset(0);
fseek(fp, 0, SEEK_SET);
}
void ResetFP() {
fseek(fp, 0, SEEK_SET);
}
FILE *fp;
const vixl::aarch64::Instruction* BufferBegin;
std::unique_ptr<vixl::aarch64::PrintDisassembler> Disasm;
};
#define TEST_SINGLE(emit, expected) \
{ \
CHECK((emit, DisassembleSingle()) == expected); \
}
+1 -1
+103 -103
View File
@@ -98,7 +98,14 @@ static bool LoadFile(std::vector<char> &Data, const std::string &Filename) {
return true;
}
void FileManager::LoadThunkDatabase(bool Global) {
struct ThunkDBObject {
std::string LibraryName;
std::unordered_set<std::string> Depends;
std::vector<std::string> Overlays;
bool Enabled{};
};
static void LoadThunkDatabase(std::unordered_map<std::string, ThunkDBObject>& ThunkDB, bool Is64BitMode, bool Global) {
auto ThunkDBPath = FEXCore::Config::GetConfigDirectory(Global) + "ThunksDB.json";
std::vector<char> FileData;
if (LoadFile(FileData, ThunkDBPath)) {
@@ -127,13 +134,13 @@ void FileManager::LoadThunkDatabase(bool Global) {
// Walk the libraries items to get the data
for (json_t const* LibraryItem = json_getChild(Library); LibraryItem != nullptr; LibraryItem = json_getSibling(LibraryItem)) {
const char* ItemName = json_getName(LibraryItem);
std::string_view ItemName = json_getName(LibraryItem);
if (strcmp(ItemName, "Library") == 0) {
if (ItemName == "Library") {
// "Library": "libGL-guest.so"
DBObject->second.LibraryName = json_getValue(LibraryItem);
}
else if (strcmp(ItemName, "Depends") == 0) {
else if (ItemName == "Depends") {
jsonType_t PropertyType = json_getType(LibraryItem);
if (PropertyType == JSON_TEXT) {
DBObject->second.Depends.insert(json_getValue(LibraryItem));
@@ -144,9 +151,8 @@ void FileManager::LoadThunkDatabase(bool Global) {
}
}
}
else if (strcmp(ItemName, "Overlay") == 0) {
auto AddWithReplacement = [DBObject, HomeDirectory](json_t const* Value) {
else if (ItemName == "Overlay") {
auto AddWithReplacement = [Is64BitMode, HomeDirectory](ThunkDBObject& DBObject, std::string LibraryItem) {
constexpr static std::array<std::string_view, 4> LibPrefixes = {
"/usr/lib",
"/usr/local/lib",
@@ -154,64 +160,53 @@ void FileManager::LoadThunkDatabase(bool Global) {
"/usr/lib/pressure-vessel/overrides/lib",
};
auto FindAndReplacePrefixes = [DBObject](std::string_view String, std::string_view Prefix, auto NewPrefixes) -> bool {
auto it = String.find(Prefix);
if (it != String.npos) {
size_t SizeOfOldPrefix = Prefix.size();
constexpr static std::array<std::string_view, 2> ArchPrefixes = {
"i386",
"x86_64",
};
for (auto& prefix : NewPrefixes) {
std::string Replacement {String};
Replacement.replace(it, SizeOfOldPrefix, prefix);
DBObject->second.Overlays.emplace_back(std::move(Replacement));
// Walk through template string and fill in prefixes from right to left
using namespace std::string_view_literals;
const std::pair PrefixArch { "@PREFIX_ARCH@"sv, LibraryItem.find("@PREFIX_ARCH@") };
const std::pair PrefixHome { "@HOME@"sv, LibraryItem.find("@HOME@") };
const std::pair PrefixLib { "@PREFIX_LIB@"sv, LibraryItem.find("@PREFIX_LIB@") };
std::string::size_type PrefixPositions[] = {
PrefixArch.second, PrefixHome.second, PrefixLib.second,
};
// Sort offsets in descending order to enable safe in-place replacement
std::sort(std::begin(PrefixPositions), std::end(PrefixPositions), std::greater<>{});
for (auto& LibPrefix : LibPrefixes) {
std::string Replacement = LibraryItem;
for (auto PrefixPos : PrefixPositions) {
if (PrefixPos == std::string::npos) {
continue;
} else if (PrefixPos == PrefixArch.second) {
Replacement.replace(PrefixPos, PrefixArch.first.size(), ArchPrefixes[Is64BitMode]);
} else if (PrefixPos == PrefixHome.second) {
Replacement.replace(PrefixPos, PrefixHome.first.size(), HomeDirectory);
} else if (PrefixPos == PrefixLib.second) {
Replacement.replace(PrefixPos, PrefixLib.first.size(), LibPrefix);
}
return true;
}
else {
return false;
DBObject.Overlays.emplace_back(std::move(Replacement));
if (PrefixLib.second == std::string::npos) {
// Don't repeat for other LibPrefixes entries if the prefix wasn't used
break;
}
};
auto FindAndReplaceSingleNewPrefix = [DBObject](std::string_view String, std::string_view Prefix, auto NewPrefix) -> bool {
auto it = String.find(Prefix);
if (it != String.npos) {
size_t SizeOfOldPrefix = Prefix.size();
std::string Replacement {String};
Replacement.replace(it, SizeOfOldPrefix, NewPrefix);
DBObject->second.Overlays.emplace_back(std::move(Replacement));
return true;
}
else {
return false;
}
};
std::string NonModifiedLibraryItem {static_cast<const char*>(json_getValue(Value))};
// Prefixes are mutually exclusive currently.
// Walk through each individual library item and attempt to change prefixes before inserting this in to our overlay system.
// Attempt to replace @PREFIX_LIB@ first
bool Inserted = FindAndReplacePrefixes(NonModifiedLibraryItem, "@PREFIX_LIB@", LibPrefixes);
// Attempt to replace @HOME@ second
if (!Inserted) {
Inserted = FindAndReplaceSingleNewPrefix(NonModifiedLibraryItem, "@HOME@", HomeDirectory);
}
// Failing to replace prefixes, insert the item unmodifed
if (!Inserted) {
DBObject->second.Overlays.emplace_back(NonModifiedLibraryItem);
}
};
jsonType_t PropertyType = json_getType(LibraryItem);
if (PropertyType == JSON_TEXT) {
AddWithReplacement(LibraryItem);
AddWithReplacement(DBObject->second, json_getValue(LibraryItem));
}
else if (PropertyType == JSON_ARRAY) {
for (json_t const* Overlay = json_getChild(LibraryItem); Overlay != nullptr; Overlay = json_getSibling(Overlay)) {
AddWithReplacement(Overlay);
AddWithReplacement(DBObject->second, json_getValue(Overlay));
}
}
}
@@ -223,31 +218,7 @@ void FileManager::LoadThunkDatabase(bool Global) {
FileManager::FileManager(FEXCore::Context::Context *ctx)
: EmuFD {ctx} {
bool LoadedThunkDatabase{};
auto ThunkConfigFile = ThunkConfig();
auto ThunkGuestPath = std::filesystem::path(ThunkGuestLibs());
auto LoadThunksDB = [this, ThunkGuestPath](bool *LoadedThunkDatabase, json_t const* ThunksDB) {
// If a thunks DB property exists then we pull in data from the thunks database
// Load the initial thunks database
if (!*LoadedThunkDatabase) {
LoadThunkDatabase(true);
LoadThunkDatabase(false);
*LoadedThunkDatabase = true;
}
// Now load this property
for (json_t const* Item = json_getChild(ThunksDB); Item != nullptr; Item = json_getSibling(Item)) {
const char *LibraryName = json_getName(Item);
bool LibraryEnabled = json_getInteger(Item) != 0;
// If the library is enabled then find it in the DB
// Enable the overlay and all the dependencies in one go
auto DBObject = ThunkDB.find(LibraryName);
if (DBObject != ThunkDB.end()) {
DBObject->second.Enabled = LibraryEnabled;
}
}
};
// We try to load ThunksDB from:
// - FEX global config
@@ -283,6 +254,10 @@ FileManager::FileManager(FEXCore::Context::Context *ctx)
ConfigPaths.emplace_back(FEXCore::Config::GetApplicationConfig(AppName, false));
}
std::unordered_map<std::string, ThunkDBObject> ThunkDB;
LoadThunkDatabase(ThunkDB, Is64BitMode(), true);
LoadThunkDatabase(ThunkDB, Is64BitMode(), false);
for (const auto &Path : ConfigPaths) {
std::vector<char> FileData;
if (LoadFile(FileData, Path)) {
@@ -293,51 +268,76 @@ FileManager::FileManager(FEXCore::Context::Context *ctx)
},
};
// If a thunks DB property exists then we pull in data from the thunks database
json_t const *json = json_createWithPool(&FileData.at(0), &Pool.PoolObject);
json_t const* ThunksDB = json_getProperty( json, "ThunksDB" );
if (ThunksDB) {
LoadThunksDB(&LoadedThunkDatabase, ThunksDB);
if (!ThunksDB) {
continue;
}
for (json_t const* Item = json_getChild(ThunksDB); Item != nullptr; Item = json_getSibling(Item)) {
const char *LibraryName = json_getName(Item);
bool LibraryEnabled = json_getInteger(Item) != 0;
// If the library is enabled then find it in the DB
auto DBObject = ThunkDB.find(LibraryName);
if (DBObject != ThunkDB.end()) {
DBObject->second.Enabled = LibraryEnabled;
}
}
}
}
// Now that we loaded the thunks object, walk through and ensure dependencies are enabled as well.
// Now that we loaded the thunks object, walk through and ensure dependencies are enabled as well
auto ThunkGuestPath = std::filesystem::path { Is64BitMode() ? ThunkGuestLibs() : ThunkGuestLibs32() };
for (auto const &DBObject : ThunkDB) {
if (!DBObject.second.Enabled) {
continue;
}
// Now walk the dependencies and set them up as well
// Make sure to enable each one as we go to remove circular dependencies
std::function<void(const std::unordered_set<std::string> &Depends, bool AlreadyEnabled)> InsertDependencies
= [this, &ThunkGuestPath, &InsertDependencies](const std::unordered_set<std::string> &Depends, bool AlreadyEnabled) -> void {
for (auto const &Depend : Depends) {
auto DBDepend = ThunkDB.find(Depend);
if (DBDepend != ThunkDB.end() &&
(DBDepend->second.Enabled == false || AlreadyEnabled)) {
// Recursively add paths for this thunk library and its dependencies to ThunkOverlays.
// Using a local struct for this is slightly less ugly than using self-capturing lambdas
struct {
decltype(FileManager::ThunkOverlays)& ThunkOverlays;
decltype(ThunkDB)& ThunkDB;
const std::filesystem::path& ThunkGuestPath;
bool Is64BitMode;
auto ThunkPath = ThunkGuestPath / DBDepend->second.LibraryName;
if (std::filesystem::exists(ThunkPath)) {
for (const auto& Overlay : DBDepend->second.Overlays) {
// Direct full path in guest RootFS to our overlay file
ThunkOverlays.emplace(Overlay, ThunkPath);
void SetupOverlay(const ThunkDBObject& DBDepend) {
auto ThunkPath = ThunkGuestPath / DBDepend.LibraryName;
if (!std::filesystem::exists(ThunkPath)) {
if (!Is64BitMode) {
// Guest libraries not existing is expected since not all libraries are thunked on 32-bit
return;
}
ERROR_AND_DIE_FMT("Requested thunking via guest library \"{}\" that does not exist", ThunkPath.string());
}
// Enabled, now walk this dependencies
DBDepend->second.Enabled = true;
InsertDependencies(DBDepend->second.Depends, false);
for (const auto& Overlay : DBDepend.Overlays) {
// Direct full path in guest RootFS to our overlay file
ThunkOverlays.emplace(Overlay, ThunkPath);
}
};
void InsertDependencies(const std::unordered_set<std::string> &Depends) {
for (auto const &Depend : Depends) {
auto& DBDepend = ThunkDB.at(Depend);
if (DBDepend.Enabled) {
continue;
}
SetupOverlay(DBDepend);
// Mark enabled and recurse into dependencies
DBDepend.Enabled = true;
InsertDependencies(DBDepend.Depends);
}
}
};
};
} DBObjectHandler { ThunkOverlays, ThunkDB, ThunkGuestPath, Is64BitMode() };
InsertDependencies({DBObject.first}, true);
InsertDependencies(DBObject.second.Depends, false);
DBObjectHandler.SetupOverlay(DBObject.second);
DBObjectHandler.InsertDependencies(DBObject.second.Depends);
}
// Now clear the thunk database since we're loaded
ThunkDB.clear();
if (false) {
// Useful for debugging
if (ThunkOverlays.size()) {
+2 -10
View File
@@ -90,19 +90,11 @@ private:
FEX_CONFIG_OPT(Filename, APP_FILENAME);
FEX_CONFIG_OPT(LDPath, ROOTFS);
FEX_CONFIG_OPT(ThunkHostLibs, THUNKHOSTLIBS);
FEX_CONFIG_OPT(ThunkGuestLibs, THUNKGUESTLIBS);
FEX_CONFIG_OPT(ThunkGuestLibs32, THUNKGUESTLIBS32);
FEX_CONFIG_OPT(ThunkConfig, THUNKCONFIG);
FEX_CONFIG_OPT(AppConfigName, APP_CONFIG_NAME);
FEX_CONFIG_OPT(Is64BitMode, IS64BIT_MODE);
uint32_t CurrentPID{};
void LoadThunkDatabase(bool Global);
struct ThunkDBObject {
std::string LibraryName;
std::unordered_set<std::string> Depends;
std::vector<std::string> Overlays;
bool Enabled{};
};
std::unordered_map<std::string, ThunkDBObject> ThunkDB{};
};
}
+2 -2
View File
@@ -111,8 +111,8 @@ function(add_guest_lib NAME SONAME)
if (BITNESS EQUAL 32)
# Makes the GOT/PLT lookups slightly less painful
target_compile_options(${NAME}-guest PRIVATE -fno-plt)
target_link_options(VDSO-guest PRIVATE "LINKER:-z,now" "LINKER:-z,relro" "LINKER:-z,notext")
target_compile_options(${NAME}-guest PRIVATE -fno-plt -fno-stack-protector)
target_link_options(${NAME}-guest PRIVATE "LINKER:-z,now" "LINKER:-z,relro" "LINKER:-z,notext")
endif()
# Add linker script if set
+89 -80
View File
@@ -15,15 +15,15 @@ endif()
# - custom command: Main build step that runs the thunk generator on the given interface definition
# - libxyz-interface: Target for IDE integration (making sure libxyz-interface.cpp shows up as a source file in the project tree)
# - libxyz-deps: Interface target to read include directories from which are passed to libclang when parsing the interface definition
function(generate NAME SOURCE_FILE)
function(generate NAME SOURCE_FILE GUEST_BITNESS)
# Interface target for the user to add include directories
add_library(${NAME}-deps INTERFACE)
target_include_directories(${NAME}-deps INTERFACE "${CMAKE_CURRENT_SOURCE_DIR}/../include")
target_link_libraries(${NAME}-deps INTERFACE FEXLoader)
add_library(${NAME}-${GUEST_BITNESS}-deps INTERFACE)
target_include_directories(${NAME}-${GUEST_BITNESS}-deps INTERFACE "${CMAKE_CURRENT_SOURCE_DIR}/../include")
target_link_libraries(${NAME}-${GUEST_BITNESS}-deps INTERFACE FEXLoader)
# Shorthand for the include directories added after calling this function.
# This is not evaluated directly, hence directories added after return are still picked up
set(prop "$<TARGET_PROPERTY:${NAME}-deps,INTERFACE_INCLUDE_DIRECTORIES>")
set(compile_prop "$<TARGET_PROPERTY:${NAME}-deps,INTERFACE_COMPILE_DEFINITIONS>")
set(prop "$<TARGET_PROPERTY:${NAME}-${GUEST_BITNESS}-deps,INTERFACE_INCLUDE_DIRECTORIES>")
set(compile_prop "$<TARGET_PROPERTY:${NAME}-${GUEST_BITNESS}-deps,INTERFACE_COMPILE_DEFINITIONS>")
if (CMAKE_SYSTEM_PROCESSOR MATCHES "x86_64")
list(APPEND compile_prop _M_X86_64=1)
elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "aarch64")
@@ -31,11 +31,11 @@ function(generate NAME SOURCE_FILE)
endif()
# Target for IDE integration
add_library(${NAME}-interface EXCLUDE_FROM_ALL ${SOURCE_FILE})
target_link_libraries(${NAME}-interface PRIVATE ${NAME}-deps)
add_library(${NAME}-${GUEST_BITNESS}-interface EXCLUDE_FROM_ALL ${SOURCE_FILE})
target_link_libraries(${NAME}-${GUEST_BITNESS}-interface PRIVATE ${NAME}-${GUEST_BITNESS}-deps)
# Run thunk generator for each of the given output files
set(OUTFOLDER "${CMAKE_CURRENT_BINARY_DIR}/gen")
set(OUTFOLDER "${CMAKE_CURRENT_BINARY_DIR}/gen_${GUEST_BITNESS}")
set(OUTFILE "${OUTFOLDER}/thunkgen_host_${NAME}.inl")
file(MAKE_DIRECTORY "${OUTFOLDER}")
@@ -57,7 +57,7 @@ function(generate NAME SOURCE_FILE)
set(GEN_${NAME} ${OUTPUTS} PARENT_SCOPE)
endfunction()
function(add_host_lib NAME)
function(add_host_lib NAME GUEST_BITNESS)
set (SOURCE_FILE ../lib${NAME}/lib${NAME}_Host.cpp)
get_filename_component(SOURCE_FILE_ABS "${SOURCE_FILE}" ABSOLUTE)
if (NOT EXISTS "${SOURCE_FILE_ABS}")
@@ -68,105 +68,114 @@ function(add_host_lib NAME)
endif()
endif()
add_library(${NAME}-host SHARED ${SOURCE_FILE} ${GEN_lib${NAME}})
target_include_directories(${NAME}-host PRIVATE "${CMAKE_CURRENT_BINARY_DIR}/gen/")
target_link_libraries(${NAME}-host PRIVATE dl)
target_link_libraries(${NAME}-host PRIVATE lib${NAME}-deps)
add_library(${NAME}-host-${GUEST_BITNESS} SHARED ${SOURCE_FILE} ${GEN_lib${NAME}})
set_target_properties(${NAME}-host-${GUEST_BITNESS} PROPERTIES OUTPUT_NAME "${NAME}-host")
set_target_properties(${NAME}-host-${GUEST_BITNESS} PROPERTIES LIBRARY_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/HostLibs_${GUEST_BITNESS}")
target_include_directories(${NAME}-host-${GUEST_BITNESS} PRIVATE "${CMAKE_CURRENT_BINARY_DIR}/gen_${GUEST_BITNESS}/")
target_link_libraries(${NAME}-host-${GUEST_BITNESS} PRIVATE dl)
target_link_libraries(${NAME}-host-${GUEST_BITNESS} PRIVATE lib${NAME}-${GUEST_BITNESS}-deps)
## Make signed overflow well defined 2's complement overflow
target_compile_options(${NAME}-host PRIVATE -fwrapv)
target_compile_options(${NAME}-host-${GUEST_BITNESS} PRIVATE -fwrapv)
# generated files forward-declare functions that need to be implemented manually, so pass --no-undefined to make sure errors are detected at compile-time rather than runtime
target_link_options(${NAME}-host PRIVATE "LINKER:--no-undefined")
target_link_options(${NAME}-host-${GUEST_BITNESS} PRIVATE "LINKER:--no-undefined")
install(TARGETS ${NAME}-host DESTINATION ${HOSTLIBS_DATA_DIRECTORY}/HostThunks/)
if (${GUEST_BITNESS} EQUAL 32)
install(TARGETS ${NAME}-host-${GUEST_BITNESS} DESTINATION ${HOSTLIBS_DATA_DIRECTORY}/HostThunks_32/)
else()
install(TARGETS ${NAME}-host-${GUEST_BITNESS} DESTINATION ${HOSTLIBS_DATA_DIRECTORY}/HostThunks/)
endif()
endfunction()
#add_host_lib(fex_malloc_symbols)
set (BITNESS_LIST "32;64")
foreach(GUEST_BITNESS IN LISTS BITNESS_LIST)
#add_host_lib(fex_malloc_symbols ${GUEST_BITNESS})
#generate(libfex_malloc)
#add_host_lib(fex_malloc)
#generate(libfex_malloc)
#add_host_lib(fex_malloc ${GUEST_BITNESS})
generate(libasound ${CMAKE_CURRENT_SOURCE_DIR}/../libasound/libasound_interface.cpp)
add_host_lib(asound)
generate(libasound ${CMAKE_CURRENT_SOURCE_DIR}/../libasound/libasound_interface.cpp ${GUEST_BITNESS})
add_host_lib(asound ${GUEST_BITNESS})
generate(libEGL ${CMAKE_CURRENT_SOURCE_DIR}/../libEGL/libEGL_interface.cpp)
add_host_lib(EGL)
generate(libEGL ${CMAKE_CURRENT_SOURCE_DIR}/../libEGL/libEGL_interface.cpp ${GUEST_BITNESS})
add_host_lib(EGL ${GUEST_BITNESS})
generate(libGL ${CMAKE_CURRENT_SOURCE_DIR}/../libGL/libGL_interface.cpp)
add_host_lib(GL)
generate(libGL ${CMAKE_CURRENT_SOURCE_DIR}/../libGL/libGL_interface.cpp ${GUEST_BITNESS})
add_host_lib(GL ${GUEST_BITNESS})
find_package(OpenGL REQUIRED)
target_link_libraries(GL-host PRIVATE OpenGL::GL)
find_package(OpenGL REQUIRED)
target_link_libraries(GL-host-${GUEST_BITNESS} PRIVATE OpenGL::GL)
# disabled for now, headers are platform specific
# find_package(SDL2 REQUIRED)
# generate(libSDL2)
# add_host_lib(SDL2)
# target_include_directories(SDL2-host PRIVATE ${SDL2_INCLUDE_DIRS})
# disabled for now, headers are platform specific
# find_package(SDL2 REQUIRED)
# generate(libSDL2)
# add_host_lib(SDL2 ${GUEST_BITNESS})
# target_include_directories(SDL2-host PRIVATE ${SDL2_INCLUDE_DIRS})
find_package(PkgConfig)
pkg_search_module(X11 REQUIRED x11)
find_package(PkgConfig)
pkg_search_module(X11 REQUIRED x11)
string(REGEX MATCH "([0-9]*)\.([0-9]*)\.([0-9]*)" _ "${X11_VERSION}")
set(X11_VERSION_MAJOR ${CMAKE_MATCH_1})
set(X11_VERSION_MINOR ${CMAKE_MATCH_2})
set(X11_VERSION_PATCH ${CMAKE_MATCH_3})
string(REGEX MATCH "([0-9]*)\.([0-9]*)\.([0-9]*)" _ "${X11_VERSION}")
set(X11_VERSION_MAJOR ${CMAKE_MATCH_1})
set(X11_VERSION_MINOR ${CMAKE_MATCH_2})
set(X11_VERSION_PATCH ${CMAKE_MATCH_3})
generate(libX11 ${CMAKE_CURRENT_SOURCE_DIR}/../libX11/libX11_interface.cpp)
add_host_lib(X11)
generate(libX11 ${CMAKE_CURRENT_SOURCE_DIR}/../libX11/libX11_interface.cpp ${GUEST_BITNESS})
add_host_lib(X11 ${GUEST_BITNESS})
target_compile_definitions(libX11-deps INTERFACE -DX11_VERSION_MAJOR=${X11_VERSION_MAJOR})
target_compile_definitions(libX11-deps INTERFACE -DX11_VERSION_MINOR=${X11_VERSION_MINOR})
target_compile_definitions(libX11-deps INTERFACE -DX11_VERSION_PATCH=${X11_VERSION_PATCH})
target_compile_definitions(libX11-${GUEST_BITNESS}-deps INTERFACE -DX11_VERSION_MAJOR=${X11_VERSION_MAJOR})
target_compile_definitions(libX11-${GUEST_BITNESS}-deps INTERFACE -DX11_VERSION_MINOR=${X11_VERSION_MINOR})
target_compile_definitions(libX11-${GUEST_BITNESS}-deps INTERFACE -DX11_VERSION_PATCH=${X11_VERSION_PATCH})
generate(libXext ${CMAKE_CURRENT_SOURCE_DIR}/../libXext/libXext_interface.cpp)
add_host_lib(Xext)
generate(libXext ${CMAKE_CURRENT_SOURCE_DIR}/../libXext/libXext_interface.cpp ${GUEST_BITNESS})
add_host_lib(Xext ${GUEST_BITNESS})
target_compile_definitions(libXext-deps INTERFACE -DX11_VERSION_MAJOR=${X11_VERSION_MAJOR})
target_compile_definitions(libXext-deps INTERFACE -DX11_VERSION_MINOR=${X11_VERSION_MINOR})
target_compile_definitions(libXext-deps INTERFACE -DX11_VERSION_PATCH=${X11_VERSION_PATCH})
target_compile_definitions(libXext-${GUEST_BITNESS}-deps INTERFACE -DX11_VERSION_MAJOR=${X11_VERSION_MAJOR})
target_compile_definitions(libXext-${GUEST_BITNESS}-deps INTERFACE -DX11_VERSION_MINOR=${X11_VERSION_MINOR})
target_compile_definitions(libXext-${GUEST_BITNESS}-deps INTERFACE -DX11_VERSION_PATCH=${X11_VERSION_PATCH})
generate(libXrender ${CMAKE_CURRENT_SOURCE_DIR}/../libXrender/libXrender_interface.cpp)
add_host_lib(Xrender)
generate(libXrender ${CMAKE_CURRENT_SOURCE_DIR}/../libXrender/libXrender_interface.cpp ${GUEST_BITNESS})
add_host_lib(Xrender ${GUEST_BITNESS})
generate(libXfixes ${CMAKE_CURRENT_SOURCE_DIR}/../libXfixes/libXfixes_interface.cpp)
add_host_lib(Xfixes)
generate(libXfixes ${CMAKE_CURRENT_SOURCE_DIR}/../libXfixes/libXfixes_interface.cpp ${GUEST_BITNESS})
add_host_lib(Xfixes ${GUEST_BITNESS})
generate(libvulkan ${CMAKE_CURRENT_SOURCE_DIR}/../libvulkan/libvulkan_interface.cpp)
target_include_directories(libvulkan-deps INTERFACE ${FEX_PROJECT_SOURCE_DIR}/External/Vulkan-Headers/include/)
add_host_lib(vulkan)
generate(libvulkan ${CMAKE_CURRENT_SOURCE_DIR}/../libvulkan/libvulkan_interface.cpp ${GUEST_BITNESS})
target_include_directories(libvulkan-${GUEST_BITNESS}-deps INTERFACE ${FEX_PROJECT_SOURCE_DIR}/External/Vulkan-Headers/include/)
add_host_lib(vulkan ${GUEST_BITNESS})
generate(libxcb ${CMAKE_CURRENT_SOURCE_DIR}/../libxcb/libxcb_interface.cpp)
add_host_lib(xcb)
generate(libxcb ${CMAKE_CURRENT_SOURCE_DIR}/../libxcb/libxcb_interface.cpp ${GUEST_BITNESS})
add_host_lib(xcb ${GUEST_BITNESS})
generate(libxcb-dri2 ${CMAKE_CURRENT_SOURCE_DIR}/../libxcb-dri2/libxcb-dri2_interface.cpp)
add_host_lib(xcb-dri2)
generate(libxcb-dri2 ${CMAKE_CURRENT_SOURCE_DIR}/../libxcb-dri2/libxcb-dri2_interface.cpp ${GUEST_BITNESS})
add_host_lib(xcb-dri2 ${GUEST_BITNESS})
generate(libxcb-dri3 ${CMAKE_CURRENT_SOURCE_DIR}/../libxcb-dri3/libxcb-dri3_interface.cpp)
add_host_lib(xcb-dri3)
generate(libxcb-dri3 ${CMAKE_CURRENT_SOURCE_DIR}/../libxcb-dri3/libxcb-dri3_interface.cpp ${GUEST_BITNESS})
add_host_lib(xcb-dri3 ${GUEST_BITNESS})
generate(libxcb-xfixes ${CMAKE_CURRENT_SOURCE_DIR}/../libxcb-xfixes/libxcb-xfixes_interface.cpp)
add_host_lib(xcb-xfixes)
generate(libxcb-xfixes ${CMAKE_CURRENT_SOURCE_DIR}/../libxcb-xfixes/libxcb-xfixes_interface.cpp ${GUEST_BITNESS})
add_host_lib(xcb-xfixes ${GUEST_BITNESS})
generate(libxcb-shm ${CMAKE_CURRENT_SOURCE_DIR}/../libxcb-shm/libxcb-shm_interface.cpp)
add_host_lib(xcb-shm)
generate(libxcb-shm ${CMAKE_CURRENT_SOURCE_DIR}/../libxcb-shm/libxcb-shm_interface.cpp ${GUEST_BITNESS})
add_host_lib(xcb-shm ${GUEST_BITNESS})
generate(libxcb-sync ${CMAKE_CURRENT_SOURCE_DIR}/../libxcb-sync/libxcb-sync_interface.cpp)
add_host_lib(xcb-sync)
generate(libxcb-sync ${CMAKE_CURRENT_SOURCE_DIR}/../libxcb-sync/libxcb-sync_interface.cpp ${GUEST_BITNESS})
add_host_lib(xcb-sync ${GUEST_BITNESS})
generate(libxcb-present ${CMAKE_CURRENT_SOURCE_DIR}/../libxcb-present/libxcb-present_interface.cpp)
add_host_lib(xcb-present)
generate(libxcb-present ${CMAKE_CURRENT_SOURCE_DIR}/../libxcb-present/libxcb-present_interface.cpp ${GUEST_BITNESS})
add_host_lib(xcb-present ${GUEST_BITNESS})
generate(libxcb-randr ${CMAKE_CURRENT_SOURCE_DIR}/../libxcb-randr/libxcb-randr_interface.cpp)
add_host_lib(xcb-randr)
generate(libxcb-randr ${CMAKE_CURRENT_SOURCE_DIR}/../libxcb-randr/libxcb-randr_interface.cpp ${GUEST_BITNESS})
add_host_lib(xcb-randr ${GUEST_BITNESS})
generate(libxcb-glx ${CMAKE_CURRENT_SOURCE_DIR}/../libxcb-glx/libxcb-glx_interface.cpp)
add_host_lib(xcb-glx)
generate(libxcb-glx ${CMAKE_CURRENT_SOURCE_DIR}/../libxcb-glx/libxcb-glx_interface.cpp ${GUEST_BITNESS})
add_host_lib(xcb-glx ${GUEST_BITNESS})
generate(libxshmfence ${CMAKE_CURRENT_SOURCE_DIR}/../libxshmfence/libxshmfence_interface.cpp)
add_host_lib(xshmfence)
generate(libxshmfence ${CMAKE_CURRENT_SOURCE_DIR}/../libxshmfence/libxshmfence_interface.cpp ${GUEST_BITNESS})
add_host_lib(xshmfence ${GUEST_BITNESS})
generate(libdrm ${CMAKE_CURRENT_SOURCE_DIR}/../libdrm/libdrm_interface.cpp)
target_include_directories(libdrm-deps INTERFACE /usr/include/drm/)
target_include_directories(libdrm-deps INTERFACE /usr/include/libdrm/)
add_host_lib(drm)
generate(libdrm ${CMAKE_CURRENT_SOURCE_DIR}/../libdrm/libdrm_interface.cpp ${GUEST_BITNESS})
target_include_directories(libdrm-${GUEST_BITNESS}-deps INTERFACE /usr/include/drm/)
target_include_directories(libdrm-${GUEST_BITNESS}-deps INTERFACE /usr/include/libdrm/)
add_host_lib(drm ${GUEST_BITNESS})
endforeach()
+11 -3
View File
@@ -101,6 +101,7 @@ inline bool IsLibLoaded(const char *libname) {
template<auto Thunk, typename Result, typename... Args>
inline Result CallHostFunction(Args... args) {
#ifndef _M_ARM_64
#if __SIZEOF_POINTER__ == 8
// This magic incantation of using a register variable with an empty asm block is necessary for correct operation!
// If we only use inline asm that sets a variable then the compiler will reorder the function
// prologue to be BEFORE our inline asm. Which makes sense in hindsight, but for anything with 8+ arguments this
@@ -121,6 +122,13 @@ inline Result CallHostFunction(Args... args) {
// This magic incantation also works in that instance so this is about the best we can do without adding a new attribute to clang for modifying the
// ABI.
asm volatile("" : "=r" (host_addr));
#else
// Use mm0 to pass in host_addr (chosen to avoid conflicts with vectorcall).
// Note this register overlaps the x87 st(0) register (used to return float values),
// so applications that expect this register to be preserved could run into problems.
register uintptr_t host_addr asm ("mm0");
asm volatile("" : "=r" (host_addr));
#endif
#else
uintptr_t host_addr = 0;
#endif
@@ -141,9 +149,9 @@ inline Result CallHostFunction(Args... args) {
// Convenience wrapper that returns the function pointer to a CallHostFunction
// instantiation matching the function signature of `host_func`
template<typename Result, typename...Args>
static auto GetCallerForHostFunction(THUNK_ABI Result (*host_func)(Args...))
-> THUNK_ABI Result(*)(Args...) {
return &CallHostFunction<fexthunks_invoke_callback<THUNK_ABI Result(Args...)>, Result, Args...>;
static auto GetCallerForHostFunction(Result (*host_func)(Args...))
-> Result(*)(Args...) {
return &CallHostFunction<fexthunks_invoke_callback<Result(Args...)>, Result, Args...>;
}
// Ensures the given host function can safely be called from guest code.
+1 -1
View File
@@ -1,4 +1,4 @@
# FEX-2212
# FEX-2301
## External/FEXCore
See [FEXCore/Readme.md](../External/FEXCore/Readme.md) for more details
@@ -0,0 +1,20 @@
%ifdef CONFIG
{
"RegData": {
"RAX": "0x10011"
},
"Mode": "32BIT"
}
%endif
mov esp, 0xe0000010
; This is a common pattern in 32-bit PIE code.
; 32-bit GOT calculation needs to do a call+pop to do get the EIP.
; LEA doesn't work because it there is no EIP relative ops like on x86-64.
call target
target:
pop eax
hlt
+4
View File
@@ -3,6 +3,10 @@ Test_REP/F3_52.asm
Test_REP/F3_53.asm
Test_TwoByte/0F_52.asm
Test_TwoByte/0F_53.asm
Test_VEX/vrcpps.asm
Test_VEX/vrcpss.asm
Test_VEX/vrsqrtss.asm
Test_VEX/vrsqrtps.asm
# Not supported in userspace in all cases
Test_Primary/Primary_D7.asm
+13
View File
@@ -5,6 +5,12 @@ Test_H0F38/66_DD.asm
Test_H0F38/66_DE.asm
Test_H0F38/66_DF.asm
Test_H0F3A/0_66_DF.asm
Test_VEX/vaesdec.asm
Test_VEX/vaesdeclast.asm
Test_VEX/vaesenc.asm
Test_VEX/vaesenclast.asm
Test_VEX/vaesimc.asm
Test_VEX/vaeskeygenassist.asm
# PCMUL considered to be part of crypto operations. Simulator doesn't support this.
Test_H0F3A/pclmulqdq.asm
@@ -70,3 +76,10 @@ Test_H0F3A/66_09.asm
Test_H0F3A/66_0A.asm
Test_H0F3A/66_0B.asm
Test_OpSize/66_5B.asm
Test_VEX/vcvtpd2dq_inexact.asm
Test_VEX/vcvtps2dq_inexact.asm
Test_VEX/vldmxcsr.asm
Test_VEX/vroundpd.asm
Test_VEX/vroundps.asm
Test_VEX/vroundsd.asm
Test_VEX/vroundss.asm
+3
View File
@@ -15,3 +15,6 @@ Test_Secondary/15_F3_03_2.asm
# Zen+ CI doesn't support UMIP so it returns "real" values
Test_Secondary/07_XX_00.asm
# We don't emulate all of the MXCSR bits
Test_VEX/vldmxcsr.asm
+69
View File
@@ -0,0 +1,69 @@
%ifdef CONFIG
{
"HostFeatures": ["AVX"],
"RegData": {
"XMM0": ["0x4014000000000000", "0x5152535455565758", "0x0000000000000000", "0x0000000000000000"],
"XMM1": ["0x4010000000000000", "0x5152535455565758", "0x5152535455565758", "0x5152535455565758"],
"XMM2": ["0x4039000000000000", "0x5152535455565758", "0x0000000000000000", "0x0000000000000000"],
"XMM3": ["0x4030000000000000", "0x5152535455565758", "0x5152535455565758", "0x5152535455565758"],
"XMM4": ["0x403A000000000000", "0x5152535455565758", "0x0000000000000000", "0x0000000000000000"],
"XMM5": ["0x403D000000000000", "0x5152535455565758", "0x0000000000000000", "0x0000000000000000"],
"XMM6": ["0x4039000000000000", "0x5152535455565758", "0x5152535455565758", "0x5152535455565758"],
"XMM7": ["0x404B000000000000", "0x5152535455565758", "0x0000000000000000", "0x0000000000000000"]
},
"MemoryRegions": {
"0x100000000": "4096"
}
}
%endif
lea rdx, [rel .data]
vmovapd ymm0, [rdx + 32 * 0]
vmovapd ymm1, [rdx + 32 * 1]
vmovapd ymm2, [rdx + 32 * 2]
vmovapd ymm3, [rdx + 32 * 3]
vmovapd ymm4, [rdx + 32 * 4]
vmovapd ymm5, [rdx + 32 * 4]
; Register only
vaddsd xmm0, xmm0, xmm1
vaddsd xmm2, xmm2, xmm3
; Memory operand
vaddsd xmm4, xmm4, [rdx + 32 * 0]
vaddsd xmm5, xmm5, [rdx + 32 * 1]
; Merging different src into destination
vpxor xmm7, xmm7, xmm7
vmovapd ymm6, [rdx + 32 * 4]
vaddsd xmm7, xmm5, xmm6
hlt
align 32
.data:
dq 0x3FF0000000000000 ; 1.0
dq 0x5152535455565758
dq 0x5152535455565758
dq 0x5152535455565758
dq 0x4010000000000000 ; 4.0
dq 0x5152535455565758
dq 0x5152535455565758
dq 0x5152535455565758
dq 0x4022000000000000 ; 9.0
dq 0x5152535455565758
dq 0x5152535455565758
dq 0x5152535455565758
dq 0x4030000000000000 ; 16.0
dq 0x5152535455565758
dq 0x5152535455565758
dq 0x5152535455565758
dq 0x4039000000000000 ; 25.0
dq 0x5152535455565758
dq 0x5152535455565758
dq 0x5152535455565758
+69
View File
@@ -0,0 +1,69 @@
%ifdef CONFIG
{
"HostFeatures": ["AVX"],
"RegData": {
"XMM0": ["0x4142434440A00000", "0x5152535455565758", "0x0000000000000000", "0x0000000000000000"],
"XMM1": ["0x4142434440800000", "0x5152535455565758", "0x5152535455565758", "0x5152535455565758"],
"XMM2": ["0x4142434441C80000", "0x5152535455565758", "0x0000000000000000", "0x0000000000000000"],
"XMM3": ["0x4142434441800000", "0x5152535455565758", "0x5152535455565758", "0x5152535455565758"],
"XMM4": ["0x4142434441D00000", "0x5152535455565758", "0x0000000000000000", "0x0000000000000000"],
"XMM5": ["0x4142434441800000", "0x5152535455565758", "0x5152535455565758", "0x5152535455565758"],
"XMM6": ["0x4142434441C80000", "0x5152535455565758", "0x5152535455565758", "0x5152535455565758"],
"XMM7": ["0x4142434442240000", "0x5152535455565758", "0x0000000000000000", "0x0000000000000000"]
},
"MemoryRegions": {
"0x100000000": "4096"
}
}
%endif
lea rdx, [rel .data]
vmovapd ymm0, [rdx + 32 * 0]
vmovapd ymm1, [rdx + 32 * 1]
vmovapd ymm2, [rdx + 32 * 2]
vmovapd ymm3, [rdx + 32 * 3]
vmovapd ymm4, [rdx + 32 * 4]
; Register only
vaddss xmm0, xmm0, xmm1
vaddss xmm2, xmm2, xmm3
; Memory operand
vaddss xmm4, xmm4, [rdx + 32 * 0]
vaddss xmm5, xmm5, [rdx + 32 * 1]
; Merging different src into destination
vpxor xmm7, xmm7, xmm7
vmovapd ymm5, [rdx + 32 * 3]
vmovapd ymm6, [rdx + 32 * 4]
vaddss xmm7, xmm5, xmm6
hlt
align 32
.data:
dq 0x414243443F800000 ; 1.0
dq 0x5152535455565758
dq 0x5152535455565758
dq 0x5152535455565758
dq 0x4142434440800000 ; 4.0
dq 0x5152535455565758
dq 0x5152535455565758
dq 0x5152535455565758
dq 0x4142434441100000 ; 9.0
dq 0x5152535455565758
dq 0x5152535455565758
dq 0x5152535455565758
dq 0x4142434441800000 ; 16.0
dq 0x5152535455565758
dq 0x5152535455565758
dq 0x5152535455565758
dq 0x4142434441C80000 ; 25.0
dq 0x5152535455565758
dq 0x5152535455565758
dq 0x5152535455565758
+49
View File
@@ -0,0 +1,49 @@
%ifdef CONFIG
{
"HostFeatures": ["AVX"],
"RegData": {
"XMM2": ["0xBFF0000000000000", "0x4008000000000000", "0x0000000000000000", "0x0000000000000000"],
"XMM3": ["0xBFF0000000000000", "0x4008000000000000", "0x0000000000000000", "0x0000000000000000"],
"XMM4": ["0x3FF0000000000000", "0x4008000000000000", "0x0000000000000000", "0x0000000000000000"],
"XMM5": ["0x3FF0000000000000", "0x4008000000000000", "0x0000000000000000", "0x0000000000000000"],
"XMM6": ["0xBFF0000000000000", "0x4008000000000000", "0xBFF0000000000000", "0x4008000000000000"],
"XMM7": ["0xBFF0000000000000", "0x4008000000000000", "0xBFF0000000000000", "0x4008000000000000"],
"XMM8": ["0x3FF0000000000000", "0x4008000000000000", "0x3FF0000000000000", "0x4008000000000000"],
"XMM9": ["0x3FF0000000000000", "0x4008000000000000", "0x3FF0000000000000", "0x4008000000000000"]
},
"MemoryRegions": {
"0x100000000": "4096"
}
}
%endif
lea rdx, [rel .data]
vmovapd ymm0, [rdx]
vmovapd ymm1, [rdx + 32]
vaddsubpd xmm2, xmm0, [rdx + 32]
vaddsubpd xmm3, xmm0, xmm1
vaddsubpd xmm4, xmm1, [rdx]
vaddsubpd xmm5, xmm1, xmm0
vaddsubpd ymm6, ymm0, [rdx + 32]
vaddsubpd ymm7, ymm0, ymm1
vaddsubpd ymm8, ymm1, [rdx]
vaddsubpd ymm9, ymm1, ymm0
hlt
align 32
.data:
dq 0x3FF0000000000000
dq 0x3FF0000000000000
dq 0x3FF0000000000000
dq 0x3FF0000000000000
dq 0x4000000000000000
dq 0x4000000000000000
dq 0x4000000000000000
dq 0x4000000000000000
+39
View File
@@ -0,0 +1,39 @@
%ifdef CONFIG
{
"HostFeatures": ["AVX"],
"RegData": {
"XMM2": ["0x41200000C0000000", "0x41200000C0C00000", "0x0000000000000000", "0x0000000000000000"],
"XMM3": ["0x41200000C0000000", "0x41200000C0C00000", "0x41200000C0000000", "0x41200000C0C00000"],
"XMM4": ["0x41200000C0000000", "0x41200000C0C00000", "0x0000000000000000", "0x0000000000000000"],
"XMM5": ["0x41200000C0000000", "0x41200000C0C00000", "0x41200000C0000000", "0x41200000C0C00000"]
},
"MemoryRegions": {
"0x100000000": "4096"
}
}
%endif
lea rdx, [rel .data]
vmovapd ymm0, [rdx]
vmovapd ymm1, [rdx + 32]
vaddsubps xmm2, xmm0, [rdx + 32]
vaddsubps ymm3, ymm0, [rdx + 32]
vaddsubps xmm4, xmm0, xmm1
vaddsubps ymm5, ymm0, ymm1
hlt
align 32
.data:
dq 0x4080000040400000 ; 4, 3
dq 0x400000003f800000 ; 2, 1
dq 0x4080000040400000 ; 4, 3
dq 0x400000003f800000 ; 2, 1
dq 0x40C0000040A00000 ; 6, 5
dq 0x4100000040E00000 ; 8, 7
dq 0x40C0000040A00000 ; 6, 5
dq 0x4100000040E00000 ; 8, 7
+49
View File
@@ -0,0 +1,49 @@
%ifdef CONFIG
{
"HostFeatures": ["AVX"],
"RegData": {
"XMM1": ["0x7A1FC5A0A07A1FC5", "0xC5A07A1F1FC5A07A", "0x0000000000000000", "0x0000000000000000"],
"XMM2": ["0x85E03A5F5F85E03A", "0x3A5F85E0E03A5F85", "0x0000000000000000", "0x0000000000000000"],
"XMM3": ["0x7A1FC5A1A07A1FC4", "0xC5A07A1E1FC5A07B", "0x0000000000000000", "0x0000000000000000"],
"XMM4": ["0x85E03A5FA07A1FC5", "0xC5A07A1EE03A5F85", "0x0000000000000000", "0x0000000000000000"]
}
}
%endif
lea rdx, [rel .data]
vmovaps ymm0, [rdx + 32 * 4]
vaesdec xmm1, xmm0, [rdx + 32 * 0]
vaesdec xmm2, xmm0, [rdx + 32 * 1]
vaesdec xmm3, xmm0, [rdx + 32 * 2]
vaesdec xmm4, xmm0, [rdx + 32 * 3]
hlt
align 32
.data:
dq 0x0000000000000000
dq 0x0000000000000000
dq 0x0000000000000000
dq 0x0000000000000000
dq 0xFFFFFFFFFFFFFFFF
dq 0xFFFFFFFFFFFFFFFF
dq 0xFFFFFFFFFFFFFFFF
dq 0xFFFFFFFFFFFFFFFF
dq 0x0000000100000001
dq 0x0000000100000001
dq 0x0000000100000001
dq 0x0000000100000001
dq 0xFFFFFFFF00000000
dq 0x00000001FFFFFFFF
dq 0xFFFFFFFF00000000
dq 0x00000001FFFFFFFF
dq 0x0202020202020202
dq 0x0303030303030303
dq 0x0202020202020202
dq 0x0303030303030303
+49
View File
@@ -0,0 +1,49 @@
%ifdef CONFIG
{
"HostFeatures": ["AVX"],
"RegData": {
"XMM1": ["0xD5D56A6A6AD5D56A", "0x6A6AD5D5D56A6AD5", "0x0000000000000000", "0x0000000000000000"],
"XMM2": ["0x2A2A9595952A2A95", "0x95952A2A2A95952A", "0x0000000000000000", "0x0000000000000000"],
"XMM3": ["0xD5D56A6B6AD5D56B", "0x6A6AD5D4D56A6AD4", "0x0000000000000000", "0x0000000000000000"],
"XMM4": ["0x2A2A95956AD5D56A", "0x6A6AD5D42A95952A", "0x0000000000000000", "0x0000000000000000"]
}
}
%endif
lea rdx, [rel .data]
vmovaps ymm0, [rdx + 32 * 4]
vaesdeclast xmm1, xmm0, [rdx + 32 * 0]
vaesdeclast xmm2, xmm0, [rdx + 32 * 1]
vaesdeclast xmm3, xmm0, [rdx + 32 * 2]
vaesdeclast xmm4, xmm0, [rdx + 32 * 3]
hlt
align 32
.data:
dq 0x0000000000000000
dq 0x0000000000000000
dq 0x0000000000000000
dq 0x0000000000000000
dq 0xFFFFFFFFFFFFFFFF
dq 0xFFFFFFFFFFFFFFFF
dq 0xFFFFFFFFFFFFFFFF
dq 0xFFFFFFFFFFFFFFFF
dq 0x0000000100000001
dq 0x0000000100000001
dq 0x0000000100000001
dq 0x0000000100000001
dq 0xFFFFFFFF00000000
dq 0x00000001FFFFFFFF
dq 0xFFFFFFFF00000000
dq 0x00000001FFFFFFFF
dq 0x0202020202020202
dq 0x0303030303030303
dq 0x0202020202020202
dq 0x0303030303030303
+49
View File
@@ -0,0 +1,49 @@
%ifdef CONFIG
{
"HostFeatures": ["AVX"],
"RegData": {
"XMM1": ["0x77637B6F637B6F77", "0x7B6F77636F77637B", "0x0000000000000000", "0x0000000000000000"],
"XMM2": ["0x889C84909C849088", "0x8490889C90889C84", "0x0000000000000000", "0x0000000000000000"],
"XMM3": ["0x77637B6E637B6F76", "0x7B6F77626F77637A", "0x0000000000000000", "0x0000000000000000"],
"XMM4": ["0x889C8490637B6F77", "0x7B6F776290889C84", "0x0000000000000000", "0x0000000000000000"]
}
}
%endif
lea rdx, [rel .data]
vmovaps ymm0, [rdx + 32 * 4]
vaesenc xmm1, xmm0, [rdx + 32 * 0]
vaesenc xmm2, xmm0, [rdx + 32 * 1]
vaesenc xmm3, xmm0, [rdx + 32 * 2]
vaesenc xmm4, xmm0, [rdx + 32 * 3]
hlt
align 32
.data:
dq 0x0000000000000000
dq 0x0000000000000000
dq 0x0000000000000000
dq 0x0000000000000000
dq 0xFFFFFFFFFFFFFFFF
dq 0xFFFFFFFFFFFFFFFF
dq 0xFFFFFFFFFFFFFFFF
dq 0xFFFFFFFFFFFFFFFF
dq 0x0000000100000001
dq 0x0000000100000001
dq 0x0000000100000001
dq 0x0000000100000001
dq 0xFFFFFFFF00000000
dq 0x00000001FFFFFFFF
dq 0xFFFFFFFF00000000
dq 0x00000001FFFFFFFF
dq 0x0202020202020202
dq 0x0303030303030303
dq 0x0202020202020202
dq 0x0303030303030303
+49
View File
@@ -0,0 +1,49 @@
%ifdef CONFIG
{
"HostFeatures": ["AVX"],
"RegData": {
"XMM1": ["0x777B7B777B7B7777", "0x7B77777B77777B7B", "0x0000000000000000", "0x0000000000000000"],
"XMM2": ["0x8884848884848888", "0x8488888488888484", "0x0000000000000000", "0x0000000000000000"],
"XMM3": ["0x777B7B767B7B7776", "0x7B77777A77777B7A", "0x0000000000000000", "0x0000000000000000"],
"XMM4": ["0x888484887B7B7777", "0x7B77777A88888484", "0x0000000000000000", "0x0000000000000000"]
}
}
%endif
lea rdx, [rel .data]
vmovaps ymm0, [rdx + 32 * 4]
vaesenclast xmm1, xmm0, [rdx + 32 * 0]
vaesenclast xmm2, xmm0, [rdx + 32 * 1]
vaesenclast xmm3, xmm0, [rdx + 32 * 2]
vaesenclast xmm4, xmm0, [rdx + 32 * 3]
hlt
align 32
.data:
dq 0x0000000000000000
dq 0x0000000000000000
dq 0x0000000000000000
dq 0x0000000000000000
dq 0xFFFFFFFFFFFFFFFF
dq 0xFFFFFFFFFFFFFFFF
dq 0xFFFFFFFFFFFFFFFF
dq 0xFFFFFFFFFFFFFFFF
dq 0x0000000100000001
dq 0x0000000100000001
dq 0x0000000100000001
dq 0x0000000100000001
dq 0xFFFFFFFF00000000
dq 0x00000001FFFFFFFF
dq 0xFFFFFFFF00000000
dq 0x00000001FFFFFFFF
dq 0x0202020202020202
dq 0x0303030303030303
dq 0x0202020202020202
dq 0x0303030303030303
+66
View File
@@ -0,0 +1,66 @@
%ifdef CONFIG
{
"HostFeatures": ["AVX"],
"RegData": {
"XMM0": ["0x0000000000000000", "0x0000000000000000", "0x0000000000000000", "0x0000000000000000"],
"XMM1": ["0xFFFFFFFFFFFFFFFF", "0xFFFFFFFFFFFFFFFF", "0x0000000000000000", "0x0000000000000000"],
"XMM2": ["0x0B0D090E0B0D090E", "0x0B0D090E0B0D090E", "0x0000000000000000", "0x0000000000000000"],
"XMM3": ["0xFFFFFFFF00000000", "0x0B0D090EFFFFFFFF", "0x0000000000000000", "0x0000000000000000"],
"XMM4": ["0x0202020202020202", "0x0303030303030303", "0x0000000000000000", "0x0000000000000000"],
"XMM10": ["0x0000000000000000", "0x0000000000000000", "0x0000000000000000", "0x0000000000000000"],
"XMM11": ["0xFFFFFFFFFFFFFFFF", "0xFFFFFFFFFFFFFFFF", "0x0000000000000000", "0x0000000000000000"],
"XMM12": ["0x0B0D090E0B0D090E", "0x0B0D090E0B0D090E", "0x0000000000000000", "0x0000000000000000"],
"XMM13": ["0xFFFFFFFF00000000", "0x0B0D090EFFFFFFFF", "0x0000000000000000", "0x0000000000000000"],
"XMM14": ["0x0202020202020202", "0x0303030303030303", "0x0000000000000000", "0x0000000000000000"]
}
}
%endif
lea rdx, [rel .data]
vaesimc xmm0, [rdx + 32 * 0]
vaesimc xmm1, [rdx + 32 * 1]
vaesimc xmm2, [rdx + 32 * 2]
vaesimc xmm3, [rdx + 32 * 3]
vaesimc xmm4, [rdx + 32 * 4]
vmovapd ymm5, [rdx + 32 * 0]
vmovapd ymm6, [rdx + 32 * 1]
vmovapd ymm7, [rdx + 32 * 2]
vmovapd ymm8, [rdx + 32 * 3]
vmovapd ymm9, [rdx + 32 * 4]
vaesimc xmm10, xmm5
vaesimc xmm11, xmm6
vaesimc xmm12, xmm7
vaesimc xmm13, xmm8
vaesimc xmm14, xmm9
hlt
align 32
.data:
dq 0x0000000000000000
dq 0x0000000000000000
dq 0x0000000000000000
dq 0x0000000000000000
dq 0xFFFFFFFFFFFFFFFF
dq 0xFFFFFFFFFFFFFFFF
dq 0xFFFFFFFFFFFFFFFF
dq 0xFFFFFFFFFFFFFFFF
dq 0x0000000100000001
dq 0x0000000100000001
dq 0x0000000100000001
dq 0x0000000100000001
dq 0xFFFFFFFF00000000
dq 0x00000001FFFFFFFF
dq 0xFFFFFFFF00000000
dq 0x00000001FFFFFFFF
dq 0x0202020202020202
dq 0x0303030303030303
dq 0x0202020202020202
dq 0x0303030303030303
+64
View File
@@ -0,0 +1,64 @@
%ifdef CONFIG
{
"HostFeatures": ["AVX"],
"RegData": {
"XMM0": ["0x6363636363636363", "0x6363636363636363", "0x0000000000000000", "0x0000000000000000"],
"XMM1": ["0x1616161616161616", "0x1616161616161616", "0x0000000000000000", "0x0000000000000000"],
"XMM2": ["0x7C6363636363637C", "0x7C6363636363637C", "0x0000000000000000", "0x0000000000000000"],
"XMM3": ["0x1616161616161616", "0x7C6363636363637C", "0x0000000000000000", "0x0000000000000000"],
"XMM4": ["0x6363636263636363", "0x6363636263636363", "0x0000000000000000", "0x0000000000000000"],
"XMM5": ["0x1616161416161616", "0x1616161416161616", "0x0000000000000000", "0x0000000000000000"],
"XMM6": ["0x7C6363606363637C", "0x7C6363606363637C", "0x0000000000000000", "0x0000000000000000"],
"XMM7": ["0x1616161216161616", "0x7C6363676363637C", "0x0000000000000000", "0x0000000000000000"],
"XMM8": ["0x6363636663636363", "0x6363636663636363", "0x0000000000000000", "0x0000000000000000"],
"XMM9": ["0x1616161016161616", "0x1616161016161616", "0x0000000000000000", "0x0000000000000000"],
"XMM10": ["0x7C6363646363637C", "0x7C6363646363637C", "0x0000000000000000", "0x0000000000000000"],
"XMM11": ["0x1616161E16161616", "0x7C63636B6363637C", "0x0000000000000000", "0x0000000000000000"],
"XMM12": ["0x6363636A63636363", "0x6363636A63636363", "0x0000000000000000", "0x0000000000000000"],
"XMM13": ["0x1616161C16161616", "0x1616161C16161616", "0x0000000000000000", "0x0000000000000000"],
"XMM14": ["0x7C6363686363637C", "0x7C6363686363637C", "0x0000000000000000", "0x0000000000000000"],
"XMM15": ["0x1616161A16161616", "0x7C63636F6363637C", "0x0000000000000000", "0x0000000000000000"]
}
}
%endif
lea rdx, [rel .data]
vaeskeygenassist xmm0, [rdx + 16 * 0], 0
vaeskeygenassist xmm1, [rdx + 16 * 1], 0
vaeskeygenassist xmm2, [rdx + 16 * 2], 0
vaeskeygenassist xmm3, [rdx + 16 * 3], 0
vaeskeygenassist xmm4, [rdx + 16 * 0], 1
vaeskeygenassist xmm5, [rdx + 16 * 1], 2
vaeskeygenassist xmm6, [rdx + 16 * 2], 3
vaeskeygenassist xmm7, [rdx + 16 * 3], 4
vaeskeygenassist xmm8, [rdx + 16 * 0], 5
vaeskeygenassist xmm9, [rdx + 16 * 1], 6
vaeskeygenassist xmm10, [rdx + 16 * 2], 7
vaeskeygenassist xmm11, [rdx + 16 * 3], 8
vaeskeygenassist xmm12, [rdx + 16 * 0], 9
vaeskeygenassist xmm13, [rdx + 16 * 1], 10
vaeskeygenassist xmm14, [rdx + 16 * 2], 11
vaeskeygenassist xmm15, [rdx + 16 * 3], 12
hlt
align 16
.data:
dq 0x0000000000000000
dq 0x0000000000000000
dq 0xFFFFFFFFFFFFFFFF
dq 0xFFFFFFFFFFFFFFFF
dq 0x0000000100000001
dq 0x0000000100000001
dq 0xFFFFFFFF00000000
dq 0x00000001FFFFFFFF
dq 0x0202020202020202
dq 0x0303030303030303
+21
View File
@@ -0,0 +1,21 @@
%ifdef CONFIG
{
"HostFeatures": ["AVX"],
"RegData": {
"XMM0": ["0xA76C4F06A12BFCE0", "0x9B80767F1E6A060F", "0xA76C4F06A12BFCE0", "0x9B80767F1E6A060F"]
}
}
%endif
lea rdx, [rel .data]
vbroadcastf128 ymm0, [rdx]
hlt
align 32
.data:
dq 0xA76C4F06A12BFCE0
dq 0x9B80767F1E6A060F
dq 0xFFFFFFFFFFFFFFFF
dq 0xEEEEEEEEEEEEEEEE
+21
View File
@@ -0,0 +1,21 @@
%ifdef CONFIG
{
"HostFeatures": ["AVX"],
"RegData": {
"XMM0": ["0xA76C4F06A12BFCE0", "0x9B80767F1E6A060F", "0xA76C4F06A12BFCE0", "0x9B80767F1E6A060F"]
}
}
%endif
lea rdx, [rel .data]
vbroadcasti128 ymm0, [rdx]
hlt
align 32
.data:
dq 0xA76C4F06A12BFCE0
dq 0x9B80767F1E6A060F
dq 0xFFFFFFFFFFFFFFFF
dq 0xEEEEEEEEEEEEEEEE
+40
View File
@@ -0,0 +1,40 @@
%ifdef CONFIG
{
"HostFeatures": ["AVX"],
"RegData": {
"XMM0": ["0xA76C4F06A12BFCE0", "0x9B80767F1E6A060F", "0xFFFFFFFFFFFFFFFF", "0xEEEEEEEEEEEEEEEE"],
"XMM1": ["0x6868C3F3AAED56E0", "0xF0FCE9E294E6E6DE", "0xDDDDDDDDDDDDDDDD", "0xCCCCCCCCCCCCCCCC"],
"XMM2": ["0xA76C4F06A12BFCE0", "0xA76C4F06A12BFCE0", "0xA76C4F06A12BFCE0", "0xA76C4F06A12BFCE0"],
"XMM3": ["0x6868C3F3AAED56E0", "0x6868C3F3AAED56E0", "0x6868C3F3AAED56E0", "0x6868C3F3AAED56E0"],
"XMM4": ["0xFFFFFFFFFFFFFFFF", "0xFFFFFFFFFFFFFFFF", "0xFFFFFFFFFFFFFFFF", "0xFFFFFFFFFFFFFFFF"],
"XMM5": ["0xEEEEEEEEEEEEEEEE", "0xEEEEEEEEEEEEEEEE", "0xEEEEEEEEEEEEEEEE", "0xEEEEEEEEEEEEEEEE"]
}
}
%endif
lea rdx, [rel .data]
vmovaps ymm0, [rdx + 32 * 0]
vmovaps ymm1, [rdx + 32 * 1]
; Register broadcasting
vbroadcastsd ymm2, xmm0
vbroadcastsd ymm3, xmm1
; Memory broadcasting
vbroadcastsd ymm4, [rdx + 16]
vbroadcastsd ymm5, [rdx + 24]
hlt
align 32
.data:
dq 0xA76C4F06A12BFCE0
dq 0x9B80767F1E6A060F
dq 0xFFFFFFFFFFFFFFFF
dq 0xEEEEEEEEEEEEEEEE
dq 0x6868C3F3AAED56E0
dq 0xF0FCE9E294E6E6DE
dq 0xDDDDDDDDDDDDDDDD
dq 0xCCCCCCCCCCCCCCCC
+40
View File
@@ -0,0 +1,40 @@
%ifdef CONFIG
{
"HostFeatures": ["AVX"],
"RegData": {
"XMM0": ["0xA76C4F06A12BFCE0", "0x9B80767F1E6A060F", "0xFFFFFFFFFFFFFFFF", "0xEEEEEEEEEEEEEEEE"],
"XMM1": ["0x6868C3F3AAED56E0", "0xF0FCE9E294E6E6DE", "0xDDDDDDDDDDDDDDDD", "0xCCCCCCCCCCCCCCCC"],
"XMM2": ["0xA12BFCE0A12BFCE0", "0xA12BFCE0A12BFCE0", "0xA12BFCE0A12BFCE0", "0xA12BFCE0A12BFCE0"],
"XMM3": ["0xAAED56E0AAED56E0", "0xAAED56E0AAED56E0", "0x0000000000000000", "0x0000000000000000"],
"XMM4": ["0xFFFFFFFFFFFFFFFF", "0xFFFFFFFFFFFFFFFF", "0xFFFFFFFFFFFFFFFF", "0xFFFFFFFFFFFFFFFF"],
"XMM5": ["0xEEEEEEEEEEEEEEEE", "0xEEEEEEEEEEEEEEEE", "0x0000000000000000", "0x0000000000000000"]
}
}
%endif
lea rdx, [rel .data]
vmovaps ymm0, [rdx + 32 * 0]
vmovaps ymm1, [rdx + 32 * 1]
; Register broadcasting
vbroadcastss ymm2, xmm0
vbroadcastss xmm3, xmm1
; Memory broadcasting
vbroadcastss ymm4, [rdx + 16]
vbroadcastss xmm5, [rdx + 24]
hlt
align 32
.data:
dq 0xA76C4F06A12BFCE0
dq 0x9B80767F1E6A060F
dq 0xFFFFFFFFFFFFFFFF
dq 0xEEEEEEEEEEEEEEEE
dq 0x6868C3F3AAED56E0
dq 0xF0FCE9E294E6E6DE
dq 0xDDDDDDDDDDDDDDDD
dq 0xCCCCCCCCCCCCCCCC
+76
View File
@@ -0,0 +1,76 @@
%ifdef CONFIG
{
"HostFeatures": ["AVX"],
"RegData": {
"XMM2": ["0xFFFFFFFFFFFFFFFF", "0x0000000000000000", "0x0000000000000000", "0x0000000000000000"],
"XMM3": ["0x0000000000000000", "0xFFFFFFFFFFFFFFFF", "0x0000000000000000", "0x0000000000000000"],
"XMM4": ["0xFFFFFFFFFFFFFFFF", "0xFFFFFFFFFFFFFFFF", "0x0000000000000000", "0x0000000000000000"],
"XMM5": ["0x0000000000000000", "0xFFFFFFFFFFFFFFFF", "0x0000000000000000", "0x0000000000000000"],
"XMM6": ["0xFFFFFFFFFFFFFFFF", "0x0000000000000000", "0x0000000000000000", "0x0000000000000000"],
"XMM7": ["0x0000000000000000", "0x0000000000000000", "0x0000000000000000", "0x0000000000000000"],
"XMM10": ["0xFFFFFFFFFFFFFFFF", "0xFFFFFFFFFFFFFFFF", "0x0000000000000000", "0x0000000000000000"],
"XMM11": ["0x0000000000000000", "0x0000000000000000", "0x0000000000000000", "0x0000000000000000"],
"XMM12": ["0x0000000000000000", "0xFFFFFFFFFFFFFFFF", "0x0000000000000000", "0x0000000000000000"],
"XMM13": ["0xFFFFFFFFFFFFFFFF", "0x0000000000000000", "0x0000000000000000", "0x0000000000000000"]
},
"MemoryRegions": {
"0x100000000": "4096"
}
}
%endif
lea rdx, [rel .data]
vmovapd ymm0, [rdx + 32 * 0]
vmovapd ymm1, [rdx + 32 * 1]
vcmppd xmm2, xmm0, xmm1, 0x00 ; EQ
vcmppd xmm3, xmm0, xmm1, 0x01 ; LT
vcmppd xmm4, xmm0, xmm1, 0x02 ; LTE
vcmppd xmm5, xmm0, xmm1, 0x04 ; NEQ
vcmppd xmm6, xmm0, xmm1, 0x05 ; NLT
vcmppd xmm7, xmm0, xmm1, 0x06 ; NLTE
; Unordered and Ordered tests need to be special cased
vmovapd ymm8, [rdx + 32 * 2]
vmovapd ymm9, [rdx + 32 * 3]
; Unordered will return true when either input is nan
; [0.0, nan] unord [nan, 0.0] = [1, 1]
vcmppd xmm10, xmm8, xmm9, 0x03 ; Unordered
; Ordered will return true when both inputs are NOT nan
; [0.0, nan] ord [nan, 0.0] = [0, 0]
vcmppd xmm11, xmm8, xmm9, 0x07 ; Ordered
; Ordered will return true when both inputs are NOT nan
; [nan, 0.0] ord [nan, 0.0] = [0, 1]
vcmppd xmm12, xmm9, xmm9, 0x07 ; Ordered
; Ordered will return true when both inputs are NOT nan
; [0.0, nan] ord [0.0, nan] = [1, 0]
vcmppd xmm13, xmm8, xmm8, 0x07 ; Ordered
hlt
align 32
.data:
dq 0x3FF0000000000000
dq 0x4000000000000000
dq 0x3FF0000000000000
dq 0x4000000000000000
dq 0x3FF0000000000000
dq 0x4008000000000000
dq 0x3FF0000000000000
dq 0x4008000000000000
dq 0x0000000000000000
dq 0x7FF8000000000000
dq 0x0000000000000000
dq 0x7FF8000000000000
dq 0x7FF8000000000000
dq 0x0000000000000000
dq 0x7FF8000000000000
dq 0x0000000000000000
+76
View File
@@ -0,0 +1,76 @@
%ifdef CONFIG
{
"HostFeatures": ["AVX"],
"RegData": {
"XMM2": ["0xFFFFFFFFFFFFFFFF", "0x0000000000000000", "0xFFFFFFFFFFFFFFFF", "0x0000000000000000"],
"XMM3": ["0x0000000000000000", "0xFFFFFFFFFFFFFFFF", "0x0000000000000000", "0xFFFFFFFFFFFFFFFF"],
"XMM4": ["0xFFFFFFFFFFFFFFFF", "0xFFFFFFFFFFFFFFFF", "0xFFFFFFFFFFFFFFFF", "0xFFFFFFFFFFFFFFFF"],
"XMM5": ["0x0000000000000000", "0xFFFFFFFFFFFFFFFF", "0x0000000000000000", "0xFFFFFFFFFFFFFFFF"],
"XMM6": ["0xFFFFFFFFFFFFFFFF", "0x0000000000000000", "0xFFFFFFFFFFFFFFFF", "0x0000000000000000"],
"XMM7": ["0x0000000000000000", "0x0000000000000000", "0x0000000000000000", "0x0000000000000000"],
"XMM10": ["0xFFFFFFFFFFFFFFFF", "0xFFFFFFFFFFFFFFFF", "0xFFFFFFFFFFFFFFFF", "0xFFFFFFFFFFFFFFFF"],
"XMM11": ["0x0000000000000000", "0x0000000000000000", "0x0000000000000000", "0x0000000000000000"],
"XMM12": ["0x0000000000000000", "0xFFFFFFFFFFFFFFFF", "0x0000000000000000", "0xFFFFFFFFFFFFFFFF"],
"XMM13": ["0xFFFFFFFFFFFFFFFF", "0x0000000000000000", "0xFFFFFFFFFFFFFFFF", "0x0000000000000000"]
},
"MemoryRegions": {
"0x100000000": "4096"
}
}
%endif
lea rdx, [rel .data]
vmovapd ymm0, [rdx + 32 * 0]
vmovapd ymm1, [rdx + 32 * 1]
vcmppd ymm2, ymm0, ymm1, 0x00 ; EQ
vcmppd ymm3, ymm0, ymm1, 0x01 ; LT
vcmppd ymm4, ymm0, ymm1, 0x02 ; LTE
vcmppd ymm5, ymm0, ymm1, 0x04 ; NEQ
vcmppd ymm6, ymm0, ymm1, 0x05 ; NLT
vcmppd ymm7, ymm0, ymm1, 0x06 ; NLTE
; Unordered and Ordered tests need to be special cased
vmovapd ymm8, [rdx + 32 * 2]
vmovapd ymm9, [rdx + 32 * 3]
; Unordered will return true when either input is nan
; [0.0, nan] unord [nan, 0.0] = [1, 1]
vcmppd ymm10, ymm8, ymm9, 0x03 ; Unordered
; Ordered will return true when both inputs are NOT nan
; [0.0, nan] ord [nan, 0.0] = [0, 0]
vcmppd ymm11, ymm8, ymm9, 0x07 ; Ordered
; Ordered will return true when both inputs are NOT nan
; [nan, 0.0] ord [nan, 0.0] = [0, 1]
vcmppd ymm12, ymm9, ymm9, 0x07 ; Ordered
; Ordered will return true when both inputs are NOT nan
; [0.0, nan] ord [0.0, nan] = [1, 0]
vcmppd ymm13, ymm8, ymm8, 0x07 ; Ordered
hlt
align 32
.data:
dq 0x3FF0000000000000
dq 0x4000000000000000
dq 0x3FF0000000000000
dq 0x4000000000000000
dq 0x3FF0000000000000
dq 0x4008000000000000
dq 0x3FF0000000000000
dq 0x4008000000000000
dq 0x0000000000000000
dq 0x7FF8000000000000
dq 0x0000000000000000
dq 0x7FF8000000000000
dq 0x7FF8000000000000
dq 0x0000000000000000
dq 0x7FF8000000000000
dq 0x0000000000000000
+66
View File
@@ -0,0 +1,66 @@
%ifdef CONFIG
{
"HostFeatures": ["AVX"],
"RegData": {
"XMM2": ["0xFFFFFFFFFFFFFFFF", "0x0000000000000000", "0x0000000000000000", "0x0000000000000000"],
"XMM3": ["0x0000000000000000", "0xFFFFFFFF00000000", "0x0000000000000000", "0x0000000000000000"],
"XMM4": ["0xFFFFFFFFFFFFFFFF", "0xFFFFFFFF00000000", "0x0000000000000000", "0x0000000000000000"],
"XMM5": ["0x0000000000000000", "0xFFFFFFFFFFFFFFFF", "0x0000000000000000", "0x0000000000000000"],
"XMM6": ["0xFFFFFFFFFFFFFFFF", "0x00000000FFFFFFFF", "0x0000000000000000", "0x0000000000000000"],
"XMM7": ["0x0000000000000000", "0x00000000FFFFFFFF", "0x0000000000000000", "0x0000000000000000"],
"XMM10": ["0xFFFFFFFF00000000", "0xFFFFFFFFFFFFFFFF", "0x0000000000000000", "0x0000000000000000"],
"XMM11": ["0x00000000FFFFFFFF", "0x0000000000000000", "0x0000000000000000", "0x0000000000000000"]
},
"MemoryRegions": {
"0x100000000": "4096"
}
}
%endif
lea rdx, [rel .data]
vmovapd ymm0, [rdx + 32 * 0]
vmovapd ymm1, [rdx + 32 * 1]
vcmpps xmm2, xmm0, xmm1, 0x00 ; EQ
vcmpps xmm3, xmm0, xmm1, 0x01 ; LT
vcmpps xmm4, xmm0, xmm1, 0x02 ; LTE
vcmpps xmm5, xmm0, xmm1, 0x04 ; NEQ
vcmpps xmm6, xmm0, xmm1, 0x05 ; NLT
vcmpps xmm7, xmm0, xmm1, 0x06 ; NLTE
; Unordered and Ordered tests need to be special cased
vmovapd ymm8, [rdx + 32 * 2]
vmovapd ymm9, [rdx + 32 * 3]
; Unordered will return true when either input is nan
; [0.0, 0.0, nan, nan] unord [0.0, nan, 0.0, nan] = [0, 1, 1, 1]
vcmpps xmm10, xmm8, xmm9, 0x03 ; Unordered
; Ordered will return true when both inputs are NOT nan
; [0.0, 0.0, nan, nan] ord [0.0, nan, 0.0, nan] = [1, 0, 0, 0]
vcmpps xmm11, xmm8, xmm9, 0x07 ; Ordered
hlt
align 32
.data:
dq 0x3F80000040000000
dq 0x4000000040800000
dq 0x3F80000040000000
dq 0x4000000040800000
dq 0x3F80000040000000
dq 0x40A000003F800000
dq 0x3F80000040000000
dq 0x40A000003F800000
dq 0x0000000000000000
dq 0x7FC000007FC00000
dq 0x0000000000000000
dq 0x7FC000007FC00000
dq 0x7FC0000000000000
dq 0x7FC0000000000000
dq 0x7FC0000000000000
dq 0x7FC0000000000000
+66
View File
@@ -0,0 +1,66 @@
%ifdef CONFIG
{
"HostFeatures": ["AVX"],
"RegData": {
"XMM2": ["0xFFFFFFFFFFFFFFFF", "0x0000000000000000", "0xFFFFFFFFFFFFFFFF", "0x0000000000000000"],
"XMM3": ["0x0000000000000000", "0xFFFFFFFF00000000", "0x0000000000000000", "0xFFFFFFFF00000000"],
"XMM4": ["0xFFFFFFFFFFFFFFFF", "0xFFFFFFFF00000000", "0xFFFFFFFFFFFFFFFF", "0xFFFFFFFF00000000"],
"XMM5": ["0x0000000000000000", "0xFFFFFFFFFFFFFFFF", "0x0000000000000000", "0xFFFFFFFFFFFFFFFF"],
"XMM6": ["0xFFFFFFFFFFFFFFFF", "0x00000000FFFFFFFF", "0xFFFFFFFFFFFFFFFF", "0x00000000FFFFFFFF"],
"XMM7": ["0x0000000000000000", "0x00000000FFFFFFFF", "0x0000000000000000", "0x00000000FFFFFFFF"],
"XMM10": ["0xFFFFFFFF00000000", "0xFFFFFFFFFFFFFFFF", "0xFFFFFFFF00000000", "0xFFFFFFFFFFFFFFFF"],
"XMM11": ["0x00000000FFFFFFFF", "0x0000000000000000", "0x00000000FFFFFFFF", "0x0000000000000000"]
},
"MemoryRegions": {
"0x100000000": "4096"
}
}
%endif
lea rdx, [rel .data]
vmovapd ymm0, [rdx + 32 * 0]
vmovapd ymm1, [rdx + 32 * 1]
vcmpps ymm2, ymm0, ymm1, 0x00 ; EQ
vcmpps ymm3, ymm0, ymm1, 0x01 ; LT
vcmpps ymm4, ymm0, ymm1, 0x02 ; LTE
vcmpps ymm5, ymm0, ymm1, 0x04 ; NEQ
vcmpps ymm6, ymm0, ymm1, 0x05 ; NLT
vcmpps ymm7, ymm0, ymm1, 0x06 ; NLTE
; Unordered and Ordered tests need to be special cased
vmovapd ymm8, [rdx + 32 * 2]
vmovapd ymm9, [rdx + 32 * 3]
; Unordered will return true when either input is nan
; [0.0, 0.0, nan, nan] unord [0.0, nan, 0.0, nan] = [0, 1, 1, 1]
vcmpps ymm10, ymm8, ymm9, 0x03 ; Unordered
; Ordered will return true when both inputs are NOT nan
; [0.0, 0.0, nan, nan] ord [0.0, nan, 0.0, nan] = [1, 0, 0, 0]
vcmpps ymm11, ymm8, ymm9, 0x07 ; Ordered
hlt
align 32
.data:
dq 0x3F80000040000000
dq 0x4000000040800000
dq 0x3F80000040000000
dq 0x4000000040800000
dq 0x3F80000040000000
dq 0x40A000003F800000
dq 0x3F80000040000000
dq 0x40A000003F800000
dq 0x0000000000000000
dq 0x7FC000007FC00000
dq 0x0000000000000000
dq 0x7FC000007FC00000
dq 0x7FC0000000000000
dq 0x7FC0000000000000
dq 0x7FC0000000000000
dq 0x7FC0000000000000
+76
View File
@@ -0,0 +1,76 @@
%ifdef CONFIG
{
"HostFeatures": ["AVX"],
"RegData": {
"XMM2": ["0xFFFFFFFFFFFFFFFF", "0x4000000000000000", "0x0000000000000000", "0x0000000000000000"],
"XMM3": ["0x0000000000000000", "0x4000000000000000", "0x0000000000000000", "0x0000000000000000"],
"XMM4": ["0xFFFFFFFFFFFFFFFF", "0x4000000000000000", "0x0000000000000000", "0x0000000000000000"],
"XMM5": ["0x0000000000000000", "0x4000000000000000", "0x0000000000000000", "0x0000000000000000"],
"XMM6": ["0xFFFFFFFFFFFFFFFF", "0x4000000000000000", "0x0000000000000000", "0x0000000000000000"],
"XMM7": ["0x0000000000000000", "0x4000000000000000", "0x0000000000000000", "0x0000000000000000"],
"XMM10": ["0xFFFFFFFFFFFFFFFF", "0x7FF8000000000000", "0x0000000000000000", "0x0000000000000000"],
"XMM11": ["0x0000000000000000", "0x7FF8000000000000", "0x0000000000000000", "0x0000000000000000"],
"XMM12": ["0x0000000000000000", "0x0000000000000000", "0x0000000000000000", "0x0000000000000000"],
"XMM13": ["0xFFFFFFFFFFFFFFFF", "0x7FF8000000000000", "0x0000000000000000", "0x0000000000000000"]
},
"MemoryRegions": {
"0x100000000": "4096"
}
}
%endif
lea rdx, [rel .data]
vmovapd ymm0, [rdx + 32 * 0]
vmovapd ymm1, [rdx + 32 * 1]
vcmpsd xmm2, xmm0, xmm1, 0x00 ; EQ
vcmpsd xmm3, xmm0, xmm1, 0x01 ; LT
vcmpsd xmm4, xmm0, xmm1, 0x02 ; LTE
vcmpsd xmm5, xmm0, xmm1, 0x04 ; NEQ
vcmpsd xmm6, xmm0, xmm1, 0x05 ; NLT
vcmpsd xmm7, xmm0, xmm1, 0x06 ; NLTE
; Unordered and Ordered tests need to be special cased
vmovapd ymm8, [rdx + 32 * 2]
vmovapd ymm9, [rdx + 32 * 3]
; Unordered will return true when either input is nan
; [0.0, nan] unord [nan, 0.0] = [1, 1]
vcmpsd xmm10, xmm8, xmm9, 0x03 ; Unordered
; Ordered will return true when both inputs are NOT nan
; [0.0, nan] ord [nan, 0.0] = [0, 0]
vcmpsd xmm11, xmm8, xmm9, 0x07 ; Ordered
; Ordered will return true when both inputs are NOT nan
; [nan, 0.0] ord [nan, 0.0] = [0, 1]
vcmpsd xmm12, xmm9, xmm8, 0x07 ; Ordered
; Ordered will return true when both inputs are NOT nan
; [0.0, nan] ord [0.0, nan] = [1, 0]
vcmpsd xmm13, xmm8, xmm8, 0x07 ; Ordered
hlt
align 32
.data:
dq 0x3FF0000000000000
dq 0x4000000000000000
dq 0x3FF0000000000000
dq 0x4000000000000000
dq 0x3FF0000000000000
dq 0x4008000000000000
dq 0x3FF0000000000000
dq 0x4008000000000000
dq 0x0000000000000000
dq 0x7FF8000000000000
dq 0x0000000000000000
dq 0x7FF8000000000000
dq 0x7FF8000000000000
dq 0x0000000000000000
dq 0x7FF8000000000000
dq 0x0000000000000000
+66
View File
@@ -0,0 +1,66 @@
%ifdef CONFIG
{
"HostFeatures": ["AVX"],
"RegData": {
"XMM2": ["0x51525354FFFFFFFF", "0x5152535440000000", "0x0000000000000000", "0x0000000000000000"],
"XMM3": ["0x5152535400000000", "0x5152535440000000", "0x0000000000000000", "0x0000000000000000"],
"XMM4": ["0x51525354FFFFFFFF", "0x5152535440000000", "0x0000000000000000", "0x0000000000000000"],
"XMM5": ["0x5152535400000000", "0x5152535440000000", "0x0000000000000000", "0x0000000000000000"],
"XMM6": ["0x51525354FFFFFFFF", "0x5152535440000000", "0x0000000000000000", "0x0000000000000000"],
"XMM7": ["0x5152535400000000", "0x5152535440000000", "0x0000000000000000", "0x0000000000000000"],
"XMM10": ["0x0000000000000000", "0x7FC000007FC00000", "0x0000000000000000", "0x0000000000000000"],
"XMM11": ["0x00000000FFFFFFFF", "0x7FC000007FC00000", "0x0000000000000000", "0x0000000000000000"]
},
"MemoryRegions": {
"0x100000000": "4096"
}
}
%endif
lea rdx, [rel .data]
vmovapd ymm0, [rdx + 32 * 0]
vmovapd ymm1, [rdx + 32 * 1]
vcmpss xmm2, xmm0, xmm1, 0x00 ; EQ
vcmpss xmm3, xmm0, xmm1, 0x01 ; LT
vcmpss xmm4, xmm0, xmm1, 0x02 ; LTE
vcmpss xmm5, xmm0, xmm1, 0x04 ; NEQ
vcmpss xmm6, xmm0, xmm1, 0x05 ; NLT
vcmpss xmm7, xmm0, xmm1, 0x06 ; NLTE
; Unordered and Ordered tests need to be special cased
vmovapd ymm8, [rdx + 32 * 2]
vmovapd ymm9, [rdx + 32 * 3]
; Unordered will return true when either input is nan
; [0.0, 0.0, nan, nan] unord [0.0, nan, 0.0, nan] = [0, 1, 1, 1]
vcmpss xmm10, xmm8, xmm9, 0x03 ; Unordered
; Ordered will return true when both inputs are NOT nan
; [0.0, 0.0, nan, nan] ord [0.0, nan, 0.0, nan] = [1, 0, 0, 0]
vcmpss xmm11, xmm8, xmm9, 0x07 ; Ordered
hlt
align 32
.data:
dq 0x515253543F800000
dq 0x5152535440000000
dq 0x515253543F800000
dq 0x5152535440000000
dq 0x515253543F800000
dq 0x5152535440800000
dq 0x515253543F800000
dq 0x5152535440800000
dq 0x0000000000000000
dq 0x7FC000007FC00000
dq 0x0000000000000000
dq 0x7FC000007FC00000
dq 0x7FC0000000000000
dq 0x7FC0000000000000
dq 0x7FC0000000000000
dq 0x7FC0000000000000
+61
View File
@@ -0,0 +1,61 @@
%ifdef CONFIG
{
"HostFeatures": ["AVX"],
"RegData": {
"RAX": "0x4700",
"RBX": "0x0300"
},
"MemoryRegions": {
"0x100000000": "4096"
}
}
%endif
lea rdx, [rel .data]
vmovaps xmm0, [rdx + 16 * 0]
vcomisd xmm0, [rdx + 16 * 1] ; 1.0 <comp> 4.0
; 0: CF - 00000001
; 1: - 00000010
; 2: PF - 00000100
; 3: 0 - 00000000
; 4: AF - 00000000 <- 0
; 5: 0 - 00000000
; 6: ZF - 00000000
; 7: SF - 00000000 <- 0
; ================
; 00000111
; OF: LAHF doesn't load - 0
mov rax, 0
lahf
mov rbx, rax
vcomisd xmm0, [rdx + 16 * 2] ; 1.0 <comp> NaN
; 0: CF - 00000001
; 1: - 00000010
; 2: PF - 00000100
; 3: 0 - 00000000
; 4: AF - 00000000 <- 0
; 5: 0 - 00000000
; 6: ZF - 01000000
; 7: SF - 00000000 <- 0
; ================
; 01000111
; OF: LAHF doesn't load - 0
mov rax, 0
lahf
hlt
align 16
.data:
dq 0x3FF0000000000000
dq 0x4000000000000000
dq 0x4010000000000000
dq 0x4010000000000000
dq 0x7FF8000000000000
dq 0x4010000000000000
+61
View File
@@ -0,0 +1,61 @@
%ifdef CONFIG
{
"HostFeatures": ["AVX"],
"RegData": {
"RAX": "0x4700",
"RBX": "0x0300"
},
"MemoryRegions": {
"0x100000000": "4096"
}
}
%endif
lea rdx, [rel .data]
vmovaps xmm0, [rdx + 16 * 0]
vcomiss xmm0, [rdx + 16 * 1]
; 0: CF - 00000001
; 1: - 00000010
; 2: PF - 00000000
; 3: 0 - 00000000
; 4: AF - 00000000 <- 0
; 5: 0 - 00000000
; 6: ZF - 00000000
; 7: SF - 00000000 <- 0
; ================
; 00000011
; OF: LAHF doesn't load - 0
mov rax, 0
lahf
mov rbx, rax
vcomiss xmm0, [rdx + 16 * 2]
; 0: CF - 00000001
; 1: - 00000010
; 2: PF - 00000100
; 3: 0 - 00000000
; 4: AF - 00000000 <- 0
; 5: 0 - 00000000
; 6: ZF - 01000000
; 7: SF - 00000000 <- 0
; ================
; 01000111
; OF: LAHF doesn't load - 0
mov rax, 0
lahf
hlt
align 16
.data:
dq 0x515253543F800000
dq 0x5152535440000000
dq 0x5152535440800000
dq 0x5152535440800000
dq 0x515253547FC00000
dq 0x5152535440800000
+38
View File
@@ -0,0 +1,38 @@
%ifdef CONFIG
{
"HostFeatures": ["AVX"],
"RegData": {
"XMM0": ["0x3FF0000000000000", "0x4000000000000000", "0x0000000000000000", "0x0000000000000000"],
"XMM1": ["0x4008000000000000", "0x4010000000000000", "0x0000000000000000", "0x0000000000000000"],
"XMM2": ["0x3FF0000000000000", "0x4000000000000000", "0x4008000000000000", "0x4010000000000000"],
"XMM3": ["0x4008000000000000", "0x4010000000000000", "0x3FF0000000000000", "0x4000000000000000"]
},
"MemoryRegions": {
"0x100000000": "4096"
}
}
%endif
lea rdx, [rel .data]
vmovapd ymm2, [rdx + 32]
vcvtdq2pd xmm0, xmm2
vcvtdq2pd xmm1, [rdx + 40]
vcvtdq2pd ymm2, xmm2
vcvtdq2pd ymm3, [rdx + 40]
hlt
align 32
.data:
dq 0x4142434445464748
dq 0x5152535455565758
dq 0x4142434445464748
dq 0x5152535455565758
dq 0x0000000200000001
dq 0x0000000400000003
dq 0x0000000200000001
dq 0x0000000400000003
+43
View File
@@ -0,0 +1,43 @@
%ifdef CONFIG
{
"HostFeatures": ["AVX"],
"RegData": {
"XMM0": ["0xBF8000003F800000", "0x437F000000000000", "0x0000000000000000", "0x0000000000000000"],
"XMM2": ["0xBF8000003F800000", "0x437F000000000000", "0x0000000000000000", "0x0000000000000000"],
"XMM3": ["0xBF8000003F800000", "0x437F000000000000", "0xBF8000003F800000", "0x437F000000000000"],
"XMM4": ["0xBF8000003F800000", "0x437F000000000000", "0xBF8000003F800000", "0x437F000000000000"],
"XMM6": ["0x4E8000004E7E0000", "0x4E8100004E808000", "0x0000000000000000", "0x0000000000000000"],
"XMM7": ["0x4E8000004E7E0000", "0x4E8100004E808000", "0x4E8000004E7E0000", "0x4E8100004E808000"]
},
"MemoryRegions": {
"0x100000000": "4096"
}
}
%endif
lea rdx, [rel .data]
vmovapd ymm1, [rdx + 32]
vmovapd ymm5, [rdx]
vcvtdq2ps xmm2, [rdx + 32]
vcvtdq2ps ymm3, [rdx + 32]
vcvtdq2ps xmm6, [rdx]
vcvtdq2ps ymm7, [rdx]
vcvtdq2ps xmm0, xmm1
vcvtdq2ps ymm4, ymm1
hlt
align 32
.data:
dq 0x400000003F800000 ; 2, 1
dq 0x4080000040400000 ; 4, 3
dq 0x400000003F800000 ; 2, 1
dq 0x4080000040400000 ; 4, 3
dq 0xFFFFFFFF00000001 ; -1, 1
dq 0x000000FF00000000 ; 255, 0
dq 0xFFFFFFFF00000001 ; -1, 1
dq 0x000000FF00000000 ; 255, 0
Loaded 100 of 272 files, more files were not shown because too many files have changed in this diff. Show more