Compare commits

...
976 Commits
Author SHA1 Message Date
Ryan Houdek cac3767c20 Docs: Update for release FEX-2410 2024-10-07 11:14:53 -07:00
Ryan Houdek e82d2c70c3 Merge pull request #4095 from alyssarosenzweig/opt/jit-time
speed up the JIT
2024-10-04 09:00:03 -07:00
LC 9716fc73d3 Merge pull request #4099 from Sonicadvance1/v6.11
LinuxSyscalls: Update max reported kernel version to 6.11
2024-10-04 11:58:20 -04:00
LC aaa8eef9f1 Merge pull request #4098 from Sonicadvance1/linux_v6.11_syscalls
LinuxEmulation: Update syscalls for v6.11
2024-10-04 11:57:52 -04:00
LC c036868938 Merge pull request #4097 from Sonicadvance1/linux_v6.11_drm
LinuxEmulation: Update DRM for v6.11
2024-10-04 11:57:21 -04:00
LC 2726f35a10 Merge pull request #4102 from Sonicadvance1/fix_vector_element_loadstore
OpcodeDispatcher: Fixes segment prefixing on vector element loadstores
2024-10-04 11:56:15 -04:00
Ryan Houdek 407dc5d4dd unittests: Adds TLS vector element loadstore test
Hits all the instructions that would have previously crashed.
2024-10-03 20:53:28 -07:00
Ryan Houdek 7dda75646d OpcodeDispatcher: Fixes segment prefixing on vector element loadstores
These had completely forgotten to load segment prefixes on the element
loadstore operations. This had caused a crash on the Linux version of
"Halls of Tormet" (steamid 2218750).

This game has TLS vector data which it was loading with a movhps and hit
this path.
2024-10-03 20:51:37 -07:00
Ryan Houdek b89f5b8a03 Merge pull request #4101 from Sonicadvance1/fprem_fixes
Fixes fprem
2024-10-03 15:49:22 -07:00
Ryan Houdek 16869d8954 unittests: fprem tests work since 688af49485 2024-10-03 14:03:51 -07:00
Ryan Houdek e353ae8408 unittests: Adds x87 fprem test
Exposes a bug in FEX's x87 FPREM implementation.
2024-10-03 14:03:51 -07:00
Alyssa Rosenzweig aa3c963df1 SoftFloat: fix FPREM
fixes incorrect signs on FPREM. in turn should fix end-to-end failures logging
into Steam.

Thank you to Sergio Lopez for tracking down the JavaScript fail, and Ryan for
finding the bug.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-03 13:56:14 -07:00
Ryan Houdek 0c8cfa7bb2 IoctlEmulation: Update AssignDeviceTypeToFD
Adds known good drivers to remove a log.
2024-10-02 19:29:45 -07:00
Ryan Houdek dd837fa693 LinuxSyscalls: Update max reported kernel version to 6.11
Investigated the uapi differences and nothing terrifying leapt out that
we haven't already implemented.
2024-10-02 15:36:47 -07:00
Ryan Houdek eeff198ff1 LinuxEmulation: Update syscalls for v6.11
Only thing added was uretprobe on x86-64. We can't support this so just
return ENOSYS.
2024-10-02 14:40:11 -07:00
Ryan Houdek 0cd6371c1b IoctlEmulation: Add panthor_drm 2024-10-02 14:27:24 -07:00
Ryan Houdek 5aff16f72b IoctlEmulation: Update xe_drm 2024-10-02 14:26:38 -07:00
Ryan Houdek 2c7896bce4 IoctlEmulation: Update v3d_drm 2024-10-02 14:26:30 -07:00
Ryan Houdek 6dd72a09e5 Scripts: Fixes newer clang-python
It provided this in the newer clang-python
2024-10-02 14:25:59 -07:00
Ryan Houdek 4e6a4d9b69 Linux: Update drm-headers to v6.11 2024-10-02 13:43:43 -07:00
Alyssa Rosenzweig 44484a7b05 ConstProp: drop non-loadbearing opts
every pattern costs us JIT time, but not every pattern is doing anything.
especially with the flag rework of 2023-2024, a lot of patterns just don't make
sense anymore. constant folding in particular isn't too useful now.

only instcountci change is AAD which i'm contractually forbidden from caring
about.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-02 14:01:32 -04:00
Alyssa Rosenzweig 6ac7ba388f ConstProp: rm leftover
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-02 13:38:48 -04:00
Alyssa Rosenzweig 36d0a67070 ConstProp: don't pool with a hash table
hash tables have high overhead! but individual blocks are small, so the
quadratic search is _much_ faster in practice. knocks 8% off node.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-02 13:38:48 -04:00
Alyssa Rosenzweig 0650dd1992 ConstProp: defer initialization
for blocks that don't need pooling. hopefully that's most of them.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-02 13:38:48 -04:00
Alyssa Rosenzweig e2d58809ed OpcodeDispatcher: rm dead
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-02 13:38:48 -04:00
Alyssa Rosenzweig 967a74cda9 OpcodeDispatcher: manually inline constants
less work for constprop.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-02 13:38:48 -04:00
Alyssa Rosenzweig 6cc6181261 OpcodeDispatcher/Flags: dont emit zero
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-02 13:38:48 -04:00
Alyssa Rosenzweig f175b525f4 OpcodeDispatcher/Flags: inline constants
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-02 13:38:48 -04:00
Alyssa Rosenzweig 319f1e66cb RegisterAllocationPass: eliminate a silly sxtb
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-02 13:38:48 -04:00
Alyssa Rosenzweig d16969bdce RegisterAllocationPass: simplify
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-02 13:38:48 -04:00
Alyssa Rosenzweig c17858bfeb RegisterAllocationPass: don't lookup sources
lot of silly chasing. not needed.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-02 13:38:48 -04:00
Alyssa Rosenzweig ec24fc3d5d RegisterAllocationPass: infer AnyRemapped
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-02 13:38:48 -04:00
Alyssa Rosenzweig 8819fa88d5 RegisterAllocationPass: avoid allocating remaps
shaves 6ms off.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-02 13:38:48 -04:00
Alyssa Rosenzweig 5e9c2110db Merge pull request #4096 from pmatos/x87Cleanup
x87 small cleanups; NFC
2024-10-02 12:43:13 -04:00
Alyssa Rosenzweig 5aaa18e7a2 RegisterAllocationPass: remove Class->Allocated
not needed anymore.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-02 12:41:37 -04:00
Alyssa Rosenzweig 8a551b9e64 RegisterAllocationPass: rm leftover comment
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-02 12:41:37 -04:00
Alyssa Rosenzweig aa548bd19c RegisterAllocationPass: track if we need a remap
should be faster on average.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-02 12:41:37 -04:00
Alyssa Rosenzweig 9565f16d84 RegisterAllocationPass: simplify
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-02 12:41:37 -04:00
Paulo Matos eb76dbdf4d x87 small cleanups; NFC 2024-10-02 18:27:51 +02:00
Ryan Houdek 967c04e252 Merge pull request #4088 from bylaws/winesys
ARM64EC: Dynamically determine syscall numbers under wine.
2024-10-01 09:07:35 -07:00
Ryan Houdek 49de1fac59 Merge pull request #4028 from neobrain/refactor_jemallocless_tools
Build host tools without jemalloc
2024-09-30 11:59:45 -07:00
Ryan Houdek 3c205eb35e Merge pull request #4094 from Sonicadvance1/strict_unittest_args
unittests: Be more strict with -- separator
2024-09-30 11:38:30 -07:00
Tony Wasserka 682b8ef705 Build host tools without jemalloc
Jemalloc blocks use on platforms with 16K pages.
2024-09-30 20:38:24 +02:00
Tony Wasserka 8c47a40625 Update jemalloc 2024-09-30 20:35:46 +02:00
Ryan Houdek ee5c6af868 unittests: Be more strict with -- separator 2024-09-30 11:30:56 -07:00
Ryan Houdek 1670c89b5e Merge pull request #4090 from Sonicadvance1/update_vulkan_headers
Thunks/Vulkan: Updates thunks to v1.3.296
2024-09-30 09:54:15 -07:00
Ryan Houdek e9d34c7ded Merge pull request #4089 from bylaws/wineavx
ARM64EC: AVX register save/restore support under wine
2024-09-30 09:46:33 -07:00
Ryan Houdek c2f2c58367 Merge pull request #4060 from bylaws/ucrt
Windows: Switch supported MinGW CRT to UCRT
2024-09-30 09:31:27 -07:00
Billy Laws 38acd9e18c ARM64EC: Dynamically determine syscall numbers under wine
Takes advantage of the consistent alphabetical syscall ordering that
wine follows.
2024-09-29 19:42:49 +00:00
Billy Laws 21604708ca Windows: Commonise the linker loadcfg and support CFGuard toolchains 2024-09-29 19:42:13 +00:00
Billy Laws 5fec6faeb6 Windows: Switch supported MinGW CRT to UCRT
MSVCRT is deprecated and the reasons FEX used it no longer apply, so
support UCRT and UCRT only.
2024-09-29 19:42:13 +00:00
Billy Laws b659701cef ARM64EC: AVX register save/restore support under wine
Requires corresponding a wine patch to function:
https://github.com/bylaws/wine/commit/7d411bd74bd03ba90cf202623560733cb229cc7d

Since AVX still works in almost all cases without this, I chose not to
e.g. detect the patch and disable AVX support when it is missing.
2024-09-29 18:59:22 +00:00
LC e902ad5278 Merge pull request #3946 from Sonicadvance1/vdso_getrandom
VDSO: Implements v6.11 vdso getrandom
2024-09-28 15:26:35 -04:00
Ryan Houdek 0f54369f9b VDSO: Implements v6.11 vdso getrandom
This is an interesting vdso implementation because it only exists in
x86-64 with kernel v6.11. For Arm64 the implementation is likely to land
in a couple kernel versions from now.

Some differences with vdso_getrandom versus the regular getrandom
syscall
- Has two additional arguments, opaque_state and opaque_len
- Expects the userspace to mmap/munmap this opaque state structure
- Lets userspace query information about this structure upfront
- If the opaque data structure isn't provided then it falls back to
  regular SYS_getrandom

With the "glibc" implementation, we can tell the interface to allocate a
single page that gets unused (otherwise glibc ends up not using the
interface), and fallback to the regular SYS_getrandom.

In the case of the vdso interface, currently the arguments just get
passed through.

Keeping this as a WIP until the ARM64 kernel patches land and I can
actually test the things. Running the "glibc" path works today with the
selftest, but the vdso path is currently completely untested.
2024-09-28 01:06:56 -07:00
LC f640dcc7a4 Merge pull request #4075 from Sonicadvance1/move_thunkhandler
Thunks: Move to the frontend
2024-09-28 03:26:10 -04:00
Ryan Houdek b59da0e049 Thunks/Vulkan: Updates thunks to v1.3.296
Between this version and our previous version v1.3.278 this adds a few
extensions.
- VK_EXT_device_generated_commands
- VK_EXT_depth_clamp_control
- VK_AMD_anti_lag
- VK_KHR_pipeline_binary

These extensions are almost immediately getting picked up by mesa and
the D3D layers, so we want to support them as soon as possible.

Once again updated as described in #2076. This is starting to take more
time as more features and extensions get added to Vulkan. We probably
want to update the `DefinitionExtract.py` script at some point to
contain all the tertiary data in some json so everything gets generated
automatically. Not doing that today but something to think about.

Fixes #4083
2024-09-27 17:59:31 -07:00
Ryan Houdek 4ae3aef502 Vulkan-Headers: Update to v1.3.296 2024-09-27 17:17:50 -07:00
Ryan Houdek 926fa3c24c Thunks: Just move code around for a minor cleanup
NFC
2024-09-27 15:50:23 -07:00
Ryan Houdek 8c1740f592 Thunks: Remove the temporary TLS variables
From prep commit 511103ee5dcc9474b0b7468c05f61ce10fea4393.
Now that the frontend is setup, we can remove the temporary TLS
variables and use the ThreadObject directly.
2024-09-27 15:50:23 -07:00
Ryan Houdek 611aa0a5b9 Thunks: Wire up TLS handling in the frontend
Because it was moved from the backend to the frontend, re-wire up the
TLS handling which requires going through our syscall handler.
2024-09-27 15:50:21 -07:00
Ryan Houdek 0b8b5108b9 Thunks: Moves AppendThunkDefinitions function to the frontend
Backend doesn't need access to this at all
2024-09-27 15:48:31 -07:00
Ryan Houdek e409a0afec Thunks: Shift namespace name
Moves from FEXCore to FEX::HLE. Also moves the ThunkFunctions that get
exposed to a namespace to make it more obvious that these are
thunkhandlers rather than just static functions.
2024-09-27 15:48:31 -07:00
Ryan Houdek 24211f8523 FEXCore: Move ThunksHandler class to FEXLoader
With as little changes as possible, because this is fairly tricky.
2024-09-27 15:48:31 -07:00
Ryan Houdek 94462e4dd0 Thunks: Temporarily add more TLS variables while moving
Prep commit to give Thunks the data it needs before moving.
2024-09-27 15:48:31 -07:00
Ryan Houdek aa44668a41 FEXCore/Thunks: Moves the implementation of LinkAddressToGuestFunction to ContextImpl
No functional change, just moving code here.
2024-09-27 15:48:31 -07:00
Ryan Houdek 33a675624c FEXCore/Thunks: Remove lock passing in AddCustomIREntrypoint
This was a legacy feature that is no longer necessary every since we
removed the frontend IRLoader.
2024-09-27 15:48:31 -07:00
LC 5745b419d9 Merge pull request #4045 from Sonicadvance1/stop_using_dlopen_for_vdso
VDSOEmulation: Stop using dlopen for VDSO
2024-09-27 18:20:55 -04:00
LC 9100235041 Merge pull request #4077 from Sonicadvance1/frontend_informed_supportsavx
SignalDelegator: Let the frontend inform AVX support
2024-09-27 18:15:13 -04:00
LC 4d62e75d5a Merge pull request #4085 from pmatos/FScaleZero
Fix FScale'ing zero that should return zero
2024-09-27 18:13:07 -04:00
LC b6d3df0e54 Merge pull request #4086 from Sonicadvance1/stop_breaking_x86
Stop installing binfmt_misc on x86
2024-09-27 03:34:41 -04:00
Ryan Houdek d5d1230422 Stop installing binfmt_misc on x86
With the new systemd support, we had forgotten to wrap this check in an
arm64 check. I had done an install and broke my x86 machine.

Add the check back so we stop breaking x86 machines...again. Took me
only about three hours to find the issue this time.
2024-09-26 15:49:48 -07:00
Paulo Matos f35a212f06 Fix FScale'ing zero that should return zero
Added tests will fail without the current patch.
This will properly check if we return the correct sign of zero.
2024-09-26 17:57:39 +02:00
Ryan Houdek f49d82deb3 Merge pull request #4074 from Sonicadvance1/global_rootfs
FEXConfig: Add the ability to watch and configure global rootfs
2024-09-24 03:30:47 -07:00
Ryan Houdek 77b801b8ae FEXConfig: Add the ability to watch and configure global rootfs
Global rootfs doesn't support a named rootfs (only local supports that).
So when it is a global rootfs option just give the full path to the
config path.

Just needs some special handling to ensure the full path is retained for
global paths.

A reimplementation of #4067 that doesn't hardcode paths and routes the
global data path through FEXCore.
2024-09-24 01:17:45 -07:00
Ryan Houdek 163a3fc899 FEXConfig: Convert select over to ppoll usage
Using select initially was a mistake, it has issues with FD limits and
is less efficient due to fd_set. It's recommended to switch select using
over to polling, so do it.
2024-09-24 01:17:45 -07:00
Ryan Houdek 9b627e8743 FEXCore: Add global data directory support
Currently unused as all data still comes from local installs.
2024-09-24 01:17:43 -07:00
LC 9d6865f62d Merge pull request #4073 from Sonicadvance1/constexpr_all_tables
OpcodeDispatcher: Constexpr-ify all the tables possible
2024-09-23 21:40:37 -04:00
LC d547f2b8f2 Merge pull request #4080 from Sonicadvance1/improve_createthread_stack
Linux: Optimize CreateNewThread and HandleNewClone stack usage
2024-09-23 21:28:56 -04:00
Ryan Houdek b5c9eb3463 Merge pull request #4081 from Sonicadvance1/smarter_meta_reload
Config: Be a bit smarter in ReloadMetaLayer
2024-09-23 04:52:25 -07:00
Ryan Houdek 00a3793814 Config: Be a bit smarter in ReloadMetaLayer
Reduces stack usage from ~1024 bytes to ~704 bytes and stops making a
bunch of string copies when they can just be pointers instead.
2024-09-23 04:21:10 -07:00
LC 4f8d7cb89c Merge pull request #4078 from Sonicadvance1/fix_envloader_stackusage
Config: Fixes stack usage in EnvLoader
2024-09-22 19:19:46 -04:00
LC 9310ac59f4 Merge pull request #4076 from Sonicadvance1/remove_block_sampling
FEXCore: Remove BlockSamplingData
2024-09-22 19:18:37 -04:00
Ryan Houdek 9e4de3bfe8 Merge pull request #4079 from alyssarosenzweig/bug/maxss-nan
Fix (v)maxss on non-AFP platforms
2024-09-22 11:45:30 -07:00
Alyssa Rosenzweig 598b99fe58 InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-22 14:06:27 -04:00
Alyssa Rosenzweig 9dda76ebe6 unittests: add min/max tests
we had a bug with max of nans. since x86 behaviour is weird, add a unit test to
try all the weird things. this fails on main without AFP

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-22 14:06:27 -04:00
Ryan Houdek f5def7ae1c Linux: Also optimize HandleNewClone stack usage
Drops from ~1392 bytes of stack usage to ~80 bytes
2024-09-22 10:51:25 -07:00
Ryan Houdek 58bcf691d6 Linux: Optimize CreateNewThread stack usage
We were creating a copy of the FEXCore::Core::CPUState object when we
didn't need to. We can pass the host thread's CPUState frame through to
the creation handlers since it's read-only (so modify it to be const).

We then just move the RAX and RSP setting to /after/ the CreateThread
handling instead of before.
This reduces stack usage from ~1392 bytes to ~80 bytes.
2024-09-22 10:45:21 -07:00
Alyssa Rosenzweig 33186b803d JIT: fix scalar fmax on non-AFP platforms
NaN and signed zero handling was busted.  Pretty nasty.

Fixes the following Vulkan CTS test run under emulation without thunking with a clang-built Mesa:

  dEQP-VK.spirv_assembly.instruction.graphics.float_controls.fp32.generated_args.denorm_nmax_nan_flush_to_zero_vert

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-22 12:14:21 -04:00
Alyssa Rosenzweig bb1d7d0750 JIT: rm broken dead code
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-22 12:14:21 -04:00
Ryan Houdek 4897c1e80c Config: Fixes stack usage in EnvLoader
EnvLoader::Load was generating strings on the stack and then copying
them to the FEXCore::Config state. FEXCore::Config::Set supports
string_views directly which will emplace in to the map directly.

This improves this functions stack usage from ~4464 bytes to ~288 bytes.
2024-09-22 08:45:02 -07:00
Ryan Houdek e7605c94dd SignalDelegator: Let the frontend inform AVX support
Since the frontend has changed to informing the backend if AVX is
supported, there is no reason to feed that configuration back in to
SignalDelegator from the backend.

Instead inform the SignalDelegator directly in the frontend instead of
this now weird round-about path.
2024-09-22 08:23:06 -07:00
Ryan Houdek ab5a10374b FEXCore: Remove BlockSamplingData
This was a legacy interface that has long lost its implementation and
has long lost its use. This interface was similar to an older
Dolphin-Emu interface that was useful in that project, but isn't really
useful in a world with hundreds of threads and processes running at the
same time.
2024-09-22 08:04:39 -07:00
Ryan Houdek dc2a26582f FEXCore/OpcodeDispatcher: Fixes missing static on some tables
This was causing egregious stack usage in these two functions
2024-09-16 18:52:55 -07:00
Ryan Houdek e6512fbe4d X86Tables: Remove function wraps from VEX tables 2024-09-16 18:52:55 -07:00
Ryan Houdek dbca441573 X86Tables: Remove function wraps from DDD table 2024-09-16 18:52:55 -07:00
Ryan Houdek 31adc953b7 X86Tables: Remove function wraps from H0F38/H0F3A table 2024-09-16 18:52:55 -07:00
Ryan Houdek 0dd7f5e2f7 X86Tables: Remove function wraps from SecondaryModRM table 2024-09-16 18:52:54 -07:00
Ryan Houdek 36556c4705 X86Tables: Remove function wraps from Group tables 2024-09-16 18:52:54 -07:00
Ryan Houdek e29fac3f25 X86Tables: Remove function wraps from Secondary tables 2024-09-16 18:52:54 -07:00
Ryan Houdek c984bdb42a OpcodeDispatcher: Remove previous template instantiantions and use Bind 2024-09-16 18:52:54 -07:00
Ryan Houdek 2645b374a7 OpcodeDispatcher: Deduplicate InstallToTable helper 2024-09-16 18:52:54 -07:00
Ryan Houdek 10192eebe5 X86Tables: Remove EVEX Table
Still track the telemetry if EVEX is attempted, but remove the tables
which we won't be supporting.
2024-09-16 17:40:20 -07:00
Ryan Houdek 0e57cbf5b9 OpcodeDispatcher: Remove EVEX table install
This hasn't been necessary for quite a while as we handle
invalid/unsupported opcodes in the frontend now.
2024-09-16 17:40:20 -07:00
Ryan Houdek c7413d96ad OpcodeDispatcher: Constexpr-ify VEX & VEXGroup tables 2024-09-16 17:40:20 -07:00
Ryan Houdek f3ba8cb33c OpcodeDispatcher: Constexpr-ify DDD tables 2024-09-16 17:40:20 -07:00
Ryan Houdek e714933b10 OpcodeDispatcher: Constexpr-ify H0F3A tables 2024-09-16 17:40:20 -07:00
Ryan Houdek 9a0f83cbf6 OpcodeDispatcher: Constexpr-ify H0F38 tables 2024-09-16 17:40:20 -07:00
Ryan Houdek c4d30e8437 OpcodeDispatcher: Constexpr-ify SecondaryModRM tables 2024-09-16 17:40:20 -07:00
Ryan Houdek 7b30df8a56 OpcodeDispatcher: Constexpr-ify SecondaryGroup tables 2024-09-16 17:40:20 -07:00
Ryan Houdek 86aa459dd1 OpcodeDispatcher: Constexpr-ify PrimaryGroup tables 2024-09-16 17:40:20 -07:00
Ryan Houdek 6c9a47fc71 OpcodeDispatcher: Constexpr-ify Secondary OpSizeMod tables 2024-09-16 17:40:20 -07:00
Ryan Houdek 7a72cf631b OpcodeDispatcher: Constexpr-ify Secondary RepNEMod tables 2024-09-16 17:40:20 -07:00
Ryan Houdek 3d98eef7b8 OpcodeDispatcher: Constexpr-ify Secondary RepMod tables 2024-09-16 17:40:20 -07:00
Ryan Houdek f2011b0b79 OpcodeDispatcher: Constexpr-ify Secondary base tables 2024-09-16 17:40:19 -07:00
Ryan Houdek c0b8a0d9c3 Merge pull request #4066 from Sonicadvance1/remove_imgui
Tools: Delete imgui FEXConfig
2024-09-16 17:25:46 -07:00
Ryan Houdek 5bd0afa307 Merge pull request #4068 from Sonicadvance1/update_cpuid
CPUID: Update to something a little more modern
2024-09-16 06:33:04 -07:00
Ryan Houdek 20fb0da7d9 Merge pull request #4048 from Sonicadvance1/constexpr_more_tables
FEXCore: Convert Base tables over to constexpr
2024-09-16 06:32:23 -07:00
Ryan Houdek 35cb1f710c FEXQConfig->FEXConfig 2024-09-16 02:11:27 -07:00
Ryan Houdek 7c7b99f9c5 FEXConfig: Remove 2024-09-16 02:09:25 -07:00
Ryan Houdek 09879bd962 CPUID: Update to something a little more modern
Previously reported as some old CPU without AVX and SSE4 and other
things.
Start advertising as something more modern that actually shipped with
AVX2 and other features. Should help some modern libraries that do bad
family and model checks rather that CPUID features checks.

Also removes the silly `(ES)` tag from CPU-Z.

Also moves generation in to a constexpr function that can actually range
check these 4-bit and 8-bit values.
2024-09-16 01:42:35 -07:00
Ryan Houdek c8f3fe3788 Merge pull request #4071 from Sonicadvance1/unittests_incorrect_arg
unittests: Fixes incorrect argument
2024-09-15 15:20:54 -07:00
Ryan Houdek 88e3164b55 unittests: Fixes incorrect argument
Turns out our unittests have been passing an incorrect argument for
years.
2024-09-14 23:15:52 -07:00
Ryan Houdek bebd7400c0 Merge pull request #4063 from Sonicadvance1/fallback_if_no_zenity
FEXRootFSFetcher: Fallback to TTY if zenity isn't installed
2024-09-14 16:14:32 -07:00
Ryan Houdek e190d029dc Merge pull request #4064 from Sonicadvance1/remove_binfmt_flag
FEXLoader: Drop the binfmt_misc `I` flag
2024-09-13 12:51:24 -07:00
Ryan Houdek 37a70e2ec6 FEXCore: Convert Base tables over to constexpr
Only doing the single table for review purposes. Once reviewed I will
hammer out the remaining tables.

Similar to #3320, most of the OpcodeDispatcher tables can be consteval
and made to be a compile time constant. This just requires shuffling the
code slightly. The idea is to get almost all of the table setup out of
the `InstallOpcodeHandlers` function and instead only install the
handlers that change based on 32-bit or 64-bit, just like the x86 tables
we also did.
2024-09-13 11:39:09 -07:00
Ryan Houdek 6ff073ac11 FEXRootFSFetcher: convert vector usage over to array and span
Most uses of vector could actually be converted to array and a lot of
argument passing could actually be converted over to spans instead.
2024-09-13 11:29:44 -07:00
Ryan Houdek b6e4c47abc FEXLoader: Drop the binfmt_misc I flag
Upstream has rejected this flag which would have let FEX close the gap
in functionality between binfmt_misc interpreters and PT_INTERP
interpreters for how `/proc/exe` is handled. Since we are unable to
change their opinions, just remove the code from FEX since it's never
going to happen.

Removes a little bit of confusing behaviour in execveat and
filemanagement handling. Leaving us with only the regular confusing
behaviour of trying to track accesses to `/proc/exe` instead.
2024-09-12 19:58:35 -07:00
Ryan Houdek daf8b409f3 FEXRootFSFetcher: Fallback to TTY if zenity isn't installed
If packaged improperly or if someone has just did a source install then
they may not have zenity. Fallback to TTY path if zenity isn't
installed.
2024-09-11 17:08:20 -07:00
Ryan Houdek 0a35029d9a Merge pull request #4062 from Sonicadvance1/fix_fexserver_writing_to_pipe
FEXServer/PipeScanner: Stop writing null to incoming pipes
2024-09-11 13:59:24 -07:00
Ryan Houdek e8b51ac0a4 FEXServer/PipeScanner: Stop writing null to incoming pipes
This was causing problems in a typical use case of piping stdout/stderr
to another application. So if FEXInterpreter was started, with stdout
piped to another application, and FEXServer wasn't running so a new
instance needed to be started. FEXServer would write a null to every
pipe then close it. This includes the stdout that is being piped to the
new application!

This writing was legacy behaviour before FEX was properly listening to
POLLHUP and is no longer necessary. Just remove the writing completely
to fix this issue.

Easiest test case was just `./Bin/FEXInterpreter `which ls` / | xxd | head -n 3`

Returned:
```
$ ./Bin/FEXInterpreter `which ls` / | xxd | head -n 3
00000000: 0000 0000 6269 6e0a 6269 6e2e 7573 722d  ....bin.bin.usr-
00000010: 6973 2d6d 6572 6765 640a 626f 6f74 0a64  is-merged.boot.d
00000020: 6576 0a65 7463 0a68 6f6d 650a 6c69 620a  ev.etc.home.lib.
```

That first null shouldn't be there.
2024-09-11 12:52:57 -07:00
Ryan Houdek f20e626fda Merge pull request #4059 from bylaws/log
Windows: Don't assume the log file was successfully opened
2024-09-11 10:16:14 -07:00
Ryan Houdek 2672e06aa5 Merge pull request #4058 from bylaws/cef
ARM64EC: Patch the call checker before calling any syscall exports
2024-09-11 10:16:02 -07:00
Ryan Houdek 4a2b4be59d Merge pull request #4057 from bylaws/overcommit
Support for emulated overcommit on Windows
2024-09-11 10:15:43 -07:00
Ryan Houdek 05be9445cd Merge pull request #4056 from Sonicadvance1/context_remove_unused_features
FEXCore/Context: Removes unused features
2024-09-11 10:09:24 -07:00
Ryan Houdek 4cb4c0e277 Merge pull request #4055 from Sonicadvance1/remove_header
FEXCore: Remove Context header include when unneeded
2024-09-11 10:09:14 -07:00
Billy Laws 0135e2d78c LookupCache: Use emulated overcommit if necessary
Allocates uncommited memory and calls the frontend callbacks so it can
commit pages on faults as necessary.
2024-09-10 16:08:45 +00:00
Billy Laws 1266a5a5ce Windows: Support emulated memory overcommit 2024-09-10 16:08:45 +00:00
Billy Laws 7255850e9d ARM64EC: Patch the call checker before calling any syscall exports
CEF on Windows patches these before FEX is even loaded, so this needs to
be setup as early as possible using some direct syscalls which skip any
such patches.
2024-09-10 16:05:37 +00:00
Billy Laws 002a8bebd8 Windows: Cleanup exception handling logic 2024-09-10 16:05:02 +00:00
Billy Laws 452c9fd19b AllocatorHooks: Make commiting allocated memory optional
Ignored for linux as it supports overcommit.
2024-09-10 16:05:02 +00:00
Billy Laws 0b0e15c9b4 Windows: Don't assume the log file was successfully opened 2024-09-10 15:58:03 +00:00
Ryan Houdek ae9db336e7 FEXCore/Context: Removes unused features
No functional change here.
- CoreRunningMode enum and variable wasn't used anymore.
   - Code was moved to the frontend
- CustomCPUFactory wasn't used anymore
   - All special signal handling and various features were moved to
     TestHarnessRunner
   - We also don't want to support actual custom CPU cores.
   - TestHarnessRunner just runs as a host runner if compiled on an
     x86-64 device if vixl sim isn't enabled now.
   - Removes the Core config option entirely.
- Moves VDSOPointers struct to the frontend
   - Every use of this lives in the Linux frontend instead now
2024-09-09 18:38:33 -07:00
Ryan Houdek 360ea538b7 Merge pull request #4054 from Sonicadvance1/fix_xxhash
FEXRootFSFetcher/XXHash: Fix double fd close
2024-09-09 18:03:55 -07:00
Ryan Houdek f99900bdc5 Merge pull request #4053 from Sonicadvance1/disable_vixl_compiling
CMake: Disable vixl compiling if not enabled
2024-09-09 18:03:46 -07:00
Ryan Houdek 34f2bce203 FEXCore: Remove Context header include when unneeded
This is one of the most expensive headers in FEX, averaging 1.2 seconds
of compile time per include.
We include this header 31 times around the project. Remove
the five instances where it is unused to help this issue. Next step
would be to make the header lighter if possible.
2024-09-09 17:43:35 -07:00
Ryan Houdek a05d6ae082 FEXRootFSFetcher/XXHash: Check for errors in two locations that were missed 2024-09-09 14:07:23 -07:00
Ryan Houdek 6a4eb434f7 FEXRootFSFetcher/XXHash: Fix double fd close
Since `HadError` is a lambda that handles FD closing this was a double
FD close. Just move the xxhash state freeing in there as well and remove
the FD and hash freeing. Fixes the double free.
2024-09-09 14:00:44 -07:00
Ryan Houdek 0c3eb41f57 CMake: Disable vixl compiling if not enabled
Like how FEXCore disabled vixl linking, make sure we aren't accidentally
compiling and including it when vixl stuff is disabled. Noticed this in
the build logs.
2024-09-09 13:54:35 -07:00
Ryan Houdek e4bfbda008 VDSOEmulation: Stop using dlopen for VDSO
We have been relying on the
`YesIKnowImNotSupposedToUseTheGlibcAllocator` fault avoidance for a long
while now. The plan was to rewrite the symbol fetching in the future to
avoid this since VDSO is kind of special anyway.

That time is now, I have had this VDSO symbol parsing code living in a
different project for a few months now and it is working great.

The basic things here are that the Linux kernel provides the VDSO
mapping base pointer through the auxv value `AT_SYSINFO_EHDR`. We then
need a minimal ELF parser that /only/ parses the dynamic symbol header
(and accompanying string header).

Then it's a simple case of walking the symbol table and recording the
pointers. Confirmed this fetches all the correct symbols (As I've been
using it for a while already I already knew it worked.)

This lets us stop allocating memory through the glibc allocator due to
dlopen.
2024-09-09 13:26:06 -07:00
Ryan Houdek b4c797c199 Merge pull request #4052 from Sonicadvance1/various_warnings
Bunch of little things reported by coverity
2024-09-09 12:54:22 -07:00
Ryan Houdek ebae1bf71d Merge pull request #4051 from Sonicadvance1/fix_warning
Arm64: Fix warning
2024-09-09 08:24:48 -07:00
Ryan Houdek fa293e5e8f GDBServer: Fix buffer overrun in x87 register copying 2024-09-08 20:24:17 -07:00
Ryan Houdek 720c89e648 GDBServer: Make it clear we're copying all 512-bytes to clear a warning 2024-09-08 20:23:03 -07:00
Ryan Houdek abc5f5abd4 GDBServer: Use std::move for move assignment 2024-09-08 20:22:39 -07:00
Ryan Houdek 22ab16b217 Linux/x32/ipc: Fixes version check for msgrcv
Also adds an EINVAL return if version 1 is attempted with SHMAT,
mirroring what the kernel returns.
2024-09-08 20:18:11 -07:00
Ryan Houdek b80f05dd86 Seccomp/BPFEmitter: Reorder check to avoid integer overflow warning 2024-09-08 20:13:14 -07:00
Ryan Houdek 1642a0f4d9 Seccomp/BPFEmitter: Initialize Func,FuncSize just to make coverity happy 2024-09-08 20:10:44 -07:00
Ryan Houdek 96e990dade SignalDelegator: Use const auto ref to avoid copy 2024-09-08 20:07:45 -07:00
Ryan Houdek 3423a12ab6 SignalDelegator: Remove unused function copying for delegators 2024-09-08 20:06:38 -07:00
Ryan Houdek c4a0fb6609 Ioctl/drm: Fix missing member copy 2024-09-08 20:02:51 -07:00
Ryan Houdek 8cf23eba9c FEXLoader/ELFCodeLoader: Use const ref auto to avoid copy 2024-09-08 20:02:51 -07:00
Ryan Houdek fbe817f1c4 Config: Use std::move on strings to avoid copy 2024-09-08 20:02:51 -07:00
Ryan Houdek 3b61394548 FEXRootFSFetcher: Fixes some extract logic
`Extract` was never set to false even if the user said not to overwrite.
Ensure that if the user decided not to overwrite an existing folder,
that it returns false so it correctly doesn't overwrite the folder.
2024-09-08 20:02:51 -07:00
Ryan Houdek 8ed6b36181 FileManagement: Use const ref for AppConfigName 2024-09-08 20:02:51 -07:00
Ryan Houdek 9141322666 Ioctl/drm: Adds missing flags member copying 2024-09-08 20:02:51 -07:00
Ryan Houdek 2d91c5441e Ioctl/Radeon: Adds missing func member copying 2024-09-08 20:02:51 -07:00
Ryan Houdek 2ff1589c35 FileManagement: Use const ref for Filename 2024-09-08 20:02:51 -07:00
Ryan Houdek a308b9edbe Linux/GenerateMap: Fixes FD leak
This FD was unused
2024-09-08 19:34:38 -07:00
Ryan Houdek f6a8e595df Linux/Syscalls: Add todo for futimesat 2024-09-08 19:34:38 -07:00
Ryan Houdek eaaf62e4b7 X87: Make it a bit clearer that 80-bit floats can't hit the memory path
Makes the static analysis happy.
2024-09-08 18:03:20 -07:00
Ryan Houdek dc5fc57e19 FEXServer: Check if prctl errors 2024-09-08 18:03:20 -07:00
Ryan Houdek c3c0bbb060 F80Fallbacks: Fix Uninitialized variable warning that can't occur 2024-09-08 18:03:20 -07:00
Ryan Houdek 98a91d242d FEXRootFSFetcher: Fixes resource leak on XXHash error 2024-09-08 18:03:20 -07:00
Ryan Houdek 8da6b1e729 FEXCore/Allocator: Fixes uninitialized scalar variable warning that can't occur 2024-09-08 18:03:20 -07:00
Ryan Houdek d260bce31e FEXRootFSFetcher: Fixes uninitialized variable 2024-09-08 18:03:20 -07:00
Ryan Houdek a94aaca0ec OpcodeDispatcher: Add default FEX_UNREACHABLE
This can't happen but if it changes then make sure we capture it.
2024-09-08 18:03:20 -07:00
Ryan Houdek 2ac3ca3ead pidof: Fix uninitialized variable warning 2024-09-08 18:03:20 -07:00
Ryan Houdek c559445b62 Linux/IoctlEmulation: Fixes incorrect direction copy for a few ioctls
Plus convert remaining handlers over to using CPYF/CPYT handlers so
ensure this doesn't happen again.
2024-09-08 18:03:20 -07:00
Ryan Houdek 8ea4e4f663 Linux: Adds some missing brace initializers on conversion operators 2024-09-08 18:03:20 -07:00
Ryan Houdek 2da700fe82 CodeEmitter: Removes some ternaries that are unnecessary
NFC
2024-09-08 18:03:20 -07:00
Ryan Houdek 0a32ba9b29 instcountCI: Fixes for shifts 2024-09-08 18:03:20 -07:00
Ryan Houdek f51832ca6b FEXCore: Fixes a bug with VPSRLDQ/VPSLLDQ with >= 16-byte shifts
When the shift amount is >= 16-bytes then we need to zero the register.
We had a bug where we were assigning `Result.High` to itself, which
effectively made the top 128-bits of the ymm register not modify itself.

Adds a unit test to ensure that doesn't happen again.
2024-09-08 15:39:22 -07:00
Ryan Houdek 5f9a30af7a FileManagement: Use const ref for RootFSPath
No need to make a copy
2024-09-08 15:20:04 -07:00
Ryan Houdek 453ef0b94c Linux/ExecveHandler: Move a path rather than copy
We can do move assignment instead of copy assignment here.
2024-09-08 15:17:04 -07:00
Ryan Houdek 4779e64cf5 Linux/EmulatedFiles: Check return for seal
This shouldn't be possible to occur, but have a log message just in
case.
2024-09-08 15:15:38 -07:00
Ryan Houdek e6025cc087 Arm64: Fix warning
This variable is only used in a log message no, so move it there.
2024-09-08 11:15:46 -07:00
Ryan Houdek cd4c224dc0 Merge pull request #4050 from alyssarosenzweig/sra/move-size
JIT: always use 64-bit moves for SRA
2024-09-08 10:30:17 -07:00
Ryan Houdek 07f117b957 Merge pull request #4047 from bylaws/arm64ec-sysv2
ARM64EC: Switch to exception-based native syscall dispatch
2024-09-08 10:30:05 -07:00
Ryan Houdek 36e4f9a070 Merge pull request #4049 from alyssarosenzweig/opt/constprop-merges
ConstProp: speed it up
2024-09-08 10:13:23 -07:00
Alyssa Rosenzweig 67c751d3e5 JIT: always use 64-bit moves for SRA
I see no reason not to. Simpler code and it's slightly faster on Firestorm.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-08 13:02:57 -04:00
Ryan Houdek 1c59bfeb9f Merge pull request #4027 from alyssarosenzweig/opt/global-flag
Global flag optimizations
2024-09-08 09:42:18 -07:00
Alyssa Rosenzweig 2b0041a291 InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-08 10:27:20 -04:00
Alyssa Rosenzweig 39cc1e5116 ConstProp: merge folding+inlining
2% off nodejs

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-08 10:26:58 -04:00
Alyssa Rosenzweig f46856c090 ConstProp: template constant inlining
DRY

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-08 10:26:58 -04:00
Alyssa Rosenzweig b7859052da ConstProp: don't mask shifts
done in the emitter now

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-08 10:26:58 -04:00
Alyssa Rosenzweig 3b30d8103f ConstProp: combine like cases
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-08 10:26:58 -04:00
Alyssa Rosenzweig d19fc36f8f ConstProp: delay constant pooling
more efficient to pool the output of constant folding and such. but we need to
avoid that becoming accidentally quadratic, so rework that too

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-08 10:26:58 -04:00
Alyssa Rosenzweig 5fbf76ee9e CodeEmitter: mask immediate shifts
more convenient for constprop and not really any more expensive.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-08 10:26:58 -04:00
Ryan Houdek 5f79761fe4 Merge pull request #4046 from Sonicadvance1/cpubackend_remove_unused
CPUBackend: Remove unused functions
2024-09-07 13:11:32 -07:00
Billy Laws 621af9e7e8 ARM64EC: Switch to exception-based native syscall dispatch
The previous approach assumed that ntdll wouldn't have been patched
before FEX was loaded, but that isn't necessarily the case thanks to
CEF. This takes the same approach used by xtajit which wine recently gained
support for wine. The overhead to this isn't ideal but most syscalls
aren't directly done through x86 code anyway and in the future FEX could
forward unpatched thunks to their ARM variants at JIT time.
2024-09-07 15:30:00 +00:00
Billy Laws f4d107c8f0 ARM64EC: Split out FEX -> packed EC context conversion from reconstruction 2024-09-07 15:29:48 +00:00
Ryan Houdek 59643db331 CPUBackend: Remove unused functions
Some of these were bad ideas, some ideas were something we effectively
grew out of.
2024-09-07 08:07:00 -07:00
Alyssa Rosenzweig 46f36dc89d InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 10:59:41 -04:00
Alyssa Rosenzweig 384882744c DeadStoreElimination: drop now that it's redundant
it's only really load bearing for pf/af, which is handled as a global flag opt
now. this mitigates some of the compile time hit from globalizing flag opts.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 10:59:05 -04:00
Alyssa Rosenzweig b36d1f7e7b RedundantFlagCalculationElimination: optimize parity
another global CFG-based optimization -- if we know that the raw PF is already
1-bit we can skip parity evaluation, saving work with floating point compares.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 10:59:05 -04:00
Alyssa Rosenzweig dd7c66db2e RedundantFlagCalculationElimination: use LUT for flags
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 10:59:05 -04:00
Alyssa Rosenzweig 47ac9edd7a RedundantFlagCalculationElimination: select testz
this saves uops.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 10:59:05 -04:00
Alyssa Rosenzweig 6e6d640fec RedundantFlagCalculationElimination: fold compares/axflag into branches
now that we know whether flags are killed on the edge, we can improve branch
isel

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 10:59:05 -04:00
Alyssa Rosenzweig eb88366614 RedundantFlagCalculationElimination: globalize
Gather a control flow graph and use it to propagate flags throughout the
program.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 10:59:05 -04:00
Alyssa Rosenzweig 50d6ddd591 RedundantFlagCalculationElimination: extract per-block logic
nfc

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 10:59:05 -04:00
Alyssa Rosenzweig 249de7c758 RedundantFlagCalculationElimination: drop unneeded bools
nonzero <==> not dummy

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 10:59:05 -04:00
Alyssa Rosenzweig c350888d70 RedundantFlagCalculationElimination: add missing adczero case
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 10:59:05 -04:00
Alyssa Rosenzweig f23bef1653 RedundantFlagCalculationElimination: add missing testnz case
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 10:59:05 -04:00
Alyssa Rosenzweig d17f33a922 OpcodeDispatcher: don't emit fake 0 for condjump
not needed and getting in the way

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 10:59:05 -04:00
Alyssa Rosenzweig 50a3ca0d6d IR: introduce dedicated PF/AF instructions
this makes reasoning about them a little easier, e.g. for flags.  about 1% win
in nodejs.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 10:59:05 -04:00
Alyssa Rosenzweig 8745455a5b IR: track whether parity is read
so we can gate optimizations efficiently

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 08:26:04 -04:00
Alyssa Rosenzweig e9ab514962 IR: push parity evaluation down
so we can optimize it globally

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 08:26:04 -04:00
Alyssa Rosenzweig 4eb0948451 IR: push down AXFLAG lowering
so we can get the new axflag optimizations on billy's x13s.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 08:26:04 -04:00
Alyssa Rosenzweig 8d9f19bd73 IR: add TestZ op
more optimized than TestNZ if we don't care about the sign bit.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 08:26:04 -04:00
Alyssa Rosenzweig 9fe7bc5818 InstCountCI: add ucomiss+pf opt case
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 08:26:04 -04:00
Alyssa Rosenzweig 5a590a9b11 InstCountCI: add 8-bit test cases
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 08:26:04 -04:00
Ryan Houdek a4acd64246 Merge pull request #4042 from neobrain/refactor_system_libs
CMake: Compile with system libraries for xxhash, Catch2, and fmt, if available
2024-09-06 11:48:12 -07:00
Ryan Houdek 304b5de1af Merge pull request #4038 from Sonicadvance1/move_aotir_debugdata
AOTIR: Move debugdata structure to internal header
2024-09-06 10:50:42 -07:00
Ryan Houdek ca2fe0b301 AOTIR: Move debugdata structure to internal header
This definition doesn't need to be exposed in the public API.
2024-09-06 10:19:45 -07:00
Ryan Houdek bbef4d762c Merge pull request #4044 from bylaws/int2e
OpcodeDispatcher: Do not forbid INT 2E syscalls on 64-bit Windows
2024-09-06 10:18:18 -07:00
Ryan Houdek f77841d784 Merge pull request #4041 from Sonicadvance1/fix_poll
LinuxSyscalls: With poll syscall, ensure fds is writable only if nfds is not zero
2024-09-06 10:17:55 -07:00
Ryan Houdek 219a4777c6 Merge pull request #4040 from Sonicadvance1/delete_threadsstate
FEXCore: Delete ThreadsState struct
2024-09-06 10:16:28 -07:00
Ryan Houdek 29f0e9b4d6 Merge pull request #4039 from Sonicadvance1/move_customir_entry
FEXCore: Move `CustomIRResult` to internal header
2024-09-06 10:16:18 -07:00
Ryan Houdek b75efc42bf Merge pull request #4037 from Sonicadvance1/move_symbol
FEXCore: Move `JITSymbolBuffer` to internal header
2024-09-06 10:15:54 -07:00
Tony Wasserka 90dbd4766d Merge pull request #4043 from alyssarosenzweig/gitignore/build
gitignore: ignore build symlink
2024-09-06 18:08:47 +02:00
Billy Laws f7b911ca43 OpcodeDispatcher: Do not forbid INT 2E syscalls on 64-bit Windows
This works fine on real Windows and is relied on by wine as SystemCall
is set to 1 in KUSER_SHARED_DATA, which causes the ntdll thunks to use
it over `syscall`
2024-09-06 15:58:17 +00:00
Alyssa Rosenzweig 6de0333708 gitignore: ignore build symlink
I symlink build -> Build for YouCompleteMe.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-06 11:22:31 -04:00
Tony Wasserka ab5d3ab22b CMake: Don't use pkgconfig when cross-compiling 2024-09-06 10:30:48 +02:00
Tony Wasserka 355c3428c0 CMake: Don't use system libraries when compiling with mingw 2024-09-06 10:30:24 +02:00
Tony Wasserka c76b7bfe8d CMake: Compile with system libraries for xxhash, Catch2, and fmt, if available 2024-09-06 09:47:49 +02:00
Ryan Houdek 64c5362580 LinuxSyscalls: With poll syscall, ensure fds is writable only if nfds is not zero
The pointer is allowed to be null or garbage if the number of nfds
passed in is zero.
Unify the x86 and x86-64 implementations to ensure consistency.

Fixes Warhammer 40k: Relics of War
2024-09-05 14:33:52 -07:00
Ryan Houdek 77469de247 FEXCore: Delete ThreadsState struct
All uses of this have been removed.
2024-09-05 13:39:02 -07:00
Ryan Houdek 48fd827004 FEXCore: Move CustomIRResult to internal header
This definition doesn't need to be exposed in the public API.

Stopped needing to be public once we removed our IR CI thing.
2024-09-05 13:36:15 -07:00
Ryan Houdek f09d511ac8 FEXCore: Move JITSymbolBuffer to internal header
This definition doesn't need to be exposed in the public API.
2024-09-05 13:28:45 -07:00
Ryan Houdek d5db8948db Docs: Update for release FEX-2409 2024-09-05 12:43:45 -07:00
Ryan Houdek 5013b8a0db Merge pull request #4036 from alyssarosenzweig/opt/dont-inline-pool
ConstProp: stop pooling inline constants
2024-09-05 07:30:39 -07:00
Alyssa Rosenzweig ac65deed6c ConstProp: drop CreateInlineConstant
bit of cleanup

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-05 09:48:59 -04:00
Alyssa Rosenzweig d1d4d2d876 ConstProp: stop pooling inline constants
Because we already pooled the _Constants, there's no benefit to also pooling inline constants. the robin map to do so just adds extra overhead for no benefit - drop it.

without multiblock, shaves around 2% off node. with multiblock, a bit less than
1% but still a win.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-05 09:47:35 -04:00
Ryan Houdek e234e118e0 Merge pull request #4035 from neobrain/fix_fexqonfic_polish
FEXQonfig: Fix minor saving/loading quirks
2024-09-05 06:18:51 -07:00
Tony Wasserka 1cc54312fa FEXQonfig: Recognize file: URLs in addition to file://
QUrl::fromLocalFile produces this format for relative paths.
2024-09-05 12:54:55 +02:00
Tony Wasserka de9ab6a023 FEXQonfig: Fix loading local config files by command line argument
ParentPath doesn't return a usable value for "Config.json", so just use
std::filesystem::absolute/parent_path instead.
2024-09-05 12:53:44 +02:00
Tony Wasserka 463a0b5ed4 FEXQonfig: Set a default save location when Config.json does not exist 2024-09-05 12:37:01 +02:00
Tony Wasserka 7b0365b377 FEXQonfig: Reset dirty indicator after loading a file 2024-09-05 12:36:05 +02:00
Tony Wasserka 803501526e FEXQonfig: Fix false "Failed to load config file" error when saving to a new file 2024-09-05 12:36:05 +02:00
Tony Wasserka a63a3a47e3 FEXQonfig: Clearly indicate save dialogs as such 2024-09-05 12:36:05 +02:00
Ryan Houdek a66fac614b Merge pull request #4034 from alyssarosenzweig/fix-tied-fma
IR: fix scalar FMA tied sources
2024-09-04 09:14:24 -07:00
Alyssa Rosenzweig 6d4693cbc1 IR: fix scalar FMA tied sources
needs to be modelled explicitly or else we lose information when translating

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-04 07:42:44 -04:00
Ryan Houdek 46a2a0608b Merge pull request #4033 from Sonicadvance1/dynamic_scale_support
FEXCore: Dynamically scale TSC
2024-09-03 16:11:52 -07:00
Ryan Houdek 49b40b71a6 Merge pull request #4030 from bylaws/eccfg
Windows: Load per-application configs
2024-09-03 16:11:42 -07:00
Ryan Houdek ca8f347902 InstcountCI: Skip rdtsc and rdtscp
These will differ depending on which runner they are running on. Just
disable them but keep the instructions around so show how bad rdtscp is,
but how good rdtsc is.
2024-09-03 13:50:43 -07:00
Ryan Houdek e415b9443f Merge pull request #4032 from bylaws/small
Windows: Fix missing pragma and license text
2024-09-03 13:47:43 -07:00
Ryan Houdek fd4f6b8020 FEXCore: Dynamically scale TSC
When I implemented TSC scaling originally, I chose a scale factor of 128
because it basically covered the range of devices we cared about without
going too high. I also only tested devices that had a TSC scale factor
from 19.2Mhz to 34Mhz. Turns out there is hardware that also has a 48Mhz
cycle counter, which cause them to effectively have a 6.1Ghz cycle
counter, which is kind of absurd.

Instead of a fixed scale, just calculate the amount of scaling we need
to get >= the minimum threshold of 1Ghz. This will change the shift from
7 to 5 or 6 for the faster cycle counter devices.

Of course if someone wants to know the scale factor they can still use
cpuid function 15h to know it.

Fixes #4026
2024-09-03 13:41:31 -07:00
Ryan Houdek 7a9eb01573 Merge pull request #4031 from bylaws/midr
Windows: Fill in per-core MIDR information
2024-09-03 13:40:30 -07:00
Billy Laws b0474816c7 Windows: Fix missing pragma and license text 2024-09-03 20:30:07 +00:00
Billy Laws 941b065da6 Windows: Fill in per-core MIDR information 2024-09-03 20:29:37 +00:00
Billy Laws ccba993ca9 Windows: Load per-application configs 2024-09-03 20:25:21 +00:00
Billy Laws e664f61da8 Config: Split out command line parsing from config init 2024-09-03 20:25:21 +00:00
Ryan Houdek c74df6a59b Merge pull request #4029 from neobrain/refactor_fexqonfic_followups
FEXQonfig: Minor followup changes
2024-09-03 11:53:01 -07:00
Tony Wasserka fc76d4e8c3 CMake: Suppress warning about Qt 6 not being found 2024-09-03 16:48:22 +02:00
Tony Wasserka 099211f39c FEXQonfig: Tune padding a little
The leftPadding on the radio buttons wasn't needed. Adding a little bit
of padding below them makes the grouping visually more clear however.
2024-09-03 16:48:21 +02:00
Tony Wasserka b327d7ae19 FEXQonfig: Drop superfluous include 2024-09-03 16:48:21 +02:00
Tony Wasserka 0e6a22febe FEXQonfig: Add missing license identifiers 2024-09-03 16:48:21 +02:00
Alyssa Rosenzweig b368223d50 Merge pull request #3628 from Sonicadvance1/seccomp
LinuxEmulation: Implement support for seccomp
2024-09-03 08:49:16 -04:00
Ryan Houdek 8fe1e9562d Merge pull request #4025 from Sonicadvance1/split_lock_config_fexqconfig
FEXQConfig: Add strict split-lock option
2024-09-02 14:44:15 -07:00
Ryan Houdek ae545b8cf3 FEXQConfig: Add strict split-lock option
This was missed in the initial FEXQConfig implementation since it was
implemented at the same time.
2024-09-02 14:11:04 -07:00
Ryan Houdek ac32876e4e LinuxEmulation: Implement support for seccomp
Seccomp is a relatively complex feature that was added to Linux back in
2005, and was further extended in 2013 to support BPF based protections.
Once seccomp is enabled, you can no longer disable seccomp but
additional protections can be placed on top of existing seccomp filters.
Additionally seccomp filters are inherited in child processes, which
ensures the process tree can't escape from the secure computing
environment through child processes.

The basis of this feature is a shim that lives between userspace and the
kernel at the syscall entrypoint.
In "strict" mode, seccomp only allows read, write, exit, exit_group, and {rt_,}sigreturn to function.
When in "filter" mode, a BPF filter is run on syscall entrypoint and
returns state about if the syscall should be allowed or not. Multiple
filters can be installed in this mode, all of which get executed. The
result that is the most restricted is the action that occurs at the end.

There are some significant limitations in filter mode that must be
adhered to which makes executing this code inside of kernel space a
non-issue and effectively limits how much cpu time is spent in the filters.
Although these filters are free to do basically anything with the
provided data, just can't do any loops.

FEX needs to implement seccomp because there are multiple applications
using the feature, the primary one being Chromium which some games embed
without disabling the sandbox. WINE also uses seccomp for capturing
games that do raw Windows system calls. Apparently Red Dead Redemption
is one of the games that requires this.

While FEX implements seccomp, it is not yet all encompassing, which is
one of the reasons why it isn't enabled by default and requires a config
option.

**seccomp_unotify is not implemented**
This is a relatively new feature for seccomp which lets the seccomp
filter signal an FD for multiple things. Luckily Chromium and WINE don't
use this. This will be tricky to implement under FEX since it
requires ioctl trapping and some other behaviour

**ptrace isn't supported**
One feature of seccomp is that it can raise ptrace events. Since FEX
doesn't support ptrace at all, this isn't handled. Again Chromium and
WINE don't use this.

**kill-thread not quite correct**
This isn't directly related to seccomp but more about how we do thread
shutdown in FEX. This will require some more changes around thread state
tracking before fully supporting this. Chromium and WINE don't use this.
kill-process also falls under this

Features that are supported:
- Strict mode and seccomp-bpf mode supported
- All BFP instructions that seccomp-bpf understands
- Inheriting seccomp through execve
   - This means we serialize and deserialize the calling thread's
     seccomp filters
   - An execve that escapes FEX will also escape seccomp. Not much we
     can do about it
- TSync - Allowing post-mortem seccomp insertion which allows threads to
  synchronize seccomp filters after the fact

Features that are not supported:
- Different arch qualifiers depending on syscall entrypoint
  - Just like our syscall handler, we are hardcoded to the arch that the
    application starts with
- user_notif
- ptrace
- Runtime code cache invalidation when seccomp is installed
  - Currently we must ensure all syscalls go through the frontend
    syscall handler
  - Runtime invalidation of code cache with inline syscalls will get
    fixed in the future.

This currently isn't enabled by default because of the minor feature
problems that haven't been resolved. Currently the Linux Kernel's test
application works for the features that FEX supports, and WINE's usage
can be handled by FEX. Chromium's sandbox doesn't yet work with this PR,
but it only fails due to features unrelated to seccomp.

Having this open for merging now so we can work to resolve the remaining
issues without this bitrotting.
2024-09-02 14:07:53 -07:00
Ryan Houdek 9336e35052 Merge pull request #3929 from Sonicadvance1/portable
FEXInterpreter: Support portable installs
2024-09-02 10:13:07 -07:00
Ryan Houdek 0754affb98 Merge pull request #4024 from Sonicadvance1/fix_vdso_on_vdso_host
VDSO: Fixes a pretty nasty bug where we were never using the host VDSO
2024-09-02 09:49:24 -07:00
Ryan Houdek c413d7950b FEXInterpreter: Support portable installs 2024-09-02 09:37:21 -07:00
Ryan Houdek f8eaf9c14f VDSO: Fixes a pretty nasty bug where we were never using the host VDSO
This must have happened during a refactor or something, but since we're
making a copy of the function pointers, it would have only gotten the
version /prior/ to loading host VDSO symbols.

Moves the VDSO thunk definition setting to the end after VDSO symbol
definitions in order to get host VDSO symbols working again.
2024-09-02 09:18:37 -07:00
Ryan Houdek fc677eaabf CMake: Update minimum clang version to 13
Seccomp emulator uses lambda expressions in an unevaluated operand,
which was only added in clang-13
2024-09-02 07:38:23 -07:00
Alyssa Rosenzweig 114112a716 Merge pull request #3997 from Sonicadvance1/refactor_signal_information
SignalDelegator: Refactor how thread local data is stored
2024-09-02 10:14:52 -04:00
Ryan Houdek 9056d9b9de SignalDelegator: Refactor how thread local data is stored
Two primary things here:
- Remove the static `GlobalDelegator`
- Move the thread_local SignalDelegator::ThreadState information
  directly in to ThreadStateObject

Having the ThreadStateObject and the SignalDelegator information
disjoint was confusing but was required when we didn't have any object
in the frontend that could have its own independent data. Since we fixed
this with the `ThreadStateObject` type we can now move this over.

The `GlobalDelegator` object is now instead stored in
`ThreadStateObject` instead.

Instead of using a thread_local variable, we now just consume 8-bytes of
the signal alt-stack since the kernel gives us that information about
where it lives. This then converts all the thread_local usage to use
either the passed in CPU state if it exists, or fetching it from the
alt-stack offset.

Very minor changes in behaviour here, will help when trying to improve
FEX's behaviour around signals.
2024-09-02 06:46:19 -07:00
Alyssa Rosenzweig 74e95df661 Merge pull request #3974 from Sonicadvance1/strict_inprocess_splitlocks
Arm64: Implement support for strict in-process split-locks
2024-09-02 09:26:24 -04:00
Alyssa Rosenzweig d9544e7e02 Merge pull request #4018 from Sonicadvance1/move_guest_frame_manage
LinuxEmulation: Moves guest signal frame generation to its own file
2024-09-02 09:21:06 -04:00
Alyssa Rosenzweig 62e1767ee0 Merge pull request #4019 from Sonicadvance1/more_efault_handlers
More EFAULT handlers
2024-09-02 09:20:55 -04:00
Alyssa Rosenzweig 4baeffe84f Merge pull request #4022 from Sonicadvance1/move_sigreturn_to_frontend
FEX: Moves sigreturn symbols to frontend
2024-09-02 09:20:24 -04:00
Ryan Houdek b4a67a6178 Merge pull request #4021 from Sonicadvance1/remove_global_initializer_thunks
Thunks: Removes global static initializer
2024-08-31 22:54:22 -07:00
Ryan Houdek 8296bfc7de Merge pull request #4023 from chadmed/apple-partnums
CPUID: add missing Apple core part numbers
2024-08-31 19:45:30 -07:00
James Calligeros f588304b12 CPUID: add missing Apple core part numbers
The Ultra-class SoCs are two Max-class SoCs connected via
Apple's fabric, and thus use the same core revisions as the
Max-class SoCs for both big and LITTLE cores.

Signed-off-by: James Calligeros <jcalligeros99@gmail.com>
2024-09-01 12:14:39 +10:00
Ryan Houdek c748dbf0e3 FEX: Moves sigreturn symbols to frontend
These are a Linux construct and should live here. Removes a weird
passthrough API from FEXCore and keeps it in the frontend instead.
This isn't even typically allocated in a real setup, as it's only a
fallback for if VDSO isn't loaded.

The CallbackReturn function stays in FEXCore because it would have
caused an API in the other direction instead.
2024-08-31 07:43:04 -07:00
Mai 06497fdfad Merge pull request #4020 from Sonicadvance1/add_cross_compile_toolchain
CMake: Adds an AArch64 cross-compile toolchain file
2024-08-31 05:04:41 -04:00
Ryan Houdek e2a7fef742 Thunks: Removes global static initializer
This variable can't be constexpr initialized since it requires linker
fix-ups, which changes it in to a global static initializer instead.

While this benign as it doesn't allocate any memory, just move it next
to its single use. Removes the static initializer that was there for no
reason.
2024-08-31 01:58:35 -07:00
Ryan Houdek 17692d6eef CMake: Adds an AArch64 cross-compile toolchain file
This is useful for some people.

This doesn't allow building thunks due to the cmake limitation about
tools binaries also getting cross compiled, we need to switch to meson
if we want that, since they recently started fixing this
https://github.com/mesonbuild/meson/pull/12022

But that's an okay limitation for now.
2024-08-31 00:38:59 -07:00
Tony Wasserka 3020a0db2b Merge pull request #3982 from neobrain/feature_fexqonfic
Add Qt-based config editor
2024-08-30 10:43:07 +02:00
Ryan Houdek 92ddc0041b Merge pull request #4003 from Sonicadvance1/shortcircuit_invalid_inst
Frontend: short-circuit code generation on invalid instructions with multiblock
2024-08-29 04:52:14 -07:00
Tony Wasserka 8a5388f514 CMake: Build Dear ImGui only if needed 2024-08-29 12:14:06 +02:00
Tony Wasserka a444db7dac FEXQonfig: Use consistent orthography 2024-08-29 11:53:15 +02:00
Tony Wasserka 74da2eb5fc FEXQonfig: Fix unnecessary prompts for file selection on save 2024-08-29 11:53:15 +02:00
Tony Wasserka 539d5ed26f FEXQonfig: Fix Qt6 runtime-warnings caused by unannotated return types 2024-08-29 11:53:15 +02:00
Ryan Houdek cf7ee98831 x32/Thread: Adds EFAULT handlers 2024-08-28 04:31:06 -07:00
Ryan Houdek 5529948479 x32/Time: Adds EFAULT handlers 2024-08-28 04:25:13 -07:00
Ryan Houdek 91b6aeffe1 x32/Timer: Adds EFAULT handlers 2024-08-28 04:19:45 -07:00
Ryan Houdek 3a79b61f58 x64/Thread: Adds EFAULT handlers for two syscalls
For the remaining syscalls we need to introduce some new concepts around
checking if pointers and string arrays are readable and I don't want to
do that yet.
2024-08-28 04:14:16 -07:00
Ryan Houdek e92b24302f LinuxEmulation: Moves guest signal frame generation to its own file
No functional change, just moving the code.

This is entirely freestanding from the rest of the signal delegator
handling and mostly gets in the way when I'm working with the rest of
the signal handling. Separate it out to improve readability.
2024-08-28 03:23:00 -07:00
Tony Wasserka cc9ccf881b FEXQonfig: Use Unicode/locale-aware sorting for RootFS list
This orders the entries more like a normal file browser would.
2024-08-28 09:56:20 +02:00
Tony Wasserka 72d74ae482 FEXQonfig: Set a maximum height for RootFS list and make it scrollable 2024-08-28 09:52:46 +02:00
Tony Wasserka e88c57bec7 FEXQonfig: Move fallback RootFS entry to the top of the list
This provides better visibility when the RootFS list is large.
2024-08-28 09:51:27 +02:00
Ryan Houdek ac814ac015 Merge pull request #4017 from davide125/gdb
Install GDBSymbol integration in the correct location
2024-08-27 22:04:37 -07:00
Ryan Houdek 90f7cc925d Merge pull request #4007 from Sonicadvance1/ensure_no_256bit_operations
Arm64: Ensure 256-bit operations always assert without 256-bit SVE
2024-08-27 20:14:06 -07:00
Ryan Houdek 2039950762 Merge pull request #4016 from alyssarosenzweig/opt/umul-mov
Eliminate a move in 64-bit umul
2024-08-27 20:13:49 -07:00
Ryan Houdek b526c60c73 Merge pull request #4004 from Sonicadvance1/more_syscall_checks
Adds more syscall memory access checks
2024-08-27 20:13:21 -07:00
Davide Cavalca 1bfdd031d3 Install GDBSymbol integration in the correct location 2024-08-27 14:38:36 -07:00
Alyssa Rosenzweig 49ee8ba3be InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-27 12:33:07 -04:00
Alyssa Rosenzweig 335cd9180e OpcodeDispatcher: optimize mul rax
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-27 12:32:43 -04:00
Tony Wasserka ca9a94d572 CMake: Build only one FEXConfig and make Qt/ImGui exclusive choices 2024-08-27 18:10:06 +02:00
Alyssa Rosenzweig 03832b2523 Merge pull request #4012 from alyssarosenzweig/opt/pf-af-kill
DeadStoreElimination: handle PF/AF invalidate
2024-08-27 08:01:54 -04:00
Alyssa Rosenzweig 812224a0ef Merge pull request #4011 from bylaws/arm64-mb
Fix multiblock on ARM64EC
2024-08-27 08:01:26 -04:00
Alyssa Rosenzweig 42f2851575 Merge pull request #4009 from alyssarosenzweig/opt/axflag
Optimize AXFLAG-less systems
2024-08-27 08:01:05 -04:00
Ryan Houdek 9c7f44f8d4 x32/Signals: Add EFAULT checks 2024-08-27 03:04:42 -07:00
Ryan Houdek 5117ba351e x32/Semaphore: Add EFAULT checks 2024-08-27 03:04:24 -07:00
Ryan Houdek 441fdb689d x32/Sched: Add EFAULT checks 2024-08-27 03:04:24 -07:00
Ryan Houdek e786dfc998 x32/Msg: Add EFAULT checks 2024-08-27 03:04:24 -07:00
Ryan Houdek a1d3183c14 x32/Info: Add EFAULT checks 2024-08-27 03:04:24 -07:00
Ryan Houdek c5ef0910c5 x32/IO: Add EFAULT checks 2024-08-27 03:04:24 -07:00
Ryan Houdek affa1d0efc x32/FD: Add EFAULT checks 2024-08-27 03:04:24 -07:00
Ryan Houdek a83dc27a42 x32/EPoll: Add EFAULT checks 2024-08-27 03:04:24 -07:00
Ryan Houdek 129ec63e92 x64/Time: Add EFAULT checks 2024-08-27 03:04:24 -07:00
Ryan Houdek 3459369c6e x64/Signals: Add EFAULT checks 2024-08-27 03:04:24 -07:00
Ryan Houdek 7a490a3811 x64/Semaphore: Add EFAULT checks 2024-08-27 03:04:23 -07:00
Ryan Houdek ee17fe239a x64/EPoll: Add EFAULT checks 2024-08-27 03:04:23 -07:00
Tony Wasserka 0416950aaa Merge pull request #4015 from Sonicadvance1/fix_bug
x32/Signals: Fixes bug in the sigqueue syscalls
2024-08-27 12:03:55 +02:00
Ryan Houdek 6e46383cdb x32/Signals: Fixes bug in the sigqueue syscalls
In rt_sigqueueinfo and rt_tgsigqueueinfo we were failiny to pass along
the provided siginfo_t information to the syscall.

Fix that
2024-08-27 02:54:27 -07:00
Alyssa Rosenzweig faf1b85904 DeadStoreElimination: handle PF/AF invalidate
lets us kill some AF calculations with multiblock.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-26 11:28:16 -04:00
Billy Laws bec5f4fe2a DeadStoreElimination: Fix uint64 typing for register bitmask
unsigned longs are 32 bits wide on Windows.
2024-08-26 13:12:58 +00:00
Billy Laws fe6dbbfa63 Frontend: Don't explore native ARM64EC jump targets with multiblock 2024-08-26 13:01:06 +00:00
Ryan Houdek b19440c78f Arm64: Ensure 256-bit operations always assert without 256-bit SVE
Our JIT will happily consume incorrectly formed 256-bit vector operations in a lot of cases when the host CPU doesn't support 256-bit SVE.
This is what caused the bug in #4006. For every vector operation that
can consume a 256-bit size, add an assert that always checks if 256-bit
SVE is supported in those cases.

This will ensure that #4006 doesn't happen again.
2024-08-25 18:16:31 -07:00
Alyssa Rosenzweig abf9700475 InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-25 18:38:03 -04:00
Alyssa Rosenzweig 8d6b454455 OpcodeDispatcher: optimize AXFLAG emulation
this should help on x13s which has flagm but not flagm2.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-25 18:38:03 -04:00
Alyssa Rosenzweig 205ec3e14d OpcodeDispatcher: refactor axflag
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-25 18:35:27 -04:00
Mai f897579593 Merge pull request #4005 from Sonicadvance1/fault_safe_assert
SignalDelegator: Changes AFmt to ERROR_AND_DIE_FMT
2024-08-25 13:15:04 -04:00
Mai 2478abba29 Merge pull request #4008 from Sonicadvance1/fix_avx128_vfcmp
AVX128: Fixes 256-bit float compares
2024-08-25 12:37:04 -04:00
Mai a4db585664 Merge pull request #4006 from Sonicadvance1/fix_avx128_convert
AVX128: Fixes incorrect size usage in AVX128_Vector_CVT_Int_To_Float
2024-08-25 12:34:24 -04:00
Ryan Houdek 8bf4a124c8 AVX128: Fixes 256-bit float compares
Just like the bug in #4006, we were incorrectly passing a 256-bit
comparison in to the float compare IR operations. Once again it would
"safely" decompose in to 128-bit operations without issue.

PR #4007 added asserts to ensure that 256-bit operations aren't emitted
when the host CPU doesn't support 256-bit SVE and detected this.
2024-08-25 06:33:31 -07:00
Ryan Houdek 13c3b65732 unittests/asm: Adds test for previous bug fix 2024-08-25 05:22:30 -07:00
Ryan Houdek 7e6ba184f9 AVX128: Fixes incorrect size usage in AVX128_Vector_CVT_Int_To_Float
This handler was incorrectly using 256-bit IR operation sizes. Due to a
quirk with our IR handling, this would "safely" fall back to a 128-bit
operation and work "correctly".

The problem encountered is that since the IR operation is claiming to be
256-bit, when the value got spilled due to register pressure then a
true 256-bit store and load operation would be generated. This would
then emit an SVE load and store, with the expectation of 256-bit SVE
loadstores. This caused a SIGILL on Oryon since it doesn't support SVE,
but even would generate an invalid predicated loadstore on SVE 128-bit
hardware.

Fixes Aperture Desk Job in FEX.
2024-08-25 05:18:53 -07:00
Ryan Houdek f6cb914a2a SignalDelegator: Changes AFmt to ERROR_AND_DIE_FMT
To ensure that it actually dies after logging.
2024-08-25 04:27:59 -07:00
Tony Wasserka 6c92f94ee8 FEXQonfig: Remove unused code from CMakeLists.txt 2024-08-24 11:28:52 +02:00
Tony Wasserka bebf4209d6 FEXQonfig: Fix Qt 6 build 2024-08-24 11:01:16 +02:00
Ryan Houdek c82a683987 Frontend: short-circuit code generation on invalid instructions with multiblock
A source of overhead with multiblock is hitting instructions through a
conditional branch that can never be executed. Usually AVX512
instructions in glibc. This causes us to emit partial blocks for a ton
of targets that will never get executed.

Instead, when we have multiblock enabled, if a block hits an instruction
encoding we don't support, then remove all the decoded instructions from
the block and early terminate it if it isn't the entry block. This
resolves the issue of emitting a bunch of IR and code for blocks never
executed.

If the block of code has an invalid instruction in the entry block for
decoding then it'll still emit code up to the invalid instruction and
raise a SIGILL. This has the potential for generating some additional
blocks of code if a game is abusing SIGILL, but since that's unlikely
it's a good trade-off.

Also removes a few log instructions that don't really provide anything
anymore and just show up as confusing messages when multiblock is
enabled.
2024-08-23 17:22:33 -07:00
Ryan Houdek 886d40ccfe FEXGetConfig: Add new logs for split-lock emulation 2024-08-23 16:18:20 -07:00
Ryan Houdek 8f33e56e21 FEXConfig: Add new option for strict split-lock emulation 2024-08-23 16:18:19 -07:00
Ryan Houdek b1634680fe SpinWaitLock: Ensure mutexes are move-only
This is to prevent issues with copying mutex classes around.
2024-08-23 16:18:19 -07:00
Ryan Houdek 54d332935e Arm64: Implement support for strict in-process split-locks
For atomics that cross the 16-byte or 64-byte granularity, we need to
lock a mutex to ensure strict emulation of split-locks.

I took another look at these when I found out that Zen3 actually
implements split-locks. Not sure which architecture actually added
support for from them, but I wanted to ensure we have the ability to
handle this.

One thing that we can't handle in user-space is cross-process
split-locks through shared memory. This requires a kernel SIGBUS handler
to ensure a crashing/SIGKILL'd process doesn't lock all FEX processes in
the system.

This fixes a little split-lock abusing test that I have locally. It's a
bit flakey so it isn't viable to run in CI. Considering it is explicitly
testing a race problem.
2024-08-23 16:18:19 -07:00
Mai 2829ad56a1 Merge pull request #3996 from Sonicadvance1/more_bind
OpcodeDispatcher: Convert more template handlers to Bind handlers
2024-08-23 18:27:59 -04:00
Ryan Houdek fbf62f1296 Merge pull request #3998 from Sonicadvance1/move_midr_fetch
HostFeatures: Moves MIDR querying to the frontend
2024-08-23 15:19:18 -07:00
Ryan Houdek e6aa268093 Merge pull request #4002 from bylaws/vct
OpcodeDispatcher: Allow x86 code to read CNTVCT on ARM64EC
2024-08-23 15:19:05 -07:00
Ryan Houdek cc589ba7e6 Merge pull request #4001 from bylaws/sim
ARM64EC: Always use the CPU area context for BeginSimulation
2024-08-23 15:08:43 -07:00
Ryan Houdek f009a00986 Merge pull request #4000 from bylaws/binfmt
CMake: Don't install binfmts for MinGW builds
2024-08-23 15:08:30 -07:00
Ryan Houdek 8617150a42 Merge pull request #3999 from Sonicadvance1/fix_defer_signal_mask
SignalDelegator: Make sure to defer a signal if the guest signal mask desires
2024-08-23 15:08:06 -07:00
Billy Laws ef823ce82b OpcodeDispatcher: Allow x86 code to read CNTVCT on ARM64EC
Required by newer insider preview versions, I noticed many crashes with
this exception number and QueryPeformanceCounter in the backtrace,
testing with XTA found it to not be passed through and instead write
the host CNTVCT (unscaled) into RAX. No other registers seem to be
affected.
2024-08-23 21:46:11 +00:00
Billy Laws fc2917117e ARM64EC: Always use the CPU area context for BeginSimulation
New wine tests suggest x0 holding the pointer to the correct context is
just a coincidence so avoid relying on it.
2024-08-23 21:43:09 +00:00
Tony Wasserka 68abe400a7 FEXQonfig: Use better icon for RootFS file selection 2024-08-23 17:36:56 +02:00
Tony Wasserka df86d80a85 FEXQonfig: Clean up object ownership 2024-08-23 17:36:56 +02:00
Tony Wasserka 58cff72d38 FEXQonfig: Use FolderDialog when selecting library forwarding paths 2024-08-23 17:36:56 +02:00
Tony Wasserka fd1f5643c7 FEXQonfig: Rename header to Main.h 2024-08-23 17:28:28 +02:00
Billy Laws 0ea29dfbdf CMake: Don't install binfmts for MinGW builds 2024-08-23 12:09:02 +00:00
Tony Wasserka f4fca4482f FEXQonfig: Implement environment variable editing 2024-08-23 13:30:38 +02:00
Tony Wasserka 86d1ce7f00 FEXQonfig: Implement RootFS monitoring via inotify 2024-08-23 12:03:29 +02:00
Tony Wasserka 15fef1794c FEXQonfig: Create config and RootFS folders on startup if needed 2024-08-23 12:02:20 +02:00
Ryan Houdek 8b5d9d8fdf FEXLinuxTests: Adds unit test to ensure FEX defers a signal until unmask 2024-08-23 02:32:35 -07:00
Ryan Houdek 0ec724cf1f HostFeatures: Moves MIDR querying to the frontend
The MIDR querying is inherently OS specific and needs a bit of special
casing. Instead let the frontend inform FEXCore how many CPU cores there
are and their MIDRs instead.

This lets us keep the Linux specific code in the frontend.
2024-08-23 00:53:53 -07:00
Ryan Houdek 0dc0117e86 SignalDelegator: Make sure to defer a signal if the guest signal mask desires
Fairly trivial because we already supported deferring these signals. We
had just failed the final step of blocking the signal if we can't block
the signal (like with SIGSEGV, SIGBUS, etc).

Once the guest unblocks the signal mask with sigprocmask, the signal
will fire again.

Apparently older glibc relied on this behaviour for signal raising which
this fixes.
2024-08-22 19:48:23 -07:00
Ryan Houdek 1d00ad6030 OpcodeDispatcher: Convert VectorVariableBlend to Bind handler 2024-08-22 15:09:44 -07:00
Ryan Houdek 23a076c313 OpcodeDispatcher: Convert packed vector shifts to Bind handler 2024-08-22 15:07:09 -07:00
Ryan Houdek 57eacab654 OpcodeDispatcher: Convert VBROADCASTOp to Bind handler 2024-08-22 14:59:25 -07:00
Ryan Houdek b4093a8888 OpcodeDispatcher: Convert packed HSub to Bind handler 2024-08-22 14:57:40 -07:00
Ryan Houdek 7c5a9b5d6a OpcodeDispatcher: Convert AVXVectorVariableBlend to Bind handler 2024-08-22 14:55:34 -07:00
Ryan Houdek 12b3c82d83 OpcodeDispatcher: Convert PExtr to Bind handler 2024-08-22 14:54:26 -07:00
Ryan Houdek 66520bce0a OpcodeDispatcher: Convert VPACK{U,S}S to Bind handler 2024-08-22 14:52:41 -07:00
Ryan Houdek 25cc2bdcb8 OpcodeDispatcher: Convert VPERMILImm to Bind handler 2024-08-22 14:51:02 -07:00
Ryan Houdek f98b18800c OpcodeDispatcher: Convert MOVMSK to Bind handler 2024-08-22 14:49:57 -07:00
Ryan Houdek 0dd687a7a1 OpcodeDispatcher: Convert SHUFOp to Bind handler 2024-08-22 14:47:30 -07:00
Ryan Houdek 1aff3acbb9 OpcodeDispatcher: Convert PSHUFW to Bind handler 2024-08-22 14:45:04 -07:00
Ryan Houdek a6ab2ca30d OpcodeDispatcher: Convert PUNPCKH to Bind handler 2024-08-22 14:42:24 -07:00
Ryan Houdek ca43e2a61c OpcodeDispatcher: Convert PUNPCKL to Bind handler 2024-08-22 14:39:45 -07:00
Mai 4f404160d0 Merge pull request #3995 from Sonicadvance1/missing_header
LinuxSyscalls: Adds missing header
2024-08-22 17:22:32 -04:00
Ryan Houdek 5edc69b692 LinuxSyscalls: Adds missing header 2024-08-22 14:02:17 -07:00
Ryan Houdek b6cb897896 Merge pull request #3994 from bylaws/iofix
Windows: Fix some file operations on actual Windows
2024-08-22 14:01:56 -07:00
Ryan Houdek 47b3637452 Merge pull request #3993 from bylaws/flags
ARM64EC: Fix ContextFlag member tests
2024-08-22 14:01:43 -07:00
Ryan Houdek 5b65f30c8f Merge pull request #3992 from Sonicadvance1/rootfs_fetcher_choose_ui
FEXRootFSFetcher: Allow UI override through options
2024-08-22 13:47:54 -07:00
Ryan Houdek 23572539f8 Merge pull request #3991 from Sonicadvance1/fix_feat_lrcpc_sigbus
Arm64: Fixes SIGBUS handler for FEAT_LRCPC
2024-08-22 13:47:39 -07:00
Tony Wasserka 7176c717e5 FEXQonfig: Handle load failure 2024-08-22 20:51:19 +02:00
Tony Wasserka 3a05b760d4 FEXQonfig: Further improve UI responsiveness
clear() emits the modelReset signal, which is redundant since we're
about to add new data anyway.
2024-08-22 18:42:09 +02:00
Tony Wasserka 1e0273d570 FEXQonfig: Improve UI responsiveness 2024-08-22 18:32:22 +02:00
Tony Wasserka 8232be6302 FEXQonfig: Implement RootFS selection from file dialog 2024-08-22 18:11:04 +02:00
Alyssa Rosenzweig 43cd897cf4 Merge pull request #3988 from alyssarosenzweig/opt/idea
Optimize `adc x, 0`
2024-08-22 11:05:16 -04:00
Alyssa Rosenzweig e593807856 InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-22 10:51:06 -04:00
Alyssa Rosenzweig ce8e6e5c0c OpcodeDispatcher: optimize adc 0
clang generates this.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-22 10:51:06 -04:00
Alyssa Rosenzweig 5b9a7c7845 InstCountCI: add IDEA blocks
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-22 10:51:06 -04:00
Alyssa Rosenzweig 0e5f5a2db9 Merge pull request #3990 from alyssarosenzweig/opt/tst-al-1
Optimise `test al, 1`
2024-08-22 10:48:44 -04:00
Alyssa Rosenzweig 50a9cea16e Merge pull request #3989 from alyssarosenzweig/ici/x8count
InstCountCI: include x86 instruction count
2024-08-22 10:44:44 -04:00
Billy Laws 6aa95d82f2 Windows: Query FileStandardInformation for getting file EOF info
FileEOFInformation is write-only on actual windows.
2024-08-22 14:34:45 +00:00
Billy Laws 7c34f449d1 Windows: Translate file open flags passed to NtCreateFile 2024-08-22 14:34:45 +00:00
Billy Laws e9435203a0 Windows: Fix access() implementation
Had the result inverted.
2024-08-22 14:33:06 +00:00
Billy Laws 8d9ba0e102 ARM64EC: Fix ContextFlag member tests
There is a common arch bit that is included in each individual mask,
explicitly test for equality against the whole mask to avoid just
testing for this.
2024-08-22 14:31:11 +00:00
Tony Wasserka 3fcfd2ccde FEXQonfig: Enable file creation on save 2024-08-22 15:07:26 +02:00
Tony Wasserka aa4205c2e8 FEXQonfig: Implement "Save As" 2024-08-22 14:55:11 +02:00
Tony Wasserka 5d5b411611 FEXQonfig: Tune window size constraints 2024-08-22 14:52:34 +02:00
Tony Wasserka 5f5a06fb3b FEXQonfig: Fix formatting 2024-08-22 14:52:34 +02:00
Tony Wasserka 3077addcf8 FEXQonfig: Unindent after refactor 2024-08-22 14:52:34 +02:00
Tony Wasserka 8b6fe0c4ff FEXQonfig: Make pages scrollable 2024-08-22 14:52:34 +02:00
Tony Wasserka 29fd62e9ba FEXQonfig: Drop leftover debug log 2024-08-22 14:52:34 +02:00
Ryan Houdek 2631b113da FEXRootFSFetcher: Allow UI override through options
For #3987
2024-08-21 18:45:07 -07:00
Ryan Houdek da152031b3 unittest: Adds unittest for unaligned loadstores hitting the correct SIGBUS handler 2024-08-21 18:05:31 -07:00
Ryan Houdek be9c5678c0 Arm64: Fixes SIGBUS handler for FEAT_LRCPC
When #3899 refactored some of this code, it had split an if-else chain
in to two independent if-else chains. Turns out there was a minor
dependency between them. Add back the few checks necessary to ensure
that FEAT_LRCPC falls through before hitting the FEAT_LSE checks.

This is necessary because their instructions share an encoding so the
masks between the two instruction classes overlap slightly.
2024-08-21 18:01:55 -07:00
Alyssa Rosenzweig fdd370dd1a InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-21 17:52:22 -04:00
Alyssa Rosenzweig 6951284924 OpcodeDispatcher: optimize ADCX/ADOX
rewrite to use native adcs with flag fixups.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-21 17:52:22 -04:00
Alyssa Rosenzweig b03c613fbf OpcodeDispatcher: optimize JP/JNP
fuse and+cbnz into tbz/tbnz.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-21 17:52:22 -04:00
Alyssa Rosenzweig b893bdb8df OpcodeDispatcher: optimize fcmovu
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-21 17:52:22 -04:00
Alyssa Rosenzweig db45f6eec8 OpcodeDispatcher: optimize test with small immediate
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-21 17:52:22 -04:00
Alyssa Rosenzweig d53e689e22 IR: model tbz/tbnz
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-21 17:52:22 -04:00
Tony Wasserka f55378257a FEXQonfig: Delay saving until after the user selects a filename 2024-08-21 21:55:23 +02:00
Tony Wasserka 6a7914ac56 FEXQonfig: Fix double-save due to Qt 5 bug 2024-08-21 21:54:16 +02:00
Alyssa Rosenzweig d935d25f0c InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-21 11:48:03 -04:00
Alyssa Rosenzweig 84cf1d2fd2 InstCountCI: include x86 instruction count
useful for checking blow up ratios. I keep calculating this by hand so let's
just add it to the json

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-21 11:47:55 -04:00
Tony Wasserka ed0c045c17 FEXQonfig: Add Qt 6 support 2024-08-21 13:21:08 +02:00
Tony Wasserka d585063e60 FEXQonfig: Save CacheObjectCodeCompilation on modification 2024-08-21 11:30:56 +02:00
Tony Wasserka 3077fec9ab FEXQonfig: Add status bar 2024-08-21 11:30:56 +02:00
Tony Wasserka 9500842efc FEXQonfig: Strip file:// prefixes when displaying file paths 2024-08-21 11:30:56 +02:00
Tony Wasserka a6f9c51317 FEXQonfig: Load config from default location on startup 2024-08-21 10:04:35 +02:00
Ryan Houdek cfa2ad8423 Merge pull request #3986 from alyssarosenzweig/opt/test-x-x
Optimize test
2024-08-20 19:20:57 -07:00
Alyssa Rosenzweig 19010491da InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 21:17:08 -04:00
Alyssa Rosenzweig 5d613e8716 OpcodeDispatcher: optimize test x, x
fewer uops now that we invert carry

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 21:17:08 -04:00
Alyssa Rosenzweig 894aaa980f Merge pull request #3975 from Sonicadvance1/update_wfe_comment
SpinWaitLock: Update comment about WFE spurious wakeups
2024-08-20 20:55:30 -04:00
Ryan Houdek 877b2f4fef Merge pull request #3955 from alyssarosenzweig/opt/pop-final
Rearrange SRA to let us coalesce cmpxchg moves
2024-08-20 17:36:11 -07:00
Ryan Houdek 2a170cfdec Merge pull request #3985 from alyssarosenzweig/opt/tso
OpcodeDispatcher: fix tso checks
2024-08-20 17:18:21 -07:00
Alyssa Rosenzweig a299d6b1a5 InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 20:17:06 -04:00
Alyssa Rosenzweig ffb85e6305 ArchHelpers: rearrange SRA layout to coalesce cmpxchg
linux only for now, arm64ec should do something similar.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 20:14:08 -04:00
Alyssa Rosenzweig d7a20fa28f OpcodeDispatcher: fix tso checks
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 20:11:06 -04:00
Ryan Houdek 5ac7d5dfcd Merge pull request #3980 from alyssarosenzweig/opt/avx
Optimize AVX load/store with ldp/stp
2024-08-20 16:29:55 -07:00
Alyssa Rosenzweig 75644b33df InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:36:50 -04:00
Alyssa Rosenzweig 4c4c6e7807 OpcodeDispatcher: pair loads on cortex
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:36:50 -04:00
Alyssa Rosenzweig a8c9c71ce3 OpcodeDispatcher: use loadcontextpair
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:33:41 -04:00
Alyssa Rosenzweig 8a4bd5f22c OpcodeDispatcher: pair avx128 save
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:33:41 -04:00
Alyssa Rosenzweig 138a36c69a OpcodeDispatcher: pair avx128 restore
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:33:41 -04:00
Alyssa Rosenzweig 3400ca5d42 OpcodeDispatcher: pair avx restore
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:33:41 -04:00
Alyssa Rosenzweig 5d8164da4e OpcodeDispatcher: pair restore x87
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:33:41 -04:00
Alyssa Rosenzweig 8b89b30a6f OpcodeDispatcher: pair restore SSE
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:33:41 -04:00
Alyssa Rosenzweig e66d7cfd6c OpcodeDispatcher: pair mm save
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:33:41 -04:00
Alyssa Rosenzweig 63afa29dae OpcodeDispatcher: pair save AVX
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:33:41 -04:00
Alyssa Rosenzweig 926a9b40e2 OpcodeDispatcher: pair save mxcsr
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:33:41 -04:00
Alyssa Rosenzweig 9bb43264b5 OpcodeDispatcher: pair SSE store
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:33:37 -04:00
Alyssa Rosenzweig 32d6daf558 OpcodeDispatcher: use ldp/stp for AVX load/store
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:31:19 -04:00
Alyssa Rosenzweig bebcb73c68 OpcodeDispatcher: pair AVX high writes
reduces instr count with AVX-128

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:31:19 -04:00
Alyssa Rosenzweig d0e040514f IR: add LoadContextPair
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:31:19 -04:00
Alyssa Rosenzweig 850c027d52 IR: add AllocateFPR
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:31:19 -04:00
Alyssa Rosenzweig 5df90563e2 IR: add LoadMemPair
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:31:19 -04:00
Alyssa Rosenzweig 7a489d18c6 IR: add StoreMemPair
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:31:19 -04:00
Alyssa Rosenzweig 1db092e96f IR: add StoreContextPair
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:31:19 -04:00
Alyssa Rosenzweig 65a4de221b InstructionCountCI: add AddressingLimitations for AVX
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:31:19 -04:00
Alyssa Rosenzweig 9c8df79dfb InstructionCountCI: add bytemark stringsort blocks
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:31:19 -04:00
Ryan Houdek 689b461d7b Merge pull request #3984 from Sonicadvance1/atomic_tso_subchecks
FEXCore: Splits up atomic enablement checks
2024-08-20 15:16:54 -07:00
Ryan Houdek 92c951c81f Merge pull request #3983 from Sonicadvance1/direct_sra_correlation
Arm64: Allow directly correlating an ARM register back to an x86 register
2024-08-20 15:16:46 -07:00
Ryan Houdek 9c8438f264 FEXCore: Splits up atomic enablement checks
PR #3980 is adding a feature to merge loadstores in to paired
loadstores, but it was using the incorrect atomic check to determine if
it can safely merge them or not. It was using the GPR atomic check
instead of the vector atomic check. While this would improve performance
on Apple Silicon with its hardware TSO implementation, it would have had
zero impact on Cortex and Oryon.

Instead split out the three config options to live as a boolean check in
the ContextImpl similar to how we disable "AtomicTSOEmulation". Removing
the various configs in the JIT and CPUID so that it queries from the
same context. This makes it clearer that if you are wanting the current
active configuration for memcpy, vector, or general atomic TSO
emulation, you should query one of those three getters.

This also fixes a weird edge case bug in the arm64 JIT where you could
have TSO emulation disable, but still have vector TSO enabled partially.
Just because half a config wasn't checked in {Load,Store}MemTSO for
vectors. If the global "TSOEnabled" option is disabled then TSO should
always be disabled.

Alyssa will be able to pull this in to #3980 once merged and get the
performance uplift on Cortex and Oryon, since our default configuration
is to have vector and memcpy TSO emulation disabled.
2024-08-20 14:34:34 -07:00
Ryan Houdek caf7ad53e6 Arm64: Allow directly correlating an ARM register back to an x86 register
Fixes a bug that is getting introduced in to #3955 when it rearranged
register allocation.
2024-08-20 14:17:13 -07:00
Tony Wasserka 47f0fec2f2 Add Qt-based config editor 2024-08-20 20:54:34 +02:00
Ryan Houdek eadb502059 Merge pull request #3978 from bylaws/create
Windows: Implement CreateDirectoryW CRT function
2024-08-20 11:39:57 -07:00
Ryan Houdek 8e1695afa3 Merge pull request #3977 from bylaws/wowex
WOW64: Improve exception handling
2024-08-20 11:39:22 -07:00
Ryan Houdek a7138f26b6 Merge pull request #3981 from alyssarosenzweig/opt/btc
Optimize BTC
2024-08-20 11:38:43 -07:00
Alyssa Rosenzweig f2a9ce9d4c InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 09:26:20 -04:00
Alyssa Rosenzweig 84e4960e52 OpcodeDispatcher: optimize btc
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 09:24:02 -04:00
Ryan Houdek 99afd876ba Merge pull request #3976 from alyssarosenzweig/opt/huffman-
Improvements from bytemark "huffman"
2024-08-19 16:10:29 -07:00
Ryan Houdek 1caa31c5cb Merge pull request #3962 from Sonicadvance1/remove_vixl_requirement
HostFeatures: Removes vixl usage
2024-08-19 14:31:13 -07:00
Tony Wasserka d2c82ba707 Merge pull request #3968 from Sonicadvance1/binfmt_misc_systemd
binfmt_misc: Support systemd binfmt_misc
2024-08-19 22:33:52 +02:00
Billy Laws 9067f3513a Windows: Implement CreateDirectoryW CRT function
Used when the .fex-emu config dir doesn't exist. Also fixes up
CreateFile to not leak memory.
2024-08-19 19:11:17 +00:00
Billy Laws 409d691877 WOW64: Only handle unaligned atomics in JIT code 2024-08-19 19:07:30 +00:00
Billy Laws 1762ff0393 WOW64: Spill EFlags into the FEX state when reconstructing JIT context 2024-08-19 19:07:30 +00:00
Billy Laws 4d26178f25 Windows: Commonise guest exception handling logic and use for WOW64 2024-08-19 19:07:29 +00:00
Ryan Houdek c6582a1ce5 binfmt_misc: Support systemd binfmt_misc
Adds support for binfmt_misc through systemd configuration paths. Their
configuration files are basically the raw kernel interface description
in a .conf file, quite a bit more simple than the legacy debian path.

Default enable this path since systemd is the expected default
arrangement these days.

Fixes #2417
2024-08-19 11:15:18 -07:00
Alyssa Rosenzweig a51be56c9e InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-19 10:40:53 -04:00
Alyssa Rosenzweig e5cb583cc0 OpcodeDispatcher: optimize 8-bit rol
rotate right by less than 8:

  ror(________________7654321076543120, ...)

rotate left by less than 8:

  rol(76543210________________76543210, ...)

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-19 10:40:53 -04:00
Alyssa Rosenzweig 3ef83a9e23 OpcodeDispatcher: optimize ror al, cl
rely on the masking.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-19 10:40:53 -04:00
Alyssa Rosenzweig 7de6a5cb07 OpcodeDispatcher: optimize rotate flags
introduce an IR op for it so we can reason about flags across rotate
instructions easily.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-19 10:40:53 -04:00
Alyssa Rosenzweig e06b3a4186 ConstProp: optimize and x, -1
mitigates regression from previous patch

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-19 08:54:47 -04:00
Alyssa Rosenzweig c495f82f4f OpcodeDispatcher: optimize 8/16-bit bitwise with constants
avoid masking. apparently even modern compilers will do cute tricks with 8-bit
math in hot loops ...

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-19 08:54:47 -04:00
Alyssa Rosenzweig 49e4426ed5 RedundantFlagCalculationElimination: drop register writes
this allows us to form real cmp instructions when we know PF is dead.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-19 08:54:47 -04:00
Alyssa Rosenzweig c4e2436885 ConstProp: optimize add eax, -1 and friends
need to sign extend to get the constant inlined.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-19 08:54:47 -04:00
Alyssa Rosenzweig f078b25c7d InstructionCountCI: add bytemark huffman blocks
this test case has our biggest delta to native by far. the trash code we emit
makes clear why!

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-19 08:54:47 -04:00
Tony Wasserka e5149fba57 Merge pull request #3965 from Sonicadvance1/followup_3964
Library Forwarding: Follow up from #3964
2024-08-19 11:22:43 +02:00
Tony Wasserka 96055cbde7 Merge pull request #3972 from Sonicadvance1/update_robin_map_1_3_0
External: Update robin-map from 1.2.1 to 1.3.0
2024-08-19 10:53:40 +02:00
Tony Wasserka 4abac0cac7 Merge pull request #3970 from Sonicadvance1/stop_installing_static
FEXCore: Stop installing static library
2024-08-19 10:30:26 +02:00
Tony Wasserka 1e1bcc4af2 Merge pull request #3971 from alyssarosenzweig/cleanup/aotir
FEXLoader: don't install FEXUpdateAOTIRCache
2024-08-19 10:29:34 +02:00
Tony Wasserka f1d7879365 Merge pull request #3973 from Sonicadvance1/update_fmt_11_0_2
External: Update fmt from 10.1.1 to 11.0.2
2024-08-19 10:26:22 +02:00
Ryan Houdek 0ea3de95e0 SpinWaitLock: Update comment about WFE spurious wakeups
With recent bug fixes, WFE now can sleep for roughly as long as the
programmed architecture timer of 100 microseconds. Still nowhere near as
close as what x86 CPUs can get with waitx and waitpkg, because those
don't get spuriously woken up by an architecture timer.

100 microseconds is a significantly improvement over 52ns although.
2024-08-18 17:50:20 -07:00
Ryan Houdek 33cef7c8cd CRT: New symbols that fmt uses 2024-08-18 13:21:17 -07:00
Ryan Houdek b0fd220f3e External: Update fmt from 10.1.1 to 11.0.2
Required adding a new header to FEXCore/fextl/fmt.h and Thunks gen.cpp
2024-08-18 13:13:41 -07:00
Ryan Houdek 1e3539537d External: Update robin-map from 1.2.1 to 1.3.0
Carrying forward our patch to remove their static initializer.
2024-08-18 12:38:24 -07:00
Alyssa Rosenzweig 1b13a3dd4e FEXLoader: don't install FEXUpdateAOTIRCache
not used. we'll probably rip the whole thing out at some point but for now, no
reason to pollute user systems with this.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-17 11:33:46 -04:00
Ryan Houdek d1ec242e4f FEXCore: Stop installing static library
This hasn't ever mattered. Our API isn't even stable enough to really
install the shared library.
2024-08-16 19:55:00 -07:00
Ryan Houdek 21f9841e9d Merge pull request #3967 from bylaws/lib
Windows: Install libraries to CMAKE_INSTALL_LIBDIR
2024-08-16 17:04:52 -07:00
Billy Laws ea6c05a46a Windows: Install libraries to CMAKE_INSTALL_LIBDIR 2024-08-16 23:43:55 +00:00
Ryan Houdek 226f5e2f23 Merge pull request #3899 from Sonicadvance1/thread_safe_backpatch
Arm64: Handle backpatching in a thread-safe manner
2024-08-16 10:28:09 -07:00
Ryan Houdek a82fcdecd7 PR review comments 2024-08-16 10:21:18 -07:00
Ryan Houdek 27acbe305d ArchHelpers: Adjust ClearICache for its usage
Instead of clearing a hardcoded 16 bytes, adjust for the actual number
of instructions modified. The implementation will still only clear a
single cacheline so it doesn't change behaviour.
2024-08-16 10:21:18 -07:00
Ryan Houdek cd0739a534 Arm64Helpers: Moves instruction definitions to implementation
This used to exist in the FEXCore header since the unaligned handler was
done in the frontend. Once it got moved in to FEXCore it had stayed
there. Move it over now.
2024-08-16 10:21:18 -07:00
Ryan Houdek f1055d0713 Arm64: On backpatch ensure DMB instructions get patched in first
In the case of a visibility tear when one thread is backpatching while
another is executing. The executing thread can /potentially/ see the
writing of instructions depending on coherency rules or filling of
cachelines.

By ensuring the DMB instructions are backpatched over the NOP
instructions first, this ensures correct atomic visibility even on tear.
2024-08-16 10:21:18 -07:00
Ryan Houdek 7875b20594 Arm64: Handle backpatching in a thread-safe manner
When code buffers are shared between threads, FEX needs to be careful
around backpatching its code buffers, since one thread might have
backpatched the code that another thread was also planning on
backpatching.

To handle this case, when the handler fails to find a backpatchable
instruction, check if it was already backpatched. This can be determined
by atomically reading the instructions back and seeing if they have
turned in to the non-atomic variants.

In most cases we can just return saying that it has been handled, in the
case of a store we need to back the PC up 4 bytes to ensure the DMB is
executed before the non-atomic store.
2024-08-16 10:21:18 -07:00
Ryan Houdek 25679cd319 Arm64: Moves some unaligned atomic handling before mutex lock
These handlers don't do any code backpatching so locking the spinlock
futex isn't necessary. Move them before the lock to make them a bit more
efficient once code buffers get shared.
2024-08-16 10:21:18 -07:00
Ryan Houdek fa45ec32db Arm64: Early exit in unaligned handler if not on arm64
NFC, just removes a level of indent
2024-08-16 10:21:18 -07:00
Ryan Houdek 40d12c4d0e HostFeatures: Removes vixl usage
If we query the CPU flags ourselves then vixl is no longer a
compile-time or runtime dependency unless the vixl disassembler or
simulator is built.

A bit spicy from all the feature bits we need to load up.
2024-08-16 08:36:52 -07:00
Ryan Houdek 5e706dff64 Merge pull request #3961 from Sonicadvance1/fexcore_disable_vixl
FEXCore: Disable vixl linking if vixl disasm or simulator is disabled
2024-08-16 08:34:55 -07:00
Ryan Houdek a4860d6f87 Thunks: Follow up from #3964
Thunk host libraries were still hardcoded to lib/, use the same
mechanism as #3964 for installing to the correct location.
2024-08-16 07:39:55 -07:00
Ryan Houdek ef4c4f6e9b FEXCore: Disable vixl linking if vixl disasm or simulator is disabled
This was mostly there, just needed to remove some extraneous headers and
only insert vixl in to the library list if the options were enabled.
2024-08-16 07:29:41 -07:00
Ryan Houdek abbd65549a Merge pull request #3964 from davide125/lib64
Install libraries in the correct location
2024-08-16 07:26:02 -07:00
Ryan Houdek 00ef1baead Merge pull request #3963 from alyssarosenzweig/vendor/json
Clean up our JSON dependencies
2024-08-16 07:19:56 -07:00
Davide Cavalca 75687070a5 Install libraries in the correct location 2024-08-16 10:04:25 -04:00
Alyssa Rosenzweig 9b1496c327 tiny-json: collect includes
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-15 21:29:58 -04:00
Alyssa Rosenzweig 27cce9d9b7 tiny-json: drop nosprintf handling
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-15 21:29:55 -04:00
Alyssa Rosenzweig 08b66bf827 External: merge json headers
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-15 21:26:53 -04:00
Alyssa Rosenzweig 0aa5807482 External: merge json source files
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-15 21:25:46 -04:00
Alyssa Rosenzweig 7a2d8c5c01 External: merge json targets
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-15 21:24:44 -04:00
Alyssa Rosenzweig 1b8b1a24b0 External: merge json-maker into tiny-json
They are functionally one library.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-15 21:23:40 -04:00
Alyssa Rosenzweig 86eb2b13b9 External: hard vendor json libraries
they are both tiny and seemingly unmaintained upstream, let's bring them
under our own umbrella in tree.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-15 21:20:20 -04:00
Mai c634c53434 Merge pull request #3960 from Sonicadvance1/remove_bad_rep_assumption
OpcodeDispatcher: Remove old bad assumption in INC/DEC
2024-08-15 15:42:47 -04:00
Ryan Houdek 2eb7a9ff28 OpcodeDispatcher: Remove old bad assumption in INC/DEC
Somewhere there was an assumption made that INC and DEC supported the
repeat prefix. This isn't actually the case, while the prefix can be
encoded, it is a nop and should only expect to be used for padding.

Adds a unittest to ensure that behaviour is as expected.
2024-08-15 10:22:31 -07:00
Ryan Houdek 933c65d805 Merge pull request #3956 from alyssarosenzweig/opt/pop-return
small optimizations for returns
2024-08-15 03:30:23 -07:00
Ryan Houdek df0ecad15b Merge pull request #3933 from bylaws/arm64-suspend
Support cooperative suspend on ARM64EC
2024-08-15 01:22:28 -07:00
Ryan Houdek ce88f5f948 Merge pull request #3941 from Sonicadvance1/fault_assertion_checking
LinuxSyscalls: Implements less invasive assertion only EFAULT handlers
2024-08-15 01:14:15 -07:00
Ryan Houdek 27cd399d6f Merge pull request #3944 from Sonicadvance1/minor_syscall_changes
LinuxSyscalls: Some minor cleanups
2024-08-15 00:55:06 -07:00
Ryan Houdek 7b70925acf unittests/gvisor: Updates known failures test for poll 2024-08-15 00:48:57 -07:00
Ryan Houdek 2a3e0b93e8 unittests: Update syscalls_efault test 2024-08-15 00:48:57 -07:00
Ryan Houdek 2b26d0fff5 LinuxSyscalls: Implements less invasive assertion only EFAULT handlers
With the previous Copy{To,From}User helpers we need to actually
implement the handlers correctly. We want something that is a bit
lighter so we don't need to implement the faulting path in the syscall
handlers.

Implements a handful of helpers that just check for readable and
writable capability which can be thrown in to an assertion handler that
is zero cost in release mode.

Readable is checked by just attempting to read all bytes.
Writable is checked by attempting to read each byte and writing it back
to the same location.

Uses these helpers in x64/FD.cpp to showcase how they will be used to
detect EFAULT. Tested locally that they work correctly by writing some
small tests for the syscalls that expect EFAULT.
2024-08-15 00:48:57 -07:00
Alyssa Rosenzweig 129f676610 InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 16:59:34 -04:00
Alyssa Rosenzweig 2dc92c122e BranchOps: micro-optimize ExitFunction
this should be slightly faster on Firestorm and no worse on recent cortex

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 16:59:34 -04:00
Alyssa Rosenzweig 0db17bd58a OpcodeDispatcher: optimize IRET
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 16:59:34 -04:00
Alyssa Rosenzweig aac16493f1 OpcodeDispatcher: optimize RET
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 16:59:34 -04:00
Ryan Houdek 86e5e1a15e Merge pull request #3954 from Sonicadvance1/fix_missing_comment
HostFeatures: fix clang reformat constantly with missing comment
2024-08-14 12:03:10 -07:00
Ryan Houdek aa5d2ff31c Merge pull request #3951 from alyssarosenzweig/opt/pops
Add a hack for multiple destinations & make good use of it
2024-08-14 12:03:00 -07:00
Ryan Houdek 30dd5f5c75 HostFeatures: fix clang reformat constantly with missing comment 2024-08-14 10:00:28 -07:00
Alyssa Rosenzweig f5bc06476a InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:37:06 -04:00
Alyssa Rosenzweig c70e44cf05 OpcodeDispatcher: extract Push helper
Mirrors the Pop helper we added. This cleans up a bunch

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:37:06 -04:00
Alyssa Rosenzweig ede13e37d9 OpcodeDispatcher: optimize Thunk
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:37:06 -04:00
Alyssa Rosenzweig a7dada457a OpcodeDispatcher: optimize POPF
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:37:06 -04:00
Alyssa Rosenzweig 6367554d30 OpcodeDispatcher: optimize LEAVE
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:37:06 -04:00
Alyssa Rosenzweig 81969a684e OpcodeDispatcher: optimize pop segment
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:37:06 -04:00
Alyssa Rosenzweig ee339b5960 OpcodeDispatcher: optimize POPA
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:37:06 -04:00
Alyssa Rosenzweig 881c940693 OpcodeDispatcher: optimize POP
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:37:06 -04:00
Alyssa Rosenzweig 900c62fa7b OpcodeDispatcher: add Pop helpers
hide away the allocate dance

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:37:06 -04:00
Alyssa Rosenzweig 200c6c054f IR: introduce POP operation
rmw on a source, kind of terrible.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:37:06 -04:00
Alyssa Rosenzweig bc0927b7b1 JIT: optimize moves for cmpxchg
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:37:06 -04:00
Alyssa Rosenzweig 231c28395c RegisterAllocationPass: clean up after pair removal
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:37:06 -04:00
Alyssa Rosenzweig 64a45c0d29 IR: remove pairs
They're now unused. And won't be missed.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:37:06 -04:00
Alyssa Rosenzweig cab02be637 IR: remove unused pair create/extract
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:37:06 -04:00
Alyssa Rosenzweig 74f341bc0e IR: remove pair from cpuid
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:37:06 -04:00
Alyssa Rosenzweig feaa1af1a8 IR: remove pair from XGetBV
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:37:06 -04:00
Alyssa Rosenzweig d59d040b4e IR: add design doc breadcrumb
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:17:23 -04:00
Alyssa Rosenzweig a9c26cbf71 IR: add coalescing heuristics for pair replacements
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:17:23 -04:00
Alyssa Rosenzweig f4b5c4e69a OpcodeDispatcher: allow upper garbage for cmpxchg
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:17:23 -04:00
Alyssa Rosenzweig b8cac9f7d5 JIT: avoid some moves with caspal
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:17:23 -04:00
Alyssa Rosenzweig 13974df204 IR: drop CASPair pair result
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:17:23 -04:00
Alyssa Rosenzweig fa6fe9bf06 IR: drop cmppairz pairs
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:17:23 -04:00
Alyssa Rosenzweig 746be0824e IR: drop CASPair source pairs
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:17:23 -04:00
Alyssa Rosenzweig 93120cabbb IR: drop pair from memcpy
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:17:23 -04:00
Alyssa Rosenzweig 04ae05f4ce IR: add hack for multiple destinations
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:17:23 -04:00
Alyssa Rosenzweig 83773dddc7 IR: fix size for cmppairz
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:17:23 -04:00
Alyssa Rosenzweig 9813553f02 json_ir_generator: introduce multidestination hack
We now have two types of destinations:

* regular destinations. These are SSA. You get exactly 1 per instruction. This
  is what almost every instruction should use.

* special destinations, introduced here. These are *not* SSA. They must be
  allocated with a special instruction (added later in this PR), and then they
  are mutated by the instruction. There are two types, either pure destinations
  ("out") or read-modify-write source+destinations ("in-out"). The former are
  useful for instructions that return multiple destinations, like Memcpy. The
  latter are useful for instructions that need a source tied with a special
  destination (currently just Pop, introduced later in this series).

Special destinations reuse the mechanism of sources, to get around the
limitations on regular destinations in our current IR. Ops with special
destinations desugar to ops with no destination but extra sources prefixed Out
or Inout.

They further require HasSideEffects so we don't optimize ourselves into corners.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:17:23 -04:00
Ryan Houdek 924723d433 LinuxSyscalls: Some minor cleanups
- We can have the SyscallFunctionDefinitions be the correct size out of
  the gate. Both tables are always 512 entries in size.
- In the RegisterSyscall_{32,64} handlers, just get the reference using
  operator[]. We always know we will be under the size of the array, add
  a an assert to check. Removes a bit of vector range checking overhead.
- Namespace 32-bit syscalls like 64-bit syscalls and include in the
  regular header like 64-bit. This was just an oversight
- Use std::fill for the syscall gap for the invalid syscall, just a
  minor cleanup.

No functional change.
2024-08-13 14:06:16 -07:00
Ryan Houdek 33558e63c4 Merge pull request #3950 from alyssarosenzweig/bug/flagm-ci
InstructionCountCI: explicitly enable flagm for multiinst
2024-08-13 13:49:11 -07:00
Ryan Houdek 97c229d5eb Merge pull request #3935 from bylaws/win-feats
Windows: Support CPU feature detection from ID registry keys
2024-08-13 13:40:54 -07:00
Billy Laws 16b007df33 Windows: Partially implement the registry read API 2024-08-13 13:31:32 +00:00
Billy Laws 3cbc421c7e Windows: Detect CPU features using the registry
Native Windows does not support ID instruction emulation, instead it
populates registry keys containing their values.
2024-08-13 13:31:32 +00:00
Billy Laws 890e5e1f0f FEXCore: Support disabling host cacheline clean/clear operations 2024-08-13 13:25:34 +00:00
Billy Laws 6b98454f03 HostFeatures: Split out vixl feature and CTR/MIDR detection 2024-08-13 13:25:34 +00:00
Alyssa Rosenzweig b05329c8df InstructionCountCI: explicitly enable flagm for multiinst
Closes: https://github.com/FEX-Emu/FEX/issues/3949
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-13 09:09:07 -04:00
Ryan Houdek 6f43c8ffac Merge pull request #3947 from alyssarosenzweig/opt/littles
Little opcodedispatcher optimizations
2024-08-13 05:54:47 -07:00
Alyssa Rosenzweig ffac98051f InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-13 08:41:16 -04:00
Alyssa Rosenzweig 40812efaae OpcodeDispatcher: better handle SIB indexing
if we have shift and a constant, we can save an instruction

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-13 08:41:16 -04:00
Alyssa Rosenzweig 3429321d59 OpcodeDispatcher: allow upper garbage for MOVGPR
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-13 08:41:16 -04:00
Alyssa Rosenzweig 6cddd6cbe7 OpcodeDispatcher: allow upper garbage for a2/a3
stores mask.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-13 08:41:16 -04:00
Alyssa Rosenzweig 23d07d7d0c OpcodeDispatcher: fix folding negative offsets for 32-bit
I don't know what I was thinking when I wrote that code. Drop the silly logic
and let ConstProp inline the immediates. This fixes a lot of silly code
generated for 32-bit.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-13 08:41:16 -04:00
Alyssa Rosenzweig 1351575713 InstructionCountCI: add fpemu blocks
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-13 08:41:16 -04:00
Ryan Houdek 8aa7d1a278 Merge pull request #3939 from alyssarosenzweig/opt/cfinv
Invert carry flag internally
2024-08-13 01:05:54 -07:00
Ryan Houdek fb60a8a032 Docs: Update for release FEX-2408 2024-08-12 15:05:37 -07:00
Alyssa Rosenzweig f40bc134df Merge pull request #3945 from Sonicadvance1/config_nonnullable
Config: Little assume non-null check
2024-08-11 15:23:37 -04:00
Ryan Houdek f3811f04bd Config: Little assume non-null check
Removes a simple runtime nullcheck in Config::Layer::Set. Since we never pass a
nullptr to this.
2024-08-11 10:25:53 -07:00
Alyssa Rosenzweig 94bb7eb311 Merge pull request #3937 from Sonicadvance1/fix_script
Scripts: Fix issue in aarch64_fit_native
2024-08-10 20:07:43 -04:00
Alyssa Rosenzweig 3383786205 InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 15:21:08 -04:00
Alyssa Rosenzweig 91f4c54768 OpcodeDispatcher: optimize RDRAND on flagm
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 15:21:08 -04:00
Alyssa Rosenzweig d9c779289c OpcodeDispatcher: simplify RDRAND
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 15:18:00 -04:00
Alyssa Rosenzweig 5631ff4fd5 OpcodeDispatcher: optimize variable shifts
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:43 -04:00
Alyssa Rosenzweig 34301319bf OpcodeDispatcher: optimize IncrementByCarry
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:43 -04:00
Alyssa Rosenzweig 8eac3198b6 OpcodeDispatcher: use carry increment for ADC
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:43 -04:00
Alyssa Rosenzweig 832edd4da3 OpcodeDispatcher: use carry increment for SBC
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:43 -04:00
Alyssa Rosenzweig 5823e74bcd OpcodeDispatcher: use increment carry for atomics
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:43 -04:00
Alyssa Rosenzweig a4545f493e OpcodeDispatcher: add IncrementByCarry helper
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:43 -04:00
Alyssa Rosenzweig 3b8cd44ca4 IR: add NZCVSelectIncrement
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:43 -04:00
Alyssa Rosenzweig f138d7d9b8 OpcodeDispatcher: optimize JA/JNA
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:42 -04:00
Alyssa Rosenzweig 3bb9d44bf5 OpcodeDispatcher: optimize SetCFDirect
generally same # of instructions, but potentially fewer cycles:

old:
        "rmif x4, #63, #nzCv",
        "cfinv"

new:
        "xor x20, x4, #1",
        "rmif x20, #63, #nzCv"

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:42 -04:00
Alyssa Rosenzweig 06e6a1b19e OpcodeDispatcher: optimize logic ops
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:42 -04:00
Alyssa Rosenzweig 256d166126 OpcodeDispatcher: optimize DAA
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:42 -04:00
Alyssa Rosenzweig 630285c589 OpcodeDispatcher: optimize SetPackedRFLAG
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:42 -04:00
Alyssa Rosenzweig 9621eca677 OpcodeDispatcher: fix setrflag masking
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:42 -04:00
Alyssa Rosenzweig 7eee50d929 OpcodeDispatcher: optimize mul/umul
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:42 -04:00
Alyssa Rosenzweig 1dc22e33ae OpcodeDispatcher: inline some flag calculations
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:42 -04:00
Alyssa Rosenzweig 0ef8aaebeb OpcodeDispatcher: optimize BZHI
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:42 -04:00
Alyssa Rosenzweig d17c427e47 OpcodeDispatcher: optimize non-flagm2 comiss
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:42 -04:00
Alyssa Rosenzweig f73fb62c6e OpcodeDispatcher: optimize V(P)TEST
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:42 -04:00
Alyssa Rosenzweig 5976b712ca OpcodeDispatcher: optimize bl*
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:42 -04:00
Alyssa Rosenzweig 63f5e64adb OpcodeDispatcher: optimize add/sub
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:42 -04:00
Alyssa Rosenzweig 34ee3bb8aa OpcodeDispatcher: optimize clc/stc
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:42 -04:00
Alyssa Rosenzweig 79c745929a FEXCore: invert CF internally
Flag day change to the ABI.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:42 -04:00
Alyssa Rosenzweig 8330cc6876 OpcodeDispatcher: defer carry inverts
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:05:21 -04:00
Ryan Houdek 03ca3e7e68 Merge pull request #3934 from bylaws/wow64-b
WOW64: Support the JIT API as used by Windows
2024-08-10 09:07:58 -07:00
Ryan Houdek 2c3e6cbe65 Merge pull request #3932 from bylaws/arm64-callchk
ARM64EC: Install a custom call checker to bypass NTDLL function patches
2024-08-10 09:07:31 -07:00
Alyssa Rosenzweig 16e6163677 OpcodeDispatcher: drop unused NZCVIndexMask
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 11:26:12 -04:00
Alyssa Rosenzweig bbbc0dc9dc OpcodeDispatcher: fix weird formatting
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 11:26:12 -04:00
Alyssa Rosenzweig f9bdf0bd01 Merge pull request #3938 from Sonicadvance1/fix_vpblend_test
unittests: Fixes vpblend unittest
2024-08-10 11:08:03 -04:00
Ryan Houdek 4afc7adb05 unittests: Fixes vpblend unittest
This typo was causing undefined data to be used in the unittest, showed
up in debug builds.
2024-08-10 07:43:25 -07:00
Ryan Houdek 2152d1b2e9 Scripts: Fix issue in aarch64_fit_native
Apparently I messed this up in testing, is now fixed.
2024-08-09 21:04:26 -07:00
Ryan Houdek 0ecfc651b6 Merge pull request #3931 from Sonicadvance1/move_hostfeatures_init
FEXCore: Pass HostFeatures in to CreateNewContext directly
2024-08-09 20:30:17 -07:00
Ryan Houdek 4a3250ddea Merge pull request #3928 from bylaws/winval
InvalidationTracker: Better match Windows code invalidation behaviour
2024-08-09 20:29:57 -07:00
Alyssa Rosenzweig 633f624a69 Merge pull request #3930 from Sonicadvance1/hostfeatures_only_harnessrunner
HostFeatures: Removes feature flags always supported by FEX
2024-08-09 15:12:04 -04:00
Billy Laws ea4004ec9d ARM64EC: Support cooperative suspend
When a suspend request occurs on a running thread, Windows gives the JIT
~10s to pause and NtContinue with the x86 context. Resumption is then
done through BeginSimulation.
2024-08-09 14:34:17 +00:00
Billy Laws a469047e7a FEXCore: Support cooperative suspend on ARM64EC
ARM64EC doesn't have an explicit callback for suspend, instead this task
is delegated to the kernel setting a bool at a JIT-defined memory
location, as such the faulting based approach as was done for WOW64
cannot be used.
2024-08-09 14:34:17 +00:00
Billy Laws 26a8a2717c WOW64: Mark the CPU area context as dirty initially
After thread creation, the WOW64 CPU area context needs to be flushed
into the FEX state before entering the JIT. Wine explicitly calls
BTCpuSetContext to trigger this but Windows doesn't.
2024-08-09 11:57:09 +00:00
Billy Laws 4c0e6d5779 WOW64: Shift down used TLS slots
Fixes a crash on native Windows.
2024-08-09 11:57:09 +00:00
Billy Laws a350ef5d1b WOW64: Match the Windows function protoypes 2024-08-09 11:57:09 +00:00
Billy Laws 2cc0a867a9 ARM64EC: Set the InSimulation TEB flag when inside the JIT
This is used by the kernel to determine whether to perform cooperative
suspend.
2024-08-09 11:52:03 +00:00
Billy Laws 9d9bd750e2 ARM64EC: Install a custom call checker to bypass NTDLL function patches
Some programs will hook the NTDLL exports that FEX depends on, the
regular ARM64EC call checker will detect such patches and invoke the
JIT to run them, which leads to infinite recursion if those same
exports are used during code compilation. Fix this by resolving all
patchable FFSs to their native ARM implementations for all indirect
calls performed by FEX, skipping any x86 patches.
2024-08-09 11:48:18 +00:00
Ryan Houdek 85d1b573ef Merge pull request #3927 from bylaws/winafp
ARM64EC: Set appropriate AFP and SVE256 state on JIT entry/exit
2024-08-08 22:21:23 -07:00
Ryan Houdek 2f8c5b4820 FEXCore: Pass HostFeatures in to CreateNewContext directly
The class constructor for ContextImpl::CPUID requires HostFeatures to be
available at construction time. Pass the host features struct directly
through during construction time instead, which cleans up the interface
slightly and fixes that issue.
2024-08-08 21:02:41 -07:00
Ryan Houdek a1f55f0b0b HostFeatures: Removes feature flags always supported by FEX
These are only missing if using the hostrunner and the CI machine
doesn't support that particular feature. FEX otherwise always supports
these feature flags so they don't need to exist as options.

Just check the feature bit directly in the HostRunner frontend for these
bits.
2024-08-08 19:05:55 -07:00
Ryan Houdek 7b1d9540b7 Merge pull request #3925 from bylaws/arm64ecrt
ARM64EC: Introduce FEX-side CRT and Windows API replacements
2024-08-08 17:57:25 -07:00
Ryan Houdek 1007f874bf Merge pull request #3926 from bylaws/windef
FEXCore: Drop deferred signal handling on Windows
2024-08-08 17:33:44 -07:00
Ryan Houdek 24ea4b7537 Merge pull request #3924 from bylaws/svc
ARM64EC: Handle direct syscall instructions
2024-08-08 17:32:56 -07:00
Billy Laws 93ab57454a InvalidationTracker: Better match Windows code invalidation behaviour
When given a NULL base address, Windows invalidation callbacks will
ignore the given size and invalidate all code.
2024-08-08 12:46:49 +00:00
Mai 4882f10536 Merge pull request #3888 from Sonicadvance1/avx128_optimize_blends
AVX128: Optimize blends
2024-08-07 17:08:19 -04:00
Billy Laws fe43a2bcb2 ARM64EC: Set appropriate AFP and SVE256 state on JIT entry/exit 2024-08-07 18:34:35 +00:00
Billy Laws 6700511cdf FEXCore: Drop deferred signal handling on Windows
The async signal issues this handles do not exist on Windows.
2024-08-07 18:31:48 +00:00
Billy Laws e836639427 ARM64EC: Manually define the ARM64EC linker structures 2024-08-07 15:49:41 +00:00
Billy Laws 7eb3f33162 Update jemalloc submodule 2024-08-07 15:49:41 +00:00
Billy Laws b35a514ea6 CMake: Don't link CMake's set of extra system libraries on Windows
These are already linked in by default with clang, sp having these set
here only served to prevent the -nostdlib compiler option from having
any effect, as CMake will always explicitly include the libs in the
compiler cmdline.
2024-08-07 15:49:41 +00:00
Billy Laws 8f372821f8 Windows: Use the FEX CRT 2024-08-07 15:49:41 +00:00
Billy Laws 6376d06bed Windows: Introduce a minimal Windows API replacement 2024-08-07 15:49:41 +00:00
Billy Laws 1bb293d4be Windows: Pull in some math and formatting functions from Musl 2024-08-07 15:49:41 +00:00
Billy Laws 0e069f1e97 Windows: Introduce a minimal CRT replacement
It is dangerous for FEX to rely on the system CRT as calls can have side
effects that are also visible to the running application, and if patches
are applied to any CRT exports used during compilation the call checker
would trigger a reentry into the JIT to compile the patch and hence
deadlock. Only functions that FEX actively uses are implemented, with
the rest triggering an abort.
2024-08-07 15:49:41 +00:00
Billy Laws 9074b810a9 CMake: Always enable jemalloc for MinGW builds 2024-08-07 15:49:41 +00:00
Mai c5fe8723d3 Merge pull request #3918 from Sonicadvance1/constexpr_config_maps
Config: Converts two LUT maps over linear scan arrays
2024-08-07 10:40:22 -04:00
Ryan Houdek 434bffac33 Merge pull request #3916 from Sonicadvance1/frontend_hostfeatures
FEX: Moves HostFeatures querying to the frontend
2024-08-07 05:33:55 -07:00
Ryan Houdek e84848b16b FEX: Moves HostFeatures querying to the frontend
This moves the CPU feature querying to the frontend. The primary purpose
here is for the wow64 frontend to not require linux-isms for querying
these features. This is required since non-Linux environments don't have
the "CPUID" feature for reading EL1 MSRs in EL0.

Wiring up the remaining wow64 registry querying is left for a future
exercise.

This also technically removes an xbyak requirement from FEXCore for when
building the x86 Test harness runner, but that doesn't really matter for
regular use cases.
2024-08-07 05:26:02 -07:00
Ryan Houdek 69ed39d49e Merge pull request #3892 from Sonicadvance1/optimize_vpermq
AVX128: Optimize all cases of vpermq
2024-08-06 20:07:28 -07:00
Ryan Houdek 230bde6aef InstcountCI: Adds vpermq coverage 2024-08-06 09:08:30 -07:00
Ryan Houdek c24d7aacba unittests/ASM: Adds vpermq test that covers all immediate encodings
To ensure we cover all tests when optimizing.
2024-08-06 09:08:30 -07:00
Ryan Houdek e613876e9d AVX128: Optimize all cases of vpermq
Started by cherry-picking some cases from the variants that appeared when running
Steam, games, AV1 convolve tests, openssl, ffmpeg, libjpeg-turbo,
openh264, libvpx, gemmlowp, libyuv, and dav1d.

Then turned it around and optimized them all since all variants end up
needing to be split in to two halves, that effectively means we need to
have 16 implementations, plus a couple of special cases for duplicated
results.

Fixes #3795
2024-08-06 09:08:30 -07:00
Mai 1473129a8f Merge pull request #3920 from Sonicadvance1/fix_newline_asm
SpinWaitLock: Fixes missing newline in asm
2024-08-06 12:08:14 -04:00
Alyssa Rosenzweig c42808cb70 Merge pull request #3904 from Sonicadvance1/packaging
Scripts: Workaround deprecated parse_version
2024-08-06 11:52:59 -04:00
Ryan Houdek 054c119e2e Config: Converts two LUT maps over linear scan arrays
These two maps used for environment lookup translations were getting
globally initialized and then registers with atexit handlers.

Switch over to a constexpr array and just do linear scans. This plus
short-circuiting the environment loader so it skips all entries that
don't start with `FEX_` has the side benefit of cutting the CPU time to
1/10th the time.

This plus #3917 removes the global static initializers entirely from
this file.
2024-08-06 07:44:08 -07:00
Alyssa Rosenzweig f75bd2f09b Merge pull request #3922 from bylaws/structs
Windows: Pull in additional method and structure definitions from wine
2024-08-06 09:29:03 -04:00
Alyssa Rosenzweig a7424416d9 Merge pull request #3921 from bylaws/reloadf
Arm64Emitter: Reload STATE before SRA fill on ARM64EC
2024-08-06 09:28:23 -04:00
Alyssa Rosenzweig cadb0a2ddb Merge pull request #3923 from bylaws/except
ARM64EC: Improvements to exception flag handling
2024-08-06 09:27:50 -04:00
Alyssa Rosenzweig 2da819c0f3 Merge pull request #3919 from Sonicadvance1/remove_vestigial_vixl_usage
CodeEmitter: Removes vestigial vixl usage
2024-08-06 09:26:24 -04:00
Alyssa Rosenzweig e0c783de74 Merge pull request #3917 from Sonicadvance1/remove_static_vector
Config: Removes a static vector initializer
2024-08-06 09:26:03 -04:00
Billy Laws 6c003fcb9a Windows: Pull in more method/structure definitions from wine 2024-08-05 19:23:18 +00:00
Billy Laws c4faffc0e2 Windows: Add complete NTDLL export definitions
Generated from wine's ntdll.spec
2024-08-05 19:22:12 +00:00
Billy Laws cc2d21f411 WOW64: Resolve the wine unix call dispatcher at runtime 2024-08-05 19:22:12 +00:00
Billy Laws 59686a6c60 ARM64EC: Clear TF in the exception resumption context after a trap
Matches Windows behaviour.
2024-08-05 17:38:45 +00:00
Billy Laws 0ab864da17 ARM64EC: Reset the CPU area JIT state before handling exceptions
An exception in JIT code acts as a transition to ARM64EC code (in
NTDLL for exception handling etc) as such, much like ExitFunction,
InSimulation must be unset. InSyscallCallback is unset for robustness
against exception in the JIT itself.
2024-08-05 17:38:45 +00:00
Billy Laws 3c32271dd0 ARM64EC: Merge EFlags with the current JIT flags state on a ctx sync
Only NZCV and TF are passed through to BeginSimulation as the rest are
lost when converting to a native context and back on the ntdll side. To
prevent thread suspension from wiping out the rest of the flags, only
copy these specific flags into the current JIT EFlags state.
2024-08-05 17:38:45 +00:00
Billy Laws 507a95b817 ARM64EC: Map TF to PSTATE.SS when reconstructing a native context 2024-08-05 17:38:45 +00:00
Billy Laws 9bb9e954c2 ARM64EC: Spill EFlags when reconstructing state from in the JIT 2024-08-05 17:38:45 +00:00
Billy Laws 7f3582bb23 ARM64EC: Only clear the trap flag when handling an exception
Better matches Windows emulator behaviour.
2024-08-05 17:38:45 +00:00
Billy Laws efe15ce336 ARM64EC: Handle direct syscall instructions
Most syscalls on Windows are done by calling into their NTDLL thunks,
however some DRMs parse out their numbers from NTDLL and directly call
them. Support this by redirecting to their entry thunks in the FEX
syscall handler.
2024-08-05 17:35:32 +00:00
Billy Laws 21b0f35ef4 ARM64EC: Populate a LUT mapping NTDLL FFS exports to their native impls
To prevent FEX from redirecting to x86 code when NTDLL exports it calls
into are patched, a custom call checker will be used that checks this LUT
to redirect calls rather than the FFS itself.
2024-08-05 17:35:32 +00:00
Billy Laws ccf332d48e Arm64Emitter: Reload STATE before SRA fill on ARM64EC
While ARM64EC code cannot use x28, it can be cleared by the kernel
when performing syscalls etc so restore it from the TEB to be safe.
2024-08-05 17:31:01 +00:00
Ryan Houdek 802a32ce8a SpinWaitLock: Fixes missing newline in asm
This would cause the atomic load after the wfe to be dropped,
effectively returning stale data.
2024-08-04 06:57:05 -07:00
Ryan Houdek 70c02d5c58 ARM64Emitter: Removes unused vixl CPU object 2024-08-03 22:26:00 -07:00
Ryan Houdek 2e4fb47848 HostFeatures: Read VL ourselves
Instead of calling out to vixl
2024-08-03 22:26:00 -07:00
Ryan Houdek a4d5302369 Arm64: Adds Int helpers
One more vixl step removed.
2024-08-03 21:40:28 -07:00
Ryan Houdek 6ff3c90af3 CodeEmitter: Removes vestigial vixl usage
- IsImmLogical already existed in our CodeEmitter. We just forgot to
  allow nullptr arguments and to use it.
- Adds an equivalent IsImmAddSub helper and uses it

This gets us closer to removing vixl's global initializers from FEXCore.
2024-08-03 21:04:56 -07:00
Ryan Houdek c114279118 Config: Removes a static vector initializer
Saw this vector was getting initialized at runtime, sticking around, and
installing an atexit handler. This is completely unnecessary, just use
the OPT_BASE handler directly to walk the environment variable names.
2024-08-03 19:13:29 -07:00
Ryan Houdek 7ffd3e55d5 Merge pull request #3915 from bylaws/winbase
ARM64EC: Support the JIT API as is used by Windows
2024-08-02 10:56:46 -07:00
Ryan Houdek 201fe6ee23 Merge pull request #3909 from bylaws/ec-bitmap
Directly use the EC code bitmap for determining page arch
2024-08-02 10:55:43 -07:00
Ryan Houdek 83fedd6c8f Merge pull request #3912 from bylaws/addroverride
Don't apply the address-size flag to segment addresses
2024-08-01 18:35:06 -07:00
Ryan Houdek dedf4a93d0 Merge pull request #3913 from bylaws/logcommon
Commonise logging and fallback to a log file for debug output on Windows
2024-08-01 12:06:35 -07:00
Ryan Houdek 1f59f0e226 Merge pull request #3914 from bylaws/wincfg
Config: Search more locations for the config directory on Windows
2024-08-01 12:05:50 -07:00
Ryan Houdek c3c2b6115d Merge pull request #3910 from bylaws/f80
F80: Drop dependency on state stored in TLS
2024-08-01 12:03:41 -07:00
Billy Laws 115fbb5039 ARM64EC: Implement remaining notification callbacks 2024-08-01 12:06:25 +00:00
Billy Laws 27973d5637 ARM64EC: Fix the exception dispatcher stack layout 2024-08-01 12:06:24 +00:00
Billy Laws f121be649d ARM64EC: Match the Windows BT API function prototypes 2024-08-01 12:06:24 +00:00
Billy Laws af9bcb3efd ARM64EC: Avoid syncing uninitialized context members to the JIT state
Windows can sometimes pass in incomplete contexts to BeginSimulation. So
only sync the valid parts specified in ContextFlags.
2024-08-01 12:06:05 +00:00
Billy Laws 4877bb3f19 ARM64EC: Support directly issuing the NtContinue syscall
This is required for handling SMC with the ResetToConsistentState
arguments as used in Windows, as using the NTDLL exported NtContinue
would wipe out any reserved registers in the ARM64EC ABI.

For Windows the syscall numbers are somewhat stable, and the SVC
instruction can be called directly. Since wine doesn't handle that on
ARM64, hardcode the system call number and manually call into wine
dispatcher. Once wine gains proper syscall thunks, those can be
parsed to get the number and the hardcoding dropped.
2024-08-01 12:06:05 +00:00
Billy Laws e2583249c6 ARM64EC: Allocate the emulator stack ourselves
Actual Windows does not allocate it for us.
2024-08-01 12:06:05 +00:00
Billy Laws f48071dcbe ARM64EC: Switch to the emulator stack in BeginSimulation
Windows calls this function on the guest stack for some reason.
2024-08-01 12:06:05 +00:00
Billy Laws fa72be2ec5 Windows: Don't warn for unknown CPU features
This happens regularly as wine/games will scan all features from 0 to 64.
2024-08-01 12:06:05 +00:00
Billy Laws b7ff6dd9c6 Windows: Disable logging if SilentLog is enabled 2024-08-01 12:04:59 +00:00
Billy Laws cb6d60aa87 Config: Search more locations for the config directory on Windows 2024-08-01 11:48:40 +00:00
Billy Laws 80c9a43eef Windows: Fallback to a log file for debug output on Windows
OutputDebugString etc are exception based and thus don't really work for
FEX's needs as often times logs can happen in places where exceptions
cannot be thrown.
2024-08-01 11:45:07 +00:00
Billy Laws 05155778d4 Windows: Commonise logging code 2024-08-01 11:45:07 +00:00
Tony Wasserka 49b8dae189 Merge pull request #3908 from bylaws/pdb
CMake: Add option to build PDB debug info instead of DWARF
2024-08-01 10:01:04 +02:00
Ryan Houdek 3e59fc0a8c Merge pull request #3911 from bylaws/x80bug
x87StackOptimizationPass: Default initialise StackMemberInfo members
2024-07-31 22:57:16 -07:00
Ryan Houdek f98c010854 Merge pull request #3907 from bylaws/ec-mema
AllocatorHooks: Correct memory API usage on Windows
2024-07-31 17:48:13 -07:00
Ryan Houdek 10ee963f52 Merge pull request #3906 from bylaws/ec-sysbi
FEXCore: Add a generic spill/fill-all syscall ABI and use for Windows
2024-07-31 17:46:55 -07:00
Billy Laws af7462ee6a unittests: Add test using the address-override flag with segment addressing 2024-07-31 20:04:30 +01:00
Billy Laws be4777110c OpcodeDispatcher: Don't apply the address-size flag to segment addresses
The address-size flag only applies to the offset from the segment base,
rather than the segment address itself.
2024-07-31 18:14:32 +00:00
Billy Laws 696503680a F80: Drop dependency on state stored in TLS
Windows cannot support the implicit TLS as was used prior, so introduce
a state structure and pass it in to functions where necessary.
2024-07-31 18:51:42 +01:00
Billy Laws dd4d3bcf38 AllocatorHooks: Correct memory API usage on Windows
These issues end up being tolerated by wine but not actual windows.
2024-07-31 18:36:50 +01:00
Billy Laws f26bb6bf53 x87StackOptimizationPass: Default initialise StackMemberInfo members
Not doing so is UB.
2024-07-31 17:30:29 +00:00
Billy Laws 2c4fd79304 FEXCore: Add a generic spill/fill-all syscall ABI and use for Windows
Also drop the legacy hangover ABI as it has no users.
2024-07-31 17:25:59 +00:00
Billy Laws 910ec4aadd Dispatcher: Directly use the EC code bitmap for determining page arch
The prior approach using the L2 cache was flawed as it assumed L2
page entries had a 1-1 correspondence with actual pages. While the L2
cache could be extended to handle aliases with EC, this could lead to
thrashing etc. The cost of a lookup in the actual EC code bitmap is
cheap enough to perform every time considering the infrequency of calls
to ARM64EC code when compare to X86 L2 hits.
2024-07-31 17:24:50 +00:00
Billy Laws fb7275b3d8 Revert: "LookupCache: Track ARM64EC page state in the code cache"
This reverts the commit 526e3e654f.
2024-07-31 17:24:50 +00:00
Billy Laws 51b4bfc6a6 FEXCore: Move ARM64EC TEB offset constants to Arm64Emitter
These need to be used from outside the dispatcher, and there are already
similar defines for EC registers in the emitter header.
2024-07-31 17:24:50 +00:00
Billy Laws 9f7bc94f9d CMake: Add option to build PDB debug info instead of DWARF 2024-07-31 17:23:24 +00:00
Ryan Houdek 069e2ce62a Merge pull request #3905 from pmatos/FixWarn
Fix nasm warning in Rounding.asm
2024-07-31 08:06:12 -07:00
Paulo Matos 5bbbced1bd Fix nasm warning in Rounding.asm 2024-07-31 16:23:39 +02:00
Alyssa Rosenzweig 941fd9c6ea Merge pull request #3901 from pmatos/TopUsage
Reuse Top in ReconstructFSW_Helper
2024-07-31 08:01:05 -04:00
Paulo Matos 3332220d06 instcountci: Intersperse flag retrieval and FSW insertion 2024-07-31 12:05:17 +02:00
Paulo Matos 5933a59c09 Intersperse flag retrieval and FSW insertion 2024-07-31 12:04:54 +02:00
Paulo Matos aee8c9def2 instcountci: Reuse Top in ReconstructFSW_Helper 2024-07-31 11:57:19 +02:00
Paulo Matos c2136272bf Reuse Top in ReconstructFSW_Helper
This is a non functional. Instead of fetching top again, we use the one
obtained through the fast path calculation.
2024-07-31 11:56:14 +02:00
Ryan Houdek dd26b0c879 Merge pull request #3903 from Sonicadvance1/v6.10_syscalls
Syscalls: Updates for v6.10
2024-07-31 02:22:26 -07:00
Ryan Houdek 3d9114b74b Merge pull request #3902 from Sonicadvance1/man_page_fix
man: Fixes newline issue with strenum
2024-07-31 02:21:47 -07:00
Ryan Houdek 3f3e937967 man: Fixes newline issue with strenum
In the environment section this was causing the next environment
variable line to be merged with the strenum options

Also makes it so strenum options doesn't have a spurious comma at the
end of the list.
2024-07-31 02:02:05 -07:00
Ryan Houdek 4beb29141b Scripts: Workaround deprecated parse_version
Different approach from #3579

Instead of completely ddropping the deprecated path, support the new
path and the old path using python try-except import exceptions.

This allows us to continue using old packages in CI, while supporting
the future API once pkg_resources gets deprecated and removed. Best of
both worlds.
2024-07-31 01:57:31 -07:00
Paulo Matos 9a8e7eaace instcountci: Add instcountci for fnstsw from fast path 2024-07-31 09:59:32 +02:00
Paulo Matos 4227e012aa Add instcountci for fnstsw from fast path 2024-07-31 09:59:28 +02:00
Ryan Houdek 40bbcb7061 Syscalls: Updates for v6.10
Only mseal was added and can be a simple passthrough for us. Which is
nice.
2024-07-30 19:08:27 -07:00
Ryan Houdek 67663b812e Linux: Update syscall defines for v6.10 2024-07-30 19:01:52 -07:00
Ryan Houdek d24d0a95a0 Merge pull request #3894 from pmatos/RoundingModeTests
ASM Tests: X87 Rounding modes
2024-07-29 23:26:52 -07:00
Ryan Houdek 87fbcf754d Vector: Optimize pblendw
Using a brute force solver to add in more optimized code paths

- Adds 12 single VInsElement implementations
- Adds 4 two IR operation implementations

Not adding any of the two or three IR operation implementations that use
VInsElement because SRA interacts badly and becomes worse than the VTBX
implementation.
2024-07-27 19:25:51 -07:00
Ryan Houdek 403fd62b34 Merge pull request #3890 from Sonicadvance1/refactor_frontend_threadmanager
FEXCore: Removes ThreadManager
2024-07-26 13:27:43 -07:00
Ryan Houdek d92b6a9ac4 Merge pull request #3898 from alyssarosenzweig/ir/creative-refs
OpcodeDispatcher/X87: use less creative Refs
2024-07-26 13:26:43 -07:00
Ryan Houdek c2092bfed0 Merge pull request #3893 from pmatos/FNINITFix
Fix call to FNINITF64 and refactor
2024-07-26 13:25:49 -07:00
Ryan Houdek a6cf7fa508 Merge pull request #3896 from pmatos/CTestSkip
Test running scripts tell ctest of skipped tests
2024-07-26 13:25:28 -07:00
Alyssa Rosenzweig 5ff09f5091 OpcodeDispatcher/X87: use less creative Refs
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-26 14:30:42 -04:00
Paulo Matos 9af7ee6bd2 instcountci: Fix call to FNINITF64 and refactor 2024-07-26 14:56:10 +02:00
Paulo Matos d1e36f264f Fix call to FNINITF64 and refactor 2024-07-26 14:56:06 +02:00
Paulo Matos b1ec50c7c2 Test running scripts tell ctest of skipped tests
CMake sets 125 as the skipped test exit code that the scripts use.
2024-07-26 14:04:54 +02:00
Mai 93eead243f Merge pull request #3864 from Sonicadvance1/threads_atexit_remove
Threads: Setup the stack tracker to not need global initialization
2024-07-26 06:38:29 -04:00
Paulo Matos ceac38a6ac ASM Tests: X87 Rounding modes 2024-07-26 10:07:58 +02:00
Ryan Houdek 3b2e657fd4 FEXCore: Removes ThreadManager
This has been leaked state to FEXCore for quite a while. FEXCore never
actually needed this information, moves the bits to the frontend that
are necessary.

Minor behaviour change that `RunUntilExit` now just assumes the primary
thread is using it. This behaviour is on the chopping block to get
removed next anyway.
2024-07-25 14:54:10 -07:00
Ryan Houdek 380ba0a014 Merge pull request #3889 from Sonicadvance1/refactor_frontend_exithandler
FEXCore: Refactor ExitHandler slightly
2024-07-25 14:53:25 -07:00
Mai 1fe497d1dd Merge pull request #3891 from Sonicadvance1/remove_cpubackendfeatures
FEXCore: Removes CPUBackendFeatures
2024-07-24 20:53:10 -04:00
Ryan Houdek 7816b150d0 FEXCore: Removes CPUBackendFeatures
We were only ever hardcoding true for TBL2 and Flags now. Get rid of it.
2024-07-24 17:19:30 -07:00
Ryan Houdek ce8bc9d25c FEXCore: Refactor ExitHandler slightly
Instead of passing the TID back to the exit handler, just pass the whole
thread object. This will allow some cleanups with the frontend thread
tracking soon

NFC
2024-07-24 14:39:56 -07:00
Ryan Houdek 4634688aca InstcountCI: Update for AVX128 blends 2024-07-23 19:24:19 -07:00
Ryan Houdek dd3e3ed189 unittests/ASM: Implements a vpblendw test
Runs through all immediate encodings for vpblendw and crcs the results
to ensure correct behaviour. This was just a concern because of the typo
in documentation. But it is also good to have.
2024-07-23 19:24:19 -07:00
Ryan Houdek f8ef6feff9 AVX128: Optimize blends
Optimizes the AVX128 blends by reusing the prior SSE4.1 implementation.
Only difference is the destination register isn't reused as a source
register.

One confusing thing is that Felix Cloutier's documentation has a typo on
the 256-bit VPBLENDW instruction where it had the top 128-bit lane
reusing the destination instead of sources. So I wrote a unittest to
ensure correctness.

Fixes #3796
2024-07-23 19:24:19 -07:00
Ryan Houdek 9201ac5a6b Merge pull request #3882 from Sonicadvance1/scalar_afp_fma
AVX128: Implement support for scalar FMA with AFP
2024-07-22 13:19:59 -07:00
Ryan Houdek 8ebf049fb9 InstcountCI: Update for Scalar FMA with AFP 2024-07-22 12:58:20 -07:00
Ryan Houdek 3c5b59d985 AVX128: Implement support for scalar FMA with AFP
Now that I have AFP supporting hardware I felt better implementing this
since I can run unit tests.

Fixes #3793
2024-07-22 12:58:19 -07:00
Ryan Houdek 6b91e0cb0e Merge pull request #3887 from alyssarosenzweig/ir/prefix
json_ir_generator: stop prefixing arguments
2024-07-22 12:57:09 -07:00
Alyssa Rosenzweig 587b924de9 json_ir_generator: stop prefixing arguments
stop prefixing the arguments when we generate allocate ops (in particular), this
is more convenient and simpler. in exchange we need to prefix Op to avoid a
collision on fcmpscalarinsert which has an argument named Op, but that's a local
change at least.

came up when experimenting with new IR, but I think this is probably a win by
itself.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-22 13:50:21 -04:00
Tony Wasserka d507f4c9b1 Merge pull request #3547 from pmatos/wip_x87_stack
x87 Stack Optimization
2024-07-22 14:42:02 +02:00
Paulo Matos 39bc2a82c1 instcountci: X87 Pass and refactoring 2024-07-22 08:50:01 +02:00
Paulo Matos 774325dcf2 Tests: X87 Refactoring and Pass 2024-07-22 08:44:45 +02:00
Paulo Matos a1378f94ce X87 Code Refactoring and Optimization Pass 2024-07-22 08:44:45 +02:00
Ryan Houdek 77ec950ff2 Merge pull request #3885 from alyssarosenzweig/opt/zero-flag
Optimize zero x87 flags
2024-07-21 13:06:25 -07:00
Alyssa Rosenzweig 592d6cc43f InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-21 15:50:10 -04:00
Alyssa Rosenzweig 610caf8529 ConstProp: treat StoreContext as zeroable
todo: FPR equivalent.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-21 15:49:09 -04:00
Alyssa Rosenzweig d20b46e46f IR: drop LoadFlag/StoreFlag ops
pointless, we can just load/store the context now.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-21 15:49:09 -04:00
Alyssa Rosenzweig 4094aa1b9a DeadStoreElimination: drop flag handling
now that we do everything via NZCV, this is mostly vestigial. DF/x87 flags are
sufficiently rare to be "don't care"s here, and we don't even have multiblock
enabled yet!

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-21 15:49:08 -04:00
Ryan Houdek f8c6baae97 Merge pull request #3883 from Sonicadvance1/implement_daz
Arm64: Implements support for DAZ using AFP.FIZ
2024-07-21 10:03:34 -07:00
Ryan Houdek 5c9bb6594c Merge pull request #3884 from Sonicadvance1/remove_vex_telem
Telemetry: Remove VEX flag
2024-07-20 19:44:28 -07:00
Ryan Houdek 56df57e980 InstcountCI: Update 2024-07-20 17:26:27 -07:00
Ryan Houdek f7b4d25803 Telemetry: Remove VEX flag
This is no longer necessary and it also no longer provides us any useful
information. Since we expose the AVX CPUID flag, basically everything
uses VEX encoding now, so it is basically always set.
2024-07-20 17:24:00 -07:00
Ryan Houdek 4fffe68f81 InstcountCI: Update 2024-07-20 15:57:01 -07:00
Ryan Houdek 95b15d788b Arm64: Fix filling static registers
Some locations could end up with SRA registers that only spilled one
register.
Allow passing in temporaries from the call site.
Fixes rpid and syscalls asserting.
2024-07-20 15:57:01 -07:00
Ryan Houdek ae9312bdab unittests: Implements a DAZ test
Specifically does a vector add with and without DAZ enabled and ensures
the value is different when the source values contain a denormal.
2024-07-20 15:34:54 -07:00
Ryan Houdek b78da2e5ad Arm64: Implements support for DAZ using AFP.FIZ
When AFP is supported then we can actually support DAZ. This might also
fix the audio corruption in Animal Well but I can't test it until Steam
is running on Oryon. Requires a bit of plumbing for MXCSR which we were
hacking around before but now we actually want to store the value.

Fixes #3856
2024-07-20 15:34:54 -07:00
Ryan Houdek 54fc8cb0bd TestHarnessRunner: Support querying AFP for features
Also fixes desync of flags
2024-07-20 15:34:54 -07:00
Ryan Houdek 228009c283 Merge pull request #3881 from bylaws/race
FixedSizePooledAllocation: Fix a race when unclaiming disowned buffers
2024-07-20 12:35:35 -07:00
Billy Laws 1f878ce4cd FixedSizePooledAllocation: Fix a race when unclaiming disowned buffers
A disowned buffer could be unclaimed or claimed by a different thread in
the time between the !IsFree check and locking the allocation mutex.
Fix this and prevent such errors in the future by always checking
ownership with the allocator locked before attempting to unclaim
buffers.
2024-07-20 00:12:52 +00:00
Alyssa Rosenzweig e4b7a65a49 Merge pull request #3880 from pmatos/InstCountMemcpy
Add x87 memcpy instcountci tests
2024-07-19 08:53:23 -04:00
Paulo Matos c77a707dbe Add x87 memcpy instcountci tests 2024-07-19 09:09:34 +02:00
Ryan Houdek f81fc4e4f0 Merge pull request #3866 from Sonicadvance1/ArgumentLoader_atexit_remove
ArgumentLoader: Removes static fextl::vector usage
2024-07-18 13:12:03 -07:00
Ryan Houdek d385e496d3 Merge pull request #3879 from Sonicadvance1/cpuid_leafs
EmulatedFiles: Adds a few leaf CPUID flags
2024-07-18 13:10:21 -07:00
Mai f8c4c543e3 Merge pull request #3871 from Sonicadvance1/improve_vpshufd_vpermilps
AVX128: Improve VPERMILPS/PD and VPSHUFD
2024-07-18 15:58:47 -04:00
Ryan Houdek d2f903ae55 EmulatedFiles: Adds a few leaf CPUID flags
We support leaf functions now, so add the few that were calling for it.
We will be gaining support for the xsave ones relatively soon, so its
good to have them supported.

Also deletes a couple of cdt/cqm things that aren't exposed and we won't
be supporting.
2024-07-18 07:02:49 -07:00
Ryan Houdek d1249ec5cf Merge pull request #3878 from neobrain/refactor_fix_format_oops
EmulatedFiles: Fix bad formatting
2024-07-18 06:44:44 -07:00
Tony Wasserka bf9a6d763c EmulatedFiles: Fix bad formatting 2024-07-18 15:06:57 +02:00
Ryan Houdek 0b829d2c46 unittests: Adds a test for full pshufd imm coverage 2024-07-18 04:13:03 -07:00
Ryan Houdek bddb533fa0 InstcountCI: Add some more of the cases 2024-07-18 04:13:03 -07:00
Ryan Houdek 1c35eeffeb Vector: Optimize PSHUFD with brute force search
With a brute force search of methods between 1-3 instructions we cover a
lot more cases more optimally.

There's definitely still more cases (and probably some that can reduce
from 3 instruction to 2), but covering 44 cases is a pretty good margin
already.
2024-07-18 04:10:58 -07:00
Ryan Houdek c7254e31ed InstcountCI: Update for VPERM/VPSHUFD improvements 2024-07-18 04:10:58 -07:00
Ryan Houdek b0bd8a62a2 AVX128: Improve VPERMILPS/PD and VPSHUFD
VPSHUFD and VPERMILPS are aliases of each other.

Reuses the implementation path from the PSHUFD implementation which has
a few swizzles and then a table lookup.

VPERMILPD is a very simple swizzle per 128-bit lane.

Fixes #3797
Fixes #3784
2024-07-18 04:10:58 -07:00
Ryan Houdek 17a55fbb39 Merge pull request #3876 from alyssarosenzweig/json/x87
Autogenerate LoweredX87() query, misc json_ir_generator cleanup in the area
2024-07-18 04:10:21 -07:00
Alyssa Rosenzweig dedec83881 json_ir_generator: autoderive array names
these are purely internal.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-17 15:32:51 -04:00
Alyssa Rosenzweig 9fd5c73633 json_ir_generator: generate IsLoweredX87 helper
X87 pass will use this query.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-17 15:31:46 -04:00
Alyssa Rosenzweig bdb890a8b0 json_ir_generator: rename X87 -> LoweredX87
to reflect its actual meaning

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-17 15:31:14 -04:00
Alyssa Rosenzweig 5043d09771 json_ir_generator: use textwrap.dedent, f-string
for size

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-17 15:28:59 -04:00
Ryan Houdek da51169ba9 Merge pull request #3875 from alyssarosenzweig/ir/gethostflag
IR: garbage collect premature F80Cmp optimizations
2024-07-17 03:05:48 -07:00
Ryan Houdek f72cee480f Merge pull request #3874 from alyssarosenzweig/opt/reconstructftw
X87: save uop in ReconstructFTW
2024-07-17 03:05:37 -07:00
Alyssa Rosenzweig 7546160811 InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-16 14:53:58 -04:00
Alyssa Rosenzweig e7d5a01c5f IR: remove F80Cmp flags
nothing is optimizing around this, it's just adding pointless complexity. if we
want to actually optimize F80Cmp, the right way would be to lift the
implementation into the OpcodeDispatcher or JIT. it wouldn't be terribly
difficult. This kludge doesn't get us closer there.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-16 14:53:58 -04:00
Alyssa Rosenzweig 0c3a8d0bc8 IR: remove GetHostFlag
it doesn't get host flags, it's just an extra Bfe used in x87. pointless and
confusing!

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-16 14:44:34 -04:00
Alyssa Rosenzweig 19e58cac62 InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-16 13:54:28 -04:00
Alyssa Rosenzweig c4ba7eee87 X87: save uop in ReconstructFTW
noticed while reviewing Paulo's work

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-16 13:54:09 -04:00
Ryan Houdek 09c4a5594a Merge pull request #3870 from Sonicadvance1/enable_more_tests
github: Vixl simulator enable more asm tests
2024-07-16 07:23:22 -07:00
Alyssa Rosenzweig d204155661 Merge pull request #3872 from pmatos/X87AutoMarking
X87 Stack Ops Auto-marking
2024-07-16 09:45:42 -04:00
Tony Wasserka 924b8c10a9 Merge pull request #3873 from pmatos/UnusedFunction
Remove unused function MmapOverride
2024-07-16 12:30:04 +02:00
Paulo Matos 9017cd14c8 Remove unused function MmapOverride 2024-07-16 11:16:07 +02:00
Paulo Matos 8d89adef2e Add IR stack operations
These IR operations deal implicitly with the x87 stack and are removed
by the x87 stack optimization pass.
2024-07-16 09:07:35 +02:00
Alyssa Rosenzweig 6615b55c12 json_ir_generator: call RecordX87Use when generating ops
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-16 09:07:35 +02:00
Alyssa Rosenzweig 66865dd177 json_ir_generator: alias X87 to !JITDispatch
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-16 09:07:35 +02:00
Alyssa Rosenzweig 1e709d1150 OpcodeDispatcher: add RecordX87 helper
calls will be generated.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-16 09:07:35 +02:00
Alyssa Rosenzweig 476ee0cd7d IR: track whether x87 is used in header
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-16 09:07:35 +02:00
Ryan Houdek 6df51a57b3 Merge pull request #3868 from bylaws/arm64ec-oldnew
ARM64EC frontend
2024-07-15 09:54:56 -07:00
Ryan Houdek e65545a537 Merge pull request #3867 from pmatos/NoDisableTests
Remove Disabled_Tests file
2024-07-15 09:54:08 -07:00
Ryan Houdek b8e864ffdf Merge pull request #3865 from Sonicadvance1/telemetry_atexit
Telemetry: Change how visibility of telemetry values work
2024-07-15 09:53:37 -07:00
Paulo Matos ed87c01470 Simplify Disabled_Tests and remove pr57275 from failures
Disabled_Tests was mostly a copy of Known_Failures. Leave only the race
condition on SIGPROF test (mcount_pic.c).

Also remove pr57275.c from known failures. It passes now that we support
 AVX.

Also if a test is disabled, just skip it
2024-07-13 07:08:02 +02:00
Ryan Houdek 6c21b86a8f github: Vixl simulator enable more asm tests
We were only running SVE256 and SVE128 with AVX disabled.

Enable asm tests with SVE256, SVE128, and ASIMD, all running with AVX
enabled to hit all the tests.
2024-07-12 20:39:01 -07:00
Ryan Houdek d79b7fcc49 Merge pull request #3808 from alyssarosenzweig/rclse/3
Try to delete RCLSE again
2024-07-12 20:38:06 -07:00
Ryan Houdek b9a6caea8d Merge pull request #3844 from Sonicadvance1/fix_vmovq
AVX128: Fixes vmovq loading too much data
2024-07-12 17:07:32 -07:00
Ryan Houdek 9688f5e17d Merge pull request #3863 from Sonicadvance1/remove_static_ioctl_handlers
Ioctl32: Removes static fextl::vector in ioctlemulation
2024-07-12 17:06:16 -07:00
Billy Laws f6f8d26426 Update jemalloc submodule 2024-07-12 19:24:13 +00:00
Billy Laws dba0a1d09e ARM64EC: Initialize x86 control registers on thread start 2024-07-12 18:51:31 +00:00
Billy Laws af3145674e ARM64EC: Fixup exception information for faulting x86 instructions
FEX emulates faulting instructions (e.g. ud2 or int 2d) by jumping to
the dispatcher and filling out a structure with fault details in the
thread context. Parse this out into a windows exception record structure
so the correct fault information can be seen by the guest.
2024-07-12 18:51:31 +00:00
Billy Laws 3c19e634b3 ARM64EC: Rethrow exceptions from within the JIT
As the exception dispatcher is initially invoked on the emulator stack,
control needs to be transferred to the dispatcher on the guest stack
after recovering the x86 RSP to allow for invoking x86 exception
handlers.
2024-07-12 18:41:20 +00:00
Billy Laws f964a5187e ARM64EC: Implement BeginSimulation
This is used by the kernel (or UNIX side of ntdll in wine) to jump into
x86 code with the given context as is necessary when e.g. returning from
an exception.
2024-07-12 18:41:13 +00:00
Billy Laws 8e0fdfc325 ARM64EC: Add a helper to lookup the redirected address of an export
FEX is unable to deal with reentrant compilation of any x64 hotpatches
so they need to be ignored by bypassing FFSs and calling directly into
the native target.
2024-07-12 18:41:08 +00:00
Billy Laws 839f9ecd3b Windows: Add ARM64EC image structures 2024-07-12 18:41:06 +00:00
Billy Laws 95fc69b628 ARM64EC: Handle SMC 2024-07-12 18:41:02 +00:00
Billy Laws b9da95838a ARM64EC: Handle unaligned atomic accesses 2024-07-12 18:40:43 +00:00
Billy Laws 1059279d5d ARM64EC: Handle calls into ARM64EC code with an 8-byte-aligned SP
ARM64 requires that SP is always 16-byte aligned for memory accesses,
but ARM64EC shares the SP between x64 code and ARM64 code, the former
of which doesn't enforce such a restriction. This causes crashes in
programs such as HITMAN 3 that don't correctly follow the Windows ABI
and call into system library functions with SP only 8-byte-aligned.
Fixup stack alignment in such cases by leaving the 8-byte return
address on the stack and returning to a lone 'ret' instruction instead.
2024-07-12 18:30:04 +00:00
Billy Laws 5dc85307a6 Windows: Introduce an initial ARM64EC frontend
This allows for running x64 applications under wine without having to run all
of wine under FEX. The JIT is invoked when ARM64EC code performs an indirect
branch to x64 code, and left whenever the x64 code calls into ARM64EC
code.
2024-07-12 18:07:50 +00:00
Billy Laws 549e06aade CMake: Enable assembly source file support 2024-07-12 18:01:22 +00:00
Billy Laws 3b189f6d7d WOW64: Install into lib
This convention is used by most other projects.
2024-07-12 18:01:22 +00:00
Ryan Houdek a5d3692b53 ArgumentLoader: Removes static fextl::vector usage
Removes a global initializer and atexit registration

Ownership of this data has always been the frontend and the config
system, we just used these static vectors as a side-channel.
2024-07-12 04:48:22 -07:00
Ryan Houdek 97a68cb643 Telemetry: Change how visibility of telemetry values work
Removes global initializer for telemetry values since their address is
visible and PIC relative code loading handles the address fetching for
us.
2024-07-12 03:18:23 -07:00
Ryan Houdek d1b5dfd4b1 Threads: Setup the stack tracker to not need global initialization
Also removes the atexit handler installation
This now gets tracked by an object owned by FEXLoader (and shared with
the pthreads interface)
2024-07-12 03:00:20 -07:00
Ryan Houdek 6cdaea680d Ioctl32: Removes static fextl::vector in ioctlemulation
Removes a global static initializer for the vector and its atexit
handler.

This handler array can be consteval similar to the x86 tables so it can
be generated entirely at compile time.
2024-07-12 02:05:32 -07:00
Ryan Houdek 870e395ac4 Merge pull request #3862 from Sonicadvance1/remove_atexit_logman
LogManager: Removes fextl::vector usage
2024-07-12 02:05:02 -07:00
Ryan Houdek 04592f82f5 Merge pull request #3861 from Sonicadvance1/remove_atexit_vdso
VDSO: Stop using a vector for a static
2024-07-12 02:04:25 -07:00
Ryan Houdek 19e849283f Merge pull request #3860 from Sonicadvance1/force_noinline
OpcodeDispatcher: Force noinline for the function call in the Bind helper
2024-07-12 00:14:04 -07:00
Ryan Houdek b6e1469cd4 Merge pull request #3847 from pmatos/CoverageSupport
Enable coverage configuration for FEX
2024-07-12 00:02:08 -07:00
Ryan Houdek 5ef0db994d VDSO: Stop using a vector for a static
This causes a global initializer that registers an atexit handler.

Be smarter, use an std::array and pass its data around using a span
instead.

Removes the global initializer and removes the atexit installation
2024-07-11 23:53:57 -07:00
Ryan Houdek b523407a3e LogManager: Removes fextl::vector usage
We never use more than one logging method at a time so this was
overengineered for what it is doing.

Instead only allow one handler for messages and throw messages each
which just is a pointer.

Removes a global initializer and an atexit handler being installed
2024-07-11 22:51:56 -07:00
Ryan Houdek 8021dc10a1 OpcodeDispatcher: Force noinline for the function call in the Bind helper
Clang was inlining a few of the functions it was calling. So force it
never to inline since this is supports to be a little shim trampoline
only.
2024-07-11 19:00:42 -07:00
Ryan Houdek 7e8d734e43 AVX256: Initial fixes just to get my unittest working
This is the initial split to decouple AVX256 composed operations from
their MMX/SSE counterparts. This is to work around the subtle
differences with AVX/SSE zext/insert behaviour.
2024-07-11 18:43:31 -07:00
Ryan Houdek 3d90d1ab4f InstcountCI: Update for vmovq fix 2024-07-11 18:34:06 -07:00
Ryan Houdek 3c7318d7c8 AVX128: Fixes vmovq loading too much data
This was doing a 128-bit load from memory and then a 64-bit zero extend
which looked like a spurious move but it was trying to match the
behaviour of vmovq where it needed the zero extend.

Also adds a unit test to ensure that we aren't loading too much data by
loading right up against a page boundary.

Fixes #3787
2024-07-11 18:34:05 -07:00
Ryan Houdek fc0b233046 Merge pull request #3859 from neobrain/refactor_opdispatch_templates
OpcodeDispatcher: Replace hand-written wrapper templates with a generic utility
2024-07-11 18:18:23 -07:00
Mai e25918d846 Merge pull request #3858 from Sonicadvance1/implement_nt_load
Implement support for SSE4.1/AVX NT loads
2024-07-11 14:22:41 -04:00
Alyssa Rosenzweig d78b0ea435 InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-11 13:21:14 -04:00
Alyssa Rosenzweig 3a334c4585 Reapply "IR: drop RCLSE"
This reverts commit 78aee4d96e.
2024-07-11 13:21:14 -04:00
Alyssa Rosenzweig 8dae4bcd44 OpcodeDispatcher: drop stale comment
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-11 13:21:14 -04:00
Alyssa Rosenzweig 294f10fdd0 OpcodeDispatcher: reg cache mmx
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-11 13:21:14 -04:00
Tony Wasserka b9829ed316 OpcodeDispatcher: Replace even more hand-written wrapper templates 2024-07-11 16:19:15 +02:00
Tony Wasserka 4ccec17676 OpcodeDispatcher: Replace more hand-written wrapper templates 2024-07-11 16:19:15 +02:00
Tony Wasserka f45082043b OpcodeDispatcher: Replace hand-written wrapper templates with a generic utility 2024-07-11 16:19:14 +02:00
Tony Wasserka 3222f13dde Fix comment formatting 2024-07-11 16:19:14 +02:00
Mai b282620a48 Merge pull request #3857 from Sonicadvance1/sve_bitperm
Arm64: Implement support for SVE bitperm
2024-07-11 05:05:41 -04:00
Ryan Houdek 3ff1ff8f74 InstcountCI: Update for svebitperm 2024-07-11 01:46:35 -07:00
Ryan Houdek e24b01b6cb Arm64: Implement support for SVE bitperm 2024-07-11 01:46:35 -07:00
Tony Wasserka 9a8694c2f3 Merge pull request #3853 from neobrain/refactor_warn_fixes
Fix all the warnings
2024-07-11 10:12:41 +02:00
Tony Wasserka 070a9148aa Merge pull request #3852 from neobrain/refactor_opdispatch_codesize
OpcodeDispatcher: Avoid template monomorphization to reduce FEXLoader binary size
2024-07-11 09:58:49 +02:00
Tony Wasserka f19fe3b6f3 Fix warning about an expression with side effects being passed to __builtin_assume
LOGMAN_THROW_AA_FMT has no benefit over LOGMAN_THROW_A_FMT here, so just use
the latter.
2024-07-11 09:54:31 +02:00
Tony Wasserka 8d2b15665d Fix unused-variable warnings 2024-07-11 09:54:30 +02:00
Tony Wasserka 4dec8f22f8 Fix packed-non-pod warnings 2024-07-11 09:54:30 +02:00
Tony Wasserka a39b3aca78 Fix invalid-offsetof warnings due to JsonAllocator not being standard layout
Inheritance can be used here instead, which allows the JsonAllocator to be
reconstructed using a downcast.
2024-07-11 09:54:30 +02:00
Tony Wasserka 5dc4ab062d Fix invalid-offsetof warnings due to InternalThreadState not being standard layout
See https://github.com/llvm/llvm-project/issues/53021 for more information
about unique_ptr turning non-standard-layout.
2024-07-11 09:54:30 +02:00
Ryan Houdek 31f82c1d96 InstcountCI: Update for SVE NT load support 2024-07-10 23:07:58 -07:00
Ryan Houdek 548fd9daf8 OpcodeDispatcher: Implement support for SSE4.1 NT load 2024-07-10 23:07:37 -07:00
Ryan Houdek f831f5a0e1 AVX128: Implement support for NT Load 2024-07-10 23:07:14 -07:00
Ryan Houdek 4c21aa2604 Arm64: Implement support for NT Loads with ASIMD fallback 2024-07-10 23:06:46 -07:00
Ryan Houdek c9efb75714 CodeEmitter: Implement support for SVE NT loads 2024-07-10 23:06:19 -07:00
Ryan Houdek 5e56bdc0fd InstcountCI: Add support for SVE bitperm 2024-07-10 21:48:37 -07:00
Ryan Houdek 3554d5c2f7 HostFeatures: Check for SVE bit permute extension 2024-07-10 21:45:07 -07:00
Mai 5fe405e1fb Merge pull request #3855 from neobrain/fix_aotir_uniqueptr
AOTIR: Change std::unique_ptr to fextl::unique_ptr
2024-07-10 17:04:12 -04:00
Tony Wasserka 8381d44bbd Merge pull request #3854 from neobrain/fix_default_delete
fextl: Properly handle nullptr arguments in fextl::default_delete
2024-07-10 23:00:52 +02:00
Tony Wasserka 56bb3744a5 AOTIR: Change std::unique_ptr to fextl::unique_ptr 2024-07-10 19:34:24 +02:00
Tony Wasserka 470b435afd fextl: Properly handle nullptr arguments in fextl::default_delete
This reflects behavior of std::default_delete.
2024-07-10 19:17:50 +02:00
Alyssa Rosenzweig a4f8bbff02 OpcodeDispatcher: reg cache avx high
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-10 11:36:18 -04:00
Alyssa Rosenzweig cf5ab05b90 OpcodeDispatcher: reg cache AbridgedFTW
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-10 11:36:18 -04:00
Alyssa Rosenzweig 3a2ce240f9 OpcodeDispatcher: reg cache DF
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-10 11:36:18 -04:00
Alyssa Rosenzweig 1f01dd53f7 OpcodeDispatcher: reg cache fprs
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-10 11:36:18 -04:00
Alyssa Rosenzweig 72d41d70b6 OpcodeDispatcher: introduce GPR-only reg cache
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-10 11:36:18 -04:00
Alyssa Rosenzweig 42b5b1f64c Core: partially flush register cache per instruction
This will mitigate problems later.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-10 11:36:18 -04:00
Alyssa Rosenzweig 2949bc211d OpcodeDispatcher: thunk through FlushRegisterCache
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-10 11:36:18 -04:00
Alyssa Rosenzweig 5e0952159d unittests: add test for a MMX register cache bug
this failed on an earlier version of the register cache.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-10 11:34:24 -04:00
Tony Wasserka 441187470e OpcodeDispatcher: Avoid monomorphization of some AVX functions 2024-07-10 17:01:30 +02:00
Tony Wasserka 59fd13cc2f OpcodeDispatcher: Avoid monomorphization of even more functions 2024-07-10 17:01:30 +02:00
Tony Wasserka c9e7bfdf16 OpcodeDispatcher: Avoid monomorphization of more functions 2024-07-10 17:01:30 +02:00
Tony Wasserka 2d700c381e OpcodeDispatcher: Avoid monomorphization of large functions 2024-07-10 17:01:30 +02:00
Ryan Houdek 72d6c8ebd6 Merge pull request #3820 from alyssarosenzweig/ir/drop-deferred
Drop deferred flag infrastructure
2024-07-09 17:06:25 -07:00
Ryan Houdek 991c6941c1 Merge pull request #3849 from alyssarosenzweig/ir/drop-parser-2
Scripts: drop remnant of IR parser
2024-07-09 16:48:36 -07:00
Alyssa Rosenzweig f974696e34 Scripts: drop remnant of IR parser
unused.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-09 16:08:38 -04:00
Alyssa Rosenzweig 3ef9ea94e5 Merge pull request #3848 from pmatos/FTSTX87Tests
Tests for X87 FTST
2024-07-09 09:10:29 -04:00
Paulo Matos 381ce23fd7 Tests for X87 FTST 2024-07-09 13:36:16 +02:00
Mai af6a0be832 Merge pull request #3842 from Sonicadvance1/fix_f64_to_i32
VCVT{T,}PD2DQ fixes and optimization
2024-07-09 03:49:31 -04:00
Ryan Houdek 287fe5beac InstcountCI: Update 2024-07-09 00:38:48 -07:00
Ryan Houdek b9c214e6e8 OpcodeDispatcher: Use new IR op for vcvt{t,}pd2dq
Also fixes a bug where it was failing to zero the upper bits of the
destination register in the AVX128 implementation. Which the updated
unit tests now check against.

Fixes a minor precision issue that was reported in #2995. We still don't
return correct values for overflow. x86 always returns maximum negative
int32_t on overflow, ARM will return maximum negative or positive
depending on sign of the double.
2024-07-09 00:38:47 -07:00
Ryan Houdek d3d76aa8ce IR: Adds new F64 -> I32 operation that changes behaviour depending on SVE
SVE added the ability to do F64 -> I32 conversions directly without an
fcvtn inbetween. So maybe sure to support them.
2024-07-09 00:38:47 -07:00
Ryan Houdek 3bea08da5f Merge pull request #3843 from Sonicadvance1/remove_half_moves_fma3
Arm64: Remove one move if possible in FMA operations
2024-07-09 00:25:07 -07:00
Paulo Matos 3d5cacbdc3 Enable coverage configuration for FEX 2024-07-09 08:03:41 +02:00
Mai 7ccb252069 Merge pull request #3837 from Sonicadvance1/optimize_sve_vpgatherdq
AVX128: Extends 32-bit indexes path for 128-bit operations
2024-07-08 22:01:02 -04:00
Ryan Houdek 31547462bb InstcountCI: Update for final SVE AVX128 improvements. 2024-07-08 18:44:07 -07:00
Ryan Houdek b3a7a973a1 AVX128: Extends 32-bit indexes path for 128-bit operations
The codepath from #3826 was only targeting 256-bit sized operations.
This missed the vpgatherdq/vgatherdpd 128-bit operations. By extending
the codepath to understand 128-bit operations, we now hit these
instruction variants.

With this PR, we now have SVE128 codepaths that handle ALL variants of
x86 gather instructions! There are zero ASIMD fallbacks used in this
case!

Of course depending on the instruction, the performance still leaves a
lot to be desired, and there is no way to emulate x86 TSO behaviour
without an ASIMD fallback, which we will likely need to add as a
fallback at some point.

Based on #3836 until that is merged.
2024-07-08 18:44:07 -07:00
Mai 22b26696ba Merge pull request #3836 from Sonicadvance1/optimize_sve_vpgatherdd
AVX128: Optimize the vpgatherdd/vgatherdps cases that would fall back to ASIMD
2024-07-08 21:43:36 -04:00
Ryan Houdek 495241f8ca InstcountCI: Update for wide gather vpgatherdd SVE usage 2024-07-08 18:12:28 -07:00
Ryan Houdek 4afbfcae17 AVX128: Optimize the vpgatherdd/vgatherdps cases that would fall back to ASIMD
With the introduction of the wide gathers in #3828 this has opened new
avenues for optimizing these cases that would typically fall back to
ASIMD. In the cases that 32-bit SVE scaling doesn't fit, we can instead
sign extend the elements in to double-width address registers.

This then feeds naturally in to the SVE path even though we end up
needing to allocate 512-bits worth of address registers. This ends up
being significantly better than the ASIMD path still.

Relies on #3828 to be merged first
Fixes #3829
2024-07-08 18:12:28 -07:00
Mai 3627de4cbc Merge pull request #3828 from Sonicadvance1/optimize_wide_gathers
AVX128: Optimize QPS/QD variant of gather loads!
2024-07-08 21:11:36 -04:00
Ryan Houdek 007c07e612 InstcountCI: Update for wide gathers 2024-07-08 17:19:18 -07:00
Ryan Houdek ec7c8fd922 AVX128: Optimize QPS/QD variant of gather loads!
SVE has a special version of their gather instruction that gets similar
behaviour to x86's VGATHERQPS/VPGATHERQD instructions.

The quirk of these instructions that the previous SVE implementation
didn't handle and required ASIMD fallback, was that most gather
instructions require the data element size and address element size to
match. This x86 instruction uses a 64-bit address size while loading 32-bit
elements. This matches this specific variant of the SVE instruction, but
the data is zero-extended once loaded, requiring us to shuffle the data
after it is loaded.

This isn't the worst but the implementation is different enough that
stuffing it in to the other gather load will cause headaches.

Basically gets 32 instruction variants to use the SVE version!

Fixes #3827
2024-07-08 17:19:18 -07:00
Ryan Houdek c5a0ae7b34 IR: Adds new QPS gather load variant! 2024-07-08 17:19:18 -07:00
Ryan Houdek 4bd207ebf3 Arm64: Moves 128Bit gather ASIMD emulation to its own helper
It is going to get reused.
2024-07-08 17:19:18 -07:00
Tony Wasserka 45011234d9 Merge pull request #3845 from pmatos/TESTJOBCOUNTFix
Use nproc only if TEST_JOB_COUNT not specified
2024-07-08 22:31:23 +02:00
Paulo Matos 24017f379e Use nproc only if TEST_JOB_COUNT not specified 2024-07-08 21:38:56 +02:00
Mai aad7656b38 Merge pull request #3826 from Sonicadvance1/scale_32bit_gather
AVX128: Extend 32-bit address indices when possible
2024-07-08 15:29:44 -04:00
Ryan Houdek 80de890f05 InstcountCI: Update for removed FMA moves 2024-07-08 04:50:49 -07:00
Ryan Houdek 62cec7b6b2 Arm64: Remove one move if possible in FMA operations
If the destination isn't any of the incoming sources then we can avoid
one of the moves at the end. This half works around the problem proposed
in #3794, but doesn't solve the entire problem.

To solve the other half of the moving problem means we need to solve the
SRA allocation problem for this temporary register with addsub/subadd, so it gets allocated
for both the FMA operation and the XOR operation.
2024-07-08 04:44:40 -07:00
Ryan Houdek c9c163cd7b unittests: Update vcv{t,tt}pd2dq tests to ensure upper bits of destination are cleared 2024-07-08 03:30:10 -07:00
Mai 95a9f32bf0 Merge pull request #3840 from Sonicadvance1/extend_vinsert128_tests
unittests: Extends vinsert{i,f}128 tests for garbage data
2024-07-07 13:39:20 -04:00
Mai c4ae761a0e Merge pull request #3841 from Sonicadvance1/add_missing_cpu_names
CPUID: Adds a few missing CPU names for new CPU cores
2024-07-07 13:38:27 -04:00
Ryan Houdek 0653b346e0 CPUID: Adds a few missing CPU names for new CPU cores
These should be making their way to the market sooner rather than later
so make sure we have the descriptor text for them.
2024-07-07 02:40:19 -07:00
Ryan Houdek fa587398bd unittests: Extends vinsert{i,f}128 tests for garbage data
Just to ensure we don't hit an issue with masking the immediate bits.

Fixes #3753
2024-07-07 02:16:21 -07:00
Ryan Houdek 6b67857151 InstcountCI: Adds a missing gather instruction invariant
Oops, must have accidentally deleted this while copying things around.
2024-07-06 18:32:36 -07:00
Ryan Houdek 81165f0c40 InstcountCI: Update for 32-bit gather sign extend optimization 2024-07-06 18:32:35 -07:00
Ryan Houdek df40515087 AVX128: Extend 32-bit address indices when possible
When loading 256-bits of data with only 128-bits of address indices, we
can sign extend the source indices to be 64-bit. Thus falling down the
ideal path for SVE where each 128-bit lane is loading the data to
addresses in a 1:1 element ratio.

This means we use the SVE path more often because of this.

Based on top of #3825 because the prescaling behaviour was introduced
there. This implements its own prescaling when the sign extension occurs
because ARM's SSHLL{,2} instruction gives us that for free.

This additionally fixes a bug where we were accidentally loading the top
128-bit half of the addresses for gathers when it was unnecessary, and
on the AVX256 side it was duplicating and doing some additional work
when it shouldn't have.

It'll be good to walk the commits when looking at this one, as there are
a couple of incremental changes that are easier to follow that way.

Fixes #3806
2024-07-06 18:32:35 -07:00
Ryan Houdek c77922e3e5 InstcountCI: Update for previous fix 2024-07-06 18:32:35 -07:00
Ryan Houdek 0f9abe68b9 AVX128: Fixes accidentally loading high addr register when unnnecessary
Was missing a clamp on the high half when encounting a 128-bit gather
instruction. Was causing us to unconditionally load the top half when it
was unncessary.
2024-07-06 18:32:35 -07:00
Ryan Houdek c168ee6940 Arm64: Implements VSSHLL{,2} IR ops 2024-07-06 18:32:35 -07:00
Ryan Houdek 0d4414fdd0 AVX128: Removes templated AddrElementSize and add as argument
NFC
2024-07-06 18:32:35 -07:00
Ryan Houdek 968d5e0d8f Merge pull request #3774 from bylaws/win-ci
FEXCore ARM64EC CI support
2024-07-06 18:22:57 -07:00
Ryan Houdek 635182b57c Merge pull request #3832 from bylaws/wow64-wine
WOW64: Mark the FEX dll as a wine builtin
2024-07-06 17:58:00 -07:00
Ryan Houdek 9d0b6ce75e Merge pull request #3835 from bylaws/ec-topdown
AllocatorHooks: Allocate from the top down on windows
2024-07-06 17:40:36 -07:00
Ryan Houdek 2fdd80fe3a Merge pull request #3833 from bylaws/common-tso
Windows: Commonise TSOHandlerConfig
2024-07-06 17:38:45 -07:00
Ryan Houdek dbac23b749 Merge pull request #3834 from bylaws/ec-amd64
Windows: Report as an AMD64 processor when targeting ARM64EC
2024-07-06 17:38:13 -07:00
Billy Laws 7fa7061aa5 Windows: Report as an AMD64 processor when targeting ARM64EC 2024-07-06 20:37:15 +00:00
Billy Laws e45e631199 AllocatorHooks: Allocate from the top down on windows
FEX allocations can get in the way of allocations that are 4gb-limited
even in 65-bit mode (i.e. those from LuaJIT), so allocate starting from
the top of the AS to prevent conflicts.
2024-07-06 20:35:38 +00:00
Billy Laws b21e77c1e0 Windows: Commonise TSOHandlerConfig 2024-07-06 19:20:49 +00:00
Billy Laws ba33294225 WOW64: Mark the FEX dll as a wine builtin
Allows it to be automatically picked up by wine during prefix setup,
without a manual dll override.

Thanks to AndreRH for pointing me to this.
2024-07-06 19:19:36 +00:00
Billy Laws 97c21cc3a7 CI: Add ARM64EC build CI 2024-07-06 17:27:41 +01:00
Billy Laws 7d7e6f5326 CMake: Disable WOW64 module for ARM64EC 2024-07-06 17:27:41 +01:00
Billy Laws 5e15bd935e CMake: Disable glibc jemalloc for MinGW builds 2024-07-06 17:27:41 +01:00
Ryan Houdek 9bad09c45f Merge pull request #3823 from alyssarosenzweig/bug/shl-var-small
Fix CF with small shifts
2024-07-06 01:33:57 -07:00
Ryan Houdek 47d077ff22 Merge pull request #3825 from Sonicadvance1/scale_64bit_gather
AVX128: Prescale addresses in gathers if possible
2024-07-05 19:10:43 -07:00
Ryan Houdek bbf8dde3ca Merge pull request #3824 from alyssarosenzweig/bug/rc2
OpcodeDispatcher: Fix 8/16-bit rcr masking
2024-07-05 17:01:16 -07:00
Ryan Houdek 6e8ca3bc6c InstcountCI: Update for gather prescaling 2024-07-05 16:47:11 -07:00
Ryan Houdek 11a494d7b3 AVX128: Prescale addresses in gathers if possible
If the host supports SVE128, if the address element size and data size is 64-bit, and the scale is not one of the two that is supported by SVE; Then prescale the addresses.
64-bit address overflow masks the top bits so is well defined that we
can scale the vector elements and still execute the SVE code path in
that case. Removing the ASIMD code paths from a lot of gathers.

Fixes #3805
2024-07-05 16:47:11 -07:00
Alyssa Rosenzweig 9b570de33f InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-05 18:44:21 -04:00
Ryan Houdek b67343fc5a unittests: Adds a test for small shift flags calculation
Currently we calculate CF incorrectly in the case of small shifts with
large offsets.
2024-07-05 18:38:12 -04:00
Alyssa Rosenzweig 5a3c0eb83c OpcodeDispatcher: fix shl with 8/16-bit variable
the special case here lines up with the special case of using a larger shift for
a smaller result, so we can just grab CF from the larger result.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-05 18:38:12 -04:00
Alyssa Rosenzweig 10391608a0 InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-05 18:34:18 -04:00
Ryan Houdek 51c57cc5ae unittests: More rotate with carry unit tests
Looks like we missed some edge cases with small carry rotate. Adds even
more unit tests.
2024-07-05 18:34:18 -04:00
Alyssa Rosenzweig 05e4678e65 OpcodeDispatcher: fix missing masking on smaller RCR
I probably broke this when working on eliminating crossblock liveness.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-05 18:34:18 -04:00
Alyssa Rosenzweig 0f0e402db4 OpcodeDispatcher: fix CF with 8/16-bit immediate
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-05 18:24:34 -04:00
Alyssa Rosenzweig 837bccb1d8 InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-05 17:24:51 -04:00
Alyssa Rosenzweig adc709db2f OpcodeDispatcher: drop remnants of deferred flags
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-05 17:22:41 -04:00
Alyssa Rosenzweig 395573720d OpcodeDispatcher: drop pointless flag defers for shifts
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-05 16:24:54 -04:00
Alyssa Rosenzweig 0e62759d24 OpcodeDispatcher: stop deferring logical
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-05 16:24:54 -04:00
Alyssa Rosenzweig 926b6c3117 OpcodeDispatcher: don't defer mul flags
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-05 16:24:54 -04:00
Alyssa Rosenzweig c9f9304ba5 OpcodeDispatcher: stop deferring obscure bitwise
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-05 16:24:54 -04:00
Alyssa Rosenzweig fabd6be5af OpcodeDispatcher: drop SUB defer
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-05 16:24:54 -04:00
Alyssa Rosenzweig 1bf31d20b6 OpcodeDispatcher: switch to CalculateFlags_SUB
most of these are deferred only to be calculated immediately anyway.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-05 16:24:53 -04:00
Ryan Houdek 653bf04db0 Merge pull request #3819 from alyssarosenzweig/bug/rcr-smol
Fix 8/16-bit RCR
2024-07-05 12:49:23 -07:00
Ryan Houdek b77a25b21a Merge pull request #3818 from alyssarosenzweig/jit/shiftbymaskstozero
JIT: fix ShiftFlags masking
2024-07-05 12:49:16 -07:00
Alyssa Rosenzweig 9db6931cea InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-05 10:49:12 -04:00
Ryan Houdek bad5cef52b unittests: Adds rotate with carry test for large rotates
FEX-Emu currently doesn't do large rotates for small data sources
correctly. This will fail CI until fixed in OpcodeDispatcher
2024-07-05 10:49:02 -04:00
Alyssa Rosenzweig 94bd79b2bf OpcodeDispatcher: fix 8/16-bit RCR
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-05 10:49:02 -04:00
Alyssa Rosenzweig b746146f4e InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-05 09:57:42 -04:00
Ryan Houdek 8ac9bb5c72 unittests: Adds test for flags when shifting by zero 2024-07-05 09:57:42 -04:00
Alyssa Rosenzweig 1b552a6f62 JIT: fix ShiftFlags masking
we don't update flags for a nonzero shift that masks to zero.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-05 09:57:42 -04:00
Alyssa Rosenzweig 97329ccc7a Merge pull request #3812 from Sonicadvance1/fix_rotates_with_zero
OpcodeDispatcher: Fixes rotates with zero not zero extending 32-bit result
2024-07-05 09:48:01 -04:00
Mai f2d1f2de56 Merge pull request #3817 from Sonicadvance1/fix_x87_integer_indefinite
Softfloat: Fixes Integer indefinite return for 16-bit signed values
2024-07-04 23:11:44 -04:00
Ryan Houdek 692c2fae96 Merge pull request #3813 from alyssarosenzweig/bug/fix-sbb
Fix 16-bit SBB
2024-07-04 19:52:37 -07:00
Mai 3d65b701a2 Merge pull request #3816 from Sonicadvance1/fix_long_signed_divide
Arm64: Fixes long signed divide
2024-07-04 21:43:11 -04:00
Ryan Houdek ecaca0fe15 unittests: Adds x87 integer indefinite test
Tests 16-bit, 32-bit, and 64-bit integer conversions
2024-07-04 17:53:28 -07:00
Ryan Houdek 8955f83ef6 Softfloat: Fixes Integer indefinite return for 16-bit signed values
Regardless of positive or negative value, if the converted integer
doesn't fit in to the converted int16_t then it returns INT16_MIN.
2024-07-04 17:43:28 -07:00
Ryan Houdek 1a8aaebd79 unittests: Adds long signed divide test 2024-07-04 16:43:21 -07:00
Ryan Houdek 38a823cc54 Arm64: Fixes long signed divide
The two halves are provided as two uint64_t values that shouldn't be
sign extended between them. Treat them as uint64_t until combined in to
a single int128_t. Fixes long signed divide.
2024-07-04 16:42:23 -07:00
Ryan Houdek 25306cb373 InstcountCI: Update 2024-07-04 14:35:43 -07:00
Ryan Houdek 1084a031e7 unittests: Adds test for previous fix
All of these results would have failed except for the rorx result.
2024-07-04 14:35:43 -07:00
Ryan Houdek f6ec99bede OpcodeDispatcher: Fixes rotates with zero not zero extending 32-bit result
For all the 32-bit rotates (except for RORX) we were failing to zero
extend the 32-bit result to the destination register when the rotate was
masked to zero.

Ensure we do this.
2024-07-04 14:35:42 -07:00
Ryan Houdek 90a6647fa4 Merge pull request #3811 from alyssarosenzweig/ra/fix-lsp
RA: fix interaction between SRA & shuffles
2024-07-04 14:20:46 -07:00
Alyssa Rosenzweig a926bb81a9 InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-04 16:58:45 -04:00
Alyssa Rosenzweig fbf41e3149 unittests: add test for small sbc flags
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-04 16:58:45 -04:00
Alyssa Rosenzweig a38205069b OpcodeDispatcher: fix SBB carry flag
do it the naive way, just applying the x86 definitions of SBB.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-04 16:58:45 -04:00
Alyssa Rosenzweig 2d75801024 unittests: add tricky RA test
this fails on current main with blocksize=500 due to mentioned RA bug. passes
with blocksize=1.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-04 13:37:13 -04:00
Alyssa Rosenzweig 504511fe7e RA: fix interaction between SRA & shuffles
missed a Map. tricky case hit by the unit test added in the next commit.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-07-04 13:37:13 -04:00
492 changed files with 154075 additions and 189255 deletions

No files matched your search

+7 -2
View File
@@ -16,7 +16,7 @@ jobs:
runs-on: ${{ matrix.arch }}
strategy:
matrix:
arch: [[self-hosted, ARM64, mingw]]
arch: [[self-hosted, ARM64, mingw], [self-hosted, ARM64EC, mingw, ARM64]]
fail-fast: false
steps:
@@ -38,6 +38,11 @@ jobs:
run: |
echo "MINGW_TRIPLE=aarch64-w64-mingw32" >> $GITHUB_ENV
- name: Set CC Arm64EC
if: matrix.arch[1] == 'ARM64EC'
run: |
echo "MINGW_TRIPLE=arm64ec-w64-mingw32" >> $GITHUB_ENV
- name: Set rootfs paths
run: |
echo "FEX_ROOTFS_MOUNT=/mnt/AutoNFS/rootfs/" >> $GITHUB_ENV
@@ -73,7 +78,7 @@ jobs:
# Note the current convention is to use the -S and -B options here to specify source
# and build directories, but this is only available with CMake 3.13 and higher.
# The CMake binaries on the Github Actions machines are (as of this writing) 3.12
run: cmake $GITHUB_WORKSPACE -DCMAKE_BUILD_TYPE=$BUILD_TYPE -DCMAKE_TOOLCHAIN_FILE=$GITHUB_WORKSPACE/toolchain_mingw.cmake -DMINGW_TRIPLE=$MINGW_TRIPLE -G Ninja -DENABLE_LTO=False -DENABLE_ASSERTIONS=True -DENABLE_X86_HOST_DEBUG=True -DBUILD_TESTS=False -DENABLE_JEMALLOC=False -DENABLE_JEMALLOC_GLIBC_ALLOC=False -DCMAKE_INSTALL_PREFIX=${{runner.workspace}}/build/install
run: cmake $GITHUB_WORKSPACE -DCMAKE_BUILD_TYPE=$BUILD_TYPE -DCMAKE_TOOLCHAIN_FILE=$GITHUB_WORKSPACE/toolchain_mingw.cmake -DMINGW_TRIPLE=$MINGW_TRIPLE -G Ninja -DENABLE_LTO=False -DENABLE_ASSERTIONS=True -DENABLE_X86_HOST_DEBUG=True -DBUILD_TESTS=False -DCMAKE_INSTALL_PREFIX=${{runner.workspace}}/build/install
- name: Build
working-directory: ${{runner.workspace}}/build
+19 -6
View File
@@ -72,23 +72,22 @@ jobs:
# Execute the build. You can specify a specific target with "--target <NAME>"
run: cmake --build . --config $BUILD_TYPE
- name: ASM Tests
- name: ASM Tests - SVE256
working-directory: ${{runner.workspace}}/build
shell: bash
# Execute the unit tests
run: cmake --build . --config $BUILD_TYPE --target asm_tests
- name: ASM Test Results move
- name: ASM Test SVE256 Results move
if: ${{ always() }}
shell: bash
working-directory: ${{runner.workspace}}/build
run: mv ${{runner.workspace}}/build/Testing/Temporary/LastTest.log ${{runner.workspace}}/build/Testing/Temporary/LastTest_ASM.log || true
run: mv ${{runner.workspace}}/build/Testing/Temporary/LastTest.log ${{runner.workspace}}/build/Testing/Temporary/LastTest_ASM_SVE256Bit.log || true
- name: ASM Tests 128-bit
- name: ASM Tests - SVE128
working-directory: ${{runner.workspace}}/build
shell: bash
env:
FEX_HOSTFEATURES: "disableavx"
FEX_FORCESVEWIDTH: "128"
# Execute the unit tests
run: cmake --build . --config $BUILD_TYPE --target asm_tests
@@ -97,7 +96,21 @@ jobs:
if: ${{ always() }}
shell: bash
working-directory: ${{runner.workspace}}/build
run: mv ${{runner.workspace}}/build/Testing/Temporary/LastTest.log ${{runner.workspace}}/build/Testing/Temporary/LastTest_ASM128bit.log || true
run: mv ${{runner.workspace}}/build/Testing/Temporary/LastTest.log ${{runner.workspace}}/build/Testing/Temporary/LastTest_ASM_SVE128Bit.log || true
- name: ASM Tests - ASIMD
working-directory: ${{runner.workspace}}/build
shell: bash
env:
FEX_HOSTFEATURES: "disablesve"
# Execute the unit tests
run: cmake --build . --config $BUILD_TYPE --target asm_tests
- name: ASM Test ASIMD Results move
if: ${{ always() }}
shell: bash
working-directory: ${{runner.workspace}}/build
run: mv ${{runner.workspace}}/build/Testing/Temporary/LastTest.log ${{runner.workspace}}/build/Testing/Temporary/LastTest_ASM_ASIMD.log || true
- name: Truncate test results
if: ${{ always() }}
+1 -1
View File
@@ -4,7 +4,7 @@ compile_commands.json
vim_rc
Config.json
[Bb]uild*/
[Bb]uild*
[Bb]in/
out/
.vscode/
-9
View File
@@ -5,15 +5,6 @@
[submodule "External/cpp-optparse"]
path = Source/Common/cpp-optparse
url = https://github.com/Sonicadvance1/cpp-optparse
[submodule "External/imgui"]
path = External/imgui
url = https://github.com/Sonicadvance1/imgui.git
[submodule "External/json-maker"]
path = External/json-maker
url = https://github.com/Sonicadvance1/json-maker.git
[submodule "External/tiny-json"]
path = External/tiny-json
url = https://github.com/Sonicadvance1/tiny-json.git
[submodule "External/xbyak"]
shallow = true
path = External/xbyak
+83 -34
View File
@@ -1,5 +1,5 @@
cmake_minimum_required(VERSION 3.14)
project(FEX)
project(FEX C CXX ASM)
INCLUDE (CheckIncludeFiles)
CHECK_INCLUDE_FILES ("gdb/jit-reader.h" HAVE_GDB_JIT_READER_H)
@@ -7,7 +7,7 @@ CHECK_INCLUDE_FILES ("gdb/jit-reader.h" HAVE_GDB_JIT_READER_H)
option(BUILD_TESTS "Build unit tests to ensure sanity" TRUE)
option(BUILD_FEX_LINUX_TESTS "Build FEXLinuxTests, requires x86 compiler" FALSE)
option(BUILD_THUNKS "Build thunks" FALSE)
option(BUILD_FEXCONFIG "Build FEXConfig, requires SDL2 and X11" TRUE)
option(BUILD_FEXCONFIG "Build FEXConfig" TRUE)
option(ENABLE_CLANG_THUNKS "Build thunks with clang" FALSE)
option(ENABLE_IWYU "Enables include what you use program" FALSE)
option(ENABLE_LTO "Enable LTO with compilation" TRUE)
@@ -15,6 +15,7 @@ option(ENABLE_XRAY "Enable building with LLVM X-Ray" FALSE)
set(USE_LINKER "" CACHE STRING "Allow overriding the linker path directly")
option(ENABLE_ASAN "Enables Clang ASAN" FALSE)
option(ENABLE_TSAN "Enables Clang TSAN" FALSE)
option(ENABLE_COVERAGE "Enables Coverage" FALSE)
option(ENABLE_ASSERTIONS "Enables assertions in build" FALSE)
option(ENABLE_GDB_SYMBOLS "Enables GDBSymbols integration support" ${HAVE_GDB_JIT_READER_H})
option(ENABLE_STRICT_WERROR "Enables stricter -Werror for CI" FALSE)
@@ -27,10 +28,12 @@ option(ENABLE_LIBCXX "Enables LLVM libc++" FALSE)
option(ENABLE_CCACHE "Enables ccache for compile caching" TRUE)
option(ENABLE_VIXL_SIMULATOR "Forces the FEX JIT to use the VIXL simulator" FALSE)
option(ENABLE_VIXL_DISASSEMBLER "Enables debug disassembler output with VIXL" FALSE)
option(USE_LEGACY_BINFMTMISC "Uses legacy method of setting up binfmt_misc" FALSE)
option(COMPILE_VIXL_DISASSEMBLER "Compiles the vixl disassembler in to vixl" FALSE)
option(ENABLE_FEXCORE_PROFILER "Enables use of the FEXCore timeline profiling capabilities" FALSE)
set (FEXCORE_PROFILER_BACKEND "gpuvis" CACHE STRING "Set which backend you want to use for the FEXCore profiler")
option(ENABLE_GLIBC_ALLOCATOR_HOOK_FAULT "Enables glibc memory allocation hooking with fault for CI testing")
option(USE_PDB_DEBUGINFO "Builds debug info in PDB format" FALSE)
set (X86_32_TOOLCHAIN_FILE "${CMAKE_CURRENT_SOURCE_DIR}/toolchain_x86_32.cmake" CACHE FILEPATH "Toolchain file for the (cross-)compiler targeting i686")
set (X86_64_TOOLCHAIN_FILE "${CMAKE_CURRENT_SOURCE_DIR}/toolchain_x86_64.cmake" CACHE FILEPATH "Toolchain file for the (cross-)compiler targeting x86_64")
@@ -40,12 +43,13 @@ string(FIND ${CMAKE_BASE_NAME} mingw CONTAINS_MINGW)
if (NOT CONTAINS_MINGW EQUAL -1)
message (STATUS "Mingw build")
set (MINGW_BUILD TRUE)
set (ENABLE_JEMALLOC FALSE)
set (ENABLE_JEMALLOC TRUE)
set (ENABLE_JEMALLOC_GLIBC_ALLOC FALSE)
endif()
if (NOT MINGW_BUILD)
message (STATUS "Clang version ${CMAKE_CXX_COMPILER_VERSION}")
set (CLANG_MINIMUM_VERSION 12.0)
set (CLANG_MINIMUM_VERSION 13.0)
if (CMAKE_CXX_COMPILER_VERSION VERSION_LESS ${CLANG_MINIMUM_VERSION})
message (FATAL_ERROR "Clang version too old for FEX. Need at least ${CLANG_MINIMUM_VERSION} but has ${CMAKE_CXX_COMPILER_VERSION}")
endif()
@@ -137,9 +141,39 @@ endif()
if (CMAKE_SYSTEM_PROCESSOR MATCHES "^arm64ec")
set(_M_ARM_64EC 1)
add_definitions(-D_M_ARM_64EC=1)
endif()
# Required as FEX is not allowed to lock the CRT heap lock during compilation or callbacks
set(ENABLE_JEMALLOC TRUE)
include(CheckCXXSourceCompiles)
set(CMAKE_REQUIRED_FLAGS "-std=c++11 -Wattributes -Werror=attributes")
check_cxx_source_compiles(
"
__attribute__((preserve_all))
int Testy(int a, int b, int c, int d, int e, int f) {
return a + b + c + d + e + f;
}
int main() {
return Testy(0, 1, 2, 3, 4, 5);
}"
HAS_CLANG_PRESERVE_ALL)
unset(CMAKE_REQUIRED_FLAGS)
if (HAS_CLANG_PRESERVE_ALL)
if (MINGW_BUILD)
message(STATUS "Ignoring broken clang::preserve_all support")
set(HAS_CLANG_PRESERVE_ALL FALSE)
else()
message(STATUS "Has clang::preserve_all")
endif()
endif ()
if (_M_ARM_64 AND HAS_CLANG_PRESERVE_ALL)
add_definitions("-DFEX_PRESERVE_ALL_ATTR=__attribute__((preserve_all))" "-DFEX_HAS_PRESERVE_ALL_ATTR=1")
else()
add_definitions("-DFEX_PRESERVE_ALL_ATTR=" "-DFEX_HAS_PRESERVE_ALL_ATTR=0")
endif()
if (ENABLE_VIXL_SIMULATOR)
# We can run the simulator on both x86-64 or AArch64 hosts
add_definitions(-DVIXL_SIMULATOR=1 -DVIXL_INCLUDE_SIMULATOR_AARCH64=1)
endif()
if (ENABLE_CCACHE)
@@ -189,13 +223,17 @@ if (ENABLE_TSAN)
link_libraries(-fno-omit-frame-pointer -fsanitize=thread)
endif()
if (ENABLE_COVERAGE)
add_compile_options(-fprofile-instr-generate -fcoverage-mapping)
link_libraries(-fprofile-instr-generate -fcoverage-mapping)
endif()
if (ENABLE_JEMALLOC_GLIBC_ALLOC)
# The glibc jemalloc subproject which hooks the glibc allocator.
# Required for thunks to work.
# All host native libraries will use this allocator, while *most* other FEX internal allocations will use the other jemalloc allocator.
add_definitions(-DENABLE_JEMALLOC_GLIBC=1)
add_subdirectory(External/jemalloc_glibc/)
else()
elseif (NOT MINGW_BUILD)
message (STATUS
" jemalloc glibc allocator disabled!\n"
" This is not a recommended configuration!\n"
@@ -205,10 +243,8 @@ endif()
if (ENABLE_JEMALLOC)
# The jemalloc subproject that all FEXCore fextl objects allocate through.
add_definitions(-DENABLE_JEMALLOC=1)
add_subdirectory(External/jemalloc/)
include_directories(External/jemalloc/pregen/include/)
else()
elseif (NOT MINGW_BUILD)
message (STATUS
" jemalloc disabled!\n"
" This is not a recommended configuration!\n"
@@ -216,6 +252,11 @@ else()
" Use at your own risk!")
endif()
if (USE_PDB_DEBUGINFO)
add_compile_options(-g -gcodeview)
add_link_options(-g -Wl,--pdb=)
endif()
set (CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} -fno-omit-frame-pointer")
set (CMAKE_LINKER_FLAGS_RELWITHDEBINFO "${CMAKE_LINKER_FLAGS_RELWITHDEBINFO} -fno-omit-frame-pointer")
@@ -229,8 +270,10 @@ if (BUILD_TESTS)
set(COMPILE_VIXL_DISASSEMBLER TRUE)
endif()
add_subdirectory(External/vixl/)
include_directories(SYSTEM External/vixl/src/)
if (COMPILE_VIXL_DISASSEMBLER OR ENABLE_VIXL_SIMULATOR)
add_subdirectory(External/vixl/)
include_directories(SYSTEM External/vixl/src/)
endif()
if (CMAKE_CXX_COMPILER_ID STREQUAL "GNU")
# This means we were attempted to get compiled with GCC
@@ -240,37 +283,42 @@ endif()
find_package(PkgConfig REQUIRED)
find_package(Python 3.0 REQUIRED COMPONENTS Interpreter)
set(XXHASH_BUNDLED_MODE TRUE)
set(XXHASH_BUILD_XXHSUM FALSE)
set(BUILD_SHARED_LIBS OFF)
add_subdirectory(External/xxhash/cmake_unofficial/)
pkg_search_module(xxhash IMPORTED_TARGET xxhash libxxhash)
if (TARGET PkgConfig::xxhash AND NOT CMAKE_CROSSCOMPILING)
add_library(xxHash::xxhash ALIAS PkgConfig::xxhash)
else()
set(XXHASH_BUNDLED_MODE TRUE)
set(XXHASH_BUILD_XXHSUM FALSE)
add_subdirectory(External/xxhash/cmake_unofficial/)
endif()
add_definitions(-Wno-trigraphs)
add_definitions(-DGLOBAL_DATA_DIRECTORY="${DATA_DIRECTORY}/")
if (BUILD_TESTS)
add_subdirectory(External/Catch2/)
find_package(Catch2 QUIET)
if (NOT Catch2_FOUND)
add_subdirectory(External/Catch2/)
# Pull in catch_discover_tests definition
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/External/Catch2/contrib/")
endif()
# Pull in catch_discover_tests definition
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/External/Catch2/contrib/")
include(Catch)
endif()
# Disable fmt install
set(FMT_INSTALL OFF)
add_subdirectory(External/fmt/)
add_subdirectory(External/imgui/)
include_directories(External/imgui/)
add_subdirectory(External/json-maker/)
include_directories(External/json-maker/)
find_package(fmt QUIET)
if (NOT fmt_FOUND)
# Disable fmt install
set(FMT_INSTALL OFF)
add_subdirectory(External/fmt/)
endif()
add_subdirectory(External/tiny-json/)
include_directories(External/tiny-json/)
include_directories(External/xbyak/)
include_directories(Source/)
include_directories("${CMAKE_BINARY_DIR}/Source/")
@@ -372,8 +420,7 @@ if (BUILD_TESTS)
set (TEST_JOB_COUNT "" CACHE STRING "Override number of parallel jobs to use while running tests")
if (TEST_JOB_COUNT)
message(STATUS "Running tests with ${TEST_JOB_COUNT} jobs")
endif()
if (CMAKE_VERSION VERSION_LESS "3.29")
elseif(CMAKE_VERSION VERSION_LESS "3.29")
execute_process(COMMAND "nproc" OUTPUT_STRIP_TRAILING_WHITESPACE OUTPUT_VARIABLE TEST_JOB_COUNT)
endif()
set(TEST_JOB_FLAG "-j${TEST_JOB_COUNT}")
@@ -383,8 +430,10 @@ add_subdirectory(FEXHeaderUtils/)
add_subdirectory(CodeEmitter/)
add_subdirectory(FEXCore/)
# Binfmt_misc files must be installed prior to Source/ installs
add_subdirectory(Data/binfmts/)
if (_M_ARM_64 AND NOT MINGW_BUILD)
# Binfmt_misc files must be installed prior to Source/ installs
add_subdirectory(Data/binfmts/)
endif()
add_subdirectory(Source/)
add_subdirectory(Data/AppConfig/)
+35
View File
@@ -0,0 +1,35 @@
# This is a reference AArch64 cross compile script
# Pass in to cmake when building:
# eg: cmake -DCMAKE_TOOLCHAIN_FILE=../CMakeToolchains/AArch64.cmake ..
if (NOT DEFINED ENV{SYSROOT})
message(FATAL_ERROR "Need to have SYSROOT environment variable set")
endif()
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_PROCESSOR aarch64)
set(CMAKE_CROSSCOMPILING TRUE)
# Target triple needs to match the binutils exactly
set(TARGET_TRIPLE aarch64-linux-gnu)
set(CMAKE_C_COMPILER "clang")
set(CMAKE_CXX_COMPILER "clang++")
set(CMAKE_C_COMPILER_AR "llvm-ar")
set(CMAKE_CXX_COMPILER_AR "llvm-ar")
set(CMAKE_C_COMPILER_RANLIB "llvm-ranlib")
set(CMAKE_CXX_COMPILER_RANLIB "llvm-ranlib")
set(CMAKE_LINKER "ld.lld")
set(CMAKE_C_COMPILER_TARGET ${TARGET_TRIPLE})
set(CMAKE_CXX_COMPILER_TARGET ${TARGET_TRIPLE})
# Set the environment variable SYSROOT to the aarch64 rootfs
set(CMAKE_FIND_ROOT_PATH "$ENV{SYSROOT}")
set(CMAKE_SYSROOT "$ENV{SYSROOT}")
list(APPEND CMAKE_PREFIX_PATH "$ENV{SYSROOT}/usr/lib/${TARGET_TRIPLE}/cmake/")
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
+10 -8
View File
@@ -301,8 +301,9 @@ public:
xbfiz_helper(true, s, rd, rn, lsb, width);
}
void asr(ARMEmitter::Size s, ARMEmitter::Register rd, ARMEmitter::Register rn, uint32_t shift) {
LOGMAN_THROW_A_FMT(shift <= RegSizeInBits(s), "Tried to asr a region larger than the register");
sbfm(s, rd, rn, shift, RegSizeInBits(s) - 1);
const auto RegSize_m1 = RegSizeInBits(s) - 1;
shift &= RegSize_m1;
sbfm(s, rd, rn, shift, RegSize_m1);
}
void uxtb(ARMEmitter::Size s, ARMEmitter::Register rd, ARMEmitter::Register rn) {
@@ -325,14 +326,14 @@ public:
}
void lsl(ARMEmitter::Size s, ARMEmitter::Register rd, ARMEmitter::Register rn, uint32_t shift) {
const auto RegSize = RegSizeInBits(s);
LOGMAN_THROW_A_FMT(shift < RegSize, "Tried to lsl a region larger than the register");
ubfm(s, rd, rn, (RegSize - shift) % RegSize, RegSize - shift - 1);
const auto RegSize_m1 = RegSizeInBits(s) - 1;
shift &= RegSize_m1;
ubfm(s, rd, rn, (RegSizeInBits(s) - shift) & RegSize_m1, RegSize_m1 - shift);
}
void lsr(ARMEmitter::Size s, ARMEmitter::Register rd, ARMEmitter::Register rn, uint32_t shift) {
const auto RegSize = RegSizeInBits(s);
LOGMAN_THROW_A_FMT(shift < RegSize, "Tried to lsr a region larger than the register");
ubfm(s, rd, rn, shift, RegSize - 1);
const auto RegSize_m1 = RegSizeInBits(s) - 1;
shift &= RegSize_m1;
ubfm(s, rd, rn, shift, RegSize_m1);
}
void ubfx(ARMEmitter::Size s, ARMEmitter::Register rd, ARMEmitter::Register rn, uint32_t lsb, uint32_t width) {
LOGMAN_THROW_A_FMT(width > 0, "ubfx needs width > 0");
@@ -368,6 +369,7 @@ public:
}
void ror(ARMEmitter::Size s, ARMEmitter::Register rd, ARMEmitter::Register rn, uint32_t Imm) {
Imm &= RegSizeInBits(s) - 1;
extr(s, rd, rn, rn, Imm);
}
+12 -12
View File
@@ -1070,7 +1070,7 @@ public:
constexpr uint32_t Op = 0b0000'1110'0010'0000'0000'10 << 10;
const auto ConvertedSize =
size == ARMEmitter::SubRegSize::i32Bit ? ARMEmitter::SubRegSize::i16Bit :
size == ARMEmitter::SubRegSize::i16Bit ? ARMEmitter::SubRegSize::i8Bit : ARMEmitter::SubRegSize::i8Bit;
ARMEmitter::SubRegSize::i8Bit;
ASIMD2RegMisc(Op, 0, ConvertedSize, 0b10110, rd.D(), rn.D());
}
@@ -1082,7 +1082,7 @@ public:
constexpr uint32_t Op = 0b0000'1110'0010'0000'0000'10 << 10;
const auto ConvertedSize =
size == ARMEmitter::SubRegSize::i32Bit ? ARMEmitter::SubRegSize::i16Bit :
size == ARMEmitter::SubRegSize::i16Bit ? ARMEmitter::SubRegSize::i8Bit : ARMEmitter::SubRegSize::i8Bit;
ARMEmitter::SubRegSize::i8Bit;
ASIMD2RegMisc(Op, 0, ConvertedSize, 0b10110, rd.Q(), rn.Q());
}
@@ -1095,7 +1095,7 @@ public:
constexpr uint32_t Op = 0b0000'1110'0010'0000'0000'10 << 10;
const auto ConvertedSize =
size == ARMEmitter::SubRegSize::i64Bit ? ARMEmitter::SubRegSize::i16Bit :
size == ARMEmitter::SubRegSize::i32Bit ? ARMEmitter::SubRegSize::i8Bit : ARMEmitter::SubRegSize::i8Bit;
ARMEmitter::SubRegSize::i8Bit;
ASIMD2RegMisc(Op, 0, ConvertedSize, 0b10111, rd.D(), rn.D());
}
@@ -1107,7 +1107,7 @@ public:
constexpr uint32_t Op = 0b0000'1110'0010'0000'0000'10 << 10;
const auto ConvertedSize =
size == ARMEmitter::SubRegSize::i64Bit ? ARMEmitter::SubRegSize::i16Bit :
size == ARMEmitter::SubRegSize::i32Bit ? ARMEmitter::SubRegSize::i8Bit : ARMEmitter::SubRegSize::i8Bit;
ARMEmitter::SubRegSize::i8Bit;
ASIMD2RegMisc(Op, 0, ConvertedSize, 0b10111, rd.Q(), rn.Q());
}
@@ -1123,7 +1123,7 @@ public:
constexpr uint32_t Op = 0b0000'1110'0010'0000'0000'10 << 10;
const auto ConvertedSize =
size == ARMEmitter::SubRegSize::i64Bit ? ARMEmitter::SubRegSize::i16Bit :
size == ARMEmitter::SubRegSize::i32Bit ? ARMEmitter::SubRegSize::i8Bit : ARMEmitter::SubRegSize::i8Bit;
ARMEmitter::SubRegSize::i8Bit;
ASIMD2RegMisc<T>(Op, 0, ConvertedSize, 0b11000, rd, rn);
}
@@ -1138,7 +1138,7 @@ public:
constexpr uint32_t Op = 0b0000'1110'0010'0000'0000'10 << 10;
const auto ConvertedSize =
size == ARMEmitter::SubRegSize::i64Bit ? ARMEmitter::SubRegSize::i16Bit :
size == ARMEmitter::SubRegSize::i32Bit ? ARMEmitter::SubRegSize::i8Bit : ARMEmitter::SubRegSize::i8Bit;
ARMEmitter::SubRegSize::i8Bit;
ASIMD2RegMisc<T>(Op, 0, ConvertedSize, 0b11001, rd, rn);
}
@@ -1154,7 +1154,7 @@ public:
constexpr uint32_t Op = 0b0000'1110'0010'0000'0000'10 << 10;
const auto ConvertedSize =
size == ARMEmitter::SubRegSize::i64Bit ? ARMEmitter::SubRegSize::i16Bit :
size == ARMEmitter::SubRegSize::i32Bit ? ARMEmitter::SubRegSize::i8Bit : ARMEmitter::SubRegSize::i8Bit;
ARMEmitter::SubRegSize::i8Bit;
ASIMD2RegMisc<T>(Op, 0, ConvertedSize, 0b11010, rd, rn);
}
@@ -1169,7 +1169,7 @@ public:
constexpr uint32_t Op = 0b0000'1110'0010'0000'0000'10 << 10;
const auto ConvertedSize =
size == ARMEmitter::SubRegSize::i64Bit ? ARMEmitter::SubRegSize::i16Bit :
size == ARMEmitter::SubRegSize::i32Bit ? ARMEmitter::SubRegSize::i8Bit : ARMEmitter::SubRegSize::i8Bit;
ARMEmitter::SubRegSize::i8Bit;
ASIMD2RegMisc<T>(Op, 0, ConvertedSize, 0b11011, rd, rn);
}
@@ -1184,7 +1184,7 @@ public:
constexpr uint32_t Op = 0b0000'1110'0010'0000'0000'10 << 10;
const auto ConvertedSize =
size == ARMEmitter::SubRegSize::i64Bit ? ARMEmitter::SubRegSize::i16Bit :
size == ARMEmitter::SubRegSize::i32Bit ? ARMEmitter::SubRegSize::i8Bit : ARMEmitter::SubRegSize::i8Bit;
ARMEmitter::SubRegSize::i8Bit;
ASIMD2RegMisc<T>(Op, 0, ConvertedSize, 0b11100, rd, rn);
}
@@ -1199,7 +1199,7 @@ public:
constexpr uint32_t Op = 0b0000'1110'0010'0000'0000'10 << 10;
const auto ConvertedSize =
size == ARMEmitter::SubRegSize::i64Bit ? ARMEmitter::SubRegSize::i16Bit :
size == ARMEmitter::SubRegSize::i32Bit ? ARMEmitter::SubRegSize::i8Bit : ARMEmitter::SubRegSize::i8Bit;
ARMEmitter::SubRegSize::i8Bit;
ASIMD2RegMisc<T>(Op, 0, ConvertedSize, 0b11101, rd, rn);
}
@@ -1214,7 +1214,7 @@ public:
constexpr uint32_t Op = 0b0000'1110'0010'0000'0000'10 << 10;
const auto ConvertedSize =
size == ARMEmitter::SubRegSize::i64Bit ? ARMEmitter::SubRegSize::i16Bit :
size == ARMEmitter::SubRegSize::i32Bit ? ARMEmitter::SubRegSize::i8Bit : ARMEmitter::SubRegSize::i8Bit;
ARMEmitter::SubRegSize::i8Bit;
ASIMD2RegMisc<T>(Op, 0, ConvertedSize, 0b11110, rd, rn);
}
@@ -1229,7 +1229,7 @@ public:
constexpr uint32_t Op = 0b0000'1110'0010'0000'0000'10 << 10;
const auto ConvertedSize =
size == ARMEmitter::SubRegSize::i64Bit ? ARMEmitter::SubRegSize::i16Bit :
size == ARMEmitter::SubRegSize::i32Bit ? ARMEmitter::SubRegSize::i8Bit : ARMEmitter::SubRegSize::i8Bit;
ARMEmitter::SubRegSize::i8Bit;
ASIMD2RegMisc<T>(Op, 0, ConvertedSize, 0b11111, rd, rn);
}
+12
View File
@@ -602,6 +602,18 @@ constexpr bool AreVectorsSequential(T first, const Args&... args) {
return (fn(first, args) && ...);
}
// Returns if the immediate can fit in to add/sub immediate instruction encodings.
constexpr bool IsImmAddSub(uint64_t imm) {
constexpr uint64_t U12Mask = 0xFFF;
auto FitsWithin12Bits = [](uint64_t imm) {
return (imm & ~U12Mask) == 0;
};
// Can fit in to the instruction encoding:
// - if only bits [11:0] are set.
// - if only bits [23:12] are set.
return FitsWithin12Bits(imm) || (FitsWithin12Bits(imm >> 12) && (imm & U12Mask) == 0);
}
// This is an emitter that is designed around the smallest code bloat as possible.
// Eschewing most developer convenience in order to keep code as small as possible.
+29 -1
View File
@@ -2569,7 +2569,19 @@ public:
}
// SVE contiguous non-temporal load (scalar plus immediate)
// XXX:
void ldnt1b(ZRegister zt, PRegister pg, Register rn, int32_t Imm = 0) {
SVEContiguousNontemporalLoad(0b00, zt, pg, rn, Imm);
}
void ldnt1h(ZRegister zt, PRegister pg, Register rn, int32_t Imm = 0) {
SVEContiguousNontemporalLoad(0b01, zt, pg, rn, Imm);
}
void ldnt1w(ZRegister zt, PRegister pg, Register rn, int32_t Imm = 0) {
SVEContiguousNontemporalLoad(0b10, zt, pg, rn, Imm);
}
void ldnt1d(ZRegister zt, PRegister pg, Register rn, int32_t Imm = 0) {
SVEContiguousNontemporalLoad(0b11, zt, pg, rn, Imm);
}
// SVE contiguous non-temporal load (scalar plus scalar)
// XXX:
// SVE load multiple structures (scalar plus immediate)
@@ -4492,6 +4504,22 @@ private:
dc32(Instr);
}
// SVE contiguous non-temporal load (scalar plus immediate)
void SVEContiguousNontemporalLoad(uint32_t msz, ZRegister zt, PRegister pg, Register rn, int32_t imm) {
LOGMAN_THROW_A_FMT(pg <= PReg::p7, "Can only use p0-p7 as a governing predicate");
LOGMAN_THROW_AA_FMT(imm >= -8 && imm <= 7,
"Invalid loadstore offset ({}). Must be between [-8, 7]", imm);
const auto imm4 = static_cast<uint32_t>(imm) & 0xF;
uint32_t Instr = 0b1010'0100'0000'0000'1110'0000'0000'0000;
Instr |= msz << 23;
Instr |= imm4 << 16;
Instr |= pg.Idx() << 10;
Instr |= Encode_rn(rn);
Instr |= zt.Idx();
dc32(Instr);
}
// SVE contiguous non-temporal store (scalar plus immediate)
void SVEContiguousNontemporalStore(uint32_t msz, ZRegister zt, PRegister pg, Register rn, int32_t imm) {
LOGMAN_THROW_A_FMT(pg <= PReg::p7, "Can only use p0-p7 as a governing predicate");
+43 -3
View File
@@ -36,9 +36,9 @@
// to by n, imm_s and imm_r are undefined.
static bool IsImmLogical(uint64_t value,
unsigned width,
unsigned* n,
unsigned* imm_s,
unsigned* imm_r) {
unsigned* n = nullptr,
unsigned* imm_s = nullptr,
unsigned* imm_r = nullptr) {
[[maybe_unused]] constexpr auto kBRegSize = 8;
[[maybe_unused]] constexpr auto kHRegSize = 16;
[[maybe_unused]] constexpr auto kSRegSize = 32;
@@ -243,6 +243,46 @@ static bool IsImmLogical(uint64_t value,
return true;
}
static inline bool IsIntN(unsigned n, int64_t x) {
if (n == 64) return true;
int64_t limit = INT64_C(1) << (n - 1);
return (-limit <= x) && (x < limit);
}
static inline bool IsUintN(unsigned n, int64_t x) {
// Convert to an unsigned integer to avoid implementation-defined behavior.
return !(static_cast<uint64_t>(x) >> n);
}
// clang-format off
#define INT_1_TO_32_LIST(V) \
V(1) V(2) V(3) V(4) V(5) V(6) V(7) V(8) \
V(9) V(10) V(11) V(12) V(13) V(14) V(15) V(16) \
V(17) V(18) V(19) V(20) V(21) V(22) V(23) V(24) \
V(25) V(26) V(27) V(28) V(29) V(30) V(31) V(32)
#define INT_33_TO_63_LIST(V) \
V(33) V(34) V(35) V(36) V(37) V(38) V(39) V(40) \
V(41) V(42) V(43) V(44) V(45) V(46) V(47) V(48) \
V(49) V(50) V(51) V(52) V(53) V(54) V(55) V(56) \
V(57) V(58) V(59) V(60) V(61) V(62) V(63)
#define INT_1_TO_63_LIST(V) INT_1_TO_32_LIST(V) INT_33_TO_63_LIST(V)
// clang-format on
#define DECLARE_IS_INT_N(N) \
static inline bool IsInt##N(int64_t x) { return IsIntN(N, x); }
#define DECLARE_IS_UINT_N(N) \
static inline bool IsUint##N(int64_t x) { return IsUintN(N, x); }
INT_1_TO_63_LIST(DECLARE_IS_INT_N)
INT_1_TO_63_LIST(DECLARE_IS_UINT_N)
#undef DECLARE_IS_INT_N
#undef DECLARE_IS_UINT_N
private:
template <typename V>
+11 -2
View File
@@ -13,5 +13,14 @@ function(GenBinFmt Name)
DESTINATION ${CMAKE_INSTALL_PREFIX}/share/binfmts/)
endfunction()
GenBinFmt(FEX-x86.in)
GenBinFmt(FEX-x86_64.in)
if (NOT USE_LEGACY_BINFMTMISC)
configure_file(FEX-x86.conf.in ${CMAKE_BINARY_DIR}/Data/binfmts/FEX-x86.conf)
configure_file(FEX-x86_64.conf.in ${CMAKE_BINARY_DIR}/Data/binfmts/FEX-x86_64.conf)
install(
FILES ${CMAKE_BINARY_DIR}/Data/binfmts/FEX-x86.conf ${CMAKE_BINARY_DIR}/Data/binfmts/FEX-x86_64.conf
DESTINATION ${CMAKE_INSTALL_PREFIX}/lib/binfmt.d/)
else()
GenBinFmt(FEX-x86.in)
GenBinFmt(FEX-x86_64.in)
endif()
+1
View File
@@ -0,0 +1 @@
:FEX-x86:M:0:\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x03\x00:\xff\xff\xff\xff\xff\xfe\xfe\x00\x00\x00\x00\xff\xff\xff\xff\xff\xfe\xff\xff\xff:@CMAKE_INSTALL_PREFIX@/bin/FEXInterpreter:POCF
-1
View File
@@ -6,4 +6,3 @@ mask \xff\xff\xff\xff\xff\xfe\xfe\x00\x00\x00\x00\xff\xff\xff\xff\xff\xfe\xff\xf
credentials yes
fix_binary yes
preserve yes
expose_interpreter optional
+1
View File
@@ -0,0 +1 @@
:FEX-x86_64:M:0:\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x3e\x00:\xff\xff\xff\xff\xff\xfe\xfe\x00\x00\x00\x00\xff\xff\xff\xff\xff\xfe\xff\xff\xff:@CMAKE_INSTALL_PREFIX@/bin/FEXInterpreter:POCF
-1
View File
@@ -6,4 +6,3 @@ mask \xff\xff\xff\xff\xff\xfe\xfe\x00\x00\x00\x00\xff\xff\xff\xff\xff\xfe\xff\xf
credentials yes
fix_binary yes
preserve yes
expose_interpreter optional
+1 -1
-1
Submodule External/imgui deleted from 4c986ecb8d.
Submodule External/json-maker deleted from 8ecb8ecc34.
Submodule External/tiny-json deleted from 9d09127f87.
+3
View File
@@ -0,0 +1,3 @@
set(NAME tiny-json)
set(SRCS tiny-json.c)
add_library(${NAME} ${SRCS})
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2018 Rafa Garcia
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+647
View File
@@ -0,0 +1,647 @@
/*
<https://github.com/rafagafe/tiny-json>
Licensed under the MIT License <http://opensource.org/licenses/MIT>.
SPDX-License-Identifier: MIT
Copyright (c) 2016-2018 Rafa Garcia <rafagarcia77@gmail.com>.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
#include <stdio.h>
#include <string.h>
#include <ctype.h>
#include <stddef.h> // For NULL
#include "tiny-json.h"
/** Structure to handle a heap of JSON properties. */
typedef struct jsonStaticPool_s {
json_t* const mem; /**< Pointer to array of json properties. */
unsigned int const qty; /**< Length of the array of json properties. */
unsigned int nextFree; /**< The index of the next free json property. */
jsonPool_t pool;
} jsonStaticPool_t;
/* Search a property by its name in a JSON object. */
json_t const* json_getProperty( json_t const* obj, char const* property ) {
json_t const* sibling;
for( sibling = obj->u.c.child; sibling; sibling = sibling->sibling )
if ( sibling->name && !strcmp( sibling->name, property ) )
return sibling;
return 0;
}
/* Search a property by its name in a JSON object and return its value. */
char const* json_getPropertyValue( json_t const* obj, char const* property ) {
json_t const* field = json_getProperty( obj, property );
if ( !field ) return 0;
jsonType_t type = json_getType( field );
if ( JSON_ARRAY >= type ) return 0;
return json_getValue( field );
}
/* Internal prototypes: */
static char* goBlank( char* str );
static char* goNum( char* str );
static json_t* poolInit( jsonPool_t* pool );
static json_t* poolAlloc( jsonPool_t* pool );
static char* objValue( char* ptr, json_t* obj, jsonPool_t* pool );
static char* setToNull( char* ch );
static bool isEndOfPrimitive( char ch );
/* Parse a string to get a json. */
json_t const* json_createWithPool( char *str, jsonPool_t *pool ) {
char* ptr = goBlank( str );
if ( !ptr || *ptr != '{' ) return 0;
json_t* obj = pool->init( pool );
obj->name = 0;
obj->sibling = 0;
obj->u.c.child = 0;
ptr = objValue( ptr, obj, pool );
if ( !ptr ) return 0;
return obj;
}
/* Parse a string to get a json. */
json_t const* json_create( char* str, json_t mem[], unsigned int qty ) {
jsonStaticPool_t spool = {
.mem = mem,
.qty = qty,
.pool = {
.init = poolInit,
.alloc = poolAlloc
}
};
return json_createWithPool( str, &spool.pool );
}
/** Get a special character with its escape character. Examples:
* 'b' -> '\b', 'n' -> '\n', 't' -> '\t'
* @param ch The escape character.
* @return The character code. */
static char getEscape( char ch ) {
static struct { char ch; char code; } const pair[] = {
{ '\"', '\"' }, { '\\', '\\' },
{ '/', '/' }, { 'b', '\b' },
{ 'f', '\f' }, { 'n', '\n' },
{ 'r', '\r' }, { 't', '\t' },
};
unsigned int i;
for( i = 0; i < sizeof pair / sizeof *pair; ++i )
if ( pair[i].ch == ch )
return pair[i].code;
return '\0';
}
/** Parse 4 characters.
* @Param str Pointer to first digit.
* @retval '?' If the four characters are hexadecimal digits.
* @retcal '\0' In other cases. */
static unsigned char getCharFromUnicode( unsigned char const* str ) {
unsigned int i;
for( i = 0; i < 4; ++i )
if ( !isxdigit( str[i] ) )
return '\0';
return '?';
}
/** Parse a string and replace the scape characters by their meaning characters.
* This parser stops when finds the character '\"'. Then replaces '\"' by '\0'.
* @param str Pointer to first character.
* @retval Pointer to first non white space after the string. If success.
* @retval Null pointer if any error occur. */
static char* parseString( char* str ) {
unsigned char* head = (unsigned char*)str;
unsigned char* tail = (unsigned char*)str;
for( ; *head >= ' '; ++head, ++tail ) {
if ( *head == '\"' ) {
*tail = '\0';
return (char*)++head;
}
if ( *head == '\\' ) {
if ( *++head == 'u' ) {
char const ch = getCharFromUnicode( ++head );
if ( ch == '\0' ) return 0;
*tail = ch;
head += 3;
}
else {
char const esc = getEscape( *head );
if ( esc == '\0' ) return 0;
*tail = esc;
}
}
else *tail = *head;
}
return 0;
}
/** Parse a string to get the name of a property.
* @param str Pointer to first character.
* @param property The property to assign the name.
* @retval Pointer to first of property value. If success.
* @retval Null pointer if any error occur. */
static char* propertyName( char* ptr, json_t* property ) {
property->name = ++ptr;
ptr = parseString( ptr );
if ( !ptr ) return 0;
ptr = goBlank( ptr );
if ( !ptr ) return 0;
if ( *ptr++ != ':' ) return 0;
return goBlank( ptr );
}
/** Parse a string to get the value of a property when its type is JSON_TEXT.
* @param str Pointer to first character ('\"').
* @param property The property to assign the name.
* @retval Pointer to first non white space after the string. If success.
* @retval Null pointer if any error occur. */
static char* textValue( char* ptr, json_t* property ) {
++property->u.value;
ptr = parseString( ++ptr );
if ( !ptr ) return 0;
property->type = JSON_TEXT;
return ptr;
}
/** Compare two strings until get the null character in the second one.
* @param ptr sub string
* @param str main string
* @retval Pointer to next character.
* @retval Null pointer if any error occur. */
static char* checkStr( char* ptr, char const* str ) {
while( *str )
if ( *ptr++ != *str++ )
return 0;
return ptr;
}
/** Parser a string to get a primitive value.
* If the first character after the value is different of '}' or ']' is set to '\0'.
* @param str Pointer to first character.
* @param property Property handler to set the value and the type, (true, false or null).
* @param value String with the primitive literal.
* @param type The code of the type. ( JSON_BOOLEAN or JSON_NULL )
* @retval Pointer to first non white space after the string. If success.
* @retval Null pointer if any error occur. */
static char* primitiveValue( char* ptr, json_t* property, char const* value, jsonType_t type ) {
ptr = checkStr( ptr, value );
if ( !ptr || !isEndOfPrimitive( *ptr ) ) return 0;
ptr = setToNull( ptr );
property->type = type;
return ptr;
}
/** Parser a string to get a true value.
* If the first character after the value is different of '}' or ']' is set to '\0'.
* @param str Pointer to first character.
* @param property Property handler to set the value and the type, (true, false or null).
* @retval Pointer to first non white space after the string. If success.
* @retval Null pointer if any error occur. */
static char* trueValue( char* ptr, json_t* property ) {
return primitiveValue( ptr, property, "true", JSON_BOOLEAN );
}
/** Parser a string to get a false value.
* If the first character after the value is different of '}' or ']' is set to '\0'.
* @param str Pointer to first character.
* @param property Property handler to set the value and the type, (true, false or null).
* @retval Pointer to first non white space after the string. If success.
* @retval Null pointer if any error occur. */
static char* falseValue( char* ptr, json_t* property ) {
return primitiveValue( ptr, property, "false", JSON_BOOLEAN );
}
/** Parser a string to get a null value.
* If the first character after the value is different of '}' or ']' is set to '\0'.
* @param str Pointer to first character.
* @param property Property handler to set the value and the type, (true, false or null).
* @retval Pointer to first non white space after the string. If success.
* @retval Null pointer if any error occur. */
static char* nullValue( char* ptr, json_t* property ) {
return primitiveValue( ptr, property, "null", JSON_NULL );
}
/** Analyze the exponential part of a real number.
* @param str Pointer to first character.
* @retval Pointer to first non numerical after the string. If success.
* @retval Null pointer if any error occur. */
static char* expValue( char* ptr ) {
if ( *ptr == '-' || *ptr == '+' ) ++ptr;
if ( !isdigit( *ptr ) ) return 0;
ptr = goNum( ++ptr );
return ptr;
}
/** Analyze the decimal part of a real number.
* @param str Pointer to first character.
* @retval Pointer to first non numerical after the string. If success.
* @retval Null pointer if any error occur. */
static char* fraqValue( char* ptr ) {
if ( !isdigit( *ptr ) ) return 0;
ptr = goNum( ++ptr );
if ( !ptr ) return 0;
return ptr;
}
/** Parser a string to get a numerical value.
* If the first character after the value is different of '}' or ']' is set to '\0'.
* @param str Pointer to first character.
* @param property Property handler to set the value and the type: JSON_REAL or JSON_INTEGER.
* @retval Pointer to first non white space after the string. If success.
* @retval Null pointer if any error occur. */
static char* numValue( char* ptr, json_t* property ) {
if ( *ptr == '-' ) ++ptr;
if ( !isdigit( *ptr ) ) return 0;
if ( *ptr != '0' ) {
ptr = goNum( ptr );
if ( !ptr ) return 0;
}
else if ( isdigit( *++ptr ) ) return 0;
property->type = JSON_INTEGER;
if ( *ptr == '.' ) {
ptr = fraqValue( ++ptr );
if ( !ptr ) return 0;
property->type = JSON_REAL;
}
if ( *ptr == 'e' || *ptr == 'E' ) {
ptr = expValue( ++ptr );
if ( !ptr ) return 0;
property->type = JSON_REAL;
}
if ( !isEndOfPrimitive( *ptr ) ) return 0;
if ( JSON_INTEGER == property->type ) {
char const* value = property->u.value;
bool const negative = *value == '-';
static char const min[] = "-9223372036854775808";
static char const max[] = "9223372036854775807";
unsigned int const maxdigits = ( negative? sizeof min: sizeof max ) - 1;
unsigned int const len = ptr - value;
if ( len > maxdigits ) return 0;
if ( len == maxdigits ) {
char const tmp = *ptr;
*ptr = '\0';
char const* const threshold = negative ? min: max;
if ( 0 > strcmp( threshold, value ) ) return 0;
*ptr = tmp;
}
}
ptr = setToNull( ptr );
return ptr;
}
/** Add a property to a JSON object or array.
* @param obj The handler of the JSON object or array.
* @param property The handler of the property to be added. */
static void add( json_t* obj, json_t* property ) {
property->sibling = 0;
if ( !obj->u.c.child ){
obj->u.c.child = property;
obj->u.c.last_child = property;
} else {
obj->u.c.last_child->sibling = property;
obj->u.c.last_child = property;
}
}
/** Parser a string to get a json object value.
* @param str Pointer to first character.
* @param pool The handler of a json pool for creating json instances.
* @retval Pointer to first character after the value. If success.
* @retval Null pointer if any error occur. */
static char* objValue( char* ptr, json_t* obj, jsonPool_t* pool ) {
obj->type = JSON_OBJ;
obj->u.c.child = 0;
obj->sibling = 0;
ptr++;
for(;;) {
ptr = goBlank( ptr );
if ( !ptr ) return 0;
if ( *ptr == ',' ) {
++ptr;
continue;
}
char const endchar = ( obj->type == JSON_OBJ )? '}': ']';
if ( *ptr == endchar ) {
*ptr = '\0';
json_t* parentObj = obj->sibling;
if ( !parentObj ) return ++ptr;
obj->sibling = 0;
obj = parentObj;
++ptr;
continue;
}
json_t* property = pool->alloc( pool );
if ( !property ) return 0;
if( obj->type != JSON_ARRAY ) {
if ( *ptr != '\"' ) return 0;
ptr = propertyName( ptr, property );
if ( !ptr ) return 0;
}
else property->name = 0;
add( obj, property );
property->u.value = ptr;
switch( *ptr ) {
case '{':
property->type = JSON_OBJ;
property->u.c.child = 0;
property->sibling = obj;
obj = property;
++ptr;
break;
case '[':
property->type = JSON_ARRAY;
property->u.c.child = 0;
property->sibling = obj;
obj = property;
++ptr;
break;
case '\"': ptr = textValue( ptr, property ); break;
case 't': ptr = trueValue( ptr, property ); break;
case 'f': ptr = falseValue( ptr, property ); break;
case 'n': ptr = nullValue( ptr, property ); break;
default: ptr = numValue( ptr, property ); break;
}
if ( !ptr ) return 0;
}
}
/** Initialize a json pool.
* @param pool The handler of the pool.
* @return a instance of a json. */
static json_t* poolInit( jsonPool_t* pool ) {
jsonStaticPool_t *spool = json_containerOf( pool, jsonStaticPool_t, pool );
spool->nextFree = 1;
return spool->mem;
}
/** Create an instance of a json from a pool.
* @param pool The handler of the pool.
* @retval The handler of the new instance if success.
* @retval Null pointer if the pool was empty. */
static json_t* poolAlloc( jsonPool_t* pool ) {
jsonStaticPool_t *spool = json_containerOf( pool, jsonStaticPool_t, pool );
if ( spool->nextFree >= spool->qty ) return 0;
return spool->mem + spool->nextFree++;
}
/** Checks whether an character belongs to set.
* @param ch Character value to be checked.
* @param set Set of characters. It is just a null-terminated string.
* @return true or false there is membership or not. */
static bool isOneOfThem( char ch, char const* set ) {
while( *set != '\0' )
if ( ch == *set++ )
return true;
return false;
}
/** Increases a pointer while it points to a character that belongs to a set.
* @param str The initial pointer value.
* @param set Set of characters. It is just a null-terminated string.
* @return The final pointer value or null pointer if the null character was found. */
static char* goWhile( char* str, char const* set ) {
for(; *str != '\0'; ++str ) {
if ( !isOneOfThem( *str, set ) )
return str;
}
return 0;
}
/** Set of characters that defines a blank. */
static char const* const blank = " \n\r\t\f";
/** Increases a pointer while it points to a white space character.
* @param str The initial pointer value.
* @return The final pointer value or null pointer if the null character was found. */
static char* goBlank( char* str ) {
return goWhile( str, blank );
}
/** Increases a pointer while it points to a decimal digit character.
* @param str The initial pointer value.
* @return The final pointer value or null pointer if the null character was found. */
static char* goNum( char* str ) {
for( ; *str != '\0'; ++str ) {
if ( !isdigit( *str ) )
return str;
}
return 0;
}
/** Set of characters that defines the end of an array or a JSON object. */
static char const* const endofblock = "}]";
/** Set a char to '\0' and increase its pointer if the char is different to '}' or ']'.
* @param ch Pointer to character.
* @return Final value pointer. */
static char* setToNull( char* ch ) {
if ( !isOneOfThem( *ch, endofblock ) ) *ch++ = '\0';
return ch;
}
/** Indicate if a character is the end of a primitive value. */
static bool isEndOfPrimitive( char ch ) {
return ch == ',' || isOneOfThem( ch, blank ) || isOneOfThem( ch, endofblock );
}
/** Add a character at the end of a string.
* @param dest Pointer to the null character of the string
* @param ch Value to be added.
* @return Pointer to the null character of the destination string. */
static char* chtoa( char* dest, char ch ) {
*dest = ch;
*++dest = '\0';
return dest;
}
/** Copy a null-terminated string.
* @param dest Destination memory block.
* @param src Source string.
* @return Pointer to the null character of the destination string. */
static char* atoa( char* dest, char const* src ) {
for( ; *src != '\0'; ++dest, ++src )
*dest = *src;
*dest = '\0';
return dest;
}
/* Open a JSON object in a JSON string. */
char* json_objOpen( char* dest, char const* name ) {
if ( NULL == name )
dest = chtoa( dest, '{' );
else {
dest = chtoa( dest, '\"' );
dest = atoa( dest, name );
dest = atoa( dest, "\":{" );
}
return dest;
}
/* Close a JSON object in a JSON string. */
char* json_objClose( char* dest ) {
if ( dest[-1] == ',' )
--dest;
return atoa( dest, "}," );
}
/* Open an array in a JSON string. */
char* json_arrOpen( char* dest, char const* name ) {
if ( NULL == name )
dest = chtoa( dest, '[' );
else {
dest = chtoa( dest, '\"' );
dest = atoa( dest, name );
dest = atoa( dest, "\":[" );
}
return dest;
}
/* Close an array in a JSON string. */
char* json_arrClose( char* dest ) {
if ( dest[-1] == ',' )
--dest;
return atoa( dest, "]," );
}
/** Add the name of a text property.
* @param dest Destination memory.
* @param name The name of the property.
* @return Pointer to the next char. */
static char* strname( char* dest, char const* name ) {
dest = chtoa( dest, '\"' );
if ( NULL != name ) {
dest = atoa( dest, name );
dest = atoa( dest, "\":\"" );
}
return dest;
}
/** Get the hexadecimal digit of the least significant nibble of a integer. */
static int nibbletoch( int nibble ) {
return "0123456789ABCDEF"[ nibble % 16u ];
}
/** Get the escape character of a non-printable.
* @param ch Character source.
* @return The escape character or null character if error. */
static int escape( int ch ) {
static struct { char code; char ch; } const pair[] = {
{ '\"', '\"' }, { '\\', '\\' }, { '/', '/' }, { 'b', '\b' },
{ 'f', '\f' }, { 'n', '\n' }, { 'r', '\r' }, { 't', '\t' },
};
for( int i = 0; i < sizeof pair / sizeof *pair; ++i )
if ( ch == pair[i].ch )
return pair[i].code;
return '\0';
}
/** Copy a null-terminated string inserting escape characters if needed.
* @param dest Destination memory block.
* @param src Source string.
* @return Pointer to the null character of the destination string. */
static char* atoesc( char* dest, char const* src ) {
for( ; *src != '\0'; ++dest, ++src ) {
if ( *src >= ' ' && *src != '\"' && *src != '\\' && *src != '/' )
*dest = *src;
else {
*dest++ = '\\';
int const esc = escape( *src );
if ( esc )
*dest = esc;
else {
*dest++ = 'u';
*dest++ = '0';
*dest++ = '0';
*dest++ = nibbletoch( *src / 16 );
*dest++ = nibbletoch( *src );
}
}
}
*dest = '\0';
return dest;
}
/* Add a text property in a JSON string. */
char* json_str( char* dest, char const* name, char const* value ) {
dest = strname( dest, name );
dest = atoesc( dest, value );
dest = atoa( dest, "\"," );
return dest;
}
/** Add the name of a primitive property.
* @param dest Destination memory.
* @param name The name of the property.
* @return Pointer to the next char. */
static char* primitivename( char* dest, char const* name ) {
if( NULL == name )
return dest;
dest = chtoa( dest, '\"' );
dest = atoa( dest, name );
dest = atoa( dest, "\":" );
return dest;
}
/* Add a boolean property in a JSON string. */
char* json_bool( char* dest, char const* name, int value ) {
dest = primitivename( dest, name );
dest = atoa( dest, value ? "true," : "false," );
return dest;
}
/* Add a null property in a JSON string. */
char* json_null( char* dest, char const* name ) {
dest = primitivename( dest, name );
dest = atoa( dest, "null," );
return dest;
}
/* Used to finish the root JSON object. After call json_objClose(). */
char* json_end( char* dest ) {
if ( ',' == dest[-1] ) {
dest[-1] = '\0';
--dest;
}
return dest;
}
#define ALL_TYPES \
X( json_int, int, "%d" ) \
X( json_long, long, "%ld" ) \
X( json_uint, unsigned int, "%u" ) \
X( json_ulong, unsigned long, "%lu" ) \
X( json_verylong, long long, "%lld" ) \
X( json_double, double, "%g" ) \
#define json_num( funcname, type, fmt ) \
char* funcname( char* dest, char const* name, type value ) { \
dest = primitivename( dest, name ); \
dest += sprintf( dest, fmt, value ); \
dest = chtoa( dest, ',' ); \
return dest; \
}
#define X( name, type, fmt ) json_num( name, type, fmt )
ALL_TYPES
#undef X
+270
View File
@@ -0,0 +1,270 @@
/*
<https://github.com/rafagafe/tiny-json>
Licensed under the MIT License <http://opensource.org/licenses/MIT>.
SPDX-License-Identifier: MIT
Copyright (c) 2016-2018 Rafa Garcia <rafagarcia77@gmail.com>.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
#ifndef _TINY_JSON_H_
#define _TINY_JSON_H_
#ifdef __cplusplus
extern "C" {
#endif
#include <stddef.h>
#include <stdlib.h>
#include <stdbool.h>
#include <stdint.h>
#define json_containerOf( ptr, type, member ) \
((type*)( (char*)ptr - offsetof( type, member ) ))
/** @defgroup tinyJson Tiny JSON parser.
* @{ */
/** Enumeration of codes of supported JSON properties types. */
typedef enum {
JSON_OBJ, JSON_ARRAY, JSON_TEXT, JSON_BOOLEAN,
JSON_INTEGER, JSON_REAL, JSON_NULL
} jsonType_t;
/** Structure to handle JSON properties. */
typedef struct json_s {
struct json_s* sibling;
const char* name;
union {
const char* value;
struct {
struct json_s* child;
struct json_s* last_child;
} c;
} u;
jsonType_t type;
} json_t;
/** Parse a string to get a json.
* @param str String pointer with a JSON object. It will be modified.
* @param mem Array of json properties to allocate.
* @param qty Number of elements of mem.
* @retval Null pointer if any was wrong in the parse process.
* @retval If the parser process was successfully a valid handler of a json.
* This property is always unnamed and its type is JSON_OBJ. */
const json_t* json_create(char* str, json_t mem[], unsigned int qty);
/** Get the name of a json property.
* @param json A valid handler of a json property.
* @retval Pointer to null-terminated if property has name.
* @retval Null pointer if the property is unnamed. */
static inline const char* json_getName(const json_t* json) {
return json->name;
}
/** Get the value of a json property.
* The type of property cannot be JSON_OBJ or JSON_ARRAY.
* @param json A valid handler of a json property.
* @return Pointer to null-terminated string with the value. */
static inline const char* json_getValue(const json_t* property) {
return property->u.value;
}
/** Get the type of a json property.
* @param json A valid handler of a json property.
* @return The code of type.*/
static inline jsonType_t json_getType(const json_t* json) {
return json->type;
}
/** Get the next sibling of a JSON property that is within a JSON object or array.
* @param json A valid handler of a json property.
* @retval The handler of the next sibling if found.
* @retval Null pointer if the json property is the last one. */
static inline const json_t* json_getSibling(const json_t* json) {
return json->sibling;
}
/** Search a property by its name in a JSON object.
* @param obj A valid handler of a json object. Its type must be JSON_OBJ.
* @param property The name of property to get.
* @retval The handler of the json property if found.
* @retval Null pointer if not found. */
const json_t* json_getProperty(const json_t* obj, const char* property);
/** Search a property by its name in a JSON object and return its value.
* @param obj A valid handler of a json object. Its type must be JSON_OBJ.
* @param property The name of property to get.
* @retval If found a pointer to null-terminated string with the value.
* @retval Null pointer if not found or it is an array or an object. */
const char* json_getPropertyValue(const json_t* obj, const char* property);
/** Get the first property of a JSON object or array.
* @param json A valid handler of a json property.
* Its type must be JSON_OBJ or JSON_ARRAY.
* @retval The handler of the first property if there is.
* @retval Null pointer if the json object has not properties. */
static inline const json_t* json_getChild(const json_t* json) {
return json->u.c.child;
}
/** Get the value of a json boolean property.
* @param property A valid handler of a json object. Its type must be JSON_BOOLEAN.
* @return The value stdbool. */
static inline bool json_getBoolean(const json_t* property) {
return *property->u.value == 't';
}
/** Get the value of a json integer property.
* @param property A valid handler of a json object. Its type must be JSON_INTEGER.
* @return The value stdint. */
static inline int64_t json_getInteger(const json_t* property) {
return atoll( property->u.value );
}
/** Get the value of a json real property.
* @param property A valid handler of a json object. Its type must be JSON_REAL.
* @return The value. */
static inline double json_getReal(const json_t* property) {
return atof( property->u.value );
}
/** Structure to handle a heap of JSON properties. */
typedef struct jsonPool_s jsonPool_t;
struct jsonPool_s {
json_t* (*init)( jsonPool_t* pool );
json_t* (*alloc)( jsonPool_t* pool );
};
/** Parse a string to get a json.
* @param str String pointer with a JSON object. It will be modified.
* @param pool Custom json pool pointer.
* @retval Null pointer if any was wrong in the parse process.
* @retval If the parser process was successfully a valid handler of a json.
* This property is always unnamed and its type is JSON_OBJ. */
const json_t* json_createWithPool(char* str, jsonPool_t* pool);
/** @ } */
/** @defgroup makejoson Make JSON.
* @{ */
/** Open a JSON object in a JSON string.
* @param dest Pointer to the end of JSON under construction.
* @param name Pointer to null-terminated string or null for unnamed.
* @return Pointer to the new end of JSON under construction. */
char* json_objOpen(char* dest, const char* name);
/** Close a JSON object in a JSON string.
* @param dest Pointer to the end of JSON under construction.
* @return Pointer to the new end of JSON under construction. */
char* json_objClose(char* dest);
/** Used to finish the root JSON object. After call json_objClose().
* @param dest Pointer to the end of JSON under construction.
* @return Pointer to the new end of JSON under construction. */
char* json_end(char* dest);
/** Open an array in a JSON string.
* @param dest Pointer to the end of JSON under construction.
* @param name Pointer to null-terminated string or null for unnamed.
* @return Pointer to the new end of JSON under construction. */
char* json_arrOpen(char* dest, const char* name);
/** Close an array in a JSON string.
* @param dest Pointer to the end of JSON under construction.
* @return Pointer to the new end of JSON under construction. */
char* json_arrClose(char* dest);
/** Add a text property in a JSON string.
* @param dest Pointer to the end of JSON under construction.
* @param name Pointer to null-terminated string or null for unnamed.
* @param value A valid null-terminated string with the value.
* Backslash escapes will be added for special characters.
* @return Pointer to the new end of JSON under construction. */
char* json_str(char* dest, const char* name, const char* value);
/** Add a boolean property in a JSON string.
* @param dest Pointer to the end of JSON under construction.
* @param name Pointer to null-terminated string or null for unnamed.
* @param value Zero for false. Non zero for true.
* @return Pointer to the new end of JSON under construction. */
char* json_bool(char* dest, const char* name, int value);
/** Add a null property in a JSON string.
* @param dest Pointer to the end of JSON under construction.
* @param name Pointer to null-terminated string or null for unnamed.
* @return Pointer to the new end of JSON under construction. */
char* json_null(char* dest, const char* name);
/** Add an integer property in a JSON string.
* @param dest Pointer to the end of JSON under construction.
* @param name Pointer to null-terminated string or null for unnamed.
* @param value Value of the property.
* @return Pointer to the new end of JSON under construction. */
char* json_int(char* dest, const char* name, int value);
/** Add an unsigned integer property in a JSON string.
* @param dest Pointer to the end of JSON under construction.
* @param name Pointer to null-terminated string or null for unnamed.
* @param value Value of the property.
* @return Pointer to the new end of JSON under construction. */
char* json_uint(char* dest, const char* name, unsigned int value);
/** Add a long integer property in a JSON string.
* @param dest Pointer to the end of JSON under construction.
* @param name Pointer to null-terminated string or null for unnamed.
* @param value Value of the property.
* @return Pointer to the new end of JSON under construction. */
char* json_long(char* dest, const char* name, long int value);
/** Add an unsigned long integer property in a JSON string.
* @param dest Pointer to the end of JSON under construction.
* @param name Pointer to null-terminated string or null for unnamed.
* @param value Value of the property.
* @return Pointer to the new end of JSON under construction. */
char* json_ulong(char* dest, const char* name, unsigned long int value);
/** Add a long long integer property in a JSON string.
* @param dest Pointer to the end of JSON under construction.
* @param name Pointer to null-terminated string or null for unnamed.
* @param value Value of the property.
* @return Pointer to the new end of JSON under construction. */
char* json_verylong(char* dest, const char* name, long long int value);
/** Add a double precision number property in a JSON string.
* @param dest Pointer to the end of JSON under construction.
* @param name Pointer to null-terminated string or null for unnamed.
* @param value Value of the property.
* @return Pointer to the new end of JSON under construction. */
char* json_double(char* dest, const char* name, double value);
/** @ } */
#ifdef __cplusplus
}
#endif
#endif /* _TINY_JSON_H_ */
-21
View File
@@ -24,27 +24,6 @@ include(CheckCXXCompilerFlag)
include(CheckIncludeFileCXX)
include(CheckCXXSourceCompiles)
set(CMAKE_REQUIRED_FLAGS "-std=c++11 -Wattributes -Werror=attributes")
check_cxx_source_compiles(
"
__attribute__((preserve_all))
int Testy(int a, int b, int c, int d, int e, int f) {
return a + b + c + d + e + f;
}
int main() {
return Testy(0, 1, 2, 3, 4, 5);
}"
HAS_CLANG_PRESERVE_ALL)
unset(CMAKE_REQUIRED_FLAGS)
if (HAS_CLANG_PRESERVE_ALL)
if (MINGW_BUILD)
message(STATUS "Ignoring broken clang::preserve_all support")
set(HAS_CLANG_PRESERVE_ALL FALSE)
else()
message(STATUS "Has clang::preserve_all")
endif()
endif ()
if (EXISTS ${CMAKE_CURRENT_DIR}/External/vixl/)
# Useful to have for freestanding libFEXCore
add_subdirectory(External/vixl/)
+13 -7
View File
@@ -148,9 +148,8 @@ def print_man_options(options):
if (value_type == "strenum"):
Enums = op_vals["Enums"]
output_man.write("\\fBAvailable Options:\\fR\n")
for enum_op_key, enum_op_vals in Enums.items():
output_man.write("{}, ".format(enum_op_vals))
output_man.write("\n")
output_man.write(", ".join(f"{enum_op_val}" for [_, enum_op_val] in Enums.items()))
output_man.write("\n.sp\n")
output_man.write(".El\n")
@@ -179,9 +178,8 @@ def print_man_environment(options):
if (value_type == "strenum"):
Enums = op_vals["Enums"]
output_man.write("\\fBAvailable Options:\\fR\n")
for enum_op_key, enum_op_vals in Enums.items():
output_man.write("{}, ".format(enum_op_vals))
output_man.write("\n")
output_man.write(", ".join(f"{enum_op_val}" for [_, enum_op_val] in Enums.items()))
output_man.write("\n.sp\n")
print_man_environment_tail()
output_man.write(".El\n")
@@ -219,6 +217,14 @@ def print_man_environment_tail():
],
"''", True)
print_man_env_option(
"FEX_PORTABLE",
[
"Allows FEX to run without installation. Global locations for configuration and binfmt_misc are ignored. These files are instead read from <FEXInterpreterPath>/fex-emu/ by default.",
"For further customization, see FEX_APP_CONFIG_LOCATION and FEX_APP_DATA_LOCATION."
],
"''", True)
def print_man_header():
header ='''.Dd {0}
.Dt FEX
@@ -395,7 +401,7 @@ def print_parse_argloader_options(options):
conversion_func = "FEXCore::Config::Handler::{0}(".format(op_vals["ArgumentHandler"])
if (value_type == "str"):
NeedsString = True
conversion_func = "("
conversion_func = "std::move("
if (value_type == "bool"):
# boolean values need a decimal specifier. Otherwise fmt prints strings.
conversion_func = "fextl::fmt::format(\"{:d}\", "
+82 -116
View File
@@ -2,6 +2,7 @@
import json
import sys
from dataclasses import dataclass, field
import textwrap
def ExitError(msg):
print(msg)
@@ -53,6 +54,7 @@ class OpDefinition:
SSAArgNum: int
NonSSAArgNum: int
DynamicDispatch: bool
LoweredX87: bool
JITDispatch: bool
JITDispatchOverride: str
TiedSource: int
@@ -76,6 +78,7 @@ class OpDefinition:
self.SSAArgNum = 0
self.NonSSAArgNum = 0
self.DynamicDispatch = False
self.LoweredX87 = False
self.JITDispatch = True
self.JITDispatchOverride = None
self.TiedSource = -1
@@ -122,21 +125,36 @@ def parse_ops(ops):
RHS = EqualSplit[0].strip()
if len(EqualSplit) > 1:
OpDef.HasDest = True
LHS = EqualSplit[0].strip()
RHS = EqualSplit[1].strip()
# Parse the destination, must be one type of SSA, GPR, or FPR
ResultType = EqualSplit[0].strip()
if ResultType == "SSA":
OpDef.DestType = "SSA" # We don't know this type right now
elif ResultType == "GPR":
OpDef.DestType = "GPR"
elif ResultType == "GPRPair":
OpDef.DestType = "GPRPair"
elif ResultType == "FPR":
OpDef.DestType = "FPR"
if ":" in LHS:
# Named destinations. This is a hack, but so is the entire
# multi-destination support bolten onto the old IR...
#
# Named destinations require side effects because they break
# SSA hard. Validate that.
assert("HasSideEffects" in op_val and op_val["HasSideEffects"])
for Dest in LHS.split(","):
Dest = Dest.strip()
DType, Name = Dest.split(":$")
# If the destination appears also as a source, it is
# read-modify-write.
if Dest in RHS:
# Turn RMW into an in/out source
RHS = RHS.replace(Dest.strip(), f"{DType}:$Inout{Name}")
else:
# Turn named destinations into an out source.
RHS += f", {DType}:$Out{Name}"
else:
ExitError("Unknown destination class type {}. Needs to be one of {SSA, GPR, GPRPair, FPR}".format(ResultType))
# Single anonymous destination
if LHS not in ["SSA", "GPR", "GPRPair", "FPR"]:
ExitError(f"Unknown destination class type {LHS}. Needs to be one of SSA, GPR, GPRPair, FPR")
OpDef.HasDest = True
OpDef.DestType = LHS
# IR Op needs to start with a name
RHS = RHS.split(" ", 1)
@@ -204,7 +222,7 @@ def parse_ops(ops):
(OpArg.Type == "GPR" or
OpArg.Type == "GPRPair" or
OpArg.Type == "FPR")):
OpDef.EmitValidation.append("GetOpRegClass({}) == InvalidClass || WalkFindRegClass({}) == {}Class".format(NameWithPrefix, NameWithPrefix, OpArg.Type))
OpDef.EmitValidation.append(f"GetOpRegClass({ArgName}) == InvalidClass || WalkFindRegClass({ArgName}) == {OpArg.Type}Class")
OpArg.Name = ArgName
OpArg.NameWithPrefix = NameWithPrefix
@@ -250,6 +268,13 @@ def parse_ops(ops):
if "JITDispatchOverride" in op_val:
OpDef.JITDispatchOverride = op_val["JITDispatchOverride"]
if "X87" in op_val:
OpDef.LoweredX87 = op_val["X87"]
# X87 implies !JITDispatch
assert("JITDispatch" not in op_val)
OpDef.JITDispatch = False
if "TiedSource" in op_val:
OpDef.TiedSource = op_val["TiedSource"]
@@ -258,12 +283,8 @@ def parse_ops(ops):
for i in range(len(OpDef.EmitValidation)):
# Patch up all the argument names
for Arg in OpDef.Arguments:
if Arg.Temporary:
# Temporary ops just replace all instances no prefix variant
OpDef.EmitValidation[i] = OpDef.EmitValidation[i].replace(Arg.NameWithPrefix, Arg.Name)
else:
# All other ops replace $ with _ variant for argument passed in
OpDef.EmitValidation[i] = OpDef.EmitValidation[i].replace(Arg.NameWithPrefix, "_{}".format(Arg.Name))
# Temporary ops just replace all instances no prefix variant
OpDef.EmitValidation[i] = OpDef.EmitValidation[i].replace(Arg.NameWithPrefix, Arg.Name)
#OpDef.print()
@@ -368,42 +389,28 @@ def print_ir_sizes():
if op.Name == "Last":
output_file.write("\t-1ULL,\n")
else:
output_file.write("\tsizeof(IROp_{}),\n".format(op.Name))
output_file.write(f"\tsizeof(IROp_{op.Name}),\n")
output_file.write("};\n\n")
output_file.write(textwrap.dedent("""
};
output_file.write("// Make sure our array maps directly to the IROps enum\n")
output_file.write("static_assert(IRSizes[IROps::OP_LAST] == -1ULL);\n\n")
// Make sure our array maps directly to the IROps enum
static_assert(IRSizes[IROps::OP_LAST] == -1ULL);
output_file.write("[[maybe_unused, nodiscard]] static size_t GetSize(IROps Op) { return IRSizes[Op]; }\n\n")
[[maybe_unused, nodiscard]] static size_t GetSize(IROps Op) { return IRSizes[Op]; }
[[nodiscard, gnu::const, gnu::visibility("default")]] std::string_view const& GetName(IROps Op);
[[nodiscard, gnu::const, gnu::visibility("default")]] uint8_t GetArgs(IROps Op);
[[nodiscard, gnu::const, gnu::visibility("default")]] uint8_t GetRAArgs(IROps Op);
[[nodiscard, gnu::const, gnu::visibility("default")]] FEXCore::IR::RegisterClassType GetRegClass(IROps Op);
[[nodiscard, gnu::const, gnu::visibility("default")]] bool HasSideEffects(IROps Op);
[[nodiscard, gnu::const, gnu::visibility("default")]] bool ImplicitFlagClobber(IROps Op);
[[nodiscard, gnu::const, gnu::visibility("default")]] bool GetHasDest(IROps Op);
[[nodiscard, gnu::const, gnu::visibility("default")]] bool LoweredX87(IROps Op);
[[nodiscard, gnu::const, gnu::visibility("default")]] int8_t TiedSource(IROps Op);
output_file.write(
'[[nodiscard, gnu::const, gnu::visibility("default")]] std::string_view const& GetName(IROps Op);\n'
)
output_file.write(
'[[nodiscard, gnu::const, gnu::visibility("default")]] uint8_t GetArgs(IROps Op);\n'
)
output_file.write(
'[[nodiscard, gnu::const, gnu::visibility("default")]] uint8_t GetRAArgs(IROps Op);\n'
)
output_file.write(
'[[nodiscard, gnu::const, gnu::visibility("default")]] FEXCore::IR::RegisterClassType GetRegClass(IROps Op);\n\n'
)
output_file.write(
'[[nodiscard, gnu::const, gnu::visibility("default")]] bool HasSideEffects(IROps Op);\n'
)
output_file.write(
'[[nodiscard, gnu::const, gnu::visibility("default")]] bool ImplicitFlagClobber(IROps Op);\n'
)
output_file.write(
'[[nodiscard, gnu::const, gnu::visibility("default")]] bool GetHasDest(IROps Op);\n'
)
output_file.write(
'[[nodiscard, gnu::const, gnu::visibility("default")]] int8_t TiedSource(IROps Op);\n'
)
output_file.write("#undef IROP_SIZES\n")
output_file.write("#endif\n\n")
#undef IROP_SIZES
#endif
"""))
def print_ir_reg_classes():
output_file.write("#ifdef IROP_REG_CLASSES_IMPL\n")
@@ -493,13 +500,14 @@ def print_ir_getraargs():
def print_ir_hassideeffects():
output_file.write("#ifdef IROP_HASSIDEEFFECTS_IMPL\n")
for array, prop, T in [
("SideEffects", "HasSideEffects", "bool"),
("ImplicitFlagClobbers", "ImplicitFlagClobber", "bool"),
("TiedSources", "TiedSource", "int8_t"),
for prop, T in [
("HasSideEffects", "bool"),
("ImplicitFlagClobber", "bool"),
("LoweredX87", "bool"),
("TiedSource", "int8_t"),
]:
output_file.write(
f"constexpr std::array<{'uint8_t' if T == 'bool' else T}, OP_LAST + 1> {array} = {{\n"
f"constexpr std::array<{'uint8_t' if T == 'bool' else T}, OP_LAST + 1> {prop}_ = {{\n"
)
for op in IROps:
if T == "bool":
@@ -512,7 +520,7 @@ def print_ir_hassideeffects():
output_file.write("};\n\n")
output_file.write(f"{T} {prop}(IROps Op) {{\n")
output_file.write(f" return {array}[Op];\n")
output_file.write(f" return {prop}_[Op];\n")
output_file.write("}\n")
output_file.write("#undef IROP_HASSIDEEFFECTS_IMPL\n")
@@ -671,11 +679,11 @@ def print_ir_allocator_helpers():
output_file.write("{} {}".format(CType, arg.Name));
elif arg.IsSSA:
# SSA value
output_file.write("OrderedNode *_{}".format(arg.Name))
output_file.write("OrderedNode *{}".format(arg.Name))
else:
# User defined op that is stored
CType = IRTypesToCXX[arg.Type].CXXName
output_file.write("{} _{}".format(CType, arg.Name));
output_file.write("{} {}".format(CType, arg.Name));
if arg.DefaultInitializer != None:
output_file.write(" = {}".format(arg.DefaultInitializer))
@@ -689,23 +697,28 @@ def print_ir_allocator_helpers():
if op.ImplicitFlagClobber:
output_file.write("\t\tSaveNZCV(IROps::OP_{});".format(op.Name.upper()))
output_file.write("\t\tauto Op = AllocateOp<IROp_{}, IROps::OP_{}>();\n".format(op.Name, op.Name.upper()))
# We gather the "has x87?" flag as we go. This saves the user from
# having to keep track of whether they emitted any x87.
if op.LoweredX87:
output_file.write("\t\tRecordX87Use();\n")
output_file.write("\t\tauto _Op = AllocateOp<IROp_{}, IROps::OP_{}>();\n".format(op.Name, op.Name.upper()))
if op.SSAArgNum != 0:
output_file.write("\t\tauto ListDataBegin = DualListData.ListBegin();\n")
for arg in op.Arguments:
if arg.IsSSA:
output_file.write("\t\tOp.first->{} = _{}->Wrapped(ListDataBegin);\n".format(arg.Name, arg.Name))
output_file.write("\t\t_Op.first->{} = {}->Wrapped(ListDataBegin);\n".format(arg.Name, arg.Name))
if op.SSAArgNum != 0:
for arg in op.Arguments:
if arg.IsSSA:
output_file.write("\t\t_{}->AddUse();\n".format(arg.Name))
output_file.write("\t\t{}->AddUse();\n".format(arg.Name))
if len(op.Arguments) != 0:
for arg in op.Arguments:
if not arg.Temporary and not arg.IsSSA:
output_file.write("\t\tOp.first->{} = _{};\n".format(arg.Name, arg.Name))
output_file.write("\t\t_Op.first->{} = {};\n".format(arg.Name, arg.Name))
if (op.HasDest):
# We can only infer a size if we have arguments
@@ -715,22 +728,22 @@ def print_ir_allocator_helpers():
if len(op.Arguments) != 0:
for arg in op.Arguments:
if arg.IsSSA:
output_file.write("\t\tuint8_t Size{} = GetOpSize(_{});\n".format(arg.Name, arg.Name))
output_file.write("\t\tuint8_t Size{} = GetOpSize({});\n".format(arg.Name, arg.Name))
for arg in op.Arguments:
if arg.IsSSA:
output_file.write("\t\tInferSize = std::max(InferSize, Size{});\n".format(arg.Name))
output_file.write("\t\tOp.first->Header.Size = InferSize;\n")
output_file.write("\t\t_Op.first->Header.Size = InferSize;\n")
# Some ops without a destination still need an operating size
# Effectively reusing the destination size value for operation size
if op.DestSize != None:
output_file.write("\t\tOp.first->Header.Size = {};\n".format(op.DestSize))
output_file.write("\t\t_Op.first->Header.Size = {};\n".format(op.DestSize))
if op.NumElements == None:
output_file.write("\t\tOp.first->Header.ElementSize = Op.first->Header.Size / ({});\n".format(1))
output_file.write("\t\t_Op.first->Header.ElementSize = _Op.first->Header.Size / ({});\n".format(1))
else:
output_file.write("\t\tOp.first->Header.ElementSize = Op.first->Header.Size / ({});\n".format(op.NumElements))
output_file.write("\t\t_Op.first->Header.ElementSize = _Op.first->Header.Size / ({});\n".format(op.NumElements))
# Insert validation here
if op.EmitValidation != None:
@@ -741,58 +754,12 @@ def print_ir_allocator_helpers():
output_file.write("\tLOGMAN_THROW_A_FMT({}, \"{}\");\n".format(Validation, Sanitized))
output_file.write("\t\t#endif\n")
output_file.write("\t\treturn Op;\n")
output_file.write("\t\treturn _Op;\n")
output_file.write("\t}\n\n")
output_file.write("#undef IROP_ALLOCATE_HELPERS\n")
output_file.write("#endif\n")
def print_ir_parser_switch_helper():
output_file.write("#ifdef IROP_PARSER_SWITCH_HELPERS\n")
for op in IROps:
if op.Name != "Last" and op.SwitchGen:
output_file.write("\tcase FEXCore::IR::IROps::OP_%s: {\n" % (op.Name.upper()))
for i in range(0, len(op.Arguments)):
arg = op.Arguments[i]
LastArg = len(op.Arguments) - i - 1 == 0
if arg.Temporary:
CType = IRTypesToCXX[arg.Type].CXXName
output_file.write("\t\tauto arg{} = DecodeValue<{}>(Def.Args[{}]);\n".format(i, CType, i))
output_file.write("\t\tif (!CheckPrintErrorArg(Def, arg{}.first, {})) return false;\n".format(i, i))
elif arg.IsSSA:
# SSA value
output_file.write("\t\tauto arg{} = DecodeValue<OrderedNode*>(Def.Args[{}]);\n".format(i, i))
output_file.write("\t\tif (!CheckPrintErrorArg(Def, arg{}.first, {})) return false;\n".format(i, i))
else:
# User defined op that is stored
CType = IRTypesToCXX[arg.Type].CXXName
output_file.write("\t\tauto arg{} = DecodeValue<{}>(Def.Args[{}]);\n".format(i, CType, i))
output_file.write("\t\tif (!CheckPrintErrorArg(Def, arg{}.first, {})) return false;\n".format(i, i))
output_file.write("\t\tDef.Node = _{}(\n".format(op.Name))
for i in range(0, len(op.Arguments)):
arg = op.Arguments[i]
LastArg = len(op.Arguments) - i - 1 == 0
output_file.write("\t\t\targ{}.second".format(i))
if not LastArg:
output_file.write(",\n")
else:
output_file.write("\n")
output_file.write("\t\t);\n")
output_file.write("\t\tSSANameMapper[Def.Definition] = Def.Node;\n")
output_file.write("\t\tbreak;\n")
output_file.write("\t}\n")
output_file.write("#undef IROP_PARSER_SWITCH_HELPERS\n")
output_file.write("#endif\n")
def print_ir_dispatcher_defs():
output_dispatch_file.write("#ifdef IROP_DISPATCH_DEFS\n")
for op in IROps:
@@ -851,7 +818,6 @@ print_ir_hassideeffects()
print_ir_gethasdest()
print_ir_arg_printer()
print_ir_allocator_helpers()
print_ir_parser_switch_helper()
output_file.close()
+32 -28
View File
@@ -1,3 +1,4 @@
include(GNUInstallDirs)
set (MAN_DIR share/man CACHE PATH "MAN_DIR")
set (FEXCORE_BASE_SRCS
@@ -67,7 +68,6 @@ set (SRCS
Common/SoftFloat-3e/s_approxRecipSqrt32_1.c
Common/SoftFloat-3e/s_approxRecipSqrt_1Ks.c
Common/SoftFloat-3e/softfloat_raiseFlags.c
Common/SoftFloat-3e/softfloat_state.c
Common/SoftFloat-3e/f64_to_extF80.c
Common/SoftFloat-3e/s_commonNaNToExtF80UI.c
Common/SoftFloat-3e/s_normSubnormalF64Sig.c
@@ -86,12 +86,10 @@ set (SRCS
Common/SoftFloat-3e/s_f32UIToCommonNaN.c
Interface/Context/Context.cpp
Interface/Core/LookupCache.cpp
Interface/Core/BlockSamplingData.cpp
Interface/Core/Core.cpp
Interface/Core/CPUBackend.cpp
Interface/Core/CPUID.cpp
Interface/Core/Frontend.cpp
Interface/Core/HostFeatures.cpp
Interface/Core/ObjectCache/JobHandling.cpp
Interface/Core/ObjectCache/NamedRegionObjectHandler.cpp
Interface/Core/ObjectCache/ObjectCacheService.cpp
@@ -113,7 +111,6 @@ set (SRCS
Interface/Core/JIT/Arm64/BranchOps.cpp
Interface/Core/JIT/Arm64/ConversionOps.cpp
Interface/Core/JIT/Arm64/EncryptionOps.cpp
Interface/Core/JIT/Arm64/FlagOps.cpp
Interface/Core/JIT/Arm64/MemoryOps.cpp
Interface/Core/JIT/Arm64/MiscOps.cpp
Interface/Core/JIT/Arm64/MoveOps.cpp
@@ -121,7 +118,6 @@ set (SRCS
Interface/Core/JIT/Arm64/Arm64Relocations.cpp
Interface/Core/X86Tables/BaseTables.cpp
Interface/Core/X86Tables/DDDTables.cpp
Interface/Core/X86Tables/EVEXTables.cpp
Interface/Core/X86Tables/H0F38Tables.cpp
Interface/Core/X86Tables/H0F3ATables.cpp
Interface/Core/X86Tables/PrimaryGroupTables.cpp
@@ -131,20 +127,18 @@ set (SRCS
Interface/Core/X86Tables/VEXTables.cpp
Interface/Core/X86Tables/X87Tables.cpp
Interface/Core/X86Tables/XOPTables.cpp
Interface/HLE/Thunks/Thunks.cpp
Interface/GDBJIT/GDBJIT.cpp
Interface/IR/AOTIR.cpp
Interface/IR/IRDumper.cpp
Interface/IR/IREmitter.cpp
Interface/IR/PassManager.cpp
Interface/IR/Passes/ConstProp.cpp
Interface/IR/Passes/DeadContextStoreElimination.cpp
Interface/IR/Passes/IRDumperPass.cpp
Interface/IR/Passes/IRValidation.cpp
Interface/IR/Passes/RAValidation.cpp
Interface/IR/Passes/RedundantFlagCalculationElimination.cpp
Interface/IR/Passes/DeadStoreElimination.cpp
Interface/IR/Passes/RegisterAllocationPass.cpp
Interface/IR/Passes/x87StackOptimizationPass.cpp
Utils/Telemetry.cpp
Utils/Threads.cpp
Utils/Profiler.cpp
@@ -161,7 +155,7 @@ if (ENABLE_GLIBC_ALLOCATOR_HOOK_FAULT)
Utils/AllocatorOverride.cpp)
endif()
set(DEFINES -DTHREAD_LOCAL=_Thread_local -DJIT_ARM64)
set(DEFINES -DJIT_ARM64)
if (_M_X86_64)
list(APPEND DEFINES -D_M_X86_64=1)
@@ -171,11 +165,6 @@ if (_M_ARM_64)
list(APPEND DEFINES -D_M_ARM_64=1)
endif()
if (ENABLE_VIXL_SIMULATOR)
# We can run the simulator on both x86-64 or AArch64 hosts
list(APPEND DEFINES -DVIXL_SIMULATOR=1 -DVIXL_INCLUDE_SIMULATOR_AARCH64=1)
endif()
if (ENABLE_VIXL_DISASSEMBLER)
list(APPEND DEFINES -DVIXL_DISASSEMBLER=1)
endif()
@@ -189,7 +178,11 @@ endif()
# Some defines for the softfloat library
list(APPEND DEFINES "-DSOFTFLOAT_BUILTIN_CLZ")
set (LIBS fmt::fmt vixl xxHash::xxhash FEXHeaderUtils CodeEmitter)
set (LIBS fmt::fmt xxHash::xxhash FEXHeaderUtils CodeEmitter)
if (ENABLE_VIXL_DISASSEMBLER OR ENABLE_VIXL_SIMULATOR)
list (APPEND LIBS vixl)
endif()
if (NOT MINGW_BUILD)
list (APPEND LIBS dl)
@@ -200,14 +193,6 @@ else()
endif()
endif()
if (ENABLE_JEMALLOC)
list (APPEND LIBS FEX_jemalloc)
endif()
if (ENABLE_JEMALLOC_GLIBC_ALLOC)
list (APPEND LIBS FEX_jemalloc_glibc)
endif()
# Generate config
configure_file(
${CMAKE_CURRENT_SOURCE_DIR}/Interface/Config/Config.json.in
@@ -377,11 +362,30 @@ AddLibrary(${PROJECT_NAME} STATIC)
AddLibrary(${PROJECT_NAME}_shared SHARED)
if (NOT MINGW_BUILD)
install(TARGETS ${PROJECT_NAME} ${PROJECT_NAME}_shared
install(TARGETS ${PROJECT_NAME}_shared
LIBRARY
DESTINATION lib
COMPONENT Libraries
ARCHIVE
DESTINATION lib
DESTINATION ${CMAKE_INSTALL_LIBDIR}
COMPONENT Libraries)
endif()
# Meta-library to link jemalloc libraries enabled in the build configuration.
# Only needed for targets that run emulation. For others, use JemallocDummy.
add_library(JemallocLibs STATIC Utils/AllocatorHooks.cpp)
if (ENABLE_JEMALLOC)
target_compile_definitions(JemallocLibs PRIVATE ENABLE_JEMALLOC=1 JEMALLOC_NO_RENAME=1)
target_link_libraries(JemallocLibs PUBLIC FEX_jemalloc)
endif()
if (ENABLE_JEMALLOC_GLIBC_ALLOC)
set_source_files_properties(Interface/HLE/Thunks/Thunks.cpp PROPERTIES COMPILE_DEFINITIONS ENABLE_JEMALLOC_GLIBC=1)
target_link_libraries(JemallocLibs INTERFACE FEX_jemalloc_glibc)
endif()
if (NOT MINGW_BUILD)
# Dummy project to use for host tools.
# This overrides use of jemalloc in FEXCore with the normal glibc allocator.
add_library(JemallocDummy STATIC Utils/AllocatorHooks.cpp)
target_include_directories(JemallocDummy PRIVATE "${PROJECT_SOURCE_DIR}/include/")
endif()
# The shared library should always link enabled jemalloc libraries
target_link_libraries(${PROJECT_NAME}_shared JemallocLibs)
+4 -4
View File
@@ -55,7 +55,7 @@ void JITSymbols::RegisterJITSpace(const void* HostAddr, uint32_t CodeSize) {
}
// Buffered JIT symbols.
void JITSymbols::Register(Core::JITSymbolBuffer* Buffer, const void* HostAddr, uint64_t GuestAddr, uint32_t CodeSize) {
void JITSymbols::Register(FEXCore::JITSymbolBuffer* Buffer, const void* HostAddr, uint64_t GuestAddr, uint32_t CodeSize) {
if (fd == -1) {
return;
}
@@ -79,7 +79,7 @@ void JITSymbols::Register(Core::JITSymbolBuffer* Buffer, const void* HostAddr, u
WriteBuffer(Buffer);
}
void JITSymbols::Register(Core::JITSymbolBuffer* Buffer, const void* HostAddr, uint32_t CodeSize, std::string_view Name, uintptr_t Offset) {
void JITSymbols::Register(FEXCore::JITSymbolBuffer* Buffer, const void* HostAddr, uint32_t CodeSize, std::string_view Name, uintptr_t Offset) {
if (fd == -1) {
return;
}
@@ -104,7 +104,7 @@ void JITSymbols::Register(Core::JITSymbolBuffer* Buffer, const void* HostAddr, u
WriteBuffer(Buffer);
}
void JITSymbols::RegisterNamedRegion(Core::JITSymbolBuffer* Buffer, const void* HostAddr, uint32_t CodeSize, std::string_view Name) {
void JITSymbols::RegisterNamedRegion(FEXCore::JITSymbolBuffer* Buffer, const void* HostAddr, uint32_t CodeSize, std::string_view Name) {
if (fd == -1) {
return;
}
@@ -128,7 +128,7 @@ void JITSymbols::RegisterNamedRegion(Core::JITSymbolBuffer* Buffer, const void*
WriteBuffer(Buffer);
}
void JITSymbols::WriteBuffer(Core::JITSymbolBuffer* Buffer, bool ForceWrite) {
void JITSymbols::WriteBuffer(FEXCore::JITSymbolBuffer* Buffer, bool ForceWrite) {
auto Now = std::chrono::steady_clock::now();
if (!ForceWrite) {
if (((Buffer->LastWrite - Now) < Buffer->MAXIMUM_THRESHOLD) && Buffer->Offset < Buffer->NEEDS_WRITE_DISTANCE) {
+26 -6
View File
@@ -11,6 +11,26 @@
#include <string_view>
namespace FEXCore {
// Buffered JIT symbol tracking.
struct JITSymbolBuffer {
// Maximum buffer size to ensure we are a page in size.
constexpr static size_t BUFFER_SIZE = 4096 - (8 * 2);
// Maximum distance until the end of the buffer to do a write.
constexpr static size_t NEEDS_WRITE_DISTANCE = BUFFER_SIZE - 64;
// Maximum time threshhold to wait before a buffer write occurs.
constexpr static std::chrono::milliseconds MAXIMUM_THRESHOLD {100};
JITSymbolBuffer()
: LastWrite {std::chrono::steady_clock::now()} {}
// stead_clock to ensure a monotonic increasing clock.
// In highly stressed situations this can still cause >2% CPU time in vdso_clock_gettime.
// If we need lower CPU time when JIT symbols are enabled then FEX can read the cycle counter directly.
std::chrono::steady_clock::time_point LastWrite {};
size_t Offset {};
char Buffer[BUFFER_SIZE] {};
};
static_assert(sizeof(JITSymbolBuffer) == 4096, "Ensure this is one page in size");
class JITSymbols final {
public:
JITSymbols();
@@ -21,16 +41,16 @@ public:
void RegisterJITSpace(const void* HostAddr, uint32_t CodeSize);
// Allocate JIT buffer.
static fextl::unique_ptr<Core::JITSymbolBuffer> AllocateBuffer() {
return fextl::make_unique<Core::JITSymbolBuffer>();
static fextl::unique_ptr<FEXCore::JITSymbolBuffer> AllocateBuffer() {
return fextl::make_unique<FEXCore::JITSymbolBuffer>();
}
void Register(Core::JITSymbolBuffer* Buffer, const void* HostAddr, uint64_t GuestAddr, uint32_t CodeSize);
void Register(Core::JITSymbolBuffer* Buffer, const void* HostAddr, uint32_t CodeSize, std::string_view Name, uintptr_t Offset);
void RegisterNamedRegion(Core::JITSymbolBuffer* Buffer, const void* HostAddr, uint32_t CodeSize, std::string_view Name);
void Register(FEXCore::JITSymbolBuffer* Buffer, const void* HostAddr, uint64_t GuestAddr, uint32_t CodeSize);
void Register(FEXCore::JITSymbolBuffer* Buffer, const void* HostAddr, uint32_t CodeSize, std::string_view Name, uintptr_t Offset);
void RegisterNamedRegion(FEXCore::JITSymbolBuffer* Buffer, const void* HostAddr, uint32_t CodeSize, std::string_view Name);
private:
int fd {-1};
void WriteBuffer(Core::JITSymbolBuffer* Buffer, bool ForceWrite = false);
void WriteBuffer(FEXCore::JITSymbolBuffer* Buffer, bool ForceWrite = false);
};
} // namespace FEXCore
@@ -41,7 +41,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "softfloat.h"
FEXCORE_PRESERVE_ALL_ATTR
extFloat80_t extF80_add( extFloat80_t a, extFloat80_t b )
extFloat80_t extF80_add( struct softfloat_state *state, extFloat80_t a, extFloat80_t b )
{
union { struct extFloat80M s; extFloat80_t f; } uA;
uint_fast16_t uiA64;
@@ -53,7 +53,7 @@ extFloat80_t extF80_add( extFloat80_t a, extFloat80_t b )
bool signB;
extFloat80_t
(*magsFuncPtr)(
uint_fast16_t, uint_fast64_t, uint_fast16_t, uint_fast64_t, bool );
struct softfloat_state *, uint_fast16_t, uint_fast64_t, uint_fast16_t, uint_fast64_t, bool );
uA.f = a;
uiA64 = uA.s.signExp;
@@ -65,6 +65,6 @@ extFloat80_t extF80_add( extFloat80_t a, extFloat80_t b )
signB = signExtF80UI64( uiB64 );
magsFuncPtr =
(signA == signB) ? softfloat_addMagsExtF80 : softfloat_subMagsExtF80;
return (*magsFuncPtr)( uiA64, uiA0, uiB64, uiB0, signA );
return (*magsFuncPtr)( state, uiA64, uiA0, uiB64, uiB0, signA );
}
@@ -42,7 +42,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "softfloat.h"
FEXCORE_PRESERVE_ALL_ATTR
extFloat80_t extF80_div( extFloat80_t a, extFloat80_t b )
extFloat80_t extF80_div( struct softfloat_state *state, extFloat80_t a, extFloat80_t b )
{
union { struct extFloat80M s; extFloat80_t f; } uA;
uint_fast16_t uiA64;
@@ -107,7 +107,7 @@ extFloat80_t extF80_div( extFloat80_t a, extFloat80_t b )
if ( ! (sigB & UINT64_C( 0x8000000000000000 )) ) {
if ( ! sigB ) {
if ( ! sigA ) goto invalid;
softfloat_raiseFlags( softfloat_flag_infinite );
softfloat_raiseFlags( state, softfloat_flag_infinite );
goto infinity;
}
normExpSig = softfloat_normSubnormalExtF80Sig( sigB );
@@ -169,18 +169,18 @@ extFloat80_t extF80_div( extFloat80_t a, extFloat80_t b )
sigZExtra = (uint64_t) ((uint_fast64_t) q<<41);
return
softfloat_roundPackToExtF80(
signZ, expZ, sigZ, sigZExtra, extF80_roundingPrecision );
state, signZ, expZ, sigZ, sigZExtra, state->roundingPrecision );
/*------------------------------------------------------------------------
*------------------------------------------------------------------------*/
propagateNaN:
uiZ = softfloat_propagateNaNExtF80UI( uiA64, uiA0, uiB64, uiB0 );
uiZ = softfloat_propagateNaNExtF80UI( state, uiA64, uiA0, uiB64, uiB0 );
uiZ64 = uiZ.v64;
uiZ0 = uiZ.v0;
goto uiZ;
/*------------------------------------------------------------------------
*------------------------------------------------------------------------*/
invalid:
softfloat_raiseFlags( softfloat_flag_invalid );
softfloat_raiseFlags( state, softfloat_flag_invalid );
uiZ64 = defaultNaNExtF80UI64;
uiZ0 = defaultNaNExtF80UI0;
goto uiZ;
@@ -42,7 +42,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "softfloat.h"
FEXCORE_PRESERVE_ALL_ATTR
bool extF80_eq( extFloat80_t a, extFloat80_t b )
bool extF80_eq( struct softfloat_state *state, extFloat80_t a, extFloat80_t b )
{
union { struct extFloat80M s; extFloat80_t f; } uA;
uint_fast16_t uiA64;
@@ -62,7 +62,7 @@ bool extF80_eq( extFloat80_t a, extFloat80_t b )
softfloat_isSigNaNExtF80UI( uiA64, uiA0 )
|| softfloat_isSigNaNExtF80UI( uiB64, uiB0 )
) {
softfloat_raiseFlags( softfloat_flag_invalid );
softfloat_raiseFlags( state, softfloat_flag_invalid );
}
return false;
}
@@ -42,7 +42,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "softfloat.h"
FEXCORE_PRESERVE_ALL_ATTR
bool extF80_lt( extFloat80_t a, extFloat80_t b )
bool extF80_lt( struct softfloat_state *state, extFloat80_t a, extFloat80_t b )
{
union { struct extFloat80M s; extFloat80_t f; } uA;
uint_fast16_t uiA64;
@@ -59,7 +59,7 @@ bool extF80_lt( extFloat80_t a, extFloat80_t b )
uiB64 = uB.s.signExp;
uiB0 = uB.s.signif;
if ( isNaNExtF80UI( uiA64, uiA0 ) || isNaNExtF80UI( uiB64, uiB0 ) ) {
softfloat_raiseFlags( softfloat_flag_invalid );
softfloat_raiseFlags( state, softfloat_flag_invalid );
return false;
}
signA = signExtF80UI64( uiA64 );
@@ -42,7 +42,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "softfloat.h"
FEXCORE_PRESERVE_ALL_ATTR
extFloat80_t extF80_mul( extFloat80_t a, extFloat80_t b )
extFloat80_t extF80_mul( struct softfloat_state *state, extFloat80_t a, extFloat80_t b )
{
union { struct extFloat80M s; extFloat80_t f; } uA;
uint_fast16_t uiA64;
@@ -125,11 +125,11 @@ extFloat80_t extF80_mul( extFloat80_t a, extFloat80_t b )
}
return
softfloat_roundPackToExtF80(
signZ, expZ, sig128Z.v64, sig128Z.v0, extF80_roundingPrecision );
state, signZ, expZ, sig128Z.v64, sig128Z.v0, state->roundingPrecision );
/*------------------------------------------------------------------------
*------------------------------------------------------------------------*/
propagateNaN:
uiZ = softfloat_propagateNaNExtF80UI( uiA64, uiA0, uiB64, uiB0 );
uiZ = softfloat_propagateNaNExtF80UI( state, uiA64, uiA0, uiB64, uiB0 );
uiZ64 = uiZ.v64;
uiZ0 = uiZ.v0;
goto uiZ;
@@ -137,7 +137,7 @@ extFloat80_t extF80_mul( extFloat80_t a, extFloat80_t b )
*------------------------------------------------------------------------*/
infArg:
if ( ! magBits ) {
softfloat_raiseFlags( softfloat_flag_invalid );
softfloat_raiseFlags( state, softfloat_flag_invalid );
uiZ64 = defaultNaNExtF80UI64;
uiZ0 = defaultNaNExtF80UI0;
} else {
@@ -42,7 +42,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "softfloat.h"
FEXCORE_PRESERVE_ALL_ATTR
extFloat80_t extF80_rem( extFloat80_t a, extFloat80_t b )
extFloat80_t extF80_rem( struct softfloat_state *state, extFloat80_t a, extFloat80_t b )
{
union { struct extFloat80M s; extFloat80_t f; } uA;
uint_fast16_t uiA64;
@@ -193,18 +193,18 @@ extFloat80_t extF80_rem( extFloat80_t a, extFloat80_t b )
}
return
softfloat_normRoundPackToExtF80(
signRem, rem.v64 | rem.v0 ? expB + 32 : 0, rem.v64, rem.v0, 80 );
state, signRem, rem.v64 | rem.v0 ? expB + 32 : 0, rem.v64, rem.v0, 80 );
/*------------------------------------------------------------------------
*------------------------------------------------------------------------*/
propagateNaN:
uiZ = softfloat_propagateNaNExtF80UI( uiA64, uiA0, uiB64, uiB0 );
uiZ = softfloat_propagateNaNExtF80UI( state, uiA64, uiA0, uiB64, uiB0 );
uiZ64 = uiZ.v64;
uiZ0 = uiZ.v0;
goto uiZ;
/*------------------------------------------------------------------------
*------------------------------------------------------------------------*/
invalid:
softfloat_raiseFlags( softfloat_flag_invalid );
softfloat_raiseFlags( state, softfloat_flag_invalid );
uiZ64 = defaultNaNExtF80UI64;
uiZ0 = defaultNaNExtF80UI0;
goto uiZ;
@@ -43,7 +43,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
FEXCORE_PRESERVE_ALL_ATTR
extFloat80_t
extF80_roundToInt( extFloat80_t a, uint_fast8_t roundingMode, bool exact )
extF80_roundToInt( struct softfloat_state *state, extFloat80_t a, uint_fast8_t roundingMode, bool exact )
{
union { struct extFloat80M s; extFloat80_t f; } uA;
uint_fast16_t uiA64, signUI64;
@@ -80,7 +80,7 @@ extFloat80_t
if ( 0x403E <= exp ) {
if ( exp == 0x7FFF ) {
if ( sigA & UINT64_C( 0x7FFFFFFFFFFFFFFF ) ) {
uiZ = softfloat_propagateNaNExtF80UI( uiA64, sigA, 0, 0 );
uiZ = softfloat_propagateNaNExtF80UI( state, uiA64, sigA, 0, 0 );
uiZ64 = uiZ.v64;
sigZ = uiZ.v0;
goto uiZ;
@@ -93,7 +93,7 @@ extFloat80_t
goto uiZ;
}
if ( exp <= 0x3FFE ) {
if ( exact ) softfloat_exceptionFlags |= softfloat_flag_inexact;
if ( exact ) state->exceptionFlags |= softfloat_flag_inexact;
switch ( roundingMode ) {
case softfloat_round_near_even:
if ( !(sigA & UINT64_C( 0x7FFFFFFFFFFFFFFF )) ) break;
@@ -145,7 +145,7 @@ extFloat80_t
#ifdef SOFTFLOAT_ROUND_ODD
if ( roundingMode == softfloat_round_odd ) sigZ |= lastBitMask;
#endif
if ( exact ) softfloat_exceptionFlags |= softfloat_flag_inexact;
if ( exact ) state->exceptionFlags |= softfloat_flag_inexact;
}
uiZ:
uZ.s.signExp = uiZ64;
@@ -42,7 +42,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "softfloat.h"
FEXCORE_PRESERVE_ALL_ATTR
extFloat80_t extF80_sqrt( extFloat80_t a )
extFloat80_t extF80_sqrt( struct softfloat_state *state, extFloat80_t a )
{
union { struct extFloat80M s; extFloat80_t f; } uA;
uint_fast16_t uiA64;
@@ -74,7 +74,7 @@ extFloat80_t extF80_sqrt( extFloat80_t a )
*------------------------------------------------------------------------*/
if ( expA == 0x7FFF ) {
if ( sigA & UINT64_C( 0x7FFFFFFFFFFFFFFF ) ) {
uiZ = softfloat_propagateNaNExtF80UI( uiA64, uiA0, 0, 0 );
uiZ = softfloat_propagateNaNExtF80UI( state, uiA64, uiA0, 0, 0 );
uiZ64 = uiZ.v64;
uiZ0 = uiZ.v0;
goto uiZ;
@@ -155,11 +155,11 @@ extFloat80_t extF80_sqrt( extFloat80_t a )
}
return
softfloat_roundPackToExtF80(
0, expZ, sigZ, sigZExtra, extF80_roundingPrecision );
state, 0, expZ, sigZ, sigZExtra, state->roundingPrecision );
/*------------------------------------------------------------------------
*------------------------------------------------------------------------*/
invalid:
softfloat_raiseFlags( softfloat_flag_invalid );
softfloat_raiseFlags( state, softfloat_flag_invalid );
uiZ64 = defaultNaNExtF80UI64;
uiZ0 = defaultNaNExtF80UI0;
goto uiZ;
@@ -41,7 +41,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "softfloat.h"
FEXCORE_PRESERVE_ALL_ATTR
extFloat80_t extF80_sub( extFloat80_t a, extFloat80_t b )
extFloat80_t extF80_sub( struct softfloat_state *state, extFloat80_t a, extFloat80_t b )
{
union { struct extFloat80M s; extFloat80_t f; } uA;
uint_fast16_t uiA64;
@@ -54,7 +54,7 @@ extFloat80_t extF80_sub( extFloat80_t a, extFloat80_t b )
#if ! defined INLINE_LEVEL || (INLINE_LEVEL < 2)
extFloat80_t
(*magsFuncPtr)(
uint_fast16_t, uint_fast64_t, uint_fast16_t, uint_fast64_t, bool );
struct softfloat_state *, uint_fast16_t, uint_fast64_t, uint_fast16_t, uint_fast64_t, bool );
#endif
uA.f = a;
@@ -67,14 +67,14 @@ extFloat80_t extF80_sub( extFloat80_t a, extFloat80_t b )
signB = signExtF80UI64( uiB64 );
#if defined INLINE_LEVEL && (2 <= INLINE_LEVEL)
if ( signA == signB ) {
return softfloat_subMagsExtF80( uiA64, uiA0, uiB64, uiB0, signA );
return softfloat_subMagsExtF80( state, uiA64, uiA0, uiB64, uiB0, signA );
} else {
return softfloat_addMagsExtF80( uiA64, uiA0, uiB64, uiB0, signA );
return softfloat_addMagsExtF80( state, uiA64, uiA0, uiB64, uiB0, signA );
}
#else
magsFuncPtr =
(signA == signB) ? softfloat_subMagsExtF80 : softfloat_addMagsExtF80;
return (*magsFuncPtr)( uiA64, uiA0, uiB64, uiB0, signA );
return (*magsFuncPtr)( state, uiA64, uiA0, uiB64, uiB0, signA );
#endif
}
@@ -42,7 +42,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "softfloat.h"
FEXCORE_PRESERVE_ALL_ATTR
float128_t extF80_to_f128( extFloat80_t a )
float128_t extF80_to_f128( struct softfloat_state *state, extFloat80_t a )
{
union { struct extFloat80M s; extFloat80_t f; } uA;
uint_fast16_t uiA64;
@@ -61,7 +61,7 @@ float128_t extF80_to_f128( extFloat80_t a )
exp = expExtF80UI64( uiA64 );
frac = uiA0 & UINT64_C( 0x7FFFFFFFFFFFFFFF );
if ( (exp == 0x7FFF) && frac ) {
softfloat_extF80UIToCommonNaN( uiA64, uiA0, &commonNaN );
softfloat_extF80UIToCommonNaN( state, uiA64, uiA0, &commonNaN );
uiZ = softfloat_commonNaNToF128UI( &commonNaN );
} else {
sign = signExtF80UI64( uiA64 );
@@ -42,7 +42,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "softfloat.h"
FEXCORE_PRESERVE_ALL_ATTR
float32_t extF80_to_f32( extFloat80_t a )
float32_t extF80_to_f32( struct softfloat_state *state, extFloat80_t a )
{
union { struct extFloat80M s; extFloat80_t f; } uA;
uint_fast16_t uiA64;
@@ -66,7 +66,7 @@ float32_t extF80_to_f32( extFloat80_t a )
*------------------------------------------------------------------------*/
if ( exp == 0x7FFF ) {
if ( sig & UINT64_C( 0x7FFFFFFFFFFFFFFF ) ) {
softfloat_extF80UIToCommonNaN( uiA64, uiA0, &commonNaN );
softfloat_extF80UIToCommonNaN( state, uiA64, uiA0, &commonNaN );
uiZ = softfloat_commonNaNToF32UI( &commonNaN );
} else {
uiZ = packToF32UI( sign, 0xFF, 0 );
@@ -86,7 +86,7 @@ float32_t extF80_to_f32( extFloat80_t a )
if ( sizeof (int_fast16_t) < sizeof (int_fast32_t) ) {
if ( exp < -0x1000 ) exp = -0x1000;
}
return softfloat_roundPackToF32( sign, exp, sig32 );
return softfloat_roundPackToF32( state, sign, exp, sig32 );
/*------------------------------------------------------------------------
*------------------------------------------------------------------------*/
uiZ:
@@ -42,7 +42,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "softfloat.h"
FEXCORE_PRESERVE_ALL_ATTR
float64_t extF80_to_f64( extFloat80_t a )
float64_t extF80_to_f64( struct softfloat_state *state, extFloat80_t a )
{
union { struct extFloat80M s; extFloat80_t f; } uA;
uint_fast16_t uiA64;
@@ -72,7 +72,7 @@ float64_t extF80_to_f64( extFloat80_t a )
*------------------------------------------------------------------------*/
if ( exp == 0x7FFF ) {
if ( sig & UINT64_C( 0x7FFFFFFFFFFFFFFF ) ) {
softfloat_extF80UIToCommonNaN( uiA64, uiA0, &commonNaN );
softfloat_extF80UIToCommonNaN( state, uiA64, uiA0, &commonNaN );
uiZ = softfloat_commonNaNToF64UI( &commonNaN );
} else {
uiZ = packToF64UI( sign, 0x7FF, 0 );
@@ -86,7 +86,7 @@ float64_t extF80_to_f64( extFloat80_t a )
if ( sizeof (int_fast16_t) < sizeof (int_fast32_t) ) {
if ( exp < -0x1000 ) exp = -0x1000;
}
return softfloat_roundPackToF64( sign, exp, sig );
return softfloat_roundPackToF64( state, sign, exp, sig );
/*------------------------------------------------------------------------
*------------------------------------------------------------------------*/
uiZ:
@@ -43,7 +43,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
FEXCORE_PRESERVE_ALL_ATTR
int_fast32_t
extF80_to_i32( extFloat80_t a, uint_fast8_t roundingMode, bool exact )
extF80_to_i32( struct softfloat_state *state, extFloat80_t a, uint_fast8_t roundingMode, bool exact )
{
union { struct extFloat80M s; extFloat80_t f; } uA;
uint_fast16_t uiA64;
@@ -68,7 +68,7 @@ int_fast32_t
#elif (i32_fromNaN == i32_fromNegOverflow)
sign = 1;
#else
softfloat_raiseFlags( softfloat_flag_invalid );
softfloat_raiseFlags( state, softfloat_flag_invalid );
return i32_fromNaN;
#endif
}
@@ -78,7 +78,7 @@ int_fast32_t
shiftDist = 0x4032 - exp;
if ( shiftDist <= 0 ) shiftDist = 1;
sig = softfloat_shiftRightJam64( sig, shiftDist );
return softfloat_roundToI32( sign, sig, roundingMode, exact );
return softfloat_roundToI32( state, sign, sig, roundingMode, exact );
}
@@ -43,7 +43,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
FEXCORE_PRESERVE_ALL_ATTR
int_fast64_t
extF80_to_i64( extFloat80_t a, uint_fast8_t roundingMode, bool exact )
extF80_to_i64( struct softfloat_state *state, extFloat80_t a, uint_fast8_t roundingMode, bool exact )
{
union { struct extFloat80M s; extFloat80_t f; } uA;
uint_fast16_t uiA64;
@@ -68,7 +68,7 @@ int_fast64_t
/*--------------------------------------------------------------------
*--------------------------------------------------------------------*/
if ( shiftDist ) {
softfloat_raiseFlags( softfloat_flag_invalid );
softfloat_raiseFlags( state, softfloat_flag_invalid );
return
(exp == 0x7FFF) && (sig & UINT64_C( 0x7FFFFFFFFFFFFFFF ))
? i64_fromNaN
@@ -84,7 +84,7 @@ int_fast64_t
sig = sig64Extra.v;
sigExtra = sig64Extra.extra;
}
return softfloat_roundToI64( sign, sig, sigExtra, roundingMode, exact );
return softfloat_roundToI64( state, sign, sig, sigExtra, roundingMode, exact );
}
@@ -43,7 +43,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
FEXCORE_PRESERVE_ALL_ATTR
uint_fast64_t
extF80_to_ui64( extFloat80_t a, uint_fast8_t roundingMode, bool exact )
extF80_to_ui64( struct softfloat_state *state, extFloat80_t a, uint_fast8_t roundingMode, bool exact )
{
union { struct extFloat80M s; extFloat80_t f; } uA;
uint_fast16_t uiA64;
@@ -65,7 +65,7 @@ uint_fast64_t
*------------------------------------------------------------------------*/
shiftDist = 0x403E - exp;
if ( shiftDist < 0 ) {
softfloat_raiseFlags( softfloat_flag_invalid );
softfloat_raiseFlags( state, softfloat_flag_invalid );
return
(exp == 0x7FFF) && (sig & UINT64_C( 0x7FFFFFFFFFFFFFFF ))
? ui64_fromNaN
@@ -79,7 +79,7 @@ uint_fast64_t
sig = sig64Extra.v;
sigExtra = sig64Extra.extra;
}
return softfloat_roundToUI64( sign, sig, sigExtra, roundingMode, exact );
return softfloat_roundToUI64( state, sign, sig, sigExtra, roundingMode, exact );
}
@@ -42,7 +42,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "softfloat.h"
FEXCORE_PRESERVE_ALL_ATTR
extFloat80_t f128_to_extF80( float128_t a )
extFloat80_t f128_to_extF80( struct softfloat_state *state, float128_t a )
{
union ui128_f128 uA;
uint_fast64_t uiA64, uiA0;
@@ -70,7 +70,7 @@ extFloat80_t f128_to_extF80( float128_t a )
*------------------------------------------------------------------------*/
if ( exp == 0x7FFF ) {
if ( frac64 | frac0 ) {
softfloat_f128UIToCommonNaN( uiA64, uiA0, &commonNaN );
softfloat_f128UIToCommonNaN( state, uiA64, uiA0, &commonNaN );
uiZ = softfloat_commonNaNToExtF80UI( &commonNaN );
uiZ64 = uiZ.v64;
uiZ0 = uiZ.v0;
@@ -98,7 +98,7 @@ extFloat80_t f128_to_extF80( float128_t a )
sig128 =
softfloat_shortShiftLeft128(
frac64 | UINT64_C( 0x0001000000000000 ), frac0, 15 );
return softfloat_roundPackToExtF80( sign, exp, sig128.v64, sig128.v0, 80 );
return softfloat_roundPackToExtF80( state, sign, exp, sig128.v64, sig128.v0, 80 );
/*------------------------------------------------------------------------
*------------------------------------------------------------------------*/
uiZ:
@@ -42,7 +42,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "softfloat.h"
FEXCORE_PRESERVE_ALL_ATTR
extFloat80_t f32_to_extF80( float32_t a )
extFloat80_t f32_to_extF80( struct softfloat_state *state, float32_t a )
{
union ui32_f32 uA;
uint_fast32_t uiA;
@@ -67,7 +67,7 @@ extFloat80_t f32_to_extF80( float32_t a )
*------------------------------------------------------------------------*/
if ( exp == 0xFF ) {
if ( frac ) {
softfloat_f32UIToCommonNaN( uiA, &commonNaN );
softfloat_f32UIToCommonNaN( state, uiA, &commonNaN );
uiZ = softfloat_commonNaNToExtF80UI( &commonNaN );
uiZ64 = uiZ.v64;
uiZ0 = uiZ.v0;
@@ -42,7 +42,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "softfloat.h"
FEXCORE_PRESERVE_ALL_ATTR
extFloat80_t f64_to_extF80( float64_t a )
extFloat80_t f64_to_extF80( struct softfloat_state *state, float64_t a )
{
union ui64_f64 uA;
uint_fast64_t uiA;
@@ -67,7 +67,7 @@ extFloat80_t f64_to_extF80( float64_t a )
*------------------------------------------------------------------------*/
if ( exp == 0x7FF ) {
if ( frac ) {
softfloat_f64UIToCommonNaN( uiA, &commonNaN );
softfloat_f64UIToCommonNaN( state, uiA, &commonNaN );
uiZ = softfloat_commonNaNToExtF80UI( &commonNaN );
uiZ64 = uiZ.v64;
uiZ0 = uiZ.v0;
@@ -63,19 +63,19 @@ uint_fast32_t softfloat_roundToUI32( bool, uint_fast64_t, uint_fast8_t, bool );
#ifdef SOFTFLOAT_FAST_INT64
uint_fast64_t
softfloat_roundToUI64(
bool, uint_fast64_t, uint_fast64_t, uint_fast8_t, bool );
struct softfloat_state *, bool, uint_fast64_t, uint_fast64_t, uint_fast8_t, bool );
#else
uint_fast64_t softfloat_roundMToUI64( bool, uint32_t *, uint_fast8_t, bool );
#endif
FEXCORE_PRESERVE_ALL_ATTR
int_fast32_t softfloat_roundToI32( bool, uint_fast64_t, uint_fast8_t, bool );
int_fast32_t softfloat_roundToI32( struct softfloat_state *, bool, uint_fast64_t, uint_fast8_t, bool );
#ifdef SOFTFLOAT_FAST_INT64
FEXCORE_PRESERVE_ALL_ATTR
int_fast64_t
softfloat_roundToI64(
bool, uint_fast64_t, uint_fast64_t, uint_fast8_t, bool );
struct softfloat_state *, bool, uint_fast64_t, uint_fast64_t, uint_fast8_t, bool );
#else
int_fast64_t softfloat_roundMToI64( bool, uint32_t *, uint_fast8_t, bool );
#endif
@@ -115,7 +115,7 @@ FEXCORE_PRESERVE_ALL_ATTR
struct exp16_sig32 softfloat_normSubnormalF32Sig( uint_fast32_t );
FEXCORE_PRESERVE_ALL_ATTR
float32_t softfloat_roundPackToF32( bool, int_fast16_t, uint_fast32_t );
float32_t softfloat_roundPackToF32( struct softfloat_state *, bool, int_fast16_t, uint_fast32_t );
float32_t softfloat_normRoundPackToF32( bool, int_fast16_t, uint_fast32_t );
float32_t softfloat_addMagsF32( uint_fast32_t, uint_fast32_t );
@@ -138,7 +138,7 @@ FEXCORE_PRESERVE_ALL_ATTR
struct exp16_sig64 softfloat_normSubnormalF64Sig( uint_fast64_t );
FEXCORE_PRESERVE_ALL_ATTR
float64_t softfloat_roundPackToF64( bool, int_fast16_t, uint_fast64_t );
float64_t softfloat_roundPackToF64( struct softfloat_state *, bool, int_fast16_t, uint_fast64_t );
float64_t softfloat_normRoundPackToF64( bool, int_fast16_t, uint_fast64_t );
float64_t softfloat_addMagsF64( uint_fast64_t, uint_fast64_t, bool );
@@ -167,18 +167,18 @@ struct exp32_sig64 softfloat_normSubnormalExtF80Sig( uint_fast64_t );
FEXCORE_PRESERVE_ALL_ATTR
extFloat80_t
softfloat_roundPackToExtF80(
bool, int_fast32_t, uint_fast64_t, uint_fast64_t, uint_fast8_t );
struct softfloat_state *, bool, int_fast32_t, uint_fast64_t, uint_fast64_t, uint_fast8_t );
FEXCORE_PRESERVE_ALL_ATTR
extFloat80_t
softfloat_normRoundPackToExtF80(
bool, int_fast32_t, uint_fast64_t, uint_fast64_t, uint_fast8_t );
struct softfloat_state *, bool, int_fast32_t, uint_fast64_t, uint_fast64_t, uint_fast8_t );
extFloat80_t
softfloat_addMagsExtF80(
uint_fast16_t, uint_fast64_t, uint_fast16_t, uint_fast64_t, bool );
struct softfloat_state *, uint_fast16_t, uint_fast64_t, uint_fast16_t, uint_fast64_t, bool );
extFloat80_t
softfloat_subMagsExtF80(
uint_fast16_t, uint_fast64_t, uint_fast16_t, uint_fast64_t, bool );
struct softfloat_state *, uint_fast16_t, uint_fast64_t, uint_fast16_t, uint_fast64_t, bool );
/*----------------------------------------------------------------------------
*----------------------------------------------------------------------------*/
@@ -43,6 +43,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
extFloat80_t
softfloat_addMagsExtF80(
struct softfloat_state *state,
uint_fast16_t uiA64,
uint_fast64_t uiA0,
uint_fast16_t uiB64,
@@ -140,11 +141,11 @@ extFloat80_t
roundAndPack:
return
softfloat_roundPackToExtF80(
signZ, expZ, sigZ, sigZExtra, extF80_roundingPrecision );
state, signZ, expZ, sigZ, sigZExtra, state->roundingPrecision );
/*------------------------------------------------------------------------
*------------------------------------------------------------------------*/
propagateNaN:
uiZ = softfloat_propagateNaNExtF80UI( uiA64, uiA0, uiB64, uiB0 );
uiZ = softfloat_propagateNaNExtF80UI( state, uiA64, uiA0, uiB64, uiB0 );
uiZ64 = uiZ.v64;
uiZ0 = uiZ.v0;
uiZ:
@@ -49,11 +49,11 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
FEXCORE_PRESERVE_ALL_ATTR
void
softfloat_extF80UIToCommonNaN(
uint_fast16_t uiA64, uint_fast64_t uiA0, struct commonNaN *zPtr )
struct softfloat_state *state, uint_fast16_t uiA64, uint_fast64_t uiA0, struct commonNaN *zPtr )
{
if ( softfloat_isSigNaNExtF80UI( uiA64, uiA0 ) ) {
softfloat_raiseFlags( softfloat_flag_invalid );
softfloat_raiseFlags( state, softfloat_flag_invalid );
}
zPtr->sign = uiA64>>15;
zPtr->v64 = uiA0<<1;
@@ -50,12 +50,12 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
FEXCORE_PRESERVE_ALL_ATTR
void
softfloat_f128UIToCommonNaN(
uint_fast64_t uiA64, uint_fast64_t uiA0, struct commonNaN *zPtr )
struct softfloat_state *state, uint_fast64_t uiA64, uint_fast64_t uiA0, struct commonNaN *zPtr )
{
struct uint128 NaNSig;
if ( softfloat_isSigNaNF128UI( uiA64, uiA0 ) ) {
softfloat_raiseFlags( softfloat_flag_invalid );
softfloat_raiseFlags( state, softfloat_flag_invalid );
}
NaNSig = softfloat_shortShiftLeft128( uiA64, uiA0, 16 );
zPtr->sign = uiA64>>63;
@@ -46,11 +46,11 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
| exception is raised.
*----------------------------------------------------------------------------*/
FEXCORE_PRESERVE_ALL_ATTR
void softfloat_f32UIToCommonNaN( uint_fast32_t uiA, struct commonNaN *zPtr )
void softfloat_f32UIToCommonNaN( struct softfloat_state *state, uint_fast32_t uiA, struct commonNaN *zPtr )
{
if ( softfloat_isSigNaNF32UI( uiA ) ) {
softfloat_raiseFlags( softfloat_flag_invalid );
softfloat_raiseFlags( state, softfloat_flag_invalid );
}
zPtr->sign = uiA>>31;
zPtr->v64 = (uint_fast64_t) uiA<<41;
@@ -46,11 +46,11 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
| exception is raised.
*----------------------------------------------------------------------------*/
FEXCORE_PRESERVE_ALL_ATTR
void softfloat_f64UIToCommonNaN( uint_fast64_t uiA, struct commonNaN *zPtr )
void softfloat_f64UIToCommonNaN( struct softfloat_state *state, uint_fast64_t uiA, struct commonNaN *zPtr )
{
if ( softfloat_isSigNaNF64UI( uiA ) ) {
softfloat_raiseFlags( softfloat_flag_invalid );
softfloat_raiseFlags( state, softfloat_flag_invalid );
}
zPtr->sign = uiA>>63;
zPtr->v64 = uiA<<12;
@@ -42,6 +42,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
FEXCORE_PRESERVE_ALL_ATTR
extFloat80_t
softfloat_normRoundPackToExtF80(
struct softfloat_state *state,
bool sign,
int_fast32_t exp,
uint_fast64_t sig,
@@ -66,7 +67,7 @@ extFloat80_t
}
return
softfloat_roundPackToExtF80(
sign, exp, sig, sigExtra, roundingPrecision );
state, sign, exp, sig, sigExtra, roundingPrecision );
}
@@ -53,6 +53,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
FEXCORE_PRESERVE_ALL_ATTR
struct uint128
softfloat_propagateNaNExtF80UI(
struct softfloat_state *state,
uint_fast16_t uiA64,
uint_fast64_t uiA0,
uint_fast16_t uiB64,
@@ -76,7 +77,7 @@ struct uint128
/*------------------------------------------------------------------------
*------------------------------------------------------------------------*/
if ( isSigNaNA | isSigNaNB ) {
softfloat_raiseFlags( softfloat_flag_invalid );
softfloat_raiseFlags( state, softfloat_flag_invalid );
if ( isSigNaNA ) {
if ( isSigNaNB ) goto returnLargerMag;
if ( isNaNExtF80UI( uiB64, uiB0 ) ) goto returnB;
@@ -43,6 +43,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
FEXCORE_PRESERVE_ALL_ATTR
extFloat80_t
softfloat_roundPackToExtF80(
struct softfloat_state *state,
bool sign,
int_fast32_t exp,
uint_fast64_t sig,
@@ -59,7 +60,7 @@ extFloat80_t
/*------------------------------------------------------------------------
*------------------------------------------------------------------------*/
roundingMode = softfloat_roundingMode;
roundingMode = state->roundingMode;
roundNearEven = (roundingMode == softfloat_round_near_even);
if ( roundingPrecision == 80 ) goto precision80;
if ( roundingPrecision == 64 ) {
@@ -87,15 +88,15 @@ extFloat80_t
/*----------------------------------------------------------------
*----------------------------------------------------------------*/
isTiny =
(softfloat_detectTininess
(state->detectTininess
== softfloat_tininess_beforeRounding)
|| (exp < 0)
|| (sig <= (uint64_t) (sig + roundIncrement));
sig = softfloat_shiftRightJam64( sig, 1 - exp );
roundBits = sig & roundMask;
if ( roundBits ) {
if ( isTiny ) softfloat_raiseFlags( softfloat_flag_underflow );
softfloat_exceptionFlags |= softfloat_flag_inexact;
if ( isTiny ) softfloat_raiseFlags( state, softfloat_flag_underflow );
state->exceptionFlags |= softfloat_flag_inexact;
#ifdef SOFTFLOAT_ROUND_ODD
if ( roundingMode == softfloat_round_odd ) {
sig |= roundMask + 1;
@@ -121,7 +122,7 @@ extFloat80_t
/*------------------------------------------------------------------------
*------------------------------------------------------------------------*/
if ( roundBits ) {
softfloat_exceptionFlags |= softfloat_flag_inexact;
state->exceptionFlags |= softfloat_flag_inexact;
#ifdef SOFTFLOAT_ROUND_ODD
if ( roundingMode == softfloat_round_odd ) {
sig = (sig & ~roundMask) | (roundMask + 1);
@@ -157,7 +158,7 @@ extFloat80_t
/*----------------------------------------------------------------
*----------------------------------------------------------------*/
isTiny =
(softfloat_detectTininess
(state->detectTininess
== softfloat_tininess_beforeRounding)
|| (exp < 0)
|| ! doIncrement
@@ -168,8 +169,8 @@ extFloat80_t
sig = sig64Extra.v;
sigExtra = sig64Extra.extra;
if ( sigExtra ) {
if ( isTiny ) softfloat_raiseFlags( softfloat_flag_underflow );
softfloat_exceptionFlags |= softfloat_flag_inexact;
if ( isTiny ) softfloat_raiseFlags( state, softfloat_flag_underflow );
state->exceptionFlags |= softfloat_flag_inexact;
#ifdef SOFTFLOAT_ROUND_ODD
if ( roundingMode == softfloat_round_odd ) {
sig |= 1;
@@ -207,7 +208,7 @@ extFloat80_t
roundMask = 0;
overflow:
softfloat_raiseFlags(
softfloat_flag_overflow | softfloat_flag_inexact );
state, softfloat_flag_overflow | softfloat_flag_inexact );
if (
roundNearEven
|| (roundingMode == softfloat_round_near_maxMag)
@@ -226,7 +227,7 @@ extFloat80_t
/*------------------------------------------------------------------------
*------------------------------------------------------------------------*/
if ( sigExtra ) {
softfloat_exceptionFlags |= softfloat_flag_inexact;
state->exceptionFlags |= softfloat_flag_inexact;
#ifdef SOFTFLOAT_ROUND_ODD
if ( roundingMode == softfloat_round_odd ) {
sig |= 1;
@@ -42,7 +42,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
FEXCORE_PRESERVE_ALL_ATTR
float32_t
softfloat_roundPackToF32( bool sign, int_fast16_t exp, uint_fast32_t sig )
softfloat_roundPackToF32( struct softfloat_state *state, bool sign, int_fast16_t exp, uint_fast32_t sig )
{
uint_fast8_t roundingMode;
bool roundNearEven;
@@ -53,7 +53,7 @@ float32_t
/*------------------------------------------------------------------------
*------------------------------------------------------------------------*/
roundingMode = softfloat_roundingMode;
roundingMode = state->roundingMode;
roundNearEven = (roundingMode == softfloat_round_near_even);
roundIncrement = 0x40;
if ( ! roundNearEven && (roundingMode != softfloat_round_near_maxMag) ) {
@@ -71,19 +71,19 @@ float32_t
/*----------------------------------------------------------------
*----------------------------------------------------------------*/
isTiny =
(softfloat_detectTininess == softfloat_tininess_beforeRounding)
(state->detectTininess == softfloat_tininess_beforeRounding)
|| (exp < -1) || (sig + roundIncrement < 0x80000000);
sig = softfloat_shiftRightJam32( sig, -exp );
exp = 0;
roundBits = sig & 0x7F;
if ( isTiny && roundBits ) {
softfloat_raiseFlags( softfloat_flag_underflow );
softfloat_raiseFlags( state, softfloat_flag_underflow );
}
} else if ( (0xFD < exp) || (0x80000000 <= sig + roundIncrement) ) {
/*----------------------------------------------------------------
*----------------------------------------------------------------*/
softfloat_raiseFlags(
softfloat_flag_overflow | softfloat_flag_inexact );
state, softfloat_flag_overflow | softfloat_flag_inexact );
uiZ = packToF32UI( sign, 0xFF, 0 ) - ! roundIncrement;
goto uiZ;
}
@@ -92,7 +92,7 @@ float32_t
*------------------------------------------------------------------------*/
sig = (sig + roundIncrement)>>7;
if ( roundBits ) {
softfloat_exceptionFlags |= softfloat_flag_inexact;
state->exceptionFlags |= softfloat_flag_inexact;
#ifdef SOFTFLOAT_ROUND_ODD
if ( roundingMode == softfloat_round_odd ) {
sig |= 1;
@@ -42,7 +42,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
FEXCORE_PRESERVE_ALL_ATTR
float64_t
softfloat_roundPackToF64( bool sign, int_fast16_t exp, uint_fast64_t sig )
softfloat_roundPackToF64( struct softfloat_state *state, bool sign, int_fast16_t exp, uint_fast64_t sig )
{
uint_fast8_t roundingMode;
bool roundNearEven;
@@ -53,7 +53,7 @@ float64_t
/*------------------------------------------------------------------------
*------------------------------------------------------------------------*/
roundingMode = softfloat_roundingMode;
roundingMode = state->roundingMode;
roundNearEven = (roundingMode == softfloat_round_near_even);
roundIncrement = 0x200;
if ( ! roundNearEven && (roundingMode != softfloat_round_near_maxMag) ) {
@@ -71,14 +71,14 @@ float64_t
/*----------------------------------------------------------------
*----------------------------------------------------------------*/
isTiny =
(softfloat_detectTininess == softfloat_tininess_beforeRounding)
(state->detectTininess == softfloat_tininess_beforeRounding)
|| (exp < -1)
|| (sig + roundIncrement < UINT64_C( 0x8000000000000000 ));
sig = softfloat_shiftRightJam64( sig, -exp );
exp = 0;
roundBits = sig & 0x3FF;
if ( isTiny && roundBits ) {
softfloat_raiseFlags( softfloat_flag_underflow );
softfloat_raiseFlags( state, softfloat_flag_underflow );
}
} else if (
(0x7FD < exp)
@@ -87,7 +87,7 @@ float64_t
/*----------------------------------------------------------------
*----------------------------------------------------------------*/
softfloat_raiseFlags(
softfloat_flag_overflow | softfloat_flag_inexact );
state, softfloat_flag_overflow | softfloat_flag_inexact );
uiZ = packToF64UI( sign, 0x7FF, 0 ) - ! roundIncrement;
goto uiZ;
}
@@ -96,7 +96,7 @@ float64_t
*------------------------------------------------------------------------*/
sig = (sig + roundIncrement)>>10;
if ( roundBits ) {
softfloat_exceptionFlags |= softfloat_flag_inexact;
state->exceptionFlags |= softfloat_flag_inexact;
#ifdef SOFTFLOAT_ROUND_ODD
if ( roundingMode == softfloat_round_odd ) {
sig |= 1;
@@ -44,7 +44,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
FEXCORE_PRESERVE_ALL_ATTR
int_fast32_t
softfloat_roundToI32(
bool sign, uint_fast64_t sig, uint_fast8_t roundingMode, bool exact )
struct softfloat_state *state, bool sign, uint_fast64_t sig, uint_fast8_t roundingMode, bool exact )
{
uint_fast16_t roundIncrement, roundBits;
uint_fast32_t sig32;
@@ -86,13 +86,13 @@ int_fast32_t
#ifdef SOFTFLOAT_ROUND_ODD
if ( roundingMode == softfloat_round_odd ) z |= 1;
#endif
if ( exact ) softfloat_exceptionFlags |= softfloat_flag_inexact;
if ( exact ) state->exceptionFlags |= softfloat_flag_inexact;
}
return z;
/*------------------------------------------------------------------------
*------------------------------------------------------------------------*/
invalid:
softfloat_raiseFlags( softfloat_flag_invalid );
softfloat_raiseFlags( state, softfloat_flag_invalid );
return sign ? i32_fromNegOverflow : i32_fromPosOverflow;
}
@@ -44,6 +44,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
FEXCORE_PRESERVE_ALL_ATTR
int_fast64_t
softfloat_roundToI64(
struct softfloat_state *state,
bool sign,
uint_fast64_t sig,
uint_fast64_t sigExtra,
@@ -89,13 +90,13 @@ int_fast64_t
#ifdef SOFTFLOAT_ROUND_ODD
if ( roundingMode == softfloat_round_odd ) z |= 1;
#endif
if ( exact ) softfloat_exceptionFlags |= softfloat_flag_inexact;
if ( exact ) state->exceptionFlags |= softfloat_flag_inexact;
}
return z;
/*------------------------------------------------------------------------
*------------------------------------------------------------------------*/
invalid:
softfloat_raiseFlags( softfloat_flag_invalid );
softfloat_raiseFlags( state, softfloat_flag_invalid );
return sign ? i64_fromNegOverflow : i64_fromPosOverflow;
}
@@ -43,6 +43,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
uint_fast64_t
softfloat_roundToUI64(
struct softfloat_state *state,
bool sign,
uint_fast64_t sig,
uint_fast64_t sigExtra,
@@ -84,13 +85,13 @@ uint_fast64_t
#ifdef SOFTFLOAT_ROUND_ODD
if ( roundingMode == softfloat_round_odd ) sig |= 1;
#endif
if ( exact ) softfloat_exceptionFlags |= softfloat_flag_inexact;
if ( exact ) state->exceptionFlags |= softfloat_flag_inexact;
}
return sig;
/*------------------------------------------------------------------------
*------------------------------------------------------------------------*/
invalid:
softfloat_raiseFlags( softfloat_flag_invalid );
softfloat_raiseFlags( state, softfloat_flag_invalid );
return sign ? ui64_fromNegOverflow : ui64_fromPosOverflow;
}
@@ -43,6 +43,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
extFloat80_t
softfloat_subMagsExtF80(
struct softfloat_state *state,
uint_fast16_t uiA64,
uint_fast64_t uiA0,
uint_fast16_t uiB64,
@@ -77,7 +78,7 @@ extFloat80_t
if ( (sigA | sigB) & UINT64_C( 0x7FFFFFFFFFFFFFFF ) ) {
goto propagateNaN;
}
softfloat_raiseFlags( softfloat_flag_invalid );
softfloat_raiseFlags( state, softfloat_flag_invalid );
uiZ64 = defaultNaNExtF80UI64;
uiZ0 = defaultNaNExtF80UI0;
goto uiZ;
@@ -90,7 +91,7 @@ extFloat80_t
if ( sigB < sigA ) goto aBigger;
if ( sigA < sigB ) goto bBigger;
uiZ64 =
packToExtF80UI64( (softfloat_roundingMode == softfloat_round_min), 0 );
packToExtF80UI64( (state->roundingMode == softfloat_round_min), 0 );
uiZ0 = 0;
goto uiZ;
/*------------------------------------------------------------------------
@@ -142,11 +143,11 @@ extFloat80_t
normRoundPack:
return
softfloat_normRoundPackToExtF80(
signZ, expZ, sig128.v64, sig128.v0, extF80_roundingPrecision );
state, signZ, expZ, sig128.v64, sig128.v0, state->roundingPrecision );
/*------------------------------------------------------------------------
*------------------------------------------------------------------------*/
propagateNaN:
uiZ = softfloat_propagateNaNExtF80UI( uiA64, uiA0, uiB64, uiB0 );
uiZ = softfloat_propagateNaNExtF80UI( state, uiA64, uiA0, uiB64, uiB0 );
uiZ64 = uiZ.v64;
uiZ0 = uiZ.v0;
uiZ:
+19 -64
View File
@@ -50,50 +50,11 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include <stdint.h>
#include "softfloat_types.h"
#ifndef THREAD_LOCAL
#define THREAD_LOCAL
#endif
/*----------------------------------------------------------------------------
| Software floating-point underflow tininess-detection mode.
*----------------------------------------------------------------------------*/
extern THREAD_LOCAL uint_fast8_t softfloat_detectTininess;
enum {
softfloat_tininess_beforeRounding = 0,
softfloat_tininess_afterRounding = 1
};
/*----------------------------------------------------------------------------
| Software floating-point rounding mode. (Mode "odd" is supported only if
| SoftFloat is compiled with macro 'SOFTFLOAT_ROUND_ODD' defined.)
*----------------------------------------------------------------------------*/
extern THREAD_LOCAL uint_fast8_t softfloat_roundingMode;
enum {
softfloat_round_near_even = 0,
softfloat_round_minMag = 1,
softfloat_round_min = 2,
softfloat_round_max = 3,
softfloat_round_near_maxMag = 4,
softfloat_round_odd = 6
};
/*----------------------------------------------------------------------------
| Software floating-point exception flags.
*----------------------------------------------------------------------------*/
extern THREAD_LOCAL uint_fast8_t softfloat_exceptionFlags;
enum {
softfloat_flag_inexact = 1,
softfloat_flag_underflow = 2,
softfloat_flag_overflow = 4,
softfloat_flag_infinite = 8,
softfloat_flag_invalid = 16
};
/*----------------------------------------------------------------------------
| Routine to raise any or all of the software floating-point exception flags.
*----------------------------------------------------------------------------*/
FEXCORE_PRESERVE_ALL_ATTR
void softfloat_raiseFlags( uint_fast8_t );
void softfloat_raiseFlags( struct softfloat_state *, uint_fast8_t );
/*----------------------------------------------------------------------------
| Integer-to-floating-point conversion routines.
@@ -187,7 +148,7 @@ float16_t f32_to_f16( float32_t );
float64_t f32_to_f64( float32_t );
#ifdef SOFTFLOAT_FAST_INT64
FEXCORE_PRESERVE_ALL_ATTR
extFloat80_t f32_to_extF80( float32_t );
extFloat80_t f32_to_extF80( struct softfloat_state *, float32_t );
float128_t f32_to_f128( float32_t );
#endif
void f32_to_extF80M( float32_t, extFloat80_t * );
@@ -223,7 +184,7 @@ float16_t f64_to_f16( float64_t );
float32_t f64_to_f32( float64_t );
#ifdef SOFTFLOAT_FAST_INT64
FEXCORE_PRESERVE_ALL_ATTR
extFloat80_t f64_to_extF80( float64_t );
extFloat80_t f64_to_extF80( struct softfloat_state *, float64_t );
float128_t f64_to_f128( float64_t );
#endif
void f64_to_extF80M( float64_t, extFloat80_t * );
@@ -244,53 +205,47 @@ bool f64_le_quiet( float64_t, float64_t );
bool f64_lt_quiet( float64_t, float64_t );
bool f64_isSignalingNaN( float64_t );
/*----------------------------------------------------------------------------
| Rounding precision for 80-bit extended double-precision floating-point.
| Valid values are 32, 64, and 80.
*----------------------------------------------------------------------------*/
extern THREAD_LOCAL uint_fast8_t extF80_roundingPrecision;
/*----------------------------------------------------------------------------
| 80-bit extended double-precision floating-point operations.
*----------------------------------------------------------------------------*/
#ifdef SOFTFLOAT_FAST_INT64
uint_fast32_t extF80_to_ui32( extFloat80_t, uint_fast8_t, bool );
FEXCORE_PRESERVE_ALL_ATTR
uint_fast64_t extF80_to_ui64( extFloat80_t, uint_fast8_t, bool );
uint_fast64_t extF80_to_ui64( struct softfloat_state *, extFloat80_t, uint_fast8_t, bool );
FEXCORE_PRESERVE_ALL_ATTR
int_fast32_t extF80_to_i32( extFloat80_t, uint_fast8_t, bool );
int_fast32_t extF80_to_i32( struct softfloat_state *, extFloat80_t, uint_fast8_t, bool );
FEXCORE_PRESERVE_ALL_ATTR
int_fast64_t extF80_to_i64( extFloat80_t, uint_fast8_t, bool );
int_fast64_t extF80_to_i64( struct softfloat_state *, extFloat80_t, uint_fast8_t, bool );
uint_fast32_t extF80_to_ui32_r_minMag( extFloat80_t, bool );
uint_fast64_t extF80_to_ui64_r_minMag( extFloat80_t, bool );
int_fast32_t extF80_to_i32_r_minMag( extFloat80_t, bool );
int_fast64_t extF80_to_i64_r_minMag( extFloat80_t, bool );
float16_t extF80_to_f16( extFloat80_t );
FEXCORE_PRESERVE_ALL_ATTR
float32_t extF80_to_f32( extFloat80_t );
float32_t extF80_to_f32( struct softfloat_state *, extFloat80_t );
FEXCORE_PRESERVE_ALL_ATTR
float64_t extF80_to_f64( extFloat80_t );
float64_t extF80_to_f64( struct softfloat_state *, extFloat80_t );
FEXCORE_PRESERVE_ALL_ATTR
float128_t extF80_to_f128( extFloat80_t );
float128_t extF80_to_f128( struct softfloat_state *, extFloat80_t );
FEXCORE_PRESERVE_ALL_ATTR
extFloat80_t extF80_roundToInt( extFloat80_t, uint_fast8_t, bool );
extFloat80_t extF80_roundToInt( struct softfloat_state *, extFloat80_t, uint_fast8_t, bool );
FEXCORE_PRESERVE_ALL_ATTR
extFloat80_t extF80_add( extFloat80_t, extFloat80_t );
extFloat80_t extF80_add( struct softfloat_state *, extFloat80_t, extFloat80_t );
FEXCORE_PRESERVE_ALL_ATTR
extFloat80_t extF80_sub( extFloat80_t, extFloat80_t );
extFloat80_t extF80_sub( struct softfloat_state *, extFloat80_t, extFloat80_t );
FEXCORE_PRESERVE_ALL_ATTR
extFloat80_t extF80_mul( extFloat80_t, extFloat80_t );
extFloat80_t extF80_mul( struct softfloat_state *, extFloat80_t, extFloat80_t );
FEXCORE_PRESERVE_ALL_ATTR
extFloat80_t extF80_div( extFloat80_t, extFloat80_t );
extFloat80_t extF80_div( struct softfloat_state *, extFloat80_t, extFloat80_t );
FEXCORE_PRESERVE_ALL_ATTR
extFloat80_t extF80_rem( extFloat80_t, extFloat80_t );
extFloat80_t extF80_rem( struct softfloat_state *, extFloat80_t, extFloat80_t );
FEXCORE_PRESERVE_ALL_ATTR
extFloat80_t extF80_sqrt( extFloat80_t );
extFloat80_t extF80_sqrt( struct softfloat_state *, extFloat80_t );
FEXCORE_PRESERVE_ALL_ATTR
bool extF80_eq( extFloat80_t, extFloat80_t );
bool extF80_eq( struct softfloat_state *, extFloat80_t, extFloat80_t );
bool extF80_le( extFloat80_t, extFloat80_t );
FEXCORE_PRESERVE_ALL_ATTR
bool extF80_lt( extFloat80_t, extFloat80_t );
bool extF80_lt( struct softfloat_state *, extFloat80_t, extFloat80_t );
bool extF80_eq_signaling( extFloat80_t, extFloat80_t );
bool extF80_le_quiet( extFloat80_t, extFloat80_t );
bool extF80_lt_quiet( extFloat80_t, extFloat80_t );
@@ -341,7 +296,7 @@ float16_t f128_to_f16( float128_t );
float32_t f128_to_f32( float128_t );
float64_t f128_to_f64( float128_t );
FEXCORE_PRESERVE_ALL_ATTR
extFloat80_t f128_to_extF80( float128_t );
extFloat80_t f128_to_extF80( struct softfloat_state *, float128_t );
float128_t f128_roundToInt( float128_t, uint_fast8_t, bool );
float128_t f128_add( float128_t, float128_t );
float128_t f128_sub( float128_t, float128_t );
@@ -44,10 +44,10 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
| should be simply `softfloat_exceptionFlags |= flags;'.
*----------------------------------------------------------------------------*/
FEXCORE_PRESERVE_ALL_ATTR
void softfloat_raiseFlags( uint_fast8_t flags )
void softfloat_raiseFlags( struct softfloat_state *state, uint_fast8_t flags )
{
softfloat_exceptionFlags |= flags;
state->exceptionFlags |= flags;
}
@@ -1,52 +0,0 @@
/*============================================================================
This C source file is part of the SoftFloat IEEE Floating-Point Arithmetic
Package, Release 3e, by John R. Hauser.
Copyright 2011, 2012, 2013, 2014, 2015, 2016 The Regents of the University of
California. All Rights Reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice,
this list of conditions, and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions, and the following disclaimer in the documentation
and/or other materials provided with the distribution.
3. Neither the name of the University nor the names of its contributors may
be used to endorse or promote products derived from this software without
specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS "AS IS", AND ANY
EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE, ARE
DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
=============================================================================*/
#include <stdint.h>
#include "platform.h"
#include "internals.h"
#include "specialize.h"
#include "softfloat.h"
#ifndef THREAD_LOCAL
#define THREAD_LOCAL
#endif
THREAD_LOCAL uint_fast8_t softfloat_roundingMode = softfloat_round_near_even;
THREAD_LOCAL uint_fast8_t softfloat_detectTininess = init_detectTininess;
THREAD_LOCAL uint_fast8_t softfloat_exceptionFlags = 0;
THREAD_LOCAL uint_fast8_t extF80_roundingPrecision = 80;
@@ -77,5 +77,50 @@ struct extFloat80M { uint16_t signExp; uint64_t signif; };
*----------------------------------------------------------------------------*/
typedef struct extFloat80M extFloat80_t;
enum {
softfloat_tininess_beforeRounding = 0,
softfloat_tininess_afterRounding = 1
};
enum {
softfloat_round_near_even = 0,
softfloat_round_minMag = 1,
softfloat_round_min = 2,
softfloat_round_max = 3,
softfloat_round_near_maxMag = 4,
softfloat_round_odd = 6
};
enum {
softfloat_flag_inexact = 1,
softfloat_flag_underflow = 2,
softfloat_flag_overflow = 4,
softfloat_flag_infinite = 8,
softfloat_flag_invalid = 16
};
struct softfloat_state {
/*----------------------------------------------------------------------------
| Software floating-point underflow tininess-detection mode.
*----------------------------------------------------------------------------*/
uint8_t detectTininess; /* = init_detectTininess */
/*----------------------------------------------------------------------------
| Software floating-point rounding mode. (Mode "odd" is supported only if
| SoftFloat is compiled with macro 'SOFTFLOAT_ROUND_ODD' defined.)
*----------------------------------------------------------------------------*/
uint8_t roundingMode; /* = softfloat_round_near_even */
/*----------------------------------------------------------------------------
| Software floating-point exception flags.
*----------------------------------------------------------------------------*/
uint8_t exceptionFlags; /* = 0 */
/*----------------------------------------------------------------------------
| Rounding precision for 80-bit extended double-precision floating-point.
| Valid values are 32, 64, and 80.
*----------------------------------------------------------------------------*/
uint8_t roundingPrecision; /* = 80 */
};
#endif
@@ -136,7 +136,7 @@ uint_fast16_t
| exception is raised.
*----------------------------------------------------------------------------*/
FEXCORE_PRESERVE_ALL_ATTR
void softfloat_f32UIToCommonNaN( uint_fast32_t uiA, struct commonNaN *zPtr );
void softfloat_f32UIToCommonNaN( struct softfloat_state *, uint_fast32_t uiA, struct commonNaN *zPtr );
/*----------------------------------------------------------------------------
| Converts the common NaN pointed to by 'aPtr' into a 32-bit floating-point
@@ -173,7 +173,7 @@ uint_fast32_t
| exception is raised.
*----------------------------------------------------------------------------*/
FEXCORE_PRESERVE_ALL_ATTR
void softfloat_f64UIToCommonNaN( uint_fast64_t uiA, struct commonNaN *zPtr );
void softfloat_f64UIToCommonNaN( struct softfloat_state *, uint_fast64_t uiA, struct commonNaN *zPtr );
/*----------------------------------------------------------------------------
| Converts the common NaN pointed to by 'aPtr' into a 64-bit floating-point
@@ -222,7 +222,7 @@ uint_fast64_t
FEXCORE_PRESERVE_ALL_ATTR
void
softfloat_extF80UIToCommonNaN(
uint_fast16_t uiA64, uint_fast64_t uiA0, struct commonNaN *zPtr );
struct softfloat_state *, uint_fast16_t uiA64, uint_fast64_t uiA0, struct commonNaN *zPtr );
/*----------------------------------------------------------------------------
| Converts the common NaN pointed to by 'aPtr' into an 80-bit extended
@@ -244,6 +244,7 @@ struct uint128 softfloat_commonNaNToExtF80UI( const struct commonNaN *aPtr );
FEXCORE_PRESERVE_ALL_ATTR
struct uint128
softfloat_propagateNaNExtF80UI(
struct softfloat_state *,
uint_fast16_t uiA64,
uint_fast64_t uiA0,
uint_fast16_t uiB64,
@@ -274,7 +275,7 @@ struct uint128
FEXCORE_PRESERVE_ALL_ATTR
void
softfloat_f128UIToCommonNaN(
uint_fast64_t uiA64, uint_fast64_t uiA0, struct commonNaN *zPtr );
struct softfloat_state *, uint_fast64_t uiA64, uint_fast64_t uiA0, struct commonNaN *zPtr );
/*----------------------------------------------------------------------------
| Converts the common NaN pointed to by 'aPtr' into a 128-bit floating-point
+101 -79
View File
@@ -63,7 +63,7 @@ struct FEX_PACKED X80SoftFloat {
}
// Ops
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat FADD(const X80SoftFloat& lhs, const X80SoftFloat& rhs) {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat FADD(softfloat_state* state, const X80SoftFloat& lhs, const X80SoftFloat& rhs) {
#ifdef DEBUG_X86_FLOAT
BIGFLOAT Result;
asm(R"(
@@ -79,11 +79,11 @@ struct FEX_PACKED X80SoftFloat {
return Result;
#else
return extF80_add(lhs, rhs);
return extF80_add(state, lhs, rhs);
#endif
}
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat FSUB(const X80SoftFloat& lhs, const X80SoftFloat& rhs) {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat FSUB(softfloat_state* state, const X80SoftFloat& lhs, const X80SoftFloat& rhs) {
#ifdef DEBUG_X86_FLOAT
BIGFLOAT Result;
asm(R"(
@@ -99,11 +99,11 @@ struct FEX_PACKED X80SoftFloat {
return Result;
#else
return extF80_sub(lhs, rhs);
return extF80_sub(state, lhs, rhs);
#endif
}
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat FMUL(const X80SoftFloat& lhs, const X80SoftFloat& rhs) {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat FMUL(softfloat_state* state, const X80SoftFloat& lhs, const X80SoftFloat& rhs) {
#ifdef DEBUG_X86_FLOAT
BIGFLOAT Result;
asm(R"(
@@ -119,11 +119,11 @@ struct FEX_PACKED X80SoftFloat {
return Result;
#else
return extF80_mul(lhs, rhs);
return extF80_mul(state, lhs, rhs);
#endif
}
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat FDIV(const X80SoftFloat& lhs, const X80SoftFloat& rhs) {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat FDIV(softfloat_state* state, const X80SoftFloat& lhs, const X80SoftFloat& rhs) {
#ifdef DEBUG_X86_FLOAT
BIGFLOAT Result;
asm(R"(
@@ -139,11 +139,11 @@ struct FEX_PACKED X80SoftFloat {
return Result;
#else
return extF80_div(lhs, rhs);
return extF80_div(state, lhs, rhs);
#endif
}
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat FREM(const X80SoftFloat& lhs, const X80SoftFloat& rhs) {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat FREM(softfloat_state* state, const X80SoftFloat& lhs, const X80SoftFloat& rhs) {
#if defined(DEBUG_X86_FLOAT)
BIGFLOAT Result;
asm(R"(
@@ -160,11 +160,35 @@ struct FEX_PACKED X80SoftFloat {
return Result;
#else
return extF80_rem(lhs, rhs);
/*
* FPREM is not an IEEE-754 remainder. From the spec:
*
* Computes the remainder obtained from dividing the value in the ST(0)
* register (the dividend) by the value in the ST(1) register (the divisor
* or modulus), and stores the result in ST(0). The remainder represents the
* following value:
*
* Remainder := ST(0) − (Q * ST(1))
*
* Here, Q is an integer value that is obtained by truncating the
* floating-point number quotient of [ST(0) / ST(1)] toward zero.
*
* We implement this sequence literally. softfloat_round_minMag means
* "truncate towards zero".
*/
extFloat80_t quotient = extF80_div(state, lhs, rhs);
extFloat80_t Q = extF80_roundToInt(state, quotient, softfloat_round_minMag, true);
bool Q_zero = Q.signif == 0 && (Q.signExp & ~(1 << 15)) == 0;
if (Q_zero) {
return lhs;
} else {
return extF80_sub(state, lhs, extF80_mul(state, Q, rhs));
}
#endif
}
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat FREM1(const X80SoftFloat& lhs, const X80SoftFloat& rhs) {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat FREM1(softfloat_state* state, const X80SoftFloat& lhs, const X80SoftFloat& rhs) {
#if defined(DEBUG_X86_FLOAT)
BIGFLOAT Result;
asm(R"(
@@ -181,16 +205,16 @@ struct FEX_PACKED X80SoftFloat {
return Result;
#else
return extF80_rem(lhs, rhs);
return extF80_rem(state, lhs, rhs);
#endif
}
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat FRNDINT(const X80SoftFloat& lhs) {
return extF80_roundToInt(lhs, softfloat_roundingMode, false);
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat FRNDINT(softfloat_state* state, const X80SoftFloat& lhs) {
return extF80_roundToInt(state, lhs, state->roundingMode, false);
}
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat FRNDINT(const X80SoftFloat& lhs, uint_fast8_t RoundMode) {
return extF80_roundToInt(lhs, RoundMode, false);
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat FRNDINT(softfloat_state* state, const X80SoftFloat& lhs, uint_fast8_t RoundMode) {
return extF80_roundToInt(state, lhs, RoundMode, false);
}
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat FXTRACT_SIG(const X80SoftFloat& lhs) {
@@ -237,13 +261,14 @@ struct FEX_PACKED X80SoftFloat {
#endif
}
FEXCORE_PRESERVE_ALL_ATTR static void FCMP(const X80SoftFloat& lhs, const X80SoftFloat& rhs, bool* eq, bool* lt, bool* nan) {
*eq = extF80_eq(lhs, rhs);
*lt = extF80_lt(lhs, rhs);
FEXCORE_PRESERVE_ALL_ATTR static void
FCMP(softfloat_state* state, const X80SoftFloat& lhs, const X80SoftFloat& rhs, bool* eq, bool* lt, bool* nan) {
*eq = extF80_eq(state, lhs, rhs);
*lt = extF80_lt(state, lhs, rhs);
*nan = IsNan(lhs) || IsNan(rhs);
}
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat FSCALE(const X80SoftFloat& lhs, const X80SoftFloat& rhs) {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat FSCALE(softfloat_state* state, const X80SoftFloat& lhs, const X80SoftFloat& rhs) {
WARN_ONCE_FMT("x87: Application used FSCALE which may have accuracy problems");
#ifdef DEBUG_X86_FLOAT
BIGFLOAT Result;
@@ -261,16 +286,20 @@ struct FEX_PACKED X80SoftFloat {
return Result;
#else
X80SoftFloat Int = FRNDINT(rhs, softfloat_round_minMag);
LIBRARY_PRECISION Src2_d = Int;
extFloat80_t Zero {0, 0};
if (extF80_eq(state, lhs, Zero)) {
return lhs;
}
X80SoftFloat Int = FRNDINT(state, rhs, softfloat_round_minMag);
LIBRARY_PRECISION Src2_d = Int.ToFMax(state);
Src2_d = exp2l(Src2_d);
X80SoftFloat Src2_X80 = Src2_d;
X80SoftFloat Result = extF80_mul(lhs, Src2_X80);
X80SoftFloat Src2_X80(state, Src2_d);
X80SoftFloat Result = extF80_mul(state, lhs, Src2_X80);
return Result;
#endif
}
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat F2XM1(const X80SoftFloat& lhs) {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat F2XM1(softfloat_state* state, const X80SoftFloat& lhs) {
WARN_ONCE_FMT("x87: Application used F2XM1 which may have accuracy problems");
#ifdef DEBUG_X86_FLOAT
BIGFLOAT Result;
@@ -286,14 +315,14 @@ struct FEX_PACKED X80SoftFloat {
return Result;
#else
LIBRARY_PRECISION Src1_d = lhs;
LIBRARY_PRECISION Src1_d = lhs.ToFMax(state);
LIBRARY_PRECISION Result = exp2l(Src1_d);
Result -= 1.0;
return Result;
return X80SoftFloat(state, Result);
#endif
}
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat FYL2X(const X80SoftFloat& lhs, const X80SoftFloat& rhs) {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat FYL2X(softfloat_state* state, const X80SoftFloat& lhs, const X80SoftFloat& rhs) {
WARN_ONCE_FMT("x87: Application used FYL2X which may have accuracy problems");
#ifdef DEBUG_X86_FLOAT
BIGFLOAT Result;
@@ -310,14 +339,14 @@ struct FEX_PACKED X80SoftFloat {
return Result;
#else
LIBRARY_PRECISION Src1_d = lhs;
LIBRARY_PRECISION Src2_d = rhs;
LIBRARY_PRECISION Src1_d = lhs.ToFMax(state);
LIBRARY_PRECISION Src2_d = rhs.ToFMax(state);
LIBRARY_PRECISION Tmp = Src2_d * log2l(Src1_d);
return Tmp;
return X80SoftFloat(state, Tmp);
#endif
}
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat FATAN(const X80SoftFloat& lhs, const X80SoftFloat& rhs) {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat FATAN(softfloat_state* state, const X80SoftFloat& lhs, const X80SoftFloat& rhs) {
WARN_ONCE_FMT("x87: Application used FATAN which may have accuracy problems");
#ifdef DEBUG_X86_FLOAT
BIGFLOAT Result;
@@ -334,14 +363,14 @@ struct FEX_PACKED X80SoftFloat {
return Result;
#else
LIBRARY_PRECISION Src1_d = lhs;
LIBRARY_PRECISION Src2_d = rhs;
LIBRARY_PRECISION Src1_d = lhs.ToFMax(state);
LIBRARY_PRECISION Src2_d = rhs.ToFMax(state);
LIBRARY_PRECISION Tmp = atan2l(Src1_d, Src2_d);
return Tmp;
return X80SoftFloat(state, Tmp);
#endif
}
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat FTAN(const X80SoftFloat& lhs) {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat FTAN(softfloat_state* state, const X80SoftFloat& lhs) {
WARN_ONCE_FMT("x87: Application used FTAN which may have accuracy problems");
#ifdef DEBUG_X86_FLOAT
BIGFLOAT Result;
@@ -358,13 +387,13 @@ struct FEX_PACKED X80SoftFloat {
return Result;
#else
LIBRARY_PRECISION Src_d = lhs;
LIBRARY_PRECISION Src_d = lhs.ToFMax(state);
Src_d = tanl(Src_d);
return Src_d;
return X80SoftFloat(state, Src_d);
#endif
}
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat FSIN(const X80SoftFloat& lhs) {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat FSIN(softfloat_state* state, const X80SoftFloat& lhs) {
WARN_ONCE_FMT("x87: Application used FSIN which may have accuracy problems");
#ifdef DEBUG_X86_FLOAT
BIGFLOAT Result;
@@ -380,13 +409,13 @@ struct FEX_PACKED X80SoftFloat {
return Result;
#else
LIBRARY_PRECISION Src_d = lhs;
LIBRARY_PRECISION Src_d = lhs.ToFMax(state);
Src_d = sinl(Src_d);
return Src_d;
return X80SoftFloat(state, Src_d);
#endif
}
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat FCOS(const X80SoftFloat& lhs) {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat FCOS(softfloat_state* state, const X80SoftFloat& lhs) {
WARN_ONCE_FMT("x87: Application used FCOS which may have accuracy problems");
#ifdef DEBUG_X86_FLOAT
BIGFLOAT Result;
@@ -402,13 +431,13 @@ struct FEX_PACKED X80SoftFloat {
return Result;
#else
LIBRARY_PRECISION Src_d = lhs;
LIBRARY_PRECISION Src_d = lhs.ToFMax(state);
Src_d = cosl(Src_d);
return Src_d;
return X80SoftFloat(state, Src_d);
#endif
}
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat FSQRT(const X80SoftFloat& lhs) {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat FSQRT(softfloat_state* state, const X80SoftFloat& lhs) {
#ifdef DEBUG_X86_FLOAT
BIGFLOAT Result;
asm(R"(
@@ -423,62 +452,55 @@ struct FEX_PACKED X80SoftFloat {
return Result;
#else
return extF80_sqrt(lhs);
return extF80_sqrt(state, lhs);
#endif
}
operator float() const {
const float32_t Result = extF80_to_f32(*this);
float ToF32(softfloat_state* state) const {
const float32_t Result = extF80_to_f32(state, *this);
return FEXCore::BitCast<float>(Result);
}
operator double() const {
const float64_t Result = extF80_to_f64(*this);
double ToF64(softfloat_state* state) const {
const float64_t Result = extF80_to_f64(state, *this);
return FEXCore::BitCast<double>(Result);
}
#ifndef _WIN32
operator BIGFLOAT() const {
LIBRARY_PRECISION ToFMax(softfloat_state* state) const {
#ifdef _WIN32
return ToF64(state);
#else
#if BIGFLOATSIZE == 16
const float128_t Result = extF80_to_f128(*this);
const float128_t Result = extF80_to_f128(state, *this);
return FEXCore::BitCast<BIGFLOAT>(Result);
#else
BIGFLOAT result {};
memcpy(&result, this, sizeof(result));
return result;
#endif
}
#endif
}
operator int16_t() const {
auto rv = extF80_to_i32(*this, softfloat_roundingMode, false);
if (rv > INT16_MAX) {
return INT16_MAX;
} else if (rv < INT16_MIN) {
int16_t ToI16(softfloat_state* state) const {
auto rv = extF80_to_i32(state, *this, state->roundingMode, false);
if (rv > INT16_MAX || rv < INT16_MIN) {
///< Indefinite value for 16-bit conversions.
return INT16_MIN;
} else {
return rv;
}
}
operator int32_t() const {
return extF80_to_i32(*this, softfloat_roundingMode, false);
int32_t ToI32(softfloat_state* state) const {
return extF80_to_i32(state, *this, state->roundingMode, false);
}
operator int64_t() const {
return extF80_to_i64(*this, softfloat_roundingMode, false);
int64_t ToI64(softfloat_state* state) const {
return extF80_to_i64(state, *this, state->roundingMode, false);
}
operator uint64_t() const {
return extF80_to_ui64(*this, softfloat_roundingMode, false);
}
void operator=(const float rhs) {
*this = f32_to_extF80(FEXCore::BitCast<float32_t>(rhs));
}
void operator=(const double rhs) {
*this = f64_to_extF80(FEXCore::BitCast<float64_t>(rhs));
uint64_t ToUI64(softfloat_state* state) const {
return extF80_to_ui64(state, *this, state->roundingMode, false);
}
void operator=(const int16_t rhs) {
@@ -509,18 +531,18 @@ struct FEX_PACKED X80SoftFloat {
Sign = rhs.signExp >> 15;
}
X80SoftFloat(const float rhs) {
*this = f32_to_extF80(FEXCore::BitCast<float32_t>(rhs));
X80SoftFloat(softfloat_state* state, const float rhs) {
*this = f32_to_extF80(state, FEXCore::BitCast<float32_t>(rhs));
}
X80SoftFloat(const double rhs) {
*this = f64_to_extF80(FEXCore::BitCast<float64_t>(rhs));
X80SoftFloat(softfloat_state* state, const double rhs) {
*this = f64_to_extF80(state, FEXCore::BitCast<float64_t>(rhs));
}
#ifndef _WIN32
X80SoftFloat(BIGFLOAT rhs) {
X80SoftFloat(softfloat_state* state, BIGFLOAT rhs) {
#if BIGFLOATSIZE == 16
*this = f128_to_extF80(FEXCore::BitCast<float128_t>(rhs));
*this = f128_to_extF80(state, FEXCore::BitCast<float128_t>(rhs));
#else
*this = FEXCore::BitCast<long double>(rhs);
#endif
+33 -53
View File
@@ -43,7 +43,8 @@ namespace DefaultValues {
} // namespace DefaultValues
enum Paths {
PATH_DATA_DIR = 0,
PATH_DATA_DIR_LOCAL = 0,
PATH_DATA_DIR_GLOBAL,
PATH_CONFIG_DIR_LOCAL,
PATH_CONFIG_DIR_GLOBAL,
PATH_CONFIG_FILE_LOCAL,
@@ -53,8 +54,8 @@ enum Paths {
};
static std::array<fextl::string, Paths::PATH_LAST> Paths;
void SetDataDirectory(const std::string_view Path) {
Paths[PATH_DATA_DIR] = Path;
void SetDataDirectory(const std::string_view Path, bool Global) {
Paths[PATH_DATA_DIR_LOCAL + Global] = Path;
}
void SetConfigDirectory(const std::string_view Path, bool Global) {
@@ -73,15 +74,15 @@ const fextl::string& GetTelemetryDirectory() {
Path = TelemetryDirectory;
Path += "/";
} else {
Path = Config::GetDataDirectory() + "Telemetry/";
Path = Config::GetDataDirectory(false) + "Telemetry/";
}
}
return Path;
}
const fextl::string& GetDataDirectory() {
return Paths[PATH_DATA_DIR];
const fextl::string& GetDataDirectory(bool Global) {
return Paths[PATH_DATA_DIR_LOCAL + Global];
}
const fextl::string& GetConfigDirectory(bool Global) {
@@ -230,27 +231,26 @@ void Load() {
}
}
fextl::string ExpandPath(const fextl::string& ContainerPrefix, fextl::string PathName) {
fextl::string ExpandPath(const fextl::string& ContainerPrefix, const fextl::string& PathName) {
if (PathName.empty()) {
return {};
}
// Expand home if it exists
if (FHU::Filesystem::IsRelative(PathName)) {
fextl::string Home = getenv("HOME") ?: "";
// Home expansion only works if it is the first character
// This matches bash behaviour
if (PathName.at(0) == '~') {
PathName.replace(0, 1, Home);
return PathName;
if (PathName.starts_with("~/")) {
Home.append(PathName.begin() + 1, PathName.end());
return Home;
}
// Expand relative path to absolute
char ExistsTempPath[PATH_MAX];
char* RealPath = FHU::Filesystem::Absolute(PathName.c_str(), ExistsTempPath);
if (RealPath) {
PathName = RealPath;
if (RealPath && FHU::Filesystem::Exists(RealPath)) {
return RealPath;
}
// Only return if it exists
@@ -318,81 +318,61 @@ fextl::string FindContainerPrefix() {
void ReloadMetaLayer() {
Meta->Load();
// Do configuration option fix ups after everything is reloaded
if (FEXCore::Config::Exists(FEXCore::Config::CONFIG_CORE)) {
// Sanitize Core option
FEX_CONFIG_OPT(Core, CORE);
#if (_M_X86_64)
constexpr uint32_t MaxCoreNumber = 1;
#else
constexpr uint32_t MaxCoreNumber = 0;
#endif
if (Core > MaxCoreNumber) {
// Sanitize the core option by setting the core to the JIT if invalid
FEXCore::Config::EraseSet(FEXCore::Config::CONFIG_CORE, fextl::fmt::format("{}", static_cast<uint32_t>(FEXCore::Config::CONFIG_IRJIT)));
}
}
if (FEXCore::Config::Exists(FEXCore::Config::CONFIG_CACHEOBJECTCODECOMPILATION)) {
FEX_CONFIG_OPT(CacheObjectCodeCompilation, CACHEOBJECTCODECOMPILATION);
FEX_CONFIG_OPT(Core, CORE);
}
fextl::string ContainerPrefix {FindContainerPrefix()};
auto ExpandPathIfExists = [&ContainerPrefix](FEXCore::Config::ConfigOption Config, fextl::string PathName) {
auto NewPath = ExpandPath(ContainerPrefix, PathName);
const fextl::string ContainerPrefix {FindContainerPrefix()};
auto ExpandPathIfExists = [&ContainerPrefix](FEXCore::Config::ConfigOption Config, const fextl::string& PathName) {
const auto NewPath = ExpandPath(ContainerPrefix, PathName);
if (!NewPath.empty()) {
FEXCore::Config::EraseSet(Config, NewPath);
}
};
if (FEXCore::Config::Exists(FEXCore::Config::CONFIG_ROOTFS)) {
FEX_CONFIG_OPT(PathName, ROOTFS);
auto ExpandedString = ExpandPath(ContainerPrefix, PathName());
const auto PathName = *Meta->Get(FEXCore::Config::CONFIG_ROOTFS);
const auto ExpandedString = ExpandPath(ContainerPrefix, *PathName);
if (!ExpandedString.empty()) {
// Adjust the path if it ended up being relative
FEXCore::Config::EraseSet(FEXCore::Config::CONFIG_ROOTFS, ExpandedString);
} else if (!PathName().empty()) {
} else if (!PathName->empty()) {
// If the filesystem doesn't exist then let's see if it exists in the fex-emu folder
fextl::string NamedRootFS = GetDataDirectory() + "RootFS/" + PathName();
fextl::string NamedRootFS = GetDataDirectory(false) + "RootFS/" + *PathName;
if (FHU::Filesystem::Exists(NamedRootFS)) {
FEXCore::Config::EraseSet(FEXCore::Config::CONFIG_ROOTFS, NamedRootFS);
}
}
}
if (FEXCore::Config::Exists(FEXCore::Config::CONFIG_THUNKHOSTLIBS)) {
FEX_CONFIG_OPT(PathName, THUNKHOSTLIBS);
ExpandPathIfExists(FEXCore::Config::CONFIG_THUNKHOSTLIBS, PathName());
const auto PathName = *Meta->Get(FEXCore::Config::CONFIG_THUNKHOSTLIBS);
ExpandPathIfExists(FEXCore::Config::CONFIG_THUNKHOSTLIBS, *PathName);
}
if (FEXCore::Config::Exists(FEXCore::Config::CONFIG_THUNKGUESTLIBS)) {
FEX_CONFIG_OPT(PathName, THUNKGUESTLIBS);
ExpandPathIfExists(FEXCore::Config::CONFIG_THUNKGUESTLIBS, PathName());
const auto PathName = *Meta->Get(FEXCore::Config::CONFIG_THUNKGUESTLIBS);
ExpandPathIfExists(FEXCore::Config::CONFIG_THUNKGUESTLIBS, *PathName);
}
if (FEXCore::Config::Exists(FEXCore::Config::CONFIG_THUNKCONFIG)) {
FEX_CONFIG_OPT(PathName, THUNKCONFIG);
auto ExpandedString = ExpandPath(ContainerPrefix, PathName());
const auto PathName = *Meta->Get(FEXCore::Config::CONFIG_THUNKCONFIG);
const auto ExpandedString = ExpandPath(ContainerPrefix, *PathName);
if (!ExpandedString.empty()) {
// Adjust the path if it ended up being relative
FEXCore::Config::EraseSet(FEXCore::Config::CONFIG_THUNKCONFIG, ExpandedString);
} else if (!PathName().empty()) {
} else if (!PathName->empty()) {
// If the filesystem doesn't exist then let's see if it exists in the fex-emu folder
fextl::string NamedConfig = GetDataDirectory() + "ThunkConfigs/" + PathName();
fextl::string NamedConfig = GetDataDirectory(false) + "ThunkConfigs/" + *PathName;
if (FHU::Filesystem::Exists(NamedConfig)) {
FEXCore::Config::EraseSet(FEXCore::Config::CONFIG_THUNKCONFIG, NamedConfig);
}
}
}
if (FEXCore::Config::Exists(FEXCore::Config::CONFIG_OUTPUTLOG)) {
FEX_CONFIG_OPT(PathName, OUTPUTLOG);
if (PathName() != "stdout" && PathName() != "stderr" && PathName() != "server") {
ExpandPathIfExists(FEXCore::Config::CONFIG_OUTPUTLOG, PathName());
const auto PathName = *Meta->Get(FEXCore::Config::CONFIG_OUTPUTLOG);
if (*PathName != "stdout" && *PathName != "stderr" && *PathName != "server") {
ExpandPathIfExists(FEXCore::Config::CONFIG_OUTPUTLOG, *PathName);
}
}
if (FEXCore::Config::Exists(FEXCore::Config::CONFIG_DUMPIR) && !FEXCore::Config::Exists(FEXCore::Config::CONFIG_PASSMANAGERDUMPIR)) {
// If DumpIR is set but no PassManagerDumpIR configuration is set, then default to `afteropt`
FEX_CONFIG_OPT(PathName, DUMPIR);
if (PathName() != "no") {
const auto PathName = *Meta->Get(FEXCore::Config::CONFIG_DUMPIR);
if (*PathName != "no") {
EraseSet(FEXCore::Config::ConfigOption::CONFIG_PASSMANAGERDUMPIR,
fextl::fmt::format("{}", static_cast<uint64_t>(FEXCore::Config::PassManagerDumpIR::AFTEROPT)));
}
+20 -15
View File
@@ -1,19 +1,6 @@
{
"Options": {
"CPU": {
"Core": {
"Type": "uint32",
"Default": "FEXCore::Config::ConfigCore::CONFIG_IRJIT",
"TextDefault": "irjit",
"ShortArg": "c",
"Choices": [ "irjit", "host" ],
"ArgumentHandler": "CoreHandler",
"Desc": [
"Which CPU core to use",
"host only exists on x86_64",
"[irjit, host]"
]
},
"Multiblock": {
"Type": "bool",
"Default": "false",
@@ -76,6 +63,8 @@
"DISABLECRYPTO": "disablecrypto",
"ENABLERPRES": "enablerpres",
"DISABLERPRES": "disablerpres",
"ENABLESVEBITPERM": "enablesvebitperm",
"DISABLESVEBITPERM": "disablesvebitperm",
"ENABLEPRESERVEALLABI": "enablepreserveallabi",
"DISABLEPRESERVEALLABI": "disablepreserveallabi"
},
@@ -97,6 +86,7 @@
"\t{enable,disable}flagm2: Will force enable or disable flagm2 even if the host doesn't support it",
"\t{enable,disable}crypto: Will force enable or disable crypto extensions even if the host doesn't support it",
"\t{enable,disable}rpres: Will force enable or disable rpres even if the host doesn't support it",
"\t{enable,disable}svebitperm: Will force enable or disable svebitperm even if the host doesn't support it",
"\t{enable,disable}preserveallabi: Will force enable or disable preserve_all abi even if the host doesn't support it"
]
},
@@ -139,7 +129,7 @@
},
"ThunkHostLibs": {
"Type": "str",
"Default": "@CMAKE_INSTALL_PREFIX@/lib/fex-emu/HostThunks/",
"Default": "@CMAKE_INSTALL_PREFIX@/@CMAKE_INSTALL_LIBDIR@/fex-emu/HostThunks/",
"ShortArg": "t",
"Desc": [
"Folder to find the host-side thunking libraries."
@@ -155,7 +145,7 @@
},
"ThunkHostLibs32": {
"Type": "str",
"Default": "@CMAKE_INSTALL_PREFIX@/lib/fex-emu/HostThunks_32/",
"Default": "@CMAKE_INSTALL_PREFIX@/@CMAKE_INSTALL_LIBDIR@/fex-emu/HostThunks_32/",
"Desc": [
"Folder to find the 32-bit host-side thunking libraries."
]
@@ -419,6 +409,14 @@
"Can be dangerous due to aligned loadstores through the same code now become non-atomic."
]
},
"StrictInProcessSplitLocks": {
"Type": "bool",
"Default": "false",
"Desc": [
"Strict global lock when handling an unaligned atomic that crosses a 16-byte or cacheline granularity",
"This is required to ensure a split-lock doesn't tear inside the process"
]
},
"TSOAutoMigration": {
"Type": "bool",
"Default": "true",
@@ -506,6 +504,13 @@
"Desc": [
"Override for a FEXServer socket path. Only useful for chroots."
]
},
"NeedsSeccomp": {
"Type": "bool",
"Default": "false",
"Desc": [
"Disables inline syscalls in order to support seccomp handling"
]
}
}
},
+8 -10
View File
@@ -6,7 +6,9 @@
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/Core/Context.h>
#include <FEXCore/Core/CPUID.h>
#include <FEXCore/Core/HostFeatures.h>
#include <FEXCore/Core/SignalDelegator.h>
#include <FEXCore/Core/Thunks.h>
#include "FEXCore/Debug/InternalThreadState.h"
#include <string.h>
@@ -18,8 +20,8 @@ void InitializeStaticTables(OperatingMode Mode) {
IR::InstallOpcodeHandlers(Mode);
}
fextl::unique_ptr<FEXCore::Context::Context> FEXCore::Context::Context::CreateNewContext() {
return fextl::make_unique<FEXCore::Context::ContextImpl>();
fextl::unique_ptr<FEXCore::Context::Context> FEXCore::Context::Context::CreateNewContext(const FEXCore::HostFeatures& Features) {
return fextl::make_unique<FEXCore::Context::ContextImpl>(Features);
}
void FEXCore::Context::ContextImpl::SetExitHandler(ExitHandler handler) {
@@ -38,14 +40,6 @@ void FEXCore::Context::ContextImpl::CompileRIPCount(FEXCore::Core::InternalThrea
CompileBlock(Thread->CurrentFrame, GuestRIP, MaxInst);
}
void FEXCore::Context::ContextImpl::SetCustomCPUBackendFactory(CustomCPUFactoryType Factory) {
CustomCPUFactory = std::move(Factory);
}
HostFeatures FEXCore::Context::ContextImpl::GetHostFeatures() const {
return HostFeatures;
}
void FEXCore::Context::ContextImpl::SetSignalDelegator(FEXCore::SignalDelegator* _SignalDelegation) {
SignalDelegation = _SignalDelegation;
}
@@ -55,6 +49,10 @@ void FEXCore::Context::ContextImpl::SetSyscallHandler(FEXCore::HLE::SyscallHandl
SourcecodeResolver = Handler->GetSourcecodeResolver();
}
void FEXCore::Context::ContextImpl::SetThunkHandler(FEXCore::ThunkHandler* Handler) {
ThunkHandler = Handler;
}
FEXCore::CPUID::FunctionResults FEXCore::Context::ContextImpl::RunCPUIDFunction(uint32_t Function, uint32_t Leaf) {
return CPUID.RunFunction(Function, Leaf);
}
+41 -63
View File
@@ -26,14 +26,8 @@
#include <stdint.h>
#include <atomic>
#include <condition_variable>
#include <functional>
#include <istream>
#include <memory>
#include <mutex>
#include <shared_mutex>
#include <stddef.h>
#include <queue>
namespace FEXCore {
class CodeLoader;
@@ -65,18 +59,21 @@ namespace Validation {
} // namespace FEXCore::IR
namespace FEXCore::Context {
enum CoreRunningMode {
MODE_RUN = 0,
MODE_SINGLESTEP = 1,
};
struct ExitFunctionLinkData {
uint64_t HostBranch;
uint64_t GuestRIP;
};
struct CustomIRResult {
void* Creator;
void* Data;
CustomIRResult(void* Creator, void* Data)
: Creator(Creator)
, Data(Data) {}
};
using BlockDelinkerFunc = void (*)(FEXCore::Core::CpuStateFrame* Frame, FEXCore::Context::ExitFunctionLinkData* Record);
constexpr uint32_t TSC_SCALE = 128;
constexpr uint32_t TSC_SCALE_MAXIMUM = 1'000'000'000; ///< 1Ghz
class ContextImpl final : public FEXCore::Context::Context {
@@ -94,10 +91,6 @@ public:
void CompileRIP(FEXCore::Core::InternalThreadState* Thread, uint64_t GuestRIP) override;
void CompileRIPCount(FEXCore::Core::InternalThreadState* Thread, uint64_t GuestRIP, uint64_t MaxInst) override;
void SetCustomCPUBackendFactory(CustomCPUFactoryType Factory) override;
HostFeatures GetHostFeatures() const override;
void HandleCallback(FEXCore::Core::InternalThreadState* Thread, uint64_t RIP) override;
uint64_t RestoreRIPFromHostPC(FEXCore::Core::InternalThreadState* Thread, uint64_t HostPC) override;
@@ -136,7 +129,7 @@ public:
*/
FEXCore::Core::InternalThreadState*
CreateThread(uint64_t InitialRIP, uint64_t StackPointer, FEXCore::Core::CPUState* NewThreadState, uint64_t ParentTID) override;
CreateThread(uint64_t InitialRIP, uint64_t StackPointer, const FEXCore::Core::CPUState* NewThreadState, uint64_t ParentTID) override;
// Public for threading
void ExecutionThread(FEXCore::Core::InternalThreadState* Thread) override;
@@ -154,6 +147,7 @@ public:
#endif
void SetSignalDelegator(FEXCore::SignalDelegator* SignalDelegation) override;
void SetSyscallHandler(FEXCore::HLE::SyscallHandler* Handler) override;
void SetThunkHandler(FEXCore::ThunkHandler* Handler) override;
FEXCore::CPUID::FunctionResults RunCPUIDFunction(uint32_t Function, uint32_t Leaf) override;
FEXCore::CPUID::XCRResults RunXCRFunction(uint32_t Function) override;
@@ -193,9 +187,10 @@ public:
bool IsAddressInCodeBuffer(FEXCore::Core::InternalThreadState* Thread, uintptr_t Address) const override;
// returns false if a handler was already registered
CustomIRResult AddCustomIREntrypoint(uintptr_t Entrypoint, CustomIREntrypointHandler Handler, void* Creator = nullptr, void* Data = nullptr);
std::optional<CustomIRResult>
AddCustomIREntrypoint(uintptr_t Entrypoint, CustomIREntrypointHandler Handler, void* Creator = nullptr, void* Data = nullptr);
void AppendThunkDefinitions(const fextl::vector<FEXCore::IR::ThunkDefinition>& Definitions) override;
void AddThunkTrampolineIRHandler(uintptr_t Entrypoint, uintptr_t GuestThunkEntrypoint) override;
public:
friend class FEXCore::HLE::SyscallHandler;
@@ -206,8 +201,8 @@ public:
friend class FEXCore::IR::Validation::IRValidation;
struct {
CoreRunningMode RunningMode {CoreRunningMode::MODE_RUN};
uint64_t VirtualMemSize {1ULL << 36};
uint64_t TSCScale = 0;
// Used if the JIT needs to have its interrupt fault code emitted.
bool NeedsPendingInterruptFaultCheck {false};
@@ -218,17 +213,15 @@ public:
FEX_CONFIG_OPT(Is64BitMode, IS64BIT_MODE);
FEX_CONFIG_OPT(TSOEnabled, TSOENABLED);
FEX_CONFIG_OPT(TSOAutoMigration, TSOAUTOMIGRATION);
FEX_CONFIG_OPT(VectorTSOEnabled, VECTORTSOENABLED);
FEX_CONFIG_OPT(MemcpySetTSOEnabled, MEMCPYSETTSOENABLED);
FEX_CONFIG_OPT(ABILocalFlags, ABILOCALFLAGS);
FEX_CONFIG_OPT(AOTIRCapture, AOTIRCAPTURE);
FEX_CONFIG_OPT(AOTIRGenerate, AOTIRGENERATE);
FEX_CONFIG_OPT(AOTIRLoad, AOTIRLOAD);
FEX_CONFIG_OPT(SMCChecks, SMCCHECKS);
FEX_CONFIG_OPT(Core, CORE);
FEX_CONFIG_OPT(MaxInstPerBlock, MAXINST);
FEX_CONFIG_OPT(RootFSPath, ROOTFS);
FEX_CONFIG_OPT(ThunkHostLibsPath, THUNKHOSTLIBS);
FEX_CONFIG_OPT(ThunkHostLibsPath32, THUNKHOSTLIBS32);
FEX_CONFIG_OPT(ThunkConfigFile, THUNKCONFIG);
FEX_CONFIG_OPT(GlobalJITNaming, GLOBALJITNAMING);
FEX_CONFIG_OPT(LibraryJITNaming, LIBRARYJITNAMING);
FEX_CONFIG_OPT(BlockJITNaming, BLOCKJITNAMING);
@@ -239,32 +232,29 @@ public:
FEX_CONFIG_OPT(DisableTelemetry, DISABLETELEMETRY);
FEX_CONFIG_OPT(DisableVixlIndirectCalls, DISABLE_VIXL_INDIRECT_RUNTIME_CALLS);
FEX_CONFIG_OPT(SmallTSCScale, SMALLTSCSCALE);
FEX_CONFIG_OPT(StrictInProcessSplitLocks, STRICTINPROCESSSPLITLOCKS);
} Config;
std::atomic_bool CoreShuttingDown {false};
FEXCore::ForkableSharedMutex CodeInvalidationMutex;
uint32_t StrictSplitLockMutex {};
FEXCore::HostFeatures HostFeatures;
// CPUID depends on HostFeatures so needs to be initialized after that.
FEXCore::CPUIDEmu CPUID;
FEXCore::HLE::SyscallHandler* SyscallHandler {};
FEXCore::HLE::SourcecodeResolver* SourcecodeResolver {};
fextl::unique_ptr<FEXCore::ThunkHandler> ThunkHandler;
FEXCore::ThunkHandler* ThunkHandler {};
fextl::unique_ptr<FEXCore::CPU::Dispatcher> Dispatcher;
CustomCPUFactoryType CustomCPUFactory;
FEXCore::Context::ExitHandler CustomExitHandler;
#ifdef BLOCKSTATS
fextl::unique_ptr<FEXCore::BlockSamplingData> BlockData;
#endif
SignalDelegator* SignalDelegation {};
X86GeneratedCode X86CodeGen;
ContextImpl();
ContextImpl(const FEXCore::HostFeatures& Features);
~ContextImpl();
static void ThreadRemoveCodeEntry(FEXCore::Core::InternalThreadState* Thread, uint64_t GuestRIP);
@@ -283,9 +273,6 @@ public:
// Must be called from owning thread
static void ThreadRemoveCodeEntryFromJit(FEXCore::Core::CpuStateFrame* Frame, uint64_t GuestRIP) {
auto Thread = Frame->Thread;
LOGMAN_THROW_A_FMT(Thread->ThreadManager.GetTID() == FHU::Syscalls::gettid(), "Must be called from owning thread {}, not {}",
Thread->ThreadManager.GetTID(), FHU::Syscalls::gettid());
auto lk = GuardSignalDeferringSection(static_cast<ContextImpl*>(Thread->CTX)->CodeInvalidationMutex, Thread);
ThreadRemoveCodeEntry(Thread, GuestRIP);
@@ -331,16 +318,6 @@ public:
FEXCore::JITSymbols Symbols;
void GetVDSOSigReturn(VDSOSigReturn* VDSOPointers) override {
if (VDSOPointers->VDSO_kernel_sigreturn == nullptr) {
VDSOPointers->VDSO_kernel_sigreturn = reinterpret_cast<void*>(X86CodeGen.sigreturn_32);
}
if (VDSOPointers->VDSO_kernel_rt_sigreturn == nullptr) {
VDSOPointers->VDSO_kernel_rt_sigreturn = reinterpret_cast<void*>(X86CodeGen.rt_sigreturn_32);
}
}
FEXCore::Utils::PooledAllocatorVirtual OpDispatcherAllocator;
FEXCore::Utils::PooledAllocatorVirtual FrontendAllocator;
@@ -349,27 +326,21 @@ public:
return AtomicTSOEmulationEnabled;
}
// If atomic-based TSO emulation is enabled for vector operations.
bool IsVectorAtomicTSOEnabled() const {
return VectorAtomicTSOEmulationEnabled;
}
// If atomic-based TSO emulation is enabled for memcpy operations.
bool IsMemcpyAtomicTSOEnabled() const {
return MemcpyAtomicTSOEmulationEnabled;
}
void SetHardwareTSOSupport(bool HardwareTSOSupported) override {
SupportsHardwareTSO = HardwareTSOSupported;
UpdateAtomicTSOEmulationConfig();
}
// Returns if Software TSO emulation is required.
// NOTE: This doesn't necessary return if Atomic-based TSO is currently enabled.
// This will still return true if on a single thread and TSO is currently disabled.
//
// This is to ensure that if early initialization checks CPU features and TSO /could/ be enabled, that
// we return consistent results.
//
// To check if Atomic TSO is currently enabled in the JIT, use `IsAtomicTSOEnabled` instead.
bool SoftwareTSORequired() const {
if (SupportsHardwareTSO) {
return false;
}
return Config.TSOEnabled;
}
void EnableExitOnHLT() override {
ExitOnHLT = true;
}
@@ -378,16 +349,20 @@ public:
return ExitOnHLT;
}
FEXCore::CPU::CPUBackendFeatures BackendFeatures;
protected:
void UpdateAtomicTSOEmulationConfig() {
if (SupportsHardwareTSO) {
// If the hardware supports TSO then we don't need to emulate it through atomics.
AtomicTSOEmulationEnabled = false;
VectorAtomicTSOEmulationEnabled = false;
MemcpyAtomicTSOEmulationEnabled = false;
} else {
// Atomic TSO emulation only enabled if the config option is enabled.
AtomicTSOEmulationEnabled = (IsMemoryShared || !Config.TSOAutoMigration) && Config.TSOEnabled;
// Atomic vector TSO emulation only enabled if TSO emulation is enabled and also vector TSO is enabled.
VectorAtomicTSOEmulationEnabled = (IsMemoryShared || !Config.TSOAutoMigration) && Config.TSOEnabled && Config.VectorTSOEnabled;
// Atomic memcpy TSO emulation only enabled if TSO emulation is enabled and also memcpy TSO is enabled.
MemcpyAtomicTSOEmulationEnabled = (IsMemoryShared || !Config.TSOAutoMigration) && Config.TSOEnabled && Config.MemcpySetTSOEnabled;
}
}
@@ -410,6 +385,9 @@ private:
bool IsMemoryShared = false;
bool SupportsHardwareTSO = false;
bool AtomicTSOEmulationEnabled = true;
bool VectorAtomicTSOEmulationEnabled = false;
bool MemcpyAtomicTSOEmulationEnabled = false;
bool ExitOnHLT = false;
FEX_CONFIG_OPT(AppFilename, APP_FILENAME);
@@ -4,7 +4,6 @@
#include "FEXCore/Utils/AllocatorHooks.h"
#include "Interface/Core/Dispatcher/Dispatcher.h"
#include "Interface/Context/Context.h"
#include "Interface/HLE/Thunks/Thunks.h"
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/Utils/LogManager.h>
@@ -14,10 +13,12 @@
#include <CodeEmitter/Emitter.h>
#include <CodeEmitter/Registers.h>
#ifdef VIXL_DISASSEMBLER
#include <aarch64/cpu-aarch64.h>
#include <aarch64/instructions-aarch64.h>
#include <cpu-features.h>
#include <utils-vixl.h>
#endif
#include <array>
#include <tuple>
@@ -27,11 +28,12 @@ namespace FEXCore::CPU {
namespace x64 {
#ifndef _M_ARM_64EC
// All but x19 and x29 are caller saved
// Note that rax/rdx are rearranged here so we can coalesce cmpxchg.
constexpr std::array<ARMEmitter::Register, 18> SRA = {
ARMEmitter::Reg::r4,
ARMEmitter::Reg::r7,
ARMEmitter::Reg::r5,
ARMEmitter::Reg::r6,
ARMEmitter::Reg::r7,
ARMEmitter::Reg::r8,
ARMEmitter::Reg::r9,
ARMEmitter::Reg::r10,
@@ -192,12 +194,12 @@ namespace x64 {
} // namespace x64
namespace x32 {
// All but x19 and x29 are caller saved
// All but x19 and x29 are caller saved. eax/edx rearranged for cmpxchg.
constexpr std::array<ARMEmitter::Register, 10> SRA = {
ARMEmitter::Reg::r4,
ARMEmitter::Reg::r7,
ARMEmitter::Reg::r5,
ARMEmitter::Reg::r6,
ARMEmitter::Reg::r7,
ARMEmitter::Reg::r8,
ARMEmitter::Reg::r9,
ARMEmitter::Reg::r10,
@@ -349,8 +351,6 @@ Arm64Emitter::Arm64Emitter(FEXCore::Context::ContextImpl* ctx, void* EmissionPtr
}
#endif
CPU.SetUp();
// Number of register available is dependent on what operating mode the proccess is in.
if (EmitterCTX->Config.Is64BitMode()) {
StaticRegisters = x64::SRA;
@@ -373,6 +373,20 @@ Arm64Emitter::Arm64Emitter(FEXCore::Context::ContextImpl* ctx, void* EmissionPtr
}
}
FEXCore::X86State::X86Reg Arm64Emitter::GetX86RegRelationToARMReg(ARMEmitter::Register Reg) {
for (size_t i = 0; i < StaticRegisters.size(); ++i) {
const auto& RegI = StaticRegisters[i];
if (RegI == Reg) {
// X86 Registers are mapped linerally from the StaticRegisters span.
// Directly correlating Enum index to span index.
return static_cast<FEXCore::X86State::X86Reg>(FEXCore::ToUnderlying(FEXCore::X86State::X86Reg::REG_RAX) + i);
}
}
// Unmapped register.
return FEXCore::X86State::X86Reg::REG_INVALID;
}
void Arm64Emitter::LoadConstant(ARMEmitter::Size s, ARMEmitter::Register Reg, uint64_t Constant, bool NOPPad) {
bool Is64Bit = s == ARMEmitter::Size::i64Bit;
int Segments = Is64Bit ? 4 : 2;
@@ -421,7 +435,7 @@ void Arm64Emitter::LoadConstant(ARMEmitter::Size s, ARMEmitter::Register Reg, ui
if (RequiredMoveSegments > 1) {
// Only try to use this path if the number of segments is > 1.
// `movz` is better than `orr` since hardware will rename or merge if possible when `movz` is used.
const auto IsImm = vixl::aarch64::Assembler::IsImmLogical(Constant, RegSizeInBits(s));
const auto IsImm = ARMEmitter::Emitter::IsImmLogical(Constant, RegSizeInBits(s));
if (IsImm) {
orr(s, Reg, ARMEmitter::Reg::zr, Constant);
if (NOPPad) {
@@ -458,7 +472,7 @@ void Arm64Emitter::LoadConstant(ARMEmitter::Size s, ARMEmitter::Register Reg, ui
// If the aligned offset is within the 4GB window then we can use ADRP+ADD
// and the number of move segments more than 1
if (RequiredMoveSegments > 1 && vixl::IsInt32(AlignedOffset)) {
if (RequiredMoveSegments > 1 && ARMEmitter::Emitter::IsInt32(AlignedOffset)) {
// If this is 4k page aligned then we only need ADRP
if ((AlignedOffset & 0xFFF) == 0) {
adrp(Reg, AlignedOffset >> 12);
@@ -466,7 +480,7 @@ void Arm64Emitter::LoadConstant(ARMEmitter::Size s, ARMEmitter::Register Reg, ui
// If the constant is within 1MB of PC then we can still use ADR to load in a single instruction
// 21-bit signed integer here
int64_t SmallOffset = static_cast<int64_t>(Constant) - static_cast<int64_t>(PC);
if (vixl::IsInt21(SmallOffset)) {
if (ARMEmitter::Emitter::IsInt21(SmallOffset)) {
adr(Reg, SmallOffset);
} else {
// Need to use ADRP + ADD
@@ -570,12 +584,53 @@ void Arm64Emitter::PopCalleeSavedRegisters() {
}
}
void Arm64Emitter::FillSpecialRegs(ARMEmitter::Register TmpReg, ARMEmitter::Register TmpReg2, bool SetFIZ, bool SetPredRegs) {
#ifndef VIXL_SIMULATOR
if (EmitterCTX->HostFeatures.SupportsAFP) {
// Enable AFP features when filling JIT state.
mrs(TmpReg, ARMEmitter::SystemRegister::FPCR);
// Enable FPCR.NEP and FPCR.AH
// NEP(2): Changes ASIMD scalar instructions to insert in to the lower bits of the destination.
// AH(1): Changes NaN behaviour in some instructions. Specifically fmin, fmax.
//
// Additional interesting AFP bits:
// FIZ(0): Flush Inputs to Zero
orr(ARMEmitter::Size::i64Bit, TmpReg, TmpReg,
(1U << 2) | // NEP
(1U << 1)); // AH
if (SetFIZ) {
// Insert MXCSR.DAZ in to FIZ
ldr(TmpReg2.W(), STATE.R(), offsetof(FEXCore::Core::CPUState, mxcsr));
bfxil(ARMEmitter::Size::i64Bit, TmpReg, TmpReg2, 6, 1);
}
msr(ARMEmitter::SystemRegister::FPCR, TmpReg);
}
#endif
if (SetPredRegs) {
// Set up predicate registers.
// We don't bother spilling these in SpillStaticRegs,
// since all that matters is we restore them on a fill.
// It's not a concern if they get trounced by something else.
if (EmitterCTX->HostFeatures.SupportsSVE256) {
ptrue(ARMEmitter::SubRegSize::i8Bit, PRED_TMP_32B, ARMEmitter::PredicatePattern::SVE_VL32);
}
if (EmitterCTX->HostFeatures.SupportsSVE128) {
ptrue(ARMEmitter::SubRegSize::i8Bit, PRED_TMP_16B, ARMEmitter::PredicatePattern::SVE_VL16);
}
}
}
void Arm64Emitter::SpillStaticRegs(ARMEmitter::Register TmpReg, bool FPRs, uint32_t GPRSpillMask, uint32_t FPRSpillMask) {
#ifndef VIXL_SIMULATOR
if (EmitterCTX->HostFeatures.SupportsAFP) {
// Disable AFP features when spilling registers.
//
// Disable FPCR.NEP and FPCR.AH
// Disable FPCR.NEP and FPCR.AH and FPCR.FIZ
// NEP(2): Changes ASIMD scalar instructions to insert in to the lower bits of the destination.
// AH(1): Changes NaN behaviour in some instructions. Specifically fmin, fmax.
// Also interacts with RPRES to change reciprocal/rsqrt precision from 8-bit mantissa to 12-bit.
@@ -585,7 +640,8 @@ void Arm64Emitter::SpillStaticRegs(ARMEmitter::Register TmpReg, bool FPRs, uint3
mrs(TmpReg, ARMEmitter::SystemRegister::FPCR);
bic(ARMEmitter::Size::i64Bit, TmpReg, TmpReg,
(1U << 2) | // NEP
(1U << 1)); // AH
(1U << 1) | // AH
(1U << 0)); // FIZ
msr(ARMEmitter::SystemRegister::FPCR, TmpReg);
}
#endif
@@ -663,37 +719,37 @@ void Arm64Emitter::SpillStaticRegs(ARMEmitter::Register TmpReg, bool FPRs, uint3
}
}
void Arm64Emitter::FillStaticRegs(bool FPRs, uint32_t GPRFillMask, uint32_t FPRFillMask) {
ARMEmitter::Register TmpReg = ARMEmitter::Reg::r0;
LOGMAN_THROW_A_FMT(GPRFillMask != 0, "Must fill at least 1 GPR for a temp");
[[maybe_unused]] bool FoundRegister {};
for (auto Reg : StaticRegisters) {
if (((1U << Reg.Idx()) & GPRFillMask)) {
TmpReg = Reg;
FoundRegister = true;
break;
void Arm64Emitter::FillStaticRegs(bool FPRs, uint32_t GPRFillMask, uint32_t FPRFillMask, std::optional<ARMEmitter::Register> OptionalReg,
std::optional<ARMEmitter::Register> OptionalReg2) {
auto FindTempReg = [this](uint32_t* GPRFillMask) -> std::optional<ARMEmitter::Register> {
for (auto Reg : StaticRegisters) {
if (((1U << Reg.Idx()) & *GPRFillMask)) {
*GPRFillMask &= ~(1U << Reg.Idx());
return std::make_optional(Reg);
}
}
return std::nullopt;
};
LOGMAN_THROW_A_FMT(GPRFillMask != 0, "Must fill at least 2 GPRs for a temp");
uint32_t TempGPRFillMask = GPRFillMask;
if (!OptionalReg.has_value()) {
OptionalReg = FindTempReg(&TempGPRFillMask);
}
LOGMAN_THROW_A_FMT(FoundRegister, "Didn't have an SRA register to use as a temporary while spilling!");
#ifndef VIXL_SIMULATOR
if (EmitterCTX->HostFeatures.SupportsAFP) {
// Enable AFP features when filling JIT state.
LOGMAN_THROW_A_FMT(GPRFillMask != 0, "Must fill at least 1 GPR for a temp");
mrs(TmpReg, ARMEmitter::SystemRegister::FPCR);
// Enable FPCR.NEP and FPCR.AH
// NEP(2): Changes ASIMD scalar instructions to insert in to the lower bits of the destination.
// AH(1): Changes NaN behaviour in some instructions. Specifically fmin, fmax.
//
// Additional interesting AFP bits:
// FIZ(0): Flush Inputs to Zero
orr(ARMEmitter::Size::i64Bit, TmpReg, TmpReg,
(1U << 2) | // NEP
(1U << 1)); // AH
msr(ARMEmitter::SystemRegister::FPCR, TmpReg);
if (!OptionalReg2.has_value()) {
OptionalReg2 = FindTempReg(&TempGPRFillMask);
}
LOGMAN_THROW_A_FMT(OptionalReg.has_value() && OptionalReg2.has_value(), "Didn't have an SRA register to use as a temporary while "
"spilling!");
auto TmpReg = *OptionalReg;
auto TmpReg2 = *OptionalReg2;
#ifdef _M_ARM_64EC
// Load STATE in from the CPU area as x28 is not callee saved in the ARM64EC ABI.
ldr(TmpReg.X(), ARMEmitter::Reg::r18, TEB_CPU_AREA_OFFSET);
ldr(STATE, TmpReg, CPU_AREA_EMULATOR_DATA_OFFSET);
#endif
// Regardless of what GPRs/FPRs we're filling, we need to fill NZCV since it
@@ -704,19 +760,9 @@ void Arm64Emitter::FillStaticRegs(bool FPRs, uint32_t GPRFillMask, uint32_t FPRF
ldr(TmpReg.W(), STATE.R(), offsetof(FEXCore::Core::CpuStateFrame, State.flags[24]));
msr(ARMEmitter::SystemRegister::NZCV, TmpReg);
FillSpecialRegs(TmpReg, TmpReg2, true, FPRs);
if (FPRs) {
// Set up predicate registers.
// We don't bother spilling these in SpillStaticRegs,
// since all that matters is we restore them on a fill.
// It's not a concern if they get trounced by something else.
if (EmitterCTX->HostFeatures.SupportsSVE128) {
ptrue(ARMEmitter::SubRegSize::i8Bit, PRED_TMP_16B, ARMEmitter::PredicatePattern::SVE_VL16);
}
if (EmitterCTX->HostFeatures.SupportsSVE256) {
ptrue(ARMEmitter::SubRegSize::i8Bit, PRED_TMP_32B, ARMEmitter::PredicatePattern::SVE_VL32);
}
if (EmitterCTX->HostFeatures.SupportsAVX && EmitterCTX->HostFeatures.SupportsSVE256) {
for (size_t i = 0; i < StaticFPRegisters.size(); i++) {
const auto Reg = StaticFPRegisters[i];
@@ -4,11 +4,6 @@
#include "FEXCore/Utils/EnumUtils.h"
#include "Interface/Core/ObjectCache/Relocations.h"
#include <aarch64/assembler-aarch64.h>
#include <aarch64/constants-aarch64.h>
#include <aarch64/cpu-aarch64.h>
#include <aarch64/operands-aarch64.h>
#include <platform-vixl.h>
#ifdef VIXL_DISASSEMBLER
#include <aarch64/disasm-aarch64.h>
#endif
@@ -17,6 +12,7 @@
#include <aarch64/simulator-constants-aarch64.h>
#endif
#include <FEXCore/Core/X86Enums.h>
#include <FEXCore/Config/Config.h>
#include <FEXCore/fextl/vector.h>
#include <CodeEmitter/Emitter.h>
@@ -70,6 +66,14 @@ constexpr auto VTMP2 = ARMEmitter::VReg::v17;
// Entry/Exit ABI
constexpr auto EC_CALL_CHECKER_PC_REG = ARMEmitter::XReg::x9;
constexpr auto EC_ENTRY_CPUAREA_REG = ARMEmitter::XReg::x17;
// These structures are not included in the standard Windows headers, define the offsets of members we care about for EC here.
constexpr size_t TEB_CPU_AREA_OFFSET = 0x1788;
constexpr size_t TEB_PEB_OFFSET = 0x60;
constexpr size_t PEB_EC_CODE_BITMAP_OFFSET = 0x368;
constexpr size_t CPU_AREA_IN_SYSCALL_CALLBACK_OFFSET = 0x1;
constexpr size_t CPU_AREA_EMULATOR_STACK_BASE_OFFSET = 0x8;
constexpr size_t CPU_AREA_EMULATOR_DATA_OFFSET = 0x30;
#endif
// Predicate register temporaries (used when AVX support is enabled)
@@ -86,7 +90,6 @@ protected:
Arm64Emitter(FEXCore::Context::ContextImpl* ctx, void* EmissionPtr = nullptr, size_t size = 0);
FEXCore::Context::ContextImpl* EmitterCTX;
vixl::aarch64::CPU CPU;
std::span<const ARMEmitter::Register> ConfiguredDynamicRegisterBase {};
std::span<const ARMEmitter::Register> StaticRegisters {};
@@ -97,12 +100,19 @@ protected:
void LoadConstant(ARMEmitter::Size s, ARMEmitter::Register Reg, uint64_t Constant, bool NOPPad = false);
void FillSpecialRegs(ARMEmitter::Register TmpReg, ARMEmitter::Register TmpReg2, bool SetFIZ, bool SetPredRegs);
// Correlate an ARM register back to an x86 register index.
// Returning REG_INVALID if there was no mapping.
FEXCore::X86State::X86Reg GetX86RegRelationToARMReg(ARMEmitter::Register Reg);
// NOTE: These functions WILL clobber the register TMP4 if AVX support is enabled
// and FPRs are being spilled or filled. If only GPRs are spilled/filled, then
// TMP4 is left alone.
void SpillStaticRegs(ARMEmitter::Register TmpReg, bool FPRs = true, uint32_t GPRSpillMask = ~0U, uint32_t FPRSpillMask = ~0U);
void FillStaticRegs(bool FPRs = true, uint32_t GPRFillMask = ~0U, uint32_t FPRFillMask = ~0U);
void FillStaticRegs(bool FPRs = true, uint32_t GPRFillMask = ~0U, uint32_t FPRFillMask = ~0U,
std::optional<ARMEmitter::Register> OptionalReg = std::nullopt,
std::optional<ARMEmitter::Register> OptionalReg2 = std::nullopt);
// Register 0-18 + 29 + 30 are caller saved
static constexpr uint32_t CALLER_GPR_MASK = 0b0110'0000'0000'0111'1111'1111'1111'1111U;
@@ -1,51 +0,0 @@
// SPDX-License-Identifier: MIT
#include "Interface/Core/BlockSamplingData.h"
#include <FEXCore/Utils/LogManager.h>
#include <cstring>
#include <fstream>
#include <utility>
namespace FEXCore {
void BlockSamplingData::DumpBlockData() {
std::fstream Output;
Output.open("output.csv", std::fstream::out | std::fstream::binary);
if (!Output.is_open()) {
return;
}
Output << "Entry, Min, Max, Total, Calls, Average" << std::endl;
for (auto it : SamplingMap) {
if (!it.second->TotalCalls) {
continue;
}
Output << "0x" << std::hex << it.first << ", " << std::dec << it.second->Min << ", " << std::dec << it.second->Max << ", " << std::dec
<< it.second->TotalTime << ", " << std::dec << it.second->TotalCalls << ", " << std::dec
<< ((double)it.second->TotalTime / (double)it.second->TotalCalls) << std::endl;
}
Output.close();
LogMan::Msg::DFmt("Dumped {} blocks of sampling data", SamplingMap.size());
}
BlockSamplingData::BlockData* BlockSamplingData::GetBlockData(uint64_t RIP) {
auto it = SamplingMap.find(RIP);
if (it != SamplingMap.end()) {
return it->second;
}
BlockData* NewData = new BlockData {};
memset(NewData, 0, sizeof(BlockData));
NewData->Min = ~0ULL;
SamplingMap[RIP] = NewData;
return NewData;
}
BlockSamplingData::~BlockSamplingData() {
DumpBlockData();
for (auto it : SamplingMap) {
delete it.second;
}
SamplingMap.clear();
}
} // namespace FEXCore
@@ -1,25 +0,0 @@
// SPDX-License-Identifier: MIT
#pragma once
#include <cstdint>
#include <unordered_map>
namespace FEXCore {
class BlockSamplingData {
public:
struct BlockData {
uint64_t Start, End;
uint64_t Min, Max;
uint64_t TotalTime;
uint64_t TotalCalls;
};
BlockData* GetBlockData(uint64_t RIP);
~BlockSamplingData();
void DumpBlockData();
private:
std::unordered_map<uint64_t, BlockData*> SamplingMap;
};
} // namespace FEXCore
@@ -34,11 +34,6 @@ namespace CodeSerialize {
}
namespace CPU {
struct CPUBackendFeatures {
bool SupportsFlags = false;
bool SupportsVTBL2 = false;
};
class CPUBackend {
public:
struct CodeBuffer {
@@ -53,11 +48,6 @@ namespace CPU {
CPUBackend(FEXCore::Core::InternalThreadState* ThreadState, size_t InitialCodeSize, size_t MaxCodeSize);
virtual ~CPUBackend();
/**
* @return The name of this backend
*/
[[nodiscard]]
virtual fextl::string GetName() = 0;
struct CompiledCode {
// Where this code block begins.
@@ -129,9 +119,6 @@ namespace CPU {
*
* This is a thread specific compilation unit since there is one CPUBackend per guest thread
*
* If NeedsOpDispatch is returning false then IR and DebugData may be null and the expectation is that the code will still compile
* FEXCore::Core::ThreadState* is valid at the time of compilation.
*
* @param IR - IR that maps to the IR for this RIP
* @param DebugData - Debug data that is available for this IR indirectly
*
@@ -154,26 +141,6 @@ namespace CPU {
return nullptr;
}
/**
* @brief Function for mapping memory in to the CPUBackend's visible space. Allows setting up virtual mappings if required
*
* @return Currently unused
*/
[[nodiscard]]
virtual void* MapRegion(void* HostPtr, uint64_t GuestPtr, uint64_t Size) = 0;
/**
* @brief Lets FEXCore know if this CPUBackend needs IR and DebugData for CompileCode
*
* This is useful if the FEXCore Frontend hits an x86-64 instruction that isn't understood but can continue regardless
*
* This is useful for example, a VM based CPUbackend
*
* @return true if it needs the IR
*/
[[nodiscard]]
virtual bool NeedsOpDispatch() = 0;
virtual void ClearCache() {}
/**
+81 -42
View File
@@ -34,6 +34,8 @@ namespace ProductNames {
static const char ARM_A76AE[] = "Cortex-A76AE";
static const char ARM_V1[] = "Neoverse V1";
static const char ARM_V2[] = "Neoverse V2";
static const char ARM_V3[] = "Neoverse V3";
static const char ARM_V3AE[] = "Neoverse V3AE";
static const char ARM_A77[] = "Cortex-A77";
static const char ARM_A78[] = "Cortex-A78";
static const char ARM_A78AE[] = "Cortex-A78AE";
@@ -41,13 +43,16 @@ namespace ProductNames {
static const char ARM_A710[] = "Cortex-A710";
static const char ARM_A715[] = "Cortex-A715";
static const char ARM_A720[] = "Cortex-A720";
static const char ARM_A725[] = "Cortex-A725";
static const char ARM_X1[] = "Cortex-X1";
static const char ARM_X1C[] = "Cortex-X1C";
static const char ARM_X2[] = "Cortex-X2";
static const char ARM_X3[] = "Cortex-X3";
static const char ARM_X4[] = "Cortex-X4";
static const char ARM_X925[] = "Cortex-X925";
static const char ARM_N1[] = "Neoverse N1";
static const char ARM_N2[] = "Neoverse N2";
static const char ARM_N3[] = "Neoverse N3";
static const char ARM_E1[] = "Neoverse E1";
static const char ARM_A35[] = "Cortex-A35";
static const char ARM_A53[] = "Cortex-A53";
@@ -67,8 +72,18 @@ namespace ProductNames {
static const char ARM_Denver[] = "Nvidia Denver";
static const char ARM_Carmel[] = "Nvidia Carmel";
static const char ARM_Firestorm[] = "Apple Firestorm";
static const char ARM_Icestorm[] = "Apple Icestorm";
static const char ARM_Firestorm_M1[] = "Apple Firestorm (M1)";
static const char ARM_Icestorm_M1[] = "Apple Icestorm (M1)";
static const char ARM_Firestorm_M1Pro[] = "Apple Firestorm (M1 Pro)";
static const char ARM_Icestorm_M1Pro[] = "Apple Icestorm (M1 Pro)";
static const char ARM_Firestorm_M1Max[] = "Apple Firestorm (M1 Max)";
static const char ARM_Icestorm_M1Max[] = "Apple Icestorm (M1 Max)";
static const char ARM_Avalanche_M2[] = "Apple Avalanche (M2)";
static const char ARM_Blizzard_M2[] = "Apple Blizzard (M2)";
static const char ARM_Avalanche_M2Pro[] = "Apple Avalanche (M2 Pro)";
static const char ARM_Blizzard_M2Pro[] = "Apple Blizzard (M2 Pro)";
static const char ARM_Avalanche_M2Max[] = "Apple Avalanche (M2 Max)";
static const char ARM_Blizzard_M2Max[] = "Apple Blizzard (M2 Max)";
static const char ARM_ORYON_1[] = "Oryon-1";
#else
@@ -81,20 +96,39 @@ static uint32_t GetCPUID() {
return CPU;
}
struct CPUFamily {
uint32_t Stepping : 4;
uint32_t Model : 4;
uint32_t ExtendedModel : 4;
uint32_t FamilyID : 4;
uint32_t ExtendedFamilyID : 8;
uint32_t ProcessorType : 4;
};
constexpr static uint32_t GenerateFamily(const CPUFamily Family) {
return Family.Stepping | (Family.Model << 4) | (Family.FamilyID << 8) | (Family.ProcessorType << 12) | (Family.ExtendedModel << 16) |
(Family.ExtendedFamilyID << 20);
}
#ifdef CPUID_AMD
constexpr uint32_t FAMILY_IDENTIFIER = 0 | // Stepping
(0xA << 4) | // Model
(0xF << 8) | // Family ID
(0 << 12) | // Processor type
(0 << 16) | // Extended model ID
(1 << 20); // Extended family ID
constexpr uint32_t FAMILY_IDENTIFIER = GenerateFamily(CPUFamily {
.Stepping = 0,
.Model = 0xA,
.ExtendedModel = 0,
.FamilyID = 0xF,
.ExtendedFamilyID = 1,
.ProcessorType = 0,
});
#else
constexpr uint32_t FAMILY_IDENTIFIER = 0 | // Stepping
(0x7 << 4) | // Model
(0x6 << 8) | // Family ID
(0 << 12) | // Processor type
(1 << 16) | // Extended model ID
(0x0 << 20); // Extended family ID
constexpr uint32_t FAMILY_IDENTIFIER = GenerateFamily(CPUFamily {
.Stepping = 1,
.Model = 6,
.ExtendedModel = 0xA,
.FamilyID = 6,
.ExtendedFamilyID = 0,
.ProcessorType = 0,
});
#endif
#ifdef _M_ARM_64
@@ -109,27 +143,16 @@ void CPUIDEmu::SetupHostHybridFlag() {
uint64_t MIDR {};
for (size_t i = 0; i < Cores; ++i) {
std::error_code ec {};
fextl::string MIDRPath = fextl::fmt::format("/sys/devices/system/cpu/cpu{}/regs/identification/midr_el1", i);
std::array<char, 18> Data;
// Needs to be a fixed size since depending on kernel it will try to read a full page of data and fail
// Only read 18 bytes for a 64bit value prefixed with 0x
if (FEXCore::FileLoading::LoadFileToBuffer(MIDRPath, Data) == sizeof(Data)) {
uint64_t NewMIDR {};
std::string_view MIDRView(Data.data(), sizeof(Data));
if (FEXCore::StrConv::Conv(MIDRView, &NewMIDR)) {
if (MIDR != 0 && MIDR != NewMIDR) {
// CPU mismatch, claim hybrid
Hybrid = true;
}
// Truncate to 32-bits, top 32-bits are all reserved in MIDR
PerCPUData[i].ProductName = ProductNames::ARM_UNKNOWN;
PerCPUData[i].MIDR = NewMIDR;
MIDR = NewMIDR;
}
auto NewMIDR = CTX->HostFeatures.CPUMIDRs[i];
if (MIDR != 0 && MIDR != NewMIDR) {
// CPU mismatch, claim hybrid
Hybrid = true;
}
// Truncate to 32-bits, top 32-bits are all reserved in MIDR
PerCPUData[i].ProductName = ProductNames::ARM_UNKNOWN;
PerCPUData[i].MIDR = NewMIDR;
MIDR = NewMIDR;
}
struct CPUMIDR {
@@ -142,12 +165,22 @@ void CPUIDEmu::SetupHostHybridFlag() {
// CPU priority order
// This is mostly arbitrary but will sort by some sort of CPU priority by performance
// Relative list so things they will commonly end up in big.little configurations sort of relate
static constexpr std::array<CPUMIDR, 43> CPUMIDRs = {{
static constexpr std::array<CPUMIDR, 58> CPUMIDRs = {{
// Typically big CPU cores
{0x51, 0x001, 1, ProductNames::ARM_ORYON_1}, // Qualcomm Oryon-1
{0x61, 0x023, 1, ProductNames::ARM_Firestorm}, // Apple M1 Firestorm
{0x61, 0x039, 1, ProductNames::ARM_Avalanche_M2Max}, // Apple Avalanche (M2 Max)
{0x61, 0x035, 1, ProductNames::ARM_Avalanche_M2Pro}, // Apple Avalanche (M2 Pro)
{0x61, 0x033, 1, ProductNames::ARM_Avalanche_M2}, // Apple Avalanche (M2)
{0x61, 0x029, 1, ProductNames::ARM_Firestorm_M1Max}, // Apple Firestorm (M1 Max)
{0x61, 0x025, 1, ProductNames::ARM_Firestorm_M1Pro}, // Apple Firestorm (M1 Pro)
{0x61, 0x023, 1, ProductNames::ARM_Firestorm_M1}, // Apple Firestorm (M1)
{0x41, 0xd85, 1, ProductNames::ARM_X925}, // X925
{0x41, 0xd87, 1, ProductNames::ARM_A725}, // A725
{0x41, 0xd84, 1, ProductNames::ARM_V3}, // V3
{0x41, 0xd83, 1, ProductNames::ARM_V3AE}, // V3AE
{0x41, 0xd8e, 1, ProductNames::ARM_N3}, // N3
{0x41, 0xd82, 1, ProductNames::ARM_X4}, // X4
{0x41, 0xd81, 1, ProductNames::ARM_A720}, // A720
{0x41, 0xd4e, 1, ProductNames::ARM_X3}, // X3
@@ -183,7 +216,13 @@ void CPUIDEmu::SetupHostHybridFlag() {
{0x41, 0xd07, 1, ProductNames::ARM_A57}, // A57
// Typically Little CPU cores
{0x61, 0x022, 0, ProductNames::ARM_Icestorm}, // Apple M1 Icestorm
{0x61, 0x038, 0, ProductNames::ARM_Blizzard_M2Max}, // Apple Blizzard (M2 Max)
{0x61, 0x034, 0, ProductNames::ARM_Blizzard_M2Pro}, // Apple Blizzard (M2 Pro)
{0x61, 0x032, 0, ProductNames::ARM_Blizzard_M2}, // Apple Blizzard (M2)
{0x61, 0x028, 0, ProductNames::ARM_Icestorm_M1Max}, // Apple Icestorm (M1 Max)
{0x61, 0x024, 0, ProductNames::ARM_Icestorm_M1Pro}, // Apple Icestorm (M1 Pro)
{0x61, 0x022, 0, ProductNames::ARM_Icestorm_M1}, // Apple Icestorm (M1)
{0x41, 0xd80, 0, ProductNames::ARM_A520}, // A520
{0x41, 0xd46, 0, ProductNames::ARM_A510}, // A510
{0x41, 0xd06, 0, ProductNames::ARM_A65}, // A65
@@ -598,8 +637,8 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_07h(uint32_t Leaf) const {
// Disable Enhanced REP MOVS when TSO is enabled.
// vcruntime140 memmove will use `rep movsb` in this case which completely destroys perf in Hades(appId 1145360)
// This is due to LRCPC performance on Cortex being abysmal.
// Only enable EnhancedREPMOVS if SoftwareTSO isn't required OR if MemcpySetTSO is not enabled.
const uint32_t SupportsEnhancedREPMOVS = CTX->SoftwareTSORequired() == false || MemcpySetTSOEnabled() == false;
// Only enable EnhancedREPMOVS if atomic memcpy tso emulation isn't enabled.
const uint32_t SupportsEnhancedREPMOVS = CTX->IsMemcpyAtomicTSOEnabled() == false;
const uint32_t SupportsVPCLMULQDQ = CTX->HostFeatures.SupportsPMULL_128Bit && SupportsAVX();
// Number of subfunctions
@@ -628,7 +667,7 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_07h(uint32_t Leaf) const {
(0 << 21) | // Reserved
(0 << 22) | // Reserved
(1 << 23) | // CLFLUSHOPT instruction
(CTX->HostFeatures.SupportsCLWB << 24) | // CLWB instruction
(1 << 24) | // CLWB instruction
(0 << 25) | // Intel processor trace
(0 << 26) | // Reserved
(0 << 27) | // Reserved
@@ -762,7 +801,7 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_15h(uint32_t Leaf) const {
uint32_t FrequencyHz = GetCycleCounterFrequency();
if (FrequencyHz) {
Res.eax = 1;
Res.ebx = CTX->Config.SmallTSCScale() ? FEXCore::Context::TSC_SCALE : 1;
Res.ebx = 1U << CTX->Config.TSCScale;
Res.ecx = FrequencyHz;
}
return Res;
@@ -1175,7 +1214,7 @@ FEXCore::CPUID::XCRResults CPUIDEmu::XCRFunction_0h() const {
CPUIDEmu::CPUIDEmu(const FEXCore::Context::ContextImpl* ctx)
: CTX {ctx} {
Cores = FEXCore::CPUInfo::CalculateNumberOfCPUs();
Cores = CTX->HostFeatures.CPUMIDRs.size();
// Setup some state tracking
SetupHostHybridFlag();
-2
View File
@@ -118,8 +118,6 @@ private:
bool Hybrid {};
uint32_t Cores {};
FEX_CONFIG_OPT(HideHypervisorBit, HIDEHYPERVISORBIT);
FEX_CONFIG_OPT(SmallTSCScale, SMALLTSCSCALE);
FEX_CONFIG_OPT(MemcpySetTSOEnabled, MEMCPYSETTSOENABLED);
// XFEATURE_ENABLED_MASK
// Mask that configures what features are enabled on the CPU.
+103 -75
View File
@@ -9,7 +9,6 @@ $end_info$
*/
#include <cstdint>
#include "Interface/Context/Context.h"
#include "Interface/Core/ArchHelpers//Arm64Emitter.h"
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/CPUBackend.h"
@@ -20,7 +19,6 @@ $end_info$
#include "Interface/Core/JIT/JITCore.h"
#include "Interface/Core/Dispatcher/Dispatcher.h"
#include "Interface/Core/X86Tables/X86Tables.h"
#include "Interface/HLE/Thunks/Thunks.h"
#include "Interface/IR/IR.h"
#include "Interface/IR/IREmitter.h"
#include "Interface/IR/Passes/RegisterAllocationPass.h"
@@ -29,16 +27,17 @@ $end_info$
#include "Interface/IR/RegisterAllocationData.h"
#include "Utils/Allocator.h"
#include "Utils/Allocator/HostAllocator.h"
#include "Utils/SpinWaitLock.h"
#include <FEXCore/Config/Config.h>
#include <FEXCore/Core/Context.h>
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/Core/SignalDelegator.h>
#include <FEXCore/Core/Thunks.h>
#include <FEXCore/Core/X86Enums.h>
#include <FEXCore/Debug/InternalThreadState.h>
#include <FEXCore/HLE/SyscallHandler.h>
#include <FEXCore/HLE/SourcecodeResolver.h>
#include <FEXCore/HLE/Linux/ThreadManagement.h>
#include <FEXCore/Utils/Allocator.h>
#include <FEXCore/Utils/Event.h>
#include <FEXCore/Utils/File.h>
@@ -75,12 +74,10 @@ $end_info$
#include <xxhash.h>
namespace FEXCore::Context {
ContextImpl::ContextImpl()
: CPUID {this}
ContextImpl::ContextImpl(const FEXCore::HostFeatures& Features)
: HostFeatures {Features}
, CPUID {this}
, IRCaptureCache {this} {
#ifdef BLOCKSTATS
BlockData = std::make_unique<FEXCore::BlockSamplingData>();
#endif
if (Config.CacheObjectCodeCompilation() != FEXCore::Config::ConfigObjectCodeHandler::CONFIG_NONE) {
CodeObjectCacheService = fextl::make_unique<FEXCore::CodeSerialize::CodeObjectSerializeService>(this);
}
@@ -94,8 +91,13 @@ ContextImpl::ContextImpl()
Symbols.InitFile();
}
if (FEXCore::GetCycleCounterFrequency() >= FEXCore::Context::TSC_SCALE_MAXIMUM) {
Config.SmallTSCScale = false;
uint64_t FrequencyCounter = FEXCore::GetCycleCounterFrequency();
if (FrequencyCounter && FrequencyCounter < FEXCore::Context::TSC_SCALE_MAXIMUM && Config.SmallTSCScale()) {
// Scale TSC until it is at the minimum required.
while (FrequencyCounter < FEXCore::Context::TSC_SCALE_MAXIMUM) {
FrequencyCounter <<= 1;
++Config.TSCScale;
}
}
// Track atomic TSO emulation configuration.
@@ -190,6 +192,9 @@ uint32_t ContextImpl::ReconstructCompactedEFLAGS(FEXCore::Core::InternalThreadSt
uint32_t ZF = (Packed_NZCV >> IR::OpDispatchBuilder::IndexNZCV(X86State::RFLAG_ZF_RAW_LOC)) & 1;
uint32_t SF = (Packed_NZCV >> IR::OpDispatchBuilder::IndexNZCV(X86State::RFLAG_SF_RAW_LOC)) & 1;
// CF is inverted in our representation, undo the invert here.
CF ^= 1;
// Pack in to EFLAGS
EFLAGS |= OF << X86State::RFLAG_OF_RAW_LOC;
EFLAGS |= CF << X86State::RFLAG_CF_RAW_LOC;
@@ -293,10 +298,10 @@ void ContextImpl::SetFlagsFromCompactedEFLAGS(FEXCore::Core::InternalThreadState
}
}
// Calculate packed NZCV
// Calculate packed NZCV. Note CF is inverted.
uint32_t Packed_NZCV {};
Packed_NZCV |= (EFLAGS & (1U << X86State::RFLAG_OF_RAW_LOC)) ? 1U << IR::OpDispatchBuilder::IndexNZCV(X86State::RFLAG_OF_RAW_LOC) : 0;
Packed_NZCV |= (EFLAGS & (1U << X86State::RFLAG_CF_RAW_LOC)) ? 1U << IR::OpDispatchBuilder::IndexNZCV(X86State::RFLAG_CF_RAW_LOC) : 0;
Packed_NZCV |= (EFLAGS & (1U << X86State::RFLAG_CF_RAW_LOC)) ? 0 : 1U << IR::OpDispatchBuilder::IndexNZCV(X86State::RFLAG_CF_RAW_LOC);
Packed_NZCV |= (EFLAGS & (1U << X86State::RFLAG_ZF_RAW_LOC)) ? 1U << IR::OpDispatchBuilder::IndexNZCV(X86State::RFLAG_ZF_RAW_LOC) : 0;
Packed_NZCV |= (EFLAGS & (1U << X86State::RFLAG_SF_RAW_LOC)) ? 1U << IR::OpDispatchBuilder::IndexNZCV(X86State::RFLAG_SF_RAW_LOC) : 0;
memcpy(&Frame->State.flags[X86State::RFLAG_NZCV_LOC], &Packed_NZCV, sizeof(Packed_NZCV));
@@ -309,20 +314,10 @@ void ContextImpl::SetFlagsFromCompactedEFLAGS(FEXCore::Core::InternalThreadState
bool ContextImpl::InitCore() {
// Initialize the CPU core signal handlers & DispatcherConfig
switch (Config.Core) {
case FEXCore::Config::CONFIG_IRJIT: BackendFeatures = FEXCore::CPU::GetArm64JITBackendFeatures(); break;
case FEXCore::Config::CONFIG_CUSTOM:
// Do nothing
break;
default: LogMan::Msg::EFmt("Unknown core configuration"); return false;
}
Dispatcher = FEXCore::CPU::Dispatcher::Create(this);
// Set up the SignalDelegator config since core is initialized.
FEXCore::SignalDelegator::SignalDelegatorConfig SignalConfig {
.SupportsAVX = HostFeatures.SupportsAVX,
.DispatcherBegin = Dispatcher->Start,
.DispatcherEnd = Dispatcher->End,
@@ -351,9 +346,8 @@ bool ContextImpl::InitCore() {
SignalDelegation->SetConfig(SignalConfig);
#ifndef _WIN32
ThunkHandler = FEXCore::ThunkHandler::Create();
#else
// WIN32 always needs the interrupt fault check to be enabled.
#elif !defined(_M_ARM64EC)
// WOW64 always needs the interrupt fault check to be enabled.
Config.NeedsPendingInterruptFaultCheck = true;
#endif
@@ -373,14 +367,15 @@ void ContextImpl::HandleCallback(FEXCore::Core::InternalThreadState* Thread, uin
FEXCore::Context::ExitReason ContextImpl::RunUntilExit(FEXCore::Core::InternalThreadState* Thread) {
ExecutionThread(Thread);
while (true) {
auto reason = Thread->ExitReason;
// Don't return if a custom exit handling the exit
if (!CustomExitHandler || reason == ExitReason::EXIT_SHUTDOWN) {
return reason;
}
CoreShuttingDown.store(true);
if (CustomExitHandler) {
CustomExitHandler(Thread, FEXCore::Context::ExitReason::EXIT_SHUTDOWN);
return Thread->ExitReason;
}
return FEXCore::Context::ExitReason::EXIT_SHUTDOWN;
}
void ContextImpl::ExecuteThread(FEXCore::Core::InternalThreadState* Thread) {
@@ -390,12 +385,6 @@ void ContextImpl::ExecuteThread(FEXCore::Core::InternalThreadState* Thread) {
void ContextImpl::InitializeThreadTLSData(FEXCore::Core::InternalThreadState* Thread) {
// Let's do some initial bookkeeping here
Thread->ThreadManager.TID = FHU::Syscalls::gettid();
Thread->ThreadManager.PID = ::getpid();
if (ThunkHandler) {
ThunkHandler->RegisterTLSState(Thread);
}
#ifndef _WIN32
Alloc::OSAllocator::RegisterTLSData(Thread);
#endif
@@ -421,20 +410,14 @@ void ContextImpl::InitializeCompiler(FEXCore::Core::InternalThreadState* Thread)
Thread->PassManager->RegisterSyscallHandler(SyscallHandler);
// Create CPU backend
switch (Config.Core) {
case FEXCore::Config::CONFIG_IRJIT:
Thread->PassManager->InsertRegisterAllocationPass();
Thread->CPUBackend = FEXCore::CPU::CreateArm64JITCore(this, Thread);
break;
case FEXCore::Config::CONFIG_CUSTOM: Thread->CPUBackend = CustomCPUFactory(this, Thread); break;
default: ERROR_AND_DIE_FMT("Unknown core configuration"); break;
}
Thread->PassManager->InsertRegisterAllocationPass();
Thread->CPUBackend = FEXCore::CPU::CreateArm64JITCore(this, Thread);
Thread->PassManager->Finalize();
}
FEXCore::Core::InternalThreadState*
ContextImpl::CreateThread(uint64_t InitialRIP, uint64_t StackPointer, FEXCore::Core::CPUState* NewThreadState, uint64_t ParentTID) {
ContextImpl::CreateThread(uint64_t InitialRIP, uint64_t StackPointer, const FEXCore::Core::CPUState* NewThreadState, uint64_t ParentTID) {
FEXCore::Core::InternalThreadState* Thread = new FEXCore::Core::InternalThreadState {};
Thread->CurrentFrame->State.gregs[X86State::REG_RSP] = StackPointer;
@@ -446,7 +429,6 @@ ContextImpl::CreateThread(uint64_t InitialRIP, uint64_t StackPointer, FEXCore::C
}
// Set up the thread manager state
Thread->ThreadManager.parent_tid = ParentTID;
Thread->CurrentFrame->Thread = Thread;
InitializeCompiler(Thread);
@@ -479,8 +461,14 @@ void ContextImpl::UnlockAfterFork(FEXCore::Core::InternalThreadState* LiveThread
if (Child) {
CodeInvalidationMutex.StealAndDropActiveLocks();
if (Config.StrictInProcessSplitLocks) {
StrictSplitLockMutex = 0;
}
} else {
CodeInvalidationMutex.unlock();
if (Config.StrictInProcessSplitLocks) {
FEXCore::Utils::SpinWaitLock::unlock(&StrictSplitLockMutex);
}
return;
}
}
@@ -488,6 +476,9 @@ void ContextImpl::UnlockAfterFork(FEXCore::Core::InternalThreadState* LiveThread
void ContextImpl::LockBeforeFork(FEXCore::Core::InternalThreadState* Thread) {
CodeInvalidationMutex.lock();
Allocator::LockBeforeFork(Thread);
if (Config.StrictInProcessSplitLocks) {
FEXCore::Utils::SpinWaitLock::lock(&StrictSplitLockMutex);
}
}
#endif
@@ -606,6 +597,11 @@ ContextImpl::GenerateIR(FEXCore::Core::InternalThreadState* Thread, uint64_t Gue
uint64_t InstsInBlock = Block.NumInstructions;
if (InstsInBlock == 0) {
// Special case for an empty instruction block.
Thread->OpDispatcher->ExitFunction(Thread->OpDispatcher->_EntrypointOffset(IR::SizeToOpSize(GPRSize), Block.Entry - GuestRIP));
}
for (size_t i = 0; i < InstsInBlock; ++i) {
const FEXCore::X86Tables::X86InstInfo* TableInfo {nullptr};
const FEXCore::X86Tables::DecodedInst* DecodedInfo {nullptr};
@@ -614,6 +610,20 @@ ContextImpl::GenerateIR(FEXCore::Core::InternalThreadState* Thread, uint64_t Gue
DecodedInfo = &Block.DecodedInstructions[i];
bool IsLocked = DecodedInfo->Flags & FEXCore::X86Tables::DecodeFlags::FLAG_LOCK;
// Do a partial register cache flush before every instruction. This
// prevents cross-instruction static register caching, while allowing
// context load/stores to be optimized within a block. Theoretically,
// this flush is not required for correctness, all mandatory flushes are
// included in instruction-specific handlers. Instead, this is a blunt
// heuristic to make the register cache less aggressive, as the current
// RA generates bad code in common cases with tied registers otherwise.
//
// However, it makes our exception handling behaviour more predictable.
// It is potentially correctness bearing in that sense, but that is a
// side effect here and (if that behaviour is required) we should handle
// that more explicitly later.
Thread->OpDispatcher->FlushRegisterCache(true);
if (ExtendedDebugInfo || Thread->OpDispatcher->CanHaveSideEffects(TableInfo, DecodedInfo)) {
Thread->OpDispatcher->_GuestOpcode(Block.Entry + BlockInstructionsLength - GuestRIP);
}
@@ -804,14 +814,6 @@ uintptr_t ContextImpl::CompileBlock(FEXCore::Core::CpuStateFrame* Frame, uint64_
FEXCORE_PROFILE_SCOPED("CompileBlock");
auto Thread = Frame->Thread;
#ifdef _M_ARM_64EC
// If the target PC is EC code, mark it in the L2 and return straight to the dispatcher
// so it can handle the call/return.
if (Thread->LookupCache->CheckPageEC(GuestRIP)) {
return GuestRIP;
}
#endif
// Invalidate might take a unique lock on this, to guarantee that during invalidation no code gets compiled
auto lk = GuardSignalDeferringSection<std::shared_lock>(CodeInvalidationMutex, Thread);
@@ -918,15 +920,6 @@ void ContextImpl::ExecutionThread(FEXCore::Core::InternalThreadState* Thread) {
// If it is the parent thread that died then just leave
FEX_TODO("This doesn't make sense when the parent thread doesn't outlive its children");
if (Thread->ThreadManager.parent_tid == 0) {
CoreShuttingDown.store(true);
Thread->ExitReason = FEXCore::Context::ExitReason::EXIT_SHUTDOWN;
if (CustomExitHandler) {
CustomExitHandler(Thread->ThreadManager.TID, Thread->ExitReason);
}
}
#ifndef _WIN32
Alloc::OSAllocator::UninstallTLSData(Thread);
#endif
@@ -985,7 +978,8 @@ void ContextImpl::ThreadRemoveCodeEntry(FEXCore::Core::InternalThreadState* Thre
Thread->LookupCache->Erase(Thread->CurrentFrame, GuestRIP);
}
CustomIRResult ContextImpl::AddCustomIREntrypoint(uintptr_t Entrypoint, CustomIREntrypointHandler Handler, void* Creator, void* Data) {
std::optional<CustomIRResult>
ContextImpl::AddCustomIREntrypoint(uintptr_t Entrypoint, CustomIREntrypointHandler Handler, void* Creator, void* Data) {
LOGMAN_THROW_A_FMT(Config.Is64BitMode || !(Entrypoint >> 32), "64-bit Entrypoint in 32-bit mode {:x}", Entrypoint);
std::unique_lock lk(CustomIRMutex);
@@ -995,10 +989,50 @@ CustomIRResult ContextImpl::AddCustomIREntrypoint(uintptr_t Entrypoint, CustomIR
if (!InsertedIterator.second) {
const auto& [fn, Creator, Data] = InsertedIterator.first->second;
return CustomIRResult(std::move(lk), Creator, Data);
} else {
lk.unlock();
return CustomIRResult(std::move(lk), 0, 0);
return CustomIRResult(Creator, Data);
}
return std::nullopt;
}
void ContextImpl::AddThunkTrampolineIRHandler(uintptr_t Entrypoint, uintptr_t GuestThunkEntrypoint) {
LOGMAN_THROW_AA_FMT(Entrypoint, "Tried to link null pointer address to guest function");
LOGMAN_THROW_AA_FMT(GuestThunkEntrypoint, "Tried to link address to null pointer guest function");
if (!Config.Is64BitMode) {
LOGMAN_THROW_AA_FMT((Entrypoint >> 32) == 0, "Tried to link 64-bit address in 32-bit mode");
LOGMAN_THROW_AA_FMT((GuestThunkEntrypoint >> 32) == 0, "Tried to link 64-bit address in 32-bit mode");
}
LogMan::Msg::DFmt("Thunks: Adding guest trampoline from address {:#x} to guest function {:#x}", Entrypoint, GuestThunkEntrypoint);
auto Result = AddCustomIREntrypoint(
Entrypoint,
[this, GuestThunkEntrypoint](uintptr_t Entrypoint, FEXCore::IR::IREmitter* emit) {
auto IRHeader = emit->_IRHeader(emit->Invalid(), Entrypoint, 0, 0);
auto Block = emit->CreateCodeNode();
IRHeader.first->Blocks = emit->WrapNode(Block);
emit->SetCurrentCodeBlock(Block);
const uint8_t GPRSize = GetGPRSize();
if (GPRSize == 8) {
emit->_StoreRegister(emit->_Constant(Entrypoint), X86State::REG_R11, IR::GPRClass, GPRSize);
} else {
emit->_StoreContext(GPRSize, IR::FPRClass, emit->_VCastFromGPR(8, 8, emit->_Constant(Entrypoint)), offsetof(Core::CPUState, mm[0][0]));
}
emit->_ExitFunction(emit->_Constant(GuestThunkEntrypoint));
},
ThunkHandler, (void*)GuestThunkEntrypoint);
if (Result.has_value()) {
if (Result->Creator != ThunkHandler) {
ERROR_AND_DIE_FMT("Input address for AddThunkTrampoline is already linked by another module");
}
if (Result->Data != (void*)GuestThunkEntrypoint) {
// NOTE: This may happen in Vulkan thunks if the Vulkan driver resolves two different symbols
// to the same function (e.g. vkGetPhysicalDeviceFeatures2/vkGetPhysicalDeviceFeatures2KHR)
LogMan::Msg::EFmt("Input address for AddThunkTrampoline is already linked elsewhere");
}
}
}
@@ -1021,12 +1055,6 @@ void ContextImpl::UnloadAOTIRCacheEntry(IR::AOTIRCacheEntry* Entry) {
IRCaptureCache.UnloadAOTIRCacheEntry(Entry);
}
void ContextImpl::AppendThunkDefinitions(const fextl::vector<FEXCore::IR::ThunkDefinition>& Definitions) {
if (ThunkHandler) {
ThunkHandler->AppendThunkDefinitions(Definitions);
}
}
void ContextImpl::ConfigureAOTGen(FEXCore::Core::InternalThreadState* Thread, fextl::set<uint64_t>* ExternalBranches, uint64_t SectionMaxAddress) {
Thread->FrontendDecoder->SetExternalBranches(ExternalBranches);
Thread->FrontendDecoder->SetSectionMaxAddress(SectionMaxAddress);
@@ -62,14 +62,6 @@ void Dispatcher::EmitDispatcher() {
ARMEmitter::ForwardLabel l_CTX;
ARMEmitter::SingleUseForwardLabel l_Sleep;
#ifdef _M_ARM_64EC
// These structures are not included in the standard Windows headers, define them here
static constexpr size_t TEBCPUAreaOffset = 0x1788;
static constexpr size_t CPUAreaInSyscallCallbackOffset = 0x1;
static constexpr size_t CPUAreaEmulatorStackLimitOffset = 0x8;
static constexpr size_t CPUAreaEmulatorDataOffset = 0x30;
ARMEmitter::SingleUseForwardLabel ExitEC;
#endif
ARMEmitter::SingleUseForwardLabel l_CompileBlock;
// Push all the register we need to save
@@ -94,7 +86,7 @@ void Dispatcher::EmitDispatcher() {
b(&LoopTop);
AbsoluteLoopTopAddressEnterECFillSRA = GetCursorAddress<uint64_t>();
ldr(STATE, EC_ENTRY_CPUAREA_REG, CPUAreaEmulatorDataOffset);
ldr(STATE, EC_ENTRY_CPUAREA_REG, CPU_AREA_EMULATOR_DATA_OFFSET);
FillStaticRegs();
// Enter JIT
@@ -102,17 +94,15 @@ void Dispatcher::EmitDispatcher() {
AbsoluteLoopTopAddressEnterEC = GetCursorAddress<uint64_t>();
// Load ThreadState and write the target PC there
ldr(STATE, EC_ENTRY_CPUAREA_REG, CPUAreaEmulatorDataOffset);
ldr(STATE, EC_ENTRY_CPUAREA_REG, CPU_AREA_EMULATOR_DATA_OFFSET);
str(EC_CALL_CHECKER_PC_REG, STATE_PTR(CpuStateFrame, State.rip));
// Swap stacks to the emulator stack
ldr(TMP1, EC_ENTRY_CPUAREA_REG, CPUAreaEmulatorStackLimitOffset);
ldr(TMP1, EC_ENTRY_CPUAREA_REG, CPU_AREA_EMULATOR_STACK_BASE_OFFSET);
add(ARMEmitter::Size::i64Bit, StaticRegisters[X86State::REG_RSP], ARMEmitter::Reg::rsp, 0);
add(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::rsp, TMP1, 0);
if (EmitterCTX->HostFeatures.SupportsSVE128) {
ptrue(ARMEmitter::SubRegSize::i8Bit, PRED_TMP_16B, ARMEmitter::PredicatePattern::SVE_VL16);
}
FillSpecialRegs(TMP1, TMP2, false, true);
// Enter JIT
#endif
@@ -168,10 +158,6 @@ void Dispatcher::EmitDispatcher() {
// If page pointer is zero then we have no block
cbz(ARMEmitter::Size::i64Bit, TMP1, &NoBlock);
#ifdef _M_ARM_64EC
// The LSB of an L2 page entry indicates if this page contains EC code
tbnz(TMP1, 0, &ExitEC);
#endif
// Steal the page offset
and_(ARMEmitter::Size::i64Bit, TMP2, TMP4, 0x0FFF);
@@ -198,23 +184,13 @@ void Dispatcher::EmitDispatcher() {
and_(ARMEmitter::Size::i64Bit, TMP2, RipReg.R(), LookupCache::L1_ENTRIES_MASK);
add(TMP1, TMP1, TMP2, ARMEmitter::ShiftType::LSL, 4);
stp<ARMEmitter::IndexType::OFFSET>(TMP4, TMP3, TMP1);
stp<ARMEmitter::IndexType::OFFSET>(TMP4, RipReg, TMP1);
// Jump to the block
br(TMP4);
}
}
#ifdef _M_ARM_64EC
{
Bind(&ExitEC);
add(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::rsp, StaticRegisters[X86State::REG_RSP], 0);
mov(EC_CALL_CHECKER_PC_REG, RipReg);
ldr(TMP2, STATE_PTR(CpuStateFrame, Pointers.Common.ExitFunctionEC));
br(TMP2);
}
#endif
{
ThreadStopHandlerAddressSpillSRA = GetCursorAddress<uint64_t>();
SpillStaticRegs(TMP1);
@@ -232,14 +208,16 @@ void Dispatcher::EmitDispatcher() {
ExitFunctionLinkerAddress = GetCursorAddress<uint64_t>();
SpillStaticRegs(TMP1);
#ifndef _WIN32
ldr(ARMEmitter::XReg::x0, STATE, offsetof(FEXCore::Core::CPUState, DeferredSignalRefCount));
add(ARMEmitter::Size::i64Bit, ARMEmitter::XReg::x0, ARMEmitter::XReg::x0, 1);
str(ARMEmitter::XReg::x0, STATE, offsetof(FEXCore::Core::CPUState, DeferredSignalRefCount));
#endif
#ifdef _M_ARM_64EC
ldr(ARMEmitter::XReg::x0, ARMEmitter::XReg::x18, TEBCPUAreaOffset);
ldr(ARMEmitter::XReg::x0, ARMEmitter::XReg::x18, TEB_CPU_AREA_OFFSET);
LoadConstant(ARMEmitter::Size::i32Bit, ARMEmitter::Reg::r1, 1);
strb(ARMEmitter::WReg::w1, ARMEmitter::XReg::x0, CPUAreaInSyscallCallbackOffset);
strb(ARMEmitter::WReg::w1, ARMEmitter::XReg::x0, CPU_AREA_IN_SYSCALL_CALLBACK_OFFSET);
#endif
mov(ARMEmitter::XReg::x0, STATE);
@@ -259,10 +237,11 @@ void Dispatcher::EmitDispatcher() {
FillStaticRegs();
#ifdef _M_ARM_64EC
ldr(TMP2, ARMEmitter::XReg::x18, TEBCPUAreaOffset);
strb(ARMEmitter::WReg::zr, TMP2, CPUAreaInSyscallCallbackOffset);
ldr(TMP2, ARMEmitter::XReg::x18, TEB_CPU_AREA_OFFSET);
strb(ARMEmitter::WReg::zr, TMP2, CPU_AREA_IN_SYSCALL_CALLBACK_OFFSET);
#endif
#ifndef _WIN32
ldr(TMP2, STATE, offsetof(FEXCore::Core::CPUState, DeferredSignalRefCount));
sub(ARMEmitter::Size::i64Bit, TMP2, TMP2, 1);
str(TMP2, STATE, offsetof(FEXCore::Core::CPUState, DeferredSignalRefCount));
@@ -270,6 +249,7 @@ void Dispatcher::EmitDispatcher() {
// Trigger segfault if any deferred signals are pending
strb(ARMEmitter::XReg::zr, STATE,
offsetof(FEXCore::Core::InternalThreadState, InterruptFaultPage) - offsetof(FEXCore::Core::InternalThreadState, BaseFrameState));
#endif
br(TMP1);
}
@@ -278,20 +258,43 @@ void Dispatcher::EmitDispatcher() {
{
Bind(&NoBlock);
#ifdef _M_ARM_64EC
// Check the EC code bitmap incase we need to exit the JIT to call into native code.
ARMEmitter::SingleUseForwardLabel l_NotECCode;
ldr(TMP1, ARMEmitter::XReg::x18, TEB_PEB_OFFSET);
ldr(TMP1, TMP1, PEB_EC_CODE_BITMAP_OFFSET);
lsr(ARMEmitter::Size::i64Bit, TMP2, RipReg, 15);
and_(ARMEmitter::Size::i64Bit, TMP2, TMP2, 0x1fffffffffff8);
ldr(TMP1, TMP1, TMP2, ARMEmitter::ExtendedType::LSL_64, 0);
lsr(ARMEmitter::Size::i64Bit, TMP2, RipReg, 12);
lsrv(ARMEmitter::Size::i64Bit, TMP1, TMP1, TMP2);
tbz(TMP1, 0, &l_NotECCode);
add(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::rsp, StaticRegisters[X86State::REG_RSP], 0);
mov(EC_CALL_CHECKER_PC_REG, RipReg);
ldr(TMP2, STATE_PTR(CpuStateFrame, Pointers.Common.ExitFunctionEC));
br(TMP2);
Bind(&l_NotECCode);
#endif
SpillStaticRegs(TMP1);
if (!TMP_ABIARGS) {
mov(ARMEmitter::XReg::x2, TMP3);
mov(ARMEmitter::XReg::x2, RipReg);
}
#ifndef _WIN32
ldr(ARMEmitter::XReg::x0, STATE, offsetof(FEXCore::Core::CPUState, DeferredSignalRefCount));
add(ARMEmitter::Size::i64Bit, ARMEmitter::XReg::x0, ARMEmitter::XReg::x0, 1);
str(ARMEmitter::XReg::x0, STATE, offsetof(FEXCore::Core::CPUState, DeferredSignalRefCount));
#endif
#ifdef _M_ARM_64EC
ldr(ARMEmitter::XReg::x0, ARMEmitter::XReg::x18, TEBCPUAreaOffset);
ldr(ARMEmitter::XReg::x0, ARMEmitter::XReg::x18, TEB_CPU_AREA_OFFSET);
LoadConstant(ARMEmitter::Size::i32Bit, ARMEmitter::Reg::r1, 1);
strb(ARMEmitter::WReg::w1, ARMEmitter::XReg::x0, CPUAreaInSyscallCallbackOffset);
strb(ARMEmitter::WReg::w1, ARMEmitter::XReg::x0, CPU_AREA_IN_SYSCALL_CALLBACK_OFFSET);
#endif
ldr(ARMEmitter::XReg::x0, &l_CTX);
@@ -309,10 +312,11 @@ void Dispatcher::EmitDispatcher() {
FillStaticRegs();
#ifdef _M_ARM_64EC
ldr(TMP1, ARMEmitter::XReg::x18, TEBCPUAreaOffset);
strb(ARMEmitter::WReg::zr, TMP1, CPUAreaInSyscallCallbackOffset);
ldr(TMP1, ARMEmitter::XReg::x18, TEB_CPU_AREA_OFFSET);
strb(ARMEmitter::WReg::zr, TMP1, CPU_AREA_IN_SYSCALL_CALLBACK_OFFSET);
#endif
#ifndef _WIN32
ldr(TMP1, STATE, offsetof(FEXCore::Core::CPUState, DeferredSignalRefCount));
sub(ARMEmitter::Size::i64Bit, TMP1, TMP1, 1);
str(TMP1, STATE, offsetof(FEXCore::Core::CPUState, DeferredSignalRefCount));
@@ -320,6 +324,7 @@ void Dispatcher::EmitDispatcher() {
// Trigger segfault if any deferred signals are pending
strb(ARMEmitter::XReg::zr, STATE,
offsetof(FEXCore::Core::InternalThreadState, InterruptFaultPage) - offsetof(FEXCore::Core::InternalThreadState, BaseFrameState));
#endif
b(&LoopTop);
}
+38 -18
View File
@@ -274,12 +274,10 @@ bool Decoder::NormalOp(const FEXCore::X86Tables::X86InstInfo* Info, uint16_t Op,
DecodeInst->TableInfo = Info;
if (Info->Type == FEXCore::X86Tables::TYPE_UNKNOWN) {
LogMan::Msg::DFmt("Unknown instruction: {} 0x{:04x} 0x{:x}", Info->Name ?: "UND", Op, DecodeInst->PC);
return false;
}
if (Info->Type == FEXCore::X86Tables::TYPE_INVALID) {
LogMan::Msg::DFmt("Invalid or Unknown instruction: {} 0x{:04x} 0x{:x}", Info->Name ?: "UND", Op, DecodeInst->PC);
return false;
}
@@ -557,12 +555,10 @@ bool Decoder::NormalOpHeader(const FEXCore::X86Tables::X86InstInfo* Info, uint16
DecodeInst->TableInfo = Info;
if (Info->Type == FEXCore::X86Tables::TYPE_UNKNOWN) {
LogMan::Msg::DFmt("Unknown instruction: {} 0x{:04x} 0x{:x}", Info->Name ?: "UND", Op, DecodeInst->PC);
return false;
}
if (Info->Type == FEXCore::X86Tables::TYPE_INVALID) {
LogMan::Msg::DFmt("Invalid or Unknown instruction: {} 0x{:04x} 0x{:x}", Info->Name ?: "UND", Op, DecodeInst->PC);
return false;
}
@@ -632,7 +628,6 @@ bool Decoder::NormalOpHeader(const FEXCore::X86Tables::X86InstInfo* Info, uint16
uint16_t X87Op = ((Op - 0xD8) << 8) | ModRMByte;
return NormalOp(&X87Ops[X87Op], X87Op);
} else if (Info->Type == FEXCore::X86Tables::TYPE_VEX_TABLE_PREFIX) {
FEXCORE_TELEMETRY_SET(VEXOpTelem, 1);
uint16_t map_select = 1;
uint16_t pp = 0;
const uint8_t Byte1 = ReadByte();
@@ -678,7 +673,6 @@ bool Decoder::NormalOpHeader(const FEXCore::X86Tables::X86InstInfo* Info, uint16
FEXCore::X86Tables::X86InstInfo* LocalInfo = &VEXTableOps[Op];
if (LocalInfo->Type >= FEXCore::X86Tables::TYPE_VEX_GROUP_12 && LocalInfo->Type <= FEXCore::X86Tables::TYPE_VEX_GROUP_17) {
FEXCORE_TELEMETRY_SET(VEXOpTelem, 1);
// We have ModRM
uint8_t ModRMByte = ReadByte();
DecodeInst->ModRM = ModRMByte;
@@ -696,12 +690,8 @@ bool Decoder::NormalOpHeader(const FEXCore::X86Tables::X86InstInfo* Info, uint16
}
} else if (Info->Type == FEXCore::X86Tables::TYPE_GROUP_EVEX) {
FEXCORE_TELEMETRY_SET(EVEXOpTelem, 1);
/* uint8_t P1 = */ ReadByte();
/* uint8_t P2 = */ ReadByte();
/* uint8_t P3 = */ ReadByte();
uint8_t EVEXOp = ReadByte();
return NormalOp(&EVEXTableOps[EVEXOp], EVEXOp);
// EVEX unsupported
return false;
}
LOGMAN_MSG_A_FMT("Invalid instruction decoding type");
@@ -969,8 +959,14 @@ void Decoder::BranchTargetInMultiblockRange() {
TargetRIP &= 0xFFFFFFFFU;
}
// If the target RIP is within the symbol ranges then we are golden
if (TargetRIP >= SymbolMinAddress && TargetRIP < SymbolMaxAddress) {
// If the target RIP is x86 code within the symbol ranges then we are golden
bool ValidMultiblockMember = TargetRIP >= SymbolMinAddress && TargetRIP < SymbolMaxAddress;
#ifdef _M_ARM_64EC
ValidMultiblockMember = ValidMultiblockMember && !RtlIsEcCode(TargetRIP);
#endif
if (ValidMultiblockMember) {
// Update our conditional branch ranges before we return
if (Conditional) {
MaxCondBranchForward = std::max(MaxCondBranchForward, TargetRIP);
@@ -979,12 +975,12 @@ void Decoder::BranchTargetInMultiblockRange() {
// If we are conditional then a target can be the instruction past the conditional instruction
uint64_t FallthroughRIP = DecodeInst->PC + DecodeInst->InstSize;
if (!HasBlocks.contains(FallthroughRIP)) {
BlocksToDecode.insert(FallthroughRIP);
CurrentBlockTargets.insert(FallthroughRIP);
}
}
if (!HasBlocks.contains(TargetRIP)) {
BlocksToDecode.insert(TargetRIP);
CurrentBlockTargets.insert(TargetRIP);
}
} else {
if (ExternalBranches) {
@@ -1081,6 +1077,8 @@ void Decoder::DecodeInstructionsAtEntry(const uint8_t* _InstStream, uint64_t PC,
MaxInst = CTX->Config.MaxInstPerBlock;
}
bool EntryBlock {true};
while (!BlocksToDecode.empty()) {
auto BlockDecodeIt = BlocksToDecode.begin();
uint64_t RIPToDecode = *BlockDecodeIt;
@@ -1117,7 +1115,6 @@ void Decoder::DecodeInstructionsAtEntry(const uint8_t* _InstStream, uint64_t PC,
bool ErrorDuringDecoding = !DecodeInstruction(RIPToDecode + PCOffset);
if (ErrorDuringDecoding) [[unlikely]] {
LogMan::Msg::DFmt("Couldn't Decode something at 0x{:x}, Started at 0x{:x}", RIPToDecode + PCOffset, PC);
// Put an invalid instruction in the stream so the core can raise SIGILL if hit
CurrentBlockDecoding.HasInvalidInstruction = true;
// Error while decoding instruction. We don't know the table or instruction size
@@ -1125,6 +1122,14 @@ void Decoder::DecodeInstructionsAtEntry(const uint8_t* _InstStream, uint64_t PC,
DecodeInst->InstSize = 0;
}
if (!ErrorDuringDecoding) {
// If there wasn't an error during decoding but we have no dispatcher for the instruction then claim invalid instruction.
auto TableInfo = DecodedBuffer[BlockStartOffset + BlockNumberOfInstructions].TableInfo;
if (!TableInfo || !TableInfo->OpcodeDispatcher) {
CurrentBlockDecoding.HasInvalidInstruction = true;
}
}
DecodedMinAddress = std::min(DecodedMinAddress, RIPToDecode + PCOffset);
DecodedMaxAddress = std::max(DecodedMaxAddress, RIPToDecode + PCOffset + DecodeInst->InstSize);
++TotalInstructions;
@@ -1132,7 +1137,17 @@ void Decoder::DecodeInstructionsAtEntry(const uint8_t* _InstStream, uint64_t PC,
++DecodedSize;
// Can not continue this block at all on invalid instruction
if (CurrentBlockDecoding.HasInvalidInstruction) {
if (CurrentBlockDecoding.HasInvalidInstruction) [[unlikely]] {
if (!EntryBlock) {
// In multiblock configurations, we can early terminate any non-entrypoint blocks with the expectation that this won't get hit.
// Improves compile-times.
// Just need to undo additions that this block decoding has caused.
TotalInstructions -= CurrentBlockDecoding.NumInstructions;
DecodedSize = BlockStartOffset;
BlockNumberOfInstructions = 0;
InstStream -= PCOffset;
CurrentBlockTargets.clear();
}
break;
}
@@ -1162,6 +1177,9 @@ void Decoder::DecodeInstructionsAtEntry(const uint8_t* _InstStream, uint64_t PC,
InstStream += DecodeInst->InstSize;
}
BlocksToDecode.merge(CurrentBlockTargets);
CurrentBlockTargets.clear();
BlocksToDecode.erase(BlockDecodeIt);
HasBlocks.emplace(RIPToDecode);
@@ -1169,6 +1187,8 @@ void Decoder::DecodeInstructionsAtEntry(const uint8_t* _InstStream, uint64_t PC,
CurrentBlockDecoding.NumInstructions = BlockNumberOfInstructions;
CurrentBlockDecoding.DecodedInstructions = &DecodedBuffer[BlockStartOffset];
BlockInfo.TotalInstructionCount += BlockNumberOfInstructions;
EntryBlock = false;
}
for (auto CodePage : CodePages) {
+1 -1
View File
@@ -104,6 +104,7 @@ private:
uint64_t SectionMaxAddress {~0ULL};
DecodedBlockInformation BlockInfo;
fextl::set<uint64_t> CurrentBlockTargets;
fextl::set<uint64_t> BlocksToDecode;
fextl::set<uint64_t> HasBlocks;
fextl::set<uint64_t>* ExternalBranches {nullptr};
@@ -120,7 +121,6 @@ private:
const uint8_t* AdjustAddrForSpecialRegion(const uint8_t* _InstStream, uint64_t EntryPoint, uint64_t RIP);
FEXCORE_TELEMETRY_INIT(VEXOpTelem, TYPE_USES_VEX_OPS);
FEXCORE_TELEMETRY_INIT(EVEXOpTelem, TYPE_USES_EVEX_OPS);
};
} // namespace FEXCore::Frontend
@@ -1,293 +0,0 @@
// SPDX-License-Identifier: MIT
#include "Interface/Core/CPUID.h"
#include <FEXCore/Core/HostFeatures.h>
#include "aarch64/assembler-aarch64.h"
#include "aarch64/cpu-aarch64.h"
#include "aarch64/disasm-aarch64.h"
#include "aarch64/assembler-aarch64.h"
#ifdef _M_X86_64
#define XBYAK64
#define XBYAK_NO_EXCEPTION
#include <FEXCore/fextl/list.h>
#include <FEXCore/fextl/unordered_map.h>
#include <FEXCore/fextl/unordered_set.h>
#include <xbyak/xbyak.h>
#include <xbyak/xbyak_util.h>
#endif
namespace FEXCore {
// Data Zero Prohibited flag
// 0b0 = ZVA/GVA/GZVA permitted
// 0b1 = ZVA/GVA/GZVA prohibited
[[maybe_unused]] constexpr uint32_t DCZID_DZP_MASK = 0b1'0000;
// Log2 of the blocksize in 32-bit words
[[maybe_unused]] constexpr uint32_t DCZID_BS_MASK = 0b0'1111;
#ifdef _M_ARM_64
[[maybe_unused]]
static uint32_t GetDCZID() {
uint64_t Result {};
__asm("mrs %[Res], DCZID_EL0" : [Res] "=r"(Result));
return Result;
}
static uint32_t GetFPCR() {
uint64_t Result {};
__asm("mrs %[Res], FPCR" : [Res] "=r"(Result));
return Result;
}
static void SetFPCR(uint64_t Value) {
__asm("msr FPCR, %[Value]" ::[Value] "r"(Value));
}
static uint32_t GetMIDR() {
uint64_t Result {};
__asm("mrs %[Res], MIDR_EL1" : [Res] "=r"(Result));
return Result;
}
#else
static uint32_t GetDCZID() {
// Return unsupported
return DCZID_DZP_MASK;
}
#endif
static void OverrideFeatures(HostFeatures* Features, uint64_t ForceSVEWidth) {
// Override features if the user has specifically called for it.
FEX_CONFIG_OPT(HostFeatures, HOSTFEATURES);
if (!HostFeatures()) {
// Early exit if no features are overriden.
return;
}
#define ENABLE_DISABLE_OPTION(FeatureName, name, enum_name) \
do { \
const bool Disable##name = (HostFeatures() & FEXCore::Config::HostFeatures::DISABLE##enum_name) != 0; \
const bool Enable##name = (HostFeatures() & FEXCore::Config::HostFeatures::ENABLE##enum_name) != 0; \
LogMan::Throw::AFmt(!(Disable##name && Enable##name), "Disabling and Enabling CPU feature (" #name ") is mutually exclusive"); \
const bool AlreadyEnabled = Features->FeatureName; \
const bool Result = (AlreadyEnabled | Enable##name) & !Disable##name; \
Features->FeatureName = Result; \
} while (0)
#define GET_SINGLE_OPTION(name, enum_name) \
const bool Disable##name = (HostFeatures() & FEXCore::Config::HostFeatures::DISABLE##enum_name) != 0; \
const bool Enable##name = (HostFeatures() & FEXCore::Config::HostFeatures::ENABLE##enum_name) != 0; \
LogMan::Throw::AFmt(!(Disable##name && Enable##name), "Disabling and Enabling CPU feature (" #name ") is mutually exclusive");
ENABLE_DISABLE_OPTION(SupportsAVX, AVX, AVX);
ENABLE_DISABLE_OPTION(SupportsSVE128, SVE, SVE);
ENABLE_DISABLE_OPTION(SupportsAFP, AFP, AFP);
ENABLE_DISABLE_OPTION(SupportsRCPC, LRCPC, LRCPC);
ENABLE_DISABLE_OPTION(SupportsTSOImm9, LRCPC2, LRCPC2);
ENABLE_DISABLE_OPTION(SupportsCSSC, CSSC, CSSC);
ENABLE_DISABLE_OPTION(SupportsPMULL_128Bit, PMULL128, PMULL128);
ENABLE_DISABLE_OPTION(SupportsRAND, RNG, RNG);
ENABLE_DISABLE_OPTION(SupportsCLZERO, CLZERO, CLZERO);
ENABLE_DISABLE_OPTION(SupportsAtomics, Atomics, ATOMICS);
ENABLE_DISABLE_OPTION(SupportsFCMA, FCMA, FCMA);
ENABLE_DISABLE_OPTION(SupportsFlagM, FlagM, FLAGM);
ENABLE_DISABLE_OPTION(SupportsFlagM2, FlagM2, FLAGM2);
ENABLE_DISABLE_OPTION(SupportsRPRES, RPRES, RPRES);
ENABLE_DISABLE_OPTION(SupportsPreserveAllABI, PRESERVEALLABI, PRESERVEALLABI);
GET_SINGLE_OPTION(Crypto, CRYPTO);
#undef ENABLE_DISABLE_OPTION
#undef GET_SINGLE_OPTION
if (EnableCrypto) {
Features->SupportsAES = true;
Features->SupportsCRC = true;
Features->SupportsSHA = true;
Features->SupportsPMULL_128Bit = true;
Features->SupportsAES256 = true;
} else if (DisableCrypto) {
Features->SupportsAES = false;
Features->SupportsCRC = false;
Features->SupportsSHA = false;
Features->SupportsPMULL_128Bit = false;
Features->SupportsAES256 = false;
}
///< Only force enable SVE256 if SVE is already enabled and ForceSVEWidth is set to >= 256.
Features->SupportsSVE256 = ForceSVEWidth && ForceSVEWidth >= 256;
}
HostFeatures::HostFeatures() {
#ifdef VIXL_SIMULATOR
auto Features = vixl::CPUFeatures::All();
// Vixl simulator doesn't support AFP.
Features.Remove(vixl::CPUFeatures::Feature::kAFP);
// Vixl simulator doesn't support RPRES.
Features.Remove(vixl::CPUFeatures::Feature::kRPRES);
#elif !defined(_WIN32)
auto Features = vixl::CPUFeatures::InferFromOS();
#else
// Need to use ID registers in WINE.
auto Features = vixl::CPUFeatures::InferFromIDRegisters();
#endif
FEX_CONFIG_OPT(ForceSVEWidth, FORCESVEWIDTH);
FEX_CONFIG_OPT(Is64BitMode, IS64BIT_MODE);
SupportsAES = Features.Has(vixl::CPUFeatures::Feature::kAES);
SupportsCRC = Features.Has(vixl::CPUFeatures::Feature::kCRC32);
SupportsSHA = Features.Has(vixl::CPUFeatures::Feature::kSHA1) && Features.Has(vixl::CPUFeatures::Feature::kSHA2);
SupportsAtomics = Features.Has(vixl::CPUFeatures::Feature::kAtomics);
SupportsRAND = Features.Has(vixl::CPUFeatures::Feature::kRNG);
// Only supported when FEAT_AFP is supported
SupportsAFP = Features.Has(vixl::CPUFeatures::Feature::kAFP);
SupportsRCPC = Features.Has(vixl::CPUFeatures::Feature::kRCpc);
SupportsTSOImm9 = Features.Has(vixl::CPUFeatures::Feature::kRCpcImm);
SupportsPMULL_128Bit = Features.Has(vixl::CPUFeatures::Feature::kPmull1Q);
SupportsCSSC = Features.Has(vixl::CPUFeatures::Feature::kCSSC);
SupportsFCMA = Features.Has(vixl::CPUFeatures::Feature::kFcma);
SupportsFlagM = Features.Has(vixl::CPUFeatures::Feature::kFlagM);
SupportsFlagM2 = Features.Has(vixl::CPUFeatures::Feature::kAXFlag);
SupportsRPRES = Features.Has(vixl::CPUFeatures::Feature::kRPRES);
Supports3DNow = true;
SupportsSSE4A = true;
#ifdef VIXL_SIMULATOR
// Hardcode enable SVE with 256-bit wide registers.
SupportsSVE128 = ForceSVEWidth() ? ForceSVEWidth() >= 128 : true;
SupportsSVE256 = ForceSVEWidth() ? ForceSVEWidth() >= 256 : true;
#else
SupportsSVE128 = Features.Has(vixl::CPUFeatures::Feature::kSVE2);
SupportsSVE256 = Features.Has(vixl::CPUFeatures::Feature::kSVE2) && vixl::aarch64::CPU::ReadSVEVectorLengthInBits() >= 256;
#endif
SupportsAVX = true;
SupportsAES256 = SupportsAVX && SupportsAES;
SupportsBMI1 = true;
SupportsBMI2 = true;
SupportsCLWB = true;
if (!SupportsAtomics) {
WARN_ONCE_FMT("Host CPU doesn't support atomics. Expect bad performance");
}
#ifdef _M_ARM_64
// We need to get the CPU's cache line size
// We expect sane targets that have correct cacheline sizes across clusters
uint64_t CTR;
__asm volatile("mrs %[ctr], ctr_el0" : [ctr] "=r"(CTR));
DCacheLineSize = 4 << ((CTR >> 16) & 0xF);
ICacheLineSize = 4 << (CTR & 0xF);
// Test if this CPU supports float exception trapping by attempting to enable
// On unsupported these bits are architecturally defined as RAZ/WI
constexpr uint32_t ExceptionEnableTraps = (1U << 8) | // Invalid Operation float exception trap enable
(1U << 9) | // Divide by zero float exception trap enable
(1U << 10) | // Overflow float exception trap enable
(1U << 11) | // Underflow float exception trap enable
(1U << 12) | // Inexact float exception trap enable
(1U << 15); // Input Denormal float exception trap enable
uint32_t OriginalFPCR = GetFPCR();
uint32_t FPCR = OriginalFPCR | ExceptionEnableTraps;
SetFPCR(FPCR);
FPCR = GetFPCR();
SupportsFloatExceptions = (FPCR & ExceptionEnableTraps) == ExceptionEnableTraps;
// Set FPCR back to original just in case anything changed
SetFPCR(OriginalFPCR);
if (SupportsRAND) {
const auto MIDR = GetMIDR();
constexpr uint32_t Implementer_QCOM = 0x51;
constexpr uint32_t PartNum_Oryon1 = 0x001;
const uint32_t MIDR_Implementer = (MIDR >> 24) & 0xFF;
const uint32_t MIDR_PartNum = (MIDR >> 4) & 0xFFF;
if (MIDR_Implementer == Implementer_QCOM && MIDR_PartNum == PartNum_Oryon1) {
// Work around an errata in Qualcomm's Oryon.
// While this CPU implements the RAND extension:
// - The RNDR register works.
// - The RNDRRS register will never read a random number. (Always return failure)
// This is contrary to x86 RNG behaviour where it allows spurious failure with RDSEED, but guarantees eventual success.
// This manifested itself on Linux when an x86 processor failed to guarantee forward progress and boot of services would infinite
// loop. Just disable this extension if this CPU is detected.
SupportsRAND = false;
}
}
#endif
#ifdef VIXL_SIMULATOR
// simulator doesn't support dc(ZVA)
SupportsCLZERO = false;
// Simulator doesn't support SHA
SupportsSHA = false;
#else
// Check if we can support cacheline clears
uint32_t DCZID = GetDCZID();
if ((DCZID & DCZID_DZP_MASK) == 0) {
uint32_t DCZID_Log2 = DCZID & DCZID_BS_MASK;
uint32_t DCZID_Bytes = (1 << DCZID_Log2) * sizeof(uint32_t);
// If the DC ZVA size matches the emulated cache line size
// This means we can use the instruction
SupportsCLZERO = DCZID_Bytes == CPUIDEmu::CACHELINE_SIZE;
}
#endif
#if defined(_M_X86_64)
// Hardcoded cacheline size.
DCacheLineSize = 64U;
ICacheLineSize = 64U;
#if !defined(VIXL_SIMULATOR)
Xbyak::util::Cpu X86Features {};
SupportsAES = X86Features.has(Xbyak::util::Cpu::tAESNI);
SupportsCRC = X86Features.has(Xbyak::util::Cpu::tSSE42);
SupportsRAND = X86Features.has(Xbyak::util::Cpu::tRDRAND) && X86Features.has(Xbyak::util::Cpu::tRDSEED);
SupportsRCPC = true;
SupportsTSOImm9 = true;
Supports3DNow = X86Features.has(Xbyak::util::Cpu::t3DN) && X86Features.has(Xbyak::util::Cpu::tE3DN);
SupportsSSE4A = X86Features.has(Xbyak::util::Cpu::tSSE4a);
SupportsAVX = true;
SupportsSHA = X86Features.has(Xbyak::util::Cpu::tSHA);
SupportsBMI1 = X86Features.has(Xbyak::util::Cpu::tBMI1);
SupportsBMI2 = X86Features.has(Xbyak::util::Cpu::tBMI2);
SupportsCLWB = X86Features.has(Xbyak::util::Cpu::tCLWB);
SupportsPMULL_128Bit = X86Features.has(Xbyak::util::Cpu::tPCLMULQDQ);
SupportsAES256 = SupportsAES && X86Features.has(Xbyak::util::Cpu::tVAES);
// xbyak doesn't know how to check for CLZero
// First ensure we support a new enough extended CPUID function range
uint32_t data[4];
Xbyak::util::Cpu::getCpuid(0x8000'0000, data);
if (data[0] >= 0x8000'0008U) {
// CLZero defined in 8000_00008_EBX[bit 0]
Xbyak::util::Cpu::getCpuid(0x8000'0008, data);
SupportsCLZERO = data[1] & 1;
}
SupportsAFP = true;
SupportsFloatExceptions = true;
#endif
#endif
SupportsPreserveAllABI = FEXCORE_HAS_PRESERVE_ALL_ATTR;
if (!Is64BitMode()) {
///< Always disable AVX and AVX2 in 32-bit mode.
// When AVX256 is enabled, signal frames start using significantly more stack space.
// - 16bytes * 16 registers = 256 bytes for XMM registers.
// - 32bytes * 16 registers = 512 bytes for YMM registers.
// There are known game failures on real x86 hardware where a 32-bit game is running up against the wall on stack space on non-AVX
// hardware and then explodes when run on AVX hardware. This is to guard against that.
SupportsAVX = false;
}
OverrideFeatures(this, ForceSVEWidth());
}
} // namespace FEXCore
@@ -6,54 +6,59 @@
#include "Interface/IR/IR.h"
namespace FEXCore::CPU {
FEXCORE_PRESERVE_ALL_ATTR static void LoadDeferredFCW(uint16_t NewFCW) {
auto PC = (NewFCW >> 8) & 3;
FEXCORE_PRESERVE_ALL_ATTR static softfloat_state SoftFloatStateFromFCW(uint16_t FCW) {
softfloat_state State {};
State.detectTininess = softfloat_tininess_afterRounding;
State.exceptionFlags = 0;
auto PC = (FCW >> 8) & 3;
switch (PC) {
case 0: extF80_roundingPrecision = 32; break;
case 2: extF80_roundingPrecision = 64; break;
case 3: extF80_roundingPrecision = 80; break;
case 0: State.roundingPrecision = 32; break;
case 2: State.roundingPrecision = 64; break;
case 3: State.roundingPrecision = 80; break;
case 1: LOGMAN_MSG_A_FMT("Invalid x87 precision mode, {}", PC);
}
auto RC = (NewFCW >> 10) & 3;
auto RC = (FCW >> 10) & 3;
switch (RC) {
case 0: softfloat_roundingMode = softfloat_round_near_even; break;
case 1: softfloat_roundingMode = softfloat_round_min; break;
case 2: softfloat_roundingMode = softfloat_round_max; break;
case 3: softfloat_roundingMode = softfloat_round_minMag; break;
case 0: State.roundingMode = softfloat_round_near_even; break;
case 1: State.roundingMode = softfloat_round_min; break;
case 2: State.roundingMode = softfloat_round_max; break;
case 3: State.roundingMode = softfloat_round_minMag; break;
}
return State;
}
template<>
struct OpHandlers<IR::OP_F80CVTTO> {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle4(uint16_t NewFCW, float src) {
LoadDeferredFCW(NewFCW);
return src;
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle4(uint16_t FCW, float src) {
softfloat_state State = SoftFloatStateFromFCW(FCW);
return X80SoftFloat(&State, src);
}
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle8(uint16_t NewFCW, double src) {
LoadDeferredFCW(NewFCW);
return src;
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle8(uint16_t FCW, double src) {
softfloat_state State = SoftFloatStateFromFCW(FCW);
return X80SoftFloat(&State, src);
}
};
template<>
struct OpHandlers<IR::OP_F80CMP> {
template<uint32_t Flags>
FEXCORE_PRESERVE_ALL_ATTR static uint64_t handle(uint16_t NewFCW, X80SoftFloat Src1, X80SoftFloat Src2) {
LoadDeferredFCW(NewFCW);
FEXCORE_PRESERVE_ALL_ATTR static uint64_t handle(uint16_t FCW, X80SoftFloat Src1, X80SoftFloat Src2) {
softfloat_state State = SoftFloatStateFromFCW(FCW);
bool eq, lt, nan;
uint64_t ResultFlags = 0;
X80SoftFloat::FCMP(Src1, Src2, &eq, &lt, &nan);
if (Flags & (1 << IR::FCMP_FLAG_LT) && lt) {
X80SoftFloat::FCMP(&State, Src1, Src2, &eq, &lt, &nan);
if (lt) {
ResultFlags |= (1 << IR::FCMP_FLAG_LT);
}
if (Flags & (1 << IR::FCMP_FLAG_UNORDERED) && nan) {
if (nan) {
ResultFlags |= (1 << IR::FCMP_FLAG_UNORDERED);
}
if (Flags & (1 << IR::FCMP_FLAG_EQ) && eq) {
if (eq) {
ResultFlags |= (1 << IR::FCMP_FLAG_EQ);
}
return ResultFlags;
@@ -62,292 +67,281 @@ struct OpHandlers<IR::OP_F80CMP> {
template<>
struct OpHandlers<IR::OP_F80CVT> {
FEXCORE_PRESERVE_ALL_ATTR static float handle4(uint16_t NewFCW, X80SoftFloat src) {
LoadDeferredFCW(NewFCW);
return src;
FEXCORE_PRESERVE_ALL_ATTR static float handle4(uint16_t FCW, X80SoftFloat src) {
softfloat_state State = SoftFloatStateFromFCW(FCW);
return src.ToF32(&State);
}
FEXCORE_PRESERVE_ALL_ATTR static double handle8(uint16_t NewFCW, X80SoftFloat src) {
LoadDeferredFCW(NewFCW);
return src;
FEXCORE_PRESERVE_ALL_ATTR static double handle8(uint16_t FCW, X80SoftFloat src) {
softfloat_state State = SoftFloatStateFromFCW(FCW);
return src.ToF64(&State);
}
};
template<>
struct OpHandlers<IR::OP_F80CVTINT> {
FEXCORE_PRESERVE_ALL_ATTR static int16_t handle2(uint16_t NewFCW, X80SoftFloat src) {
LoadDeferredFCW(NewFCW);
return src;
FEXCORE_PRESERVE_ALL_ATTR static int16_t handle2(uint16_t FCW, X80SoftFloat src) {
softfloat_state State = SoftFloatStateFromFCW(FCW);
return src.ToI16(&State);
}
FEXCORE_PRESERVE_ALL_ATTR static int32_t handle4(uint16_t NewFCW, X80SoftFloat src) {
LoadDeferredFCW(NewFCW);
return src;
FEXCORE_PRESERVE_ALL_ATTR static int32_t handle4(uint16_t FCW, X80SoftFloat src) {
softfloat_state State = SoftFloatStateFromFCW(FCW);
return src.ToI32(&State);
}
FEXCORE_PRESERVE_ALL_ATTR static int64_t handle8(uint16_t NewFCW, X80SoftFloat src) {
LoadDeferredFCW(NewFCW);
return src;
FEXCORE_PRESERVE_ALL_ATTR static int64_t handle8(uint16_t FCW, X80SoftFloat src) {
softfloat_state State = SoftFloatStateFromFCW(FCW);
return src.ToI64(&State);
}
FEXCORE_PRESERVE_ALL_ATTR static int16_t handle2t(uint16_t NewFCW, X80SoftFloat src) {
LoadDeferredFCW(NewFCW);
auto rv = extF80_to_i32(src, softfloat_round_minMag, false);
FEXCORE_PRESERVE_ALL_ATTR static int16_t handle2t(uint16_t FCW, X80SoftFloat src) {
softfloat_state State = SoftFloatStateFromFCW(FCW);
auto rv = extF80_to_i32(&State, src, softfloat_round_minMag, false);
if (rv > INT16_MAX) {
return INT16_MAX;
} else if (rv < INT16_MIN) {
if (rv > INT16_MAX || rv < INT16_MIN) {
///< Indefinite value for 16-bit conversions.
return INT16_MIN;
} else {
return rv;
}
}
FEXCORE_PRESERVE_ALL_ATTR static int32_t handle4t(uint16_t NewFCW, X80SoftFloat src) {
LoadDeferredFCW(NewFCW);
return extF80_to_i32(src, softfloat_round_minMag, false);
FEXCORE_PRESERVE_ALL_ATTR static int32_t handle4t(uint16_t FCW, X80SoftFloat src) {
softfloat_state State = SoftFloatStateFromFCW(FCW);
return extF80_to_i32(&State, src, softfloat_round_minMag, false);
}
FEXCORE_PRESERVE_ALL_ATTR static int64_t handle8t(uint16_t NewFCW, X80SoftFloat src) {
LoadDeferredFCW(NewFCW);
return extF80_to_i64(src, softfloat_round_minMag, false);
FEXCORE_PRESERVE_ALL_ATTR static int64_t handle8t(uint16_t FCW, X80SoftFloat src) {
softfloat_state State = SoftFloatStateFromFCW(FCW);
return extF80_to_i64(&State, src, softfloat_round_minMag, false);
}
};
template<>
struct OpHandlers<IR::OP_F80CVTTOINT> {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle2(uint16_t NewFCW, int16_t src) {
LoadDeferredFCW(NewFCW);
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle2(uint16_t FCW, int16_t src) {
return src;
}
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle4(uint16_t NewFCW, int32_t src) {
LoadDeferredFCW(NewFCW);
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle4(uint16_t FCW, int32_t src) {
return src;
}
};
template<>
struct OpHandlers<IR::OP_F80ROUND> {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t NewFCW, X80SoftFloat Src1) {
LoadDeferredFCW(NewFCW);
return X80SoftFloat::FRNDINT(Src1);
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t FCW, X80SoftFloat Src1) {
softfloat_state State = SoftFloatStateFromFCW(FCW);
return X80SoftFloat::FRNDINT(&State, Src1);
}
};
template<>
struct OpHandlers<IR::OP_F80F2XM1> {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t NewFCW, X80SoftFloat Src1) {
LoadDeferredFCW(NewFCW);
return X80SoftFloat::F2XM1(Src1);
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t FCW, X80SoftFloat Src1) {
softfloat_state State = SoftFloatStateFromFCW(FCW);
return X80SoftFloat::F2XM1(&State, Src1);
}
};
template<>
struct OpHandlers<IR::OP_F80TAN> {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t NewFCW, X80SoftFloat Src1) {
LoadDeferredFCW(NewFCW);
return X80SoftFloat::FTAN(Src1);
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t FCW, X80SoftFloat Src1) {
softfloat_state State = SoftFloatStateFromFCW(FCW);
return X80SoftFloat::FTAN(&State, Src1);
}
};
template<>
struct OpHandlers<IR::OP_F80SQRT> {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t NewFCW, X80SoftFloat Src1) {
LoadDeferredFCW(NewFCW);
return X80SoftFloat::FSQRT(Src1);
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t FCW, X80SoftFloat Src1) {
softfloat_state State = SoftFloatStateFromFCW(FCW);
return X80SoftFloat::FSQRT(&State, Src1);
}
};
template<>
struct OpHandlers<IR::OP_F80SIN> {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t NewFCW, X80SoftFloat Src1) {
LoadDeferredFCW(NewFCW);
return X80SoftFloat::FSIN(Src1);
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t FCW, X80SoftFloat Src1) {
softfloat_state State = SoftFloatStateFromFCW(FCW);
return X80SoftFloat::FSIN(&State, Src1);
}
};
template<>
struct OpHandlers<IR::OP_F80COS> {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t NewFCW, X80SoftFloat Src1) {
LoadDeferredFCW(NewFCW);
return X80SoftFloat::FCOS(Src1);
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t FCW, X80SoftFloat Src1) {
softfloat_state State = SoftFloatStateFromFCW(FCW);
return X80SoftFloat::FCOS(&State, Src1);
}
};
template<>
struct OpHandlers<IR::OP_F80XTRACT_EXP> {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t NewFCW, X80SoftFloat Src1) {
LoadDeferredFCW(NewFCW);
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t FCW, X80SoftFloat Src1) {
return X80SoftFloat::FXTRACT_EXP(Src1);
}
};
template<>
struct OpHandlers<IR::OP_F80XTRACT_SIG> {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t NewFCW, X80SoftFloat Src1) {
LoadDeferredFCW(NewFCW);
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t FCW, X80SoftFloat Src1) {
return X80SoftFloat::FXTRACT_SIG(Src1);
}
};
template<>
struct OpHandlers<IR::OP_F80ADD> {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t NewFCW, X80SoftFloat Src1, X80SoftFloat Src2) {
LoadDeferredFCW(NewFCW);
return X80SoftFloat::FADD(Src1, Src2);
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t FCW, X80SoftFloat Src1, X80SoftFloat Src2) {
softfloat_state State = SoftFloatStateFromFCW(FCW);
return X80SoftFloat::FADD(&State, Src1, Src2);
}
};
template<>
struct OpHandlers<IR::OP_F80SUB> {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t NewFCW, X80SoftFloat Src1, X80SoftFloat Src2) {
LoadDeferredFCW(NewFCW);
return X80SoftFloat::FSUB(Src1, Src2);
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t FCW, X80SoftFloat Src1, X80SoftFloat Src2) {
softfloat_state State = SoftFloatStateFromFCW(FCW);
return X80SoftFloat::FSUB(&State, Src1, Src2);
}
};
template<>
struct OpHandlers<IR::OP_F80MUL> {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t NewFCW, X80SoftFloat Src1, X80SoftFloat Src2) {
LoadDeferredFCW(NewFCW);
return X80SoftFloat::FMUL(Src1, Src2);
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t FCW, X80SoftFloat Src1, X80SoftFloat Src2) {
softfloat_state State = SoftFloatStateFromFCW(FCW);
return X80SoftFloat::FMUL(&State, Src1, Src2);
}
};
template<>
struct OpHandlers<IR::OP_F80DIV> {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t NewFCW, X80SoftFloat Src1, X80SoftFloat Src2) {
LoadDeferredFCW(NewFCW);
return X80SoftFloat::FDIV(Src1, Src2);
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t FCW, X80SoftFloat Src1, X80SoftFloat Src2) {
softfloat_state State = SoftFloatStateFromFCW(FCW);
return X80SoftFloat::FDIV(&State, Src1, Src2);
}
};
template<>
struct OpHandlers<IR::OP_F80FYL2X> {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t NewFCW, X80SoftFloat Src1, X80SoftFloat Src2) {
LoadDeferredFCW(NewFCW);
return X80SoftFloat::FYL2X(Src1, Src2);
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t FCW, X80SoftFloat Src1, X80SoftFloat Src2) {
softfloat_state State = SoftFloatStateFromFCW(FCW);
return X80SoftFloat::FYL2X(&State, Src1, Src2);
}
};
template<>
struct OpHandlers<IR::OP_F80ATAN> {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t NewFCW, X80SoftFloat Src1, X80SoftFloat Src2) {
LoadDeferredFCW(NewFCW);
return X80SoftFloat::FATAN(Src1, Src2);
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t FCW, X80SoftFloat Src1, X80SoftFloat Src2) {
softfloat_state State = SoftFloatStateFromFCW(FCW);
return X80SoftFloat::FATAN(&State, Src1, Src2);
}
};
template<>
struct OpHandlers<IR::OP_F80FPREM1> {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t NewFCW, X80SoftFloat Src1, X80SoftFloat Src2) {
LoadDeferredFCW(NewFCW);
return X80SoftFloat::FREM1(Src1, Src2);
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t FCW, X80SoftFloat Src1, X80SoftFloat Src2) {
softfloat_state State = SoftFloatStateFromFCW(FCW);
return X80SoftFloat::FREM1(&State, Src1, Src2);
}
};
template<>
struct OpHandlers<IR::OP_F80FPREM> {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t NewFCW, X80SoftFloat Src1, X80SoftFloat Src2) {
LoadDeferredFCW(NewFCW);
return X80SoftFloat::FREM(Src1, Src2);
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t FCW, X80SoftFloat Src1, X80SoftFloat Src2) {
softfloat_state State = SoftFloatStateFromFCW(FCW);
return X80SoftFloat::FREM(&State, Src1, Src2);
}
};
template<>
struct OpHandlers<IR::OP_F80SCALE> {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t NewFCW, X80SoftFloat Src1, X80SoftFloat Src2) {
LoadDeferredFCW(NewFCW);
return X80SoftFloat::FSCALE(Src1, Src2);
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t FCW, X80SoftFloat Src1, X80SoftFloat Src2) {
softfloat_state State = SoftFloatStateFromFCW(FCW);
return X80SoftFloat::FSCALE(&State, Src1, Src2);
}
};
template<>
struct OpHandlers<IR::OP_F64SIN> {
static double handle(uint16_t NewFCW, double src) {
LoadDeferredFCW(NewFCW);
static double handle(uint16_t FCW, double src) {
return sin(src);
}
};
template<>
struct OpHandlers<IR::OP_F64COS> {
static double handle(uint16_t NewFCW, double src) {
LoadDeferredFCW(NewFCW);
static double handle(uint16_t FCW, double src) {
return cos(src);
}
};
template<>
struct OpHandlers<IR::OP_F64TAN> {
static double handle(uint16_t NewFCW, double src) {
LoadDeferredFCW(NewFCW);
static double handle(uint16_t FCW, double src) {
return tan(src);
}
};
template<>
struct OpHandlers<IR::OP_F64F2XM1> {
static double handle(uint16_t NewFCW, double src) {
LoadDeferredFCW(NewFCW);
static double handle(uint16_t FCW, double src) {
return exp2(src) - 1.0;
}
};
template<>
struct OpHandlers<IR::OP_F64ATAN> {
static double handle(uint16_t NewFCW, double src1, double src2) {
LoadDeferredFCW(NewFCW);
static double handle(uint16_t FCW, double src1, double src2) {
return atan2(src1, src2);
}
};
template<>
struct OpHandlers<IR::OP_F64FPREM> {
static double handle(uint16_t NewFCW, double src1, double src2) {
LoadDeferredFCW(NewFCW);
static double handle(uint16_t FCW, double src1, double src2) {
return fmod(src1, src2);
}
};
template<>
struct OpHandlers<IR::OP_F64FPREM1> {
static double handle(uint16_t NewFCW, double src1, double src2) {
LoadDeferredFCW(NewFCW);
static double handle(uint16_t FCW, double src1, double src2) {
return remainder(src1, src2);
}
};
template<>
struct OpHandlers<IR::OP_F64FYL2X> {
static double handle(uint16_t NewFCW, double src1, double src2) {
LoadDeferredFCW(NewFCW);
static double handle(uint16_t FCW, double src1, double src2) {
return src2 * log2(src1);
}
};
template<>
struct OpHandlers<IR::OP_F64SCALE> {
static double handle(uint16_t NewFCW, double src1, double src2) {
LoadDeferredFCW(NewFCW);
double trunc = (double)(int64_t)(src2); // truncate
return src1 * exp2(trunc);
static double handle(uint16_t FCW, double src1, double src2) {
if (src1 == 0.0) { // src1 might be +/- zero
return src1; // this will return negative or positive zero if when appropriate
}
double trun = trunc(src2);
return src1 * exp2(trun);
}
};
template<>
struct OpHandlers<IR::OP_F80BCDSTORE> {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t NewFCW, X80SoftFloat Src1) {
LoadDeferredFCW(NewFCW);
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t FCW, X80SoftFloat Src1) {
softfloat_state State = SoftFloatStateFromFCW(FCW);
bool Negative = Src1.Sign;
Src1 = X80SoftFloat::FRNDINT(Src1);
Src1 = X80SoftFloat::FRNDINT(&State, Src1);
// Clear the Sign bit
Src1.Sign = 0;
uint64_t Tmp = Src1;
uint64_t Tmp = Src1.ToI64(&State);
X80SoftFloat Rv;
uint8_t* BCD = reinterpret_cast<uint8_t*>(&Rv);
memset(BCD, 0, 10);
@@ -379,8 +373,7 @@ struct OpHandlers<IR::OP_F80BCDSTORE> {
template<>
struct OpHandlers<IR::OP_F80BCDLOAD> {
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t NewFCW, X80SoftFloat Src) {
LoadDeferredFCW(NewFCW);
FEXCORE_PRESERVE_ALL_ATTR static X80SoftFloat handle(uint16_t FCW, X80SoftFloat Src) {
uint8_t* Src1 = reinterpret_cast<uint8_t*>(&Src);
uint64_t BCD {};
// We walk through each uint8_t and pull out the BCD encoding
@@ -35,14 +35,7 @@ void InterpreterOps::FillFallbackIndexPointers(uint64_t* Info) {
Info[Core::OPINDEX_F80CVTINT_TRUNC2] = reinterpret_cast<uint64_t>(&FEXCore::CPU::OpHandlers<IR::OP_F80CVTINT>::handle2t);
Info[Core::OPINDEX_F80CVTINT_TRUNC4] = reinterpret_cast<uint64_t>(&FEXCore::CPU::OpHandlers<IR::OP_F80CVTINT>::handle4t);
Info[Core::OPINDEX_F80CVTINT_TRUNC8] = reinterpret_cast<uint64_t>(&FEXCore::CPU::OpHandlers<IR::OP_F80CVTINT>::handle8t);
Info[Core::OPINDEX_F80CMP_0] = reinterpret_cast<uint64_t>(&FEXCore::CPU::OpHandlers<IR::OP_F80CMP>::handle<0>);
Info[Core::OPINDEX_F80CMP_1] = reinterpret_cast<uint64_t>(&FEXCore::CPU::OpHandlers<IR::OP_F80CMP>::handle<1>);
Info[Core::OPINDEX_F80CMP_2] = reinterpret_cast<uint64_t>(&FEXCore::CPU::OpHandlers<IR::OP_F80CMP>::handle<2>);
Info[Core::OPINDEX_F80CMP_3] = reinterpret_cast<uint64_t>(&FEXCore::CPU::OpHandlers<IR::OP_F80CMP>::handle<3>);
Info[Core::OPINDEX_F80CMP_4] = reinterpret_cast<uint64_t>(&FEXCore::CPU::OpHandlers<IR::OP_F80CMP>::handle<4>);
Info[Core::OPINDEX_F80CMP_5] = reinterpret_cast<uint64_t>(&FEXCore::CPU::OpHandlers<IR::OP_F80CMP>::handle<5>);
Info[Core::OPINDEX_F80CMP_6] = reinterpret_cast<uint64_t>(&FEXCore::CPU::OpHandlers<IR::OP_F80CMP>::handle<6>);
Info[Core::OPINDEX_F80CMP_7] = reinterpret_cast<uint64_t>(&FEXCore::CPU::OpHandlers<IR::OP_F80CMP>::handle<7>);
Info[Core::OPINDEX_F80CMP] = reinterpret_cast<uint64_t>(&FEXCore::CPU::OpHandlers<IR::OP_F80CMP>::handle);
Info[Core::OPINDEX_F80CVTTOINT_2] = reinterpret_cast<uint64_t>(&FEXCore::CPU::OpHandlers<IR::OP_F80CVTTOINT>::handle2);
Info[Core::OPINDEX_F80CVTTOINT_4] = reinterpret_cast<uint64_t>(&FEXCore::CPU::OpHandlers<IR::OP_F80CVTTOINT>::handle4);
@@ -154,17 +147,8 @@ bool InterpreterOps::GetFallbackHandler(bool SupportsPreserveAllABI, const IR::I
break;
}
case IR::OP_F80CMP: {
auto Op = IROp->C<IR::IROp_F80Cmp>();
static constexpr std::array handlers {
&FEXCore::CPU::OpHandlers<IR::OP_F80CMP>::handle<0>, &FEXCore::CPU::OpHandlers<IR::OP_F80CMP>::handle<1>,
&FEXCore::CPU::OpHandlers<IR::OP_F80CMP>::handle<2>, &FEXCore::CPU::OpHandlers<IR::OP_F80CMP>::handle<3>,
&FEXCore::CPU::OpHandlers<IR::OP_F80CMP>::handle<4>, &FEXCore::CPU::OpHandlers<IR::OP_F80CMP>::handle<5>,
&FEXCore::CPU::OpHandlers<IR::OP_F80CMP>::handle<6>, &FEXCore::CPU::OpHandlers<IR::OP_F80CMP>::handle<7>,
};
*Info = {FABI_I64_I16_F80_F80, (void*)handlers[Op->Flags], (Core::FallbackHandlerIndex)(Core::OPINDEX_F80CMP_0 + Op->Flags),
SupportsPreserveAllABI};
*Info = {FABI_I64_I16_F80_F80, (void*)&FEXCore::CPU::OpHandlers<IR::OP_F80CMP>::handle,
(Core::FallbackHandlerIndex)(Core::OPINDEX_F80CMP), SupportsPreserveAllABI};
return true;
}
+253 -102
View File
@@ -5,6 +5,7 @@ tags: backend|arm64
$end_info$
*/
#include "CodeEmitter/Emitter.h"
#include "FEXCore/IR/IR.h"
#include "Interface/Context/Context.h"
#include "Interface/Core/JIT/Arm64/JITClass.h"
@@ -41,21 +42,6 @@ DEF_BINOP_WITH_CONSTANT(Lshl, lslv, lsl)
DEF_BINOP_WITH_CONSTANT(Lshr, lsrv, lsr)
DEF_BINOP_WITH_CONSTANT(Ror, rorv, ror)
DEF_OP(TruncElementPair) {
auto Op = IROp->C<IR::IROp_TruncElementPair>();
switch (IROp->Size) {
case 4: {
auto Dst = GetRegPair(Node);
auto Src = GetRegPair(Op->Pair.ID());
mov(ARMEmitter::Size::i32Bit, Dst.first, Src.first);
mov(ARMEmitter::Size::i32Bit, Dst.second, Src.second);
break;
}
default: LOGMAN_MSG_A_FMT("Unhandled Truncation size: {}", IROp->Size); break;
}
}
DEF_OP(Constant) {
auto Op = IROp->C<IR::IROp_Constant>();
auto Dst = GetReg(Node);
@@ -130,6 +116,21 @@ DEF_OP(AdcWithFlags) {
adcs(ConvertSize48(IROp), GetReg(Node), GetZeroableReg(Op->Src1), GetReg(Op->Src2.ID()));
}
DEF_OP(AdcZeroWithFlags) {
auto Op = IROp->C<IR::IROp_AdcZeroWithFlags>();
auto Size = ConvertSize48(IROp);
cset(Size, TMP1, ARMEmitter::Condition::CC_CC);
adds(Size, GetReg(Node), GetReg(Op->Src1.ID()), TMP1);
}
DEF_OP(AdcZero) {
auto Op = IROp->C<IR::IROp_AdcZero>();
auto Size = ConvertSize48(IROp);
cinc(Size, GetReg(Node), GetReg(Op->Src1.ID()), ARMEmitter::Condition::CC_CC);
}
DEF_OP(Adc) {
auto Op = IROp->C<IR::IROp_Adc>();
@@ -190,6 +191,30 @@ DEF_OP(TestNZ) {
}
}
DEF_OP(TestZ) {
auto Op = IROp->C<IR::IROp_TestZ>();
LOGMAN_THROW_AA_FMT(IROp->Size < 4, "TestNZ used at higher sizes");
const auto EmitSize = ARMEmitter::Size::i32Bit;
uint64_t Const;
uint64_t Mask = IROp->Size == 8 ? ~0ULL : ((1ull << (IROp->Size * 8)) - 1);
auto Src1 = GetReg(Op->Src1.ID());
if (IsInlineConstant(Op->Src2, &Const)) {
// We can promote 8/16-bit tests to 32-bit since the constant is masked.
LOGMAN_THROW_AA_FMT(!(Const & ~Mask), "constant is already masked");
tst(EmitSize, Src1, Const);
} else {
const auto Src2 = GetReg(Op->Src2.ID());
if (Src1 == Src2) {
tst(EmitSize, Src1 /* Src2 */, Mask);
} else {
and_(EmitSize, TMP1, Src1, Src2);
tst(EmitSize, TMP1, Mask);
}
}
}
DEF_OP(SubShift) {
auto Op = IROp->C<IR::IROp_SubShift>();
@@ -232,10 +257,8 @@ DEF_OP(CmpPairZ) {
mrs(TMP1, ARMEmitter::SystemRegister::NZCV);
// Compare, setting Z and clobbering NzCV
const auto Src1 = GetRegPair(Op->Src1.ID());
const auto Src2 = GetRegPair(Op->Src2.ID());
cmp(EmitSize, Src1.first, Src2.first);
ccmp(EmitSize, Src1.second, Src2.second, ARMEmitter::StatusFlags::None, ARMEmitter::Condition::CC_EQ);
cmp(EmitSize, GetReg(Op->Src1Lo.ID()), GetReg(Op->Src2Lo.ID()));
ccmp(EmitSize, GetReg(Op->Src1Hi.ID()), GetReg(Op->Src2Hi.ID()), ARMEmitter::StatusFlags::None, ARMEmitter::Condition::CC_EQ);
// Restore NzCV
if (CTX->HostFeatures.SupportsFlagM) {
@@ -274,8 +297,43 @@ DEF_OP(SetSmallNZV) {
}
DEF_OP(AXFlag) {
LOGMAN_THROW_A_FMT(CTX->HostFeatures.SupportsFlagM2, "Unsupported flagm2 op");
axflag();
if (CTX->HostFeatures.SupportsFlagM2) {
axflag();
} else {
// AXFLAG is defined in the Arm spec as
//
// gt: nzCv -> nzCv
// lt: Nzcv -> nzcv <==> 1 + 0
// eq: nZCv -> nZCv <==> 1 + (~0)
// un: nzCV -> nZcv <==> 0 + 0
//
// For the latter 3 cases, we therefore get the right NZCV by adding V_inv
// to (eq ? ~0 : 0). The remaining case is forced with ccmn.
auto V_inv = GetReg(IROp->Args[0].ID());
csetm(ARMEmitter::Size::i64Bit, TMP1, ARMEmitter::Condition::CC_EQ);
ccmn(ARMEmitter::Size::i64Bit, V_inv, TMP1, ARMEmitter::StatusFlags {0x2} /* nzCv */, ARMEmitter::Condition::CC_LE);
}
}
DEF_OP(Parity) {
auto Op = IROp->C<IR::IROp_Parity>();
auto Raw = GetReg(Op->Raw.ID());
auto Dest = GetReg(Node);
// Cascade to calculate parity of bottom 8-bits to bottom bit.
eor(ARMEmitter::Size::i32Bit, TMP1, Raw, Raw, ARMEmitter::ShiftType::LSR, 4);
eor(ARMEmitter::Size::i32Bit, TMP1, TMP1, TMP1, ARMEmitter::ShiftType::LSR, 2);
if (Op->Invert) {
eon(ARMEmitter::Size::i32Bit, Dest, TMP1, TMP1, ARMEmitter::ShiftType::LSR, 1);
} else {
eor(ARMEmitter::Size::i32Bit, Dest, TMP1, TMP1, ARMEmitter::ShiftType::LSR, 1);
}
// The above sequence leaves garbage in the upper bits.
if (Op->Mask) {
and_(ARMEmitter::Size::i32Bit, Dest, Dest, 1);
}
}
DEF_OP(CondAddNZCV) {
@@ -625,13 +683,17 @@ DEF_OP(ShiftFlags) {
// Set the output outside the branch to avoid needing an extra leg of the
// branch. We specifically do not hardcode the PF register anywhere (relying
// on a tied SRA register instead) to avoid fighting with RA/RCLSE.
// on a tied SRA register instead) to avoid fighting with RA.
if (PFTemp != PFInput) {
mov(ARMEmitter::Size::i64Bit, PFTemp, PFInput);
}
// We need to mask the source before comparing it. We don't just skip flag
// updates for Src2=0 but anything that masks to zero.
and_(ARMEmitter::Size::i32Bit, TMP1, Src2, OpSize == 8 ? 0x3f : 0x1f);
ARMEmitter::SingleUseForwardLabel Done;
cbz(EmitSize, Src2, &Done);
cbz(EmitSize, TMP1, &Done);
{
// PF/SF/ZF/OF
if (OpSize >= 4) {
@@ -642,19 +704,27 @@ DEF_OP(ShiftFlags) {
mov(ARMEmitter::Size::i64Bit, PFTemp, Dst);
}
auto CFWord = TMP1;
unsigned CFBit = 0;
// Extract the last bit shifted in to CF
if (Op->Shift == IR::ShiftType::LSL) {
if (OpSize >= 4) {
neg(EmitSize, TMP1, Src2);
neg(EmitSize, CFWord, Src2);
lsrv(EmitSize, CFWord, Src1, CFWord);
} else {
mov(EmitSize, TMP1, OpSize * 8);
sub(EmitSize, TMP1, TMP1, Src2);
CFWord = Dst.X();
CFBit = (OpSize * 8);
}
} else {
sub(ARMEmitter::Size::i64Bit, TMP1, Src2, 1);
sub(ARMEmitter::Size::i64Bit, CFWord, Src2, 1);
lsrv(EmitSize, CFWord, Src1, CFWord);
}
lsrv(EmitSize, TMP1, Src1, TMP1);
if (Op->InvertCF) {
mvn(ARMEmitter::Size::i64Bit, TMP1, CFWord);
CFWord = TMP1;
}
bool SetOF = Op->Shift != IR::ShiftType::ASR;
if (SetOF) {
@@ -664,14 +734,20 @@ DEF_OP(ShiftFlags) {
}
if (CTX->HostFeatures.SupportsFlagM) {
rmif(TMP1, 63, (1 << 1) /* C */);
rmif(CFWord, (CFBit - 1) % 64, (1 << 1) /* C */);
if (SetOF) {
rmif(TMP3, OpSize * 8 - 1, (1 << 0) /* V */);
}
} else {
mrs(TMP2, ARMEmitter::SystemRegister::NZCV);
bfi(ARMEmitter::Size::i32Bit, TMP2, TMP1, 29 /* C */, 1);
if (CFBit != 0) {
lsr(ARMEmitter::Size::i64Bit, TMP1, CFWord, CFBit);
CFWord = TMP1;
}
bfi(ARMEmitter::Size::i32Bit, TMP2, CFWord, 29 /* C */, 1);
if (SetOF) {
lsr(EmitSize, TMP3, TMP3, OpSize * 8 - 1);
@@ -689,6 +765,50 @@ DEF_OP(ShiftFlags) {
}
}
DEF_OP(RotateFlags) {
auto Op = IROp->C<IR::IROp_RotateFlags>();
const auto Result = GetReg(Op->Result.ID());
const auto Shift = GetReg(Op->Shift.ID());
const bool Left = Op->Left;
const auto EmitSize = Op->Size == 8 ? ARMEmitter::Size::i64Bit : ARMEmitter::Size::i32Bit;
// If shift=0, flags are unaffected. Wrap the whole implementation in a cbz.
ARMEmitter::SingleUseForwardLabel Done;
cbz(EmitSize, Shift, &Done);
{
// Extract the last bit shifted in to CF
const auto BitSize = Op->Size * 8;
unsigned CFBit = Left ? 0 : BitSize - 1;
// For ROR, OF is the XOR of the new CF bit and the most significant bit of the result.
// For ROL, OF is the LSB and MSB XOR'd together.
// OF is architecturally only defined for 1-bit rotate.
eor(ARMEmitter::Size::i64Bit, TMP1, Result, Result, ARMEmitter::ShiftType::LSR, Left ? BitSize - 1 : 1);
unsigned OFBit = Left ? 0 : BitSize - 2;
// Invert result so we get inverted carry.
mvn(ARMEmitter::Size::i64Bit, TMP2, Result);
if (CTX->HostFeatures.SupportsFlagM) {
rmif(TMP2, (CFBit - 1) % 64, 1 << 1 /* nzCv */);
rmif(TMP1, OFBit, 1 << 0 /* nzcV */);
} else {
if (OFBit != 0) {
lsr(EmitSize, TMP1, TMP1, OFBit);
}
if (CFBit != 0) {
lsr(EmitSize, TMP2, TMP2, CFBit);
}
mrs(TMP3, ARMEmitter::SystemRegister::NZCV);
bfi(ARMEmitter::Size::i32Bit, TMP3, TMP1, 28 /* V */, 1);
bfi(ARMEmitter::Size::i32Bit, TMP3, TMP2, 29 /* C */, 1);
msr(ARMEmitter::SystemRegister::NZCV, TMP3);
}
}
Bind(&Done);
}
DEF_OP(Extr) {
auto Op = IROp->C<IR::IROp_Extr>();
const auto Dst = GetReg(Node);
@@ -704,59 +824,74 @@ DEF_OP(PDep) {
const auto Dest = GetReg(Node);
// PDep implementation follows the ideas from
// http://0x80.pl/articles/pdep-soft-emu.html ... Basically, iterate the *set*
// bits only, which will be faster than the naive implementation as long as
// there are enough holes in the mask.
//
// The specific arm64 assembly used is based on the sequence that clang
// generates for the C code, giving context to the scheduling yielding better
// ILP than I would do by hand. The registers are allocated by hand however,
// to fit within the tight constraints we have here withot spilling. Also, we
// use cbz/cbnz for conditional branching to avoid clobbering NZCV.
// We can't clobber these
const auto OrigInput = GetReg(Op->Input.ID());
const auto OrigMask = GetReg(Op->Mask.ID());
// So we have shadow as temporaries
const auto Input = TMP1.R();
const auto Mask = TMP2.R();
if (CTX->HostFeatures.SupportsSVEBitPerm) {
// SVE added support for PDEP but it needs to be done in a vector register.
if (EmitSize == ARMEmitter::Size::i32Bit) {
fmov(ARMEmitter::Size::i32Bit, VTMP1.S(), OrigInput.W());
fmov(ARMEmitter::Size::i32Bit, VTMP2.S(), OrigMask.W());
bdep(ARMEmitter::SubRegSize::i32Bit, VTMP1.Z(), VTMP1.Z(), VTMP2.Z());
umov<ARMEmitter::SubRegSize::i32Bit>(Dest, VTMP1, 0);
} else {
fmov(ARMEmitter::Size::i64Bit, VTMP1.D(), OrigInput.X());
fmov(ARMEmitter::Size::i64Bit, VTMP2.D(), OrigMask.X());
bdep(ARMEmitter::SubRegSize::i64Bit, VTMP1.Z(), VTMP1.Z(), VTMP2.Z());
umov<ARMEmitter::SubRegSize::i64Bit>(Dest, VTMP1, 0);
}
} else {
// PDep implementation follows the ideas from
// http://0x80.pl/articles/pdep-soft-emu.html ... Basically, iterate the *set*
// bits only, which will be faster than the naive implementation as long as
// there are enough holes in the mask.
//
// The specific arm64 assembly used is based on the sequence that clang
// generates for the C code, giving context to the scheduling yielding better
// ILP than I would do by hand. The registers are allocated by hand however,
// to fit within the tight constraints we have here withot spilling. Also, we
// use cbz/cbnz for conditional branching to avoid clobbering NZCV.
// these get used variously as scratch
const auto T0 = TMP3.R();
const auto T1 = TMP4.R();
// So we have shadow as temporaries
const auto Input = TMP1.R();
const auto Mask = TMP2.R();
ARMEmitter::BackwardLabel NextBit;
ARMEmitter::SingleUseForwardLabel Done;
// these get used variously as scratch
const auto T0 = TMP3.R();
const auto T1 = TMP4.R();
// First, copy the input/mask, since we'll be clobbering. Copy as 64-bit to
// make this 0-uop on Firestorm.
mov(ARMEmitter::Size::i64Bit, Input, OrigInput);
mov(ARMEmitter::Size::i64Bit, Mask, OrigMask);
ARMEmitter::BackwardLabel NextBit;
ARMEmitter::SingleUseForwardLabel Done;
// Now, they're copied, so we can start setting Dest (even if it overlaps with
// one of them). Handle early exit case
mov(EmitSize, Dest, 0);
cbz(EmitSize, OrigMask, &Done);
// First, copy the input/mask, since we'll be clobbering. Copy as 64-bit to
// make this 0-uop on Firestorm.
mov(ARMEmitter::Size::i64Bit, Input, OrigInput);
mov(ARMEmitter::Size::i64Bit, Mask, OrigMask);
// Setup for first iteration
neg(EmitSize, T0, Mask);
and_(EmitSize, T0, T0, Mask);
// Now, they're copied, so we can start setting Dest (even if it overlaps with
// one of them). Handle early exit case
mov(EmitSize, Dest, 0);
cbz(EmitSize, OrigMask, &Done);
// Main loop
Bind(&NextBit);
sbfx(EmitSize, T1, Input, 0, 1);
eor(EmitSize, Mask, Mask, T0);
and_(EmitSize, T0, T1, T0);
neg(EmitSize, T1, Mask);
orr(EmitSize, Dest, Dest, T0);
lsr(EmitSize, Input, Input, 1);
and_(EmitSize, T0, Mask, T1);
cbnz(EmitSize, T0, &NextBit);
// Setup for first iteration
neg(EmitSize, T0, Mask);
and_(EmitSize, T0, T0, Mask);
// All done with nothing to do.
Bind(&Done);
// Main loop
Bind(&NextBit);
sbfx(EmitSize, T1, Input, 0, 1);
eor(EmitSize, Mask, Mask, T0);
and_(EmitSize, T0, T1, T0);
neg(EmitSize, T1, Mask);
orr(EmitSize, Dest, Dest, T0);
lsr(EmitSize, Input, Input, 1);
and_(EmitSize, T0, Mask, T1);
cbnz(EmitSize, T0, &NextBit);
// All done with nothing to do.
Bind(&Done);
}
}
DEF_OP(PExt) {
@@ -769,35 +904,50 @@ DEF_OP(PExt) {
const auto Mask = GetReg(Op->Mask.ID());
const auto Dest = GetReg(Node);
const auto MaskReg = TMP1;
const auto BitReg = TMP2;
const auto ValueReg = TMP3;
if (CTX->HostFeatures.SupportsSVEBitPerm) {
// SVE added support for PEXT but it needs to be done in a vector register.
if (EmitSize == ARMEmitter::Size::i32Bit) {
fmov(ARMEmitter::Size::i32Bit, VTMP1.S(), Input.W());
fmov(ARMEmitter::Size::i32Bit, VTMP2.S(), Mask.W());
bext(ARMEmitter::SubRegSize::i32Bit, VTMP1.Z(), VTMP1.Z(), VTMP2.Z());
umov<ARMEmitter::SubRegSize::i32Bit>(Dest, VTMP1, 0);
} else {
fmov(ARMEmitter::Size::i64Bit, VTMP1.D(), Input.X());
fmov(ARMEmitter::Size::i64Bit, VTMP2.D(), Mask.X());
bext(ARMEmitter::SubRegSize::i64Bit, VTMP1.Z(), VTMP1.Z(), VTMP2.Z());
umov<ARMEmitter::SubRegSize::i64Bit>(Dest, VTMP1, 0);
}
} else {
const auto MaskReg = TMP1;
const auto BitReg = TMP2;
const auto ValueReg = TMP3;
ARMEmitter::SingleUseForwardLabel EarlyExit;
ARMEmitter::BackwardLabel NextBit;
ARMEmitter::SingleUseForwardLabel Done;
ARMEmitter::SingleUseForwardLabel EarlyExit;
ARMEmitter::BackwardLabel NextBit;
ARMEmitter::SingleUseForwardLabel Done;
cbz(EmitSize, Mask, &EarlyExit);
mov(EmitSize, MaskReg, Mask);
mov(EmitSize, ValueReg, Input);
mov(EmitSize, Dest, ARMEmitter::Reg::zr);
cbz(EmitSize, Mask, &EarlyExit);
mov(EmitSize, MaskReg, Mask);
mov(EmitSize, ValueReg, Input);
mov(EmitSize, Dest, ARMEmitter::Reg::zr);
// Main loop
Bind(&NextBit);
cbz(EmitSize, MaskReg, &Done);
clz(EmitSize, BitReg, MaskReg);
lslv(EmitSize, ValueReg, ValueReg, BitReg);
lslv(EmitSize, MaskReg, MaskReg, BitReg);
extr(EmitSize, Dest, Dest, ValueReg, OpSizeBitsM1);
bfc(EmitSize, MaskReg, OpSizeBitsM1, 1);
b(&NextBit);
// Main loop
Bind(&NextBit);
cbz(EmitSize, MaskReg, &Done);
clz(EmitSize, BitReg, MaskReg);
lslv(EmitSize, ValueReg, ValueReg, BitReg);
lslv(EmitSize, MaskReg, MaskReg, BitReg);
extr(EmitSize, Dest, Dest, ValueReg, OpSizeBitsM1);
bfc(EmitSize, MaskReg, OpSizeBitsM1, 1);
b(&NextBit);
// Early exit
Bind(&EarlyExit);
mov(EmitSize, Dest, ARMEmitter::Reg::zr);
// Early exit
Bind(&EarlyExit);
mov(EmitSize, Dest, ARMEmitter::Reg::zr);
// All done with nothing to do.
Bind(&Done);
// All done with nothing to do.
Bind(&Done);
}
}
DEF_OP(LDiv) {
@@ -1325,11 +1475,6 @@ DEF_OP(Select) {
const auto Src2 = GetReg(Op->Cmp2.ID());
cmp(CompareEmitSize, Src1, Src2);
}
} else if (IsGPRPair(Op->Cmp1.ID())) {
const auto Src1 = GetRegPair(Op->Cmp1.ID());
const auto Src2 = GetRegPair(Op->Cmp2.ID());
cmp(EmitSize, Src1.first, Src2.first);
ccmp(EmitSize, Src1.second, Src2.second, ARMEmitter::StatusFlags::None, cc);
} else if (IsFPR(Op->Cmp1.ID())) {
const auto Src1 = GetVReg(Op->Cmp1.ID());
const auto Src2 = GetVReg(Op->Cmp2.ID());
@@ -1390,6 +1535,12 @@ DEF_OP(NZCVSelect) {
}
}
DEF_OP(NZCVSelectIncrement) {
auto Op = IROp->C<IR::IROp_NZCVSelectIncrement>();
csinc(ConvertSize(IROp), GetReg(Node), GetReg(Op->TrueVal.ID()), GetZeroableReg(Op->FalseVal), MapCC(Op->Cond));
}
DEF_OP(VExtractToGPR) {
const auto Op = IROp->C<IR::IROp_VExtractToGPR>();
const auto OpSize = IROp->Size;
@@ -1400,6 +1551,7 @@ DEF_OP(VExtractToGPR) {
const auto Offset = ElementSizeBits * Op->Index;
const auto Is256Bit = Offset >= SSERegBitSize;
LOGMAN_THROW_A_FMT(!Is256Bit || (Is256Bit && HostSupportsSVE256), "Need SVE256 support in order to use {} with 256-bit operation", __func__);
const auto Dst = GetReg(Node);
const auto Vector = GetVReg(Op->Vector.ID());
@@ -1420,7 +1572,6 @@ DEF_OP(VExtractToGPR) {
// when acting on larger register sizes.
PerformMove(Vector, Op->Index);
} else {
LOGMAN_THROW_AA_FMT(HostSupportsSVE256, "Host doesn't support SVE. Cannot perform 256-bit operation.");
LOGMAN_THROW_AA_FMT(Is256Bit, "Can't perform 256-bit extraction with op side: {}", OpSize);
LOGMAN_THROW_AA_FMT(Offset < AVXRegBitSize, "Trying to extract element outside bounds of register. Offset={}, Index={}", Offset, Op->Index);
@@ -7,7 +7,8 @@ $end_info$
*/
#include "Interface/Context/Context.h"
#include "Interface/Core/JIT/Arm64/JITClass.h"
#include "Interface/HLE/Thunks/Thunks.h"
#include <FEXCore/Core/Thunks.h>
namespace FEXCore::CPU {
@@ -15,19 +15,43 @@ DEF_OP(CASPair) {
auto Op = IROp->C<IR::IROp_CASPair>();
LOGMAN_THROW_AA_FMT(IROp->ElementSize == 4 || IROp->ElementSize == 8, "Wrong element size");
// Size is the size of each pair element
auto Dst = GetRegPair(Node);
auto Expected = GetRegPair(Op->Expected.ID());
auto Desired = GetRegPair(Op->Desired.ID());
auto Dst0 = GetReg(Op->OutLo.ID());
auto Dst1 = GetReg(Op->OutHi.ID());
auto Expected0 = GetReg(Op->ExpectedLo.ID());
auto Expected1 = GetReg(Op->ExpectedHi.ID());
auto Desired0 = GetReg(Op->DesiredLo.ID());
auto Desired1 = GetReg(Op->DesiredHi.ID());
auto MemSrc = GetReg(Op->Addr.ID());
const auto EmitSize = IROp->ElementSize == 8 ? ARMEmitter::Size::i64Bit : ARMEmitter::Size::i32Bit;
if (CTX->HostFeatures.SupportsAtomics) {
mov(EmitSize, TMP3, Expected.first);
mov(EmitSize, TMP4, Expected.second);
// RA has heuristics to try to pair sources, but we need to handle the cases
// where they fail. We do so by moving to temporaries. Note we use 64-bit
// moves here even for 32-bit cmpxchg, for the Firestorm register renamer.
if (Desired1.Idx() != (Desired0.Idx() + 1) || Desired0.Idx() & 1) {
mov(ARMEmitter::Size::i64Bit, TMP1, Desired0);
mov(ARMEmitter::Size::i64Bit, TMP2, Desired1);
Desired0 = TMP1;
Desired1 = TMP2;
}
caspal(EmitSize, TMP3, TMP4, Desired.first, Desired.second, MemSrc);
mov(EmitSize, Dst.first, TMP3.R());
mov(EmitSize, Dst.second, TMP4.R());
auto CaspalDst0 = Dst0;
auto CaspalDst1 = Dst1;
if (CaspalDst1.Idx() != (CaspalDst0.Idx() + 1) || CaspalDst0.Idx() & 1) {
CaspalDst0 = TMP3;
CaspalDst1 = TMP4;
}
// We can't clobber the source, these moves are inherently required due to
// ISA limitations. But by making them 64-bit, Firestorm can rename.
mov(ARMEmitter::Size::i64Bit, CaspalDst0, Expected0);
mov(ARMEmitter::Size::i64Bit, CaspalDst1, Expected1);
caspal(EmitSize, CaspalDst0, CaspalDst1, Desired0, Desired1, MemSrc);
if (CaspalDst0 != Dst0) {
mov(ARMEmitter::Size::i64Bit, Dst0, CaspalDst0);
mov(ARMEmitter::Size::i64Bit, Dst1, CaspalDst1);
}
} else {
// Save NZCV so we don't have to mark this op as clobbering NZCV (the
// SupportsAtomics does not clobber atomics and this !SupportsAtomics path
@@ -43,19 +67,19 @@ DEF_OP(CASPair) {
// This instruction sequence must be synced with HandleCASPAL_Armv8.
ldaxp(EmitSize, TMP2, TMP3, MemSrc);
cmp(EmitSize, TMP2, Expected.first);
ccmp(EmitSize, TMP3, Expected.second, ARMEmitter::StatusFlags::None, ARMEmitter::Condition::CC_EQ);
cmp(EmitSize, TMP2, Expected0);
ccmp(EmitSize, TMP3, Expected1, ARMEmitter::StatusFlags::None, ARMEmitter::Condition::CC_EQ);
b(ARMEmitter::Condition::CC_NE, &LoopNotExpected);
stlxp(EmitSize, TMP2, Desired.first, Desired.second, MemSrc);
stlxp(EmitSize, TMP2, Desired0, Desired1, MemSrc);
cbnz(EmitSize, TMP2, &LoopTop);
mov(EmitSize, Dst.first, Expected.first);
mov(EmitSize, Dst.second, Expected.second);
mov(EmitSize, Dst0, Expected0);
mov(EmitSize, Dst1, Expected1);
b(&LoopExpected);
Bind(&LoopNotExpected);
mov(EmitSize, Dst.first, TMP2.R());
mov(EmitSize, Dst.second, TMP3.R());
mov(EmitSize, Dst0, TMP2.R());
mov(EmitSize, Dst1, TMP3.R());
// exclusive monitor needs to be cleared here
// Might have hit the case where ldaxr was hit but stlxr wasn't
clrex();
@@ -11,11 +11,11 @@ $end_info$
#include "Interface/Core/JIT/Arm64/JITClass.h"
#include <FEXCore/Core/Thunks.h>
#include <FEXCore/Core/X86Enums.h>
#include <FEXCore/Debug/InternalThreadState.h>
#include <FEXCore/HLE/SyscallHandler.h>
#include <FEXCore/Utils/MathUtils.h>
#include <Interface/HLE/Thunks/Thunks.h>
namespace FEXCore::CPU {
#define DEF_OP(x) void Arm64JITCore::Op_##x(IR::IROp_Header const* IROp, IR::NodeID Node)
@@ -78,8 +78,12 @@ DEF_OP(ExitFunction) {
// L1 Cache
ldr(TMP1, STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.L1Pointer));
and_(ARMEmitter::Size::i64Bit, TMP4, RipReg, LookupCache::L1_ENTRIES_MASK);
add(TMP1, TMP1, TMP4, ARMEmitter::ShiftType::LSL, 4);
// Calculate (tmp1 + ((ripreg & L1_ENTRIES_MASK) << 4)) for the address
// arithmetic. ubfiz+add is marginally faster on Firestorm than
// and+add(shift). Same performance on Cortex.
static_assert(LookupCache::L1_ENTRIES_MASK == ((1u << 20) - 1));
ubfiz(ARMEmitter::Size::i64Bit, TMP4, RipReg, 4, 20);
add(TMP1, TMP1, TMP4);
// Note: sub+cbnz used over cmp+br to preserve flags.
ldp<ARMEmitter::IndexType::OFFSET>(TMP2, TMP1, TMP1, 0);
@@ -112,20 +116,27 @@ DEF_OP(CondJump) {
[[maybe_unused]] uint64_t Const;
[[maybe_unused]] const bool isConst = IsInlineConstant(Op->Cmp2, &Const);
auto Reg = GetReg(Op->Cmp1.ID());
const auto Size = Op->CompareSize == 4 ? ARMEmitter::Size::i32Bit : ARMEmitter::Size::i64Bit;
LOGMAN_THROW_A_FMT(IsGPR(Op->Cmp1.ID()), "CondJump: Expected GPR");
LOGMAN_THROW_A_FMT(isConst && Const == 0, "CondJump: Expected 0 source");
LOGMAN_THROW_A_FMT(Op->Cond.Val == FEXCore::IR::COND_EQ || Op->Cond.Val == FEXCore::IR::COND_NEQ, "CondJump: Expected simple "
"condition");
LOGMAN_THROW_A_FMT(isConst, "CondJump: Expected constant source");
if (Op->Cond.Val == FEXCore::IR::COND_EQ) {
cbz(Size, GetReg(Op->Cmp1.ID()), TrueTargetLabel);
LOGMAN_THROW_A_FMT(Const == 0, "CondJump: Expected 0 source");
cbz(Size, Reg, TrueTargetLabel);
} else if (Op->Cond.Val == FEXCore::IR::COND_NEQ) {
LOGMAN_THROW_A_FMT(Const == 0, "CondJump: Expected 0 source");
cbnz(Size, Reg, TrueTargetLabel);
} else if (Op->Cond.Val == FEXCore::IR::COND_TSTZ) {
LOGMAN_THROW_A_FMT(Const < 64, "CondJump: Expected valid bit source");
tbz(Reg, Const, TrueTargetLabel);
} else if (Op->Cond.Val == FEXCore::IR::COND_TSTNZ) {
LOGMAN_THROW_A_FMT(Const < 64, "CondJump: Expected valid bit source");
tbnz(Reg, Const, TrueTargetLabel);
} else {
cbnz(Size, GetReg(Op->Cmp1.ID()), TrueTargetLabel);
LOGMAN_THROW_A_FMT(false, "CondJump expected simple condition");
}
// TODO: Wire up tbz/tbnz
}
PendingTargetLabel = &JumpTargets.try_emplace(Op->FalseBlock.ID()).first->second;
@@ -184,7 +195,7 @@ DEF_OP(Syscall) {
if ((Flags & FEXCore::IR::SyscallFlags::NORETURN) != FEXCore::IR::SyscallFlags::NORETURN) {
// Result is now in x0
// Fix the stack and any values that were stepped on
FillStaticRegs(true, GPRSpillMask, FPRSpillMask);
FillStaticRegs(true, GPRSpillMask, FPRSpillMask, ARMEmitter::Reg::r1, ARMEmitter::Reg::r2);
// Now the registers we've spilled are back in their original host registers
// We can safely claim we are no longer in a syscall
@@ -255,15 +266,13 @@ DEF_OP(InlineSyscall) {
}
auto Reg = GetReg(Op->Header.Args[i].ID());
// In the case of intersection with x4, x5, or x8 then these are currently SRA
// for registers RAX, RBX, and RSI. Which have just been spilled
// Just load back from the context. Could be slightly smarter but this is fairly uncommon
if (Reg == ARMEmitter::Reg::r8) {
ldr(EmitSubSize, RegArgs[i].R(), STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[X86State::REG_RSP]));
} else if (Reg == ARMEmitter::Reg::r4) {
ldr(EmitSubSize, RegArgs[i].R(), STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[X86State::REG_RAX]));
} else if (Reg == ARMEmitter::Reg::r5) {
ldr(EmitSubSize, RegArgs[i].R(), STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[X86State::REG_RCX]));
if (SpillMask & (1U << Reg.Idx())) {
// In the case of intersection with x4, x5, or x8 then these are currently SRA
// for registers RAX, RDX, and RSP. Which have just been spilled
// Just load back from the context.
auto Correlation = GetX86RegRelationToARMReg(Reg);
LOGMAN_THROW_A_FMT(Correlation != X86State::REG_INVALID, "Invalid register mapping");
ldr(EmitSubSize, RegArgs[i].R(), STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[Correlation]));
} else {
mov(EmitSize, RegArgs[i].R(), Reg);
}
@@ -285,7 +294,7 @@ DEF_OP(InlineSyscall) {
if ((Op->Flags & FEXCore::IR::SyscallFlags::NORETURN) != FEXCore::IR::SyscallFlags::NORETURN) {
// Now that we are done in the syscall we need to carefully peel back the state
// First unspill the registers from before
FillStaticRegs(false, SpillMask);
FillStaticRegs(false, SpillMask, ~0U, ARMEmitter::Reg::r8, ARMEmitter::Reg::r1);
// Now the registers we've spilled are back in their original host registers
// We can safely claim we are no longer in a syscall
@@ -427,10 +436,11 @@ DEF_OP(CPUID) {
PopDynamicRegsAndLR();
// Results are in x0, x1
// Results want to be in a i64v2 vector
auto Dst = GetRegPair(Node);
mov(ARMEmitter::Size::i64Bit, Dst.first, TMP1);
mov(ARMEmitter::Size::i64Bit, Dst.second, TMP2);
// Results want to be 4xi32 scalars
mov(ARMEmitter::Size::i32Bit, GetReg(Op->OutEAX.ID()), TMP1);
mov(ARMEmitter::Size::i32Bit, GetReg(Op->OutECX.ID()), TMP2);
ubfx(ARMEmitter::Size::i64Bit, GetReg(Op->OutEBX.ID()), TMP1, 32, 32);
ubfx(ARMEmitter::Size::i64Bit, GetReg(Op->OutEDX.ID()), TMP2, 32, 32);
}
DEF_OP(XGetBV) {
@@ -459,11 +469,9 @@ DEF_OP(XGetBV) {
PopDynamicRegsAndLR();
// Results are in x0
// Results want to be in a i32v2 vector
auto Dst = GetRegPair(Node);
mov(ARMEmitter::Size::i32Bit, Dst.first, TMP1);
lsr(ARMEmitter::Size::i64Bit, Dst.second, TMP1, 32);
// Results are in x0, need to split into i32 parts
mov(ARMEmitter::Size::i32Bit, GetReg(Op->OutEAX.ID()), TMP1);
ubfx(ARMEmitter::Size::i64Bit, GetReg(Op->OutEDX.ID()), TMP1, 32, 32);
}
#undef DEF_OP
@@ -16,6 +16,7 @@ DEF_OP(VInsGPR) {
const auto DestIdx = Op->DestIdx;
const auto ElementSize = Op->Header.ElementSize;
const auto Is256Bit = OpSize == Core::CPUState::XMM_AVX_REG_SIZE;
LOGMAN_THROW_A_FMT(!Is256Bit || (Is256Bit && HostSupportsSVE256), "Need SVE256 support in order to use {} with 256-bit operation", __func__);
const auto SubEmitSize = ConvertSubRegSize8(IROp);
const auto ElementsPer128Bit = 16 / ElementSize;
@@ -111,6 +112,8 @@ DEF_OP(VDupFromGPR) {
const auto Src = GetReg(Op->Src.ID());
const auto Is256Bit = OpSize == Core::CPUState::XMM_AVX_REG_SIZE;
LOGMAN_THROW_A_FMT(!Is256Bit || (Is256Bit && HostSupportsSVE256), "Need SVE256 support in order to use {} with 256-bit operation", __func__);
const auto SubEmitSize = ConvertSubRegSize8(IROp);
if (HostSupportsSVE256 && Is256Bit) {
@@ -203,6 +206,7 @@ DEF_OP(Vector_SToF) {
const auto ElementSize = Op->Header.ElementSize;
const auto SubEmitSize = ConvertSubRegSize248(IROp);
const auto Is256Bit = OpSize == Core::CPUState::XMM_AVX_REG_SIZE;
LOGMAN_THROW_A_FMT(!Is256Bit || (Is256Bit && HostSupportsSVE256), "Need SVE256 support in order to use {} with 256-bit operation", __func__);
const auto Dst = GetVReg(Node);
const auto Vector = GetVReg(Op->Vector.ID());
@@ -235,6 +239,7 @@ DEF_OP(Vector_FToZS) {
const auto ElementSize = Op->Header.ElementSize;
const auto SubEmitSize = ConvertSubRegSize248(IROp);
const auto Is256Bit = OpSize == Core::CPUState::XMM_AVX_REG_SIZE;
LOGMAN_THROW_A_FMT(!Is256Bit || (Is256Bit && HostSupportsSVE256), "Need SVE256 support in order to use {} with 256-bit operation", __func__);
const auto Dst = GetVReg(Node);
const auto Vector = GetVReg(Op->Vector.ID());
@@ -265,6 +270,8 @@ DEF_OP(Vector_FToS) {
const auto OpSize = IROp->Size;
const auto Is256Bit = OpSize == Core::CPUState::XMM_AVX_REG_SIZE;
LOGMAN_THROW_A_FMT(!Is256Bit || (Is256Bit && HostSupportsSVE256), "Need SVE256 support in order to use {} with 256-bit operation", __func__);
const auto SubEmitSize = ConvertSubRegSize248(IROp);
const auto Dst = GetVReg(Node);
@@ -294,6 +301,8 @@ DEF_OP(Vector_FToF) {
const auto ElementSize = Op->Header.ElementSize;
const auto SubEmitSize = ConvertSubRegSize248(IROp);
const auto Is256Bit = OpSize == Core::CPUState::XMM_AVX_REG_SIZE;
LOGMAN_THROW_A_FMT(!Is256Bit || (Is256Bit && HostSupportsSVE256), "Need SVE256 support in order to use {} with 256-bit operation", __func__);
const auto Conv = (ElementSize << 8) | Op->SrcElementSize;
const auto Dst = GetVReg(Node);
@@ -395,6 +404,7 @@ DEF_OP(Vector_FToI) {
const auto ElementSize = Op->Header.ElementSize;
const auto SubEmitSize = ConvertSubRegSize248(IROp);
const auto Is256Bit = OpSize == Core::CPUState::XMM_AVX_REG_SIZE;
LOGMAN_THROW_A_FMT(!Is256Bit || (Is256Bit && HostSupportsSVE256), "Need SVE256 support in order to use {} with 256-bit operation", __func__);
const auto Dst = GetVReg(Node);
const auto Vector = GetVReg(Op->Vector.ID());
@@ -449,5 +459,64 @@ DEF_OP(Vector_FToI) {
}
}
DEF_OP(Vector_F64ToI32) {
const auto Op = IROp->C<IR::IROp_Vector_F64ToI32>();
const auto OpSize = IROp->Size;
const auto Round = Op->Round;
const auto Is256Bit = OpSize == Core::CPUState::XMM_AVX_REG_SIZE;
LOGMAN_THROW_A_FMT(!Is256Bit || (Is256Bit && HostSupportsSVE256), "Need SVE256 support in order to use {} with 256-bit operation", __func__);
const auto Dst = GetVReg(Node);
const auto Vector = GetVReg(Op->Vector.ID());
if (HostSupportsSVE128 || HostSupportsSVE256) {
const auto Mask = Is256Bit ? PRED_TMP_32B.Merging() : PRED_TMP_16B.Merging();
// First step is to round the f64 values to integrals (frint*)
// Then convert to integers using fcvtzs.
auto CVTReg = Dst.Z();
switch (Round) {
case IR::Round_Nearest.Val: frintn(ARMEmitter::SubRegSize::i64Bit, Dst.Z(), Mask, Vector.Z()); break;
case IR::Round_Negative_Infinity.Val: frintm(ARMEmitter::SubRegSize::i64Bit, Dst.Z(), Mask, Vector.Z()); break;
case IR::Round_Positive_Infinity.Val: frintp(ARMEmitter::SubRegSize::i64Bit, Dst.Z(), Mask, Vector.Z()); break;
case IR::Round_Towards_Zero.Val: CVTReg = Vector.Z(); break;
case IR::Round_Host.Val: frinti(ARMEmitter::SubRegSize::i64Bit, Dst.Z(), Mask, Vector.Z()); break;
}
fcvtzs(Dst.Z(), ARMEmitter::SubRegSize::i32Bit, Mask, CVTReg, ARMEmitter::SubRegSize::i64Bit);
///< Fixup format of register that fcvtzs returns.
uzp1(ARMEmitter::SubRegSize::i32Bit, Dst.Z(), Dst.Z(), Dst.Z());
if (Op->EnsureZeroUpperHalf) {
///< Match CVTPD2DQ/CVTTPD2DQ behaviour if necessary by zeroing the upper bits here.
if (Is256Bit) {
mov(Dst.Q(), Dst.Q());
} else {
mov(Dst.D(), Dst.D());
}
}
} else {
// This has a known precision issue that isn't easily resolvable without throwing away performance.
// Doing the conversion in multi-stage steps has an issue that you can lose precision in the f32->i32 step if your source was f64.
// To get around this with ASIMD FEX needs to use fcvtzs (Scalar, Integer, to GPR) for each F64 to be directly converted to i32.
// This is a very costly transform that the SVE path doesn't need to do since it supports f64->i32 directly.
// If this precision issue is necessary then we can add an option for it in the future.
///< Round float to integral depending on rounding mode.
switch (Round) {
case FEXCore::IR::Round_Nearest.Val: frintn(ARMEmitter::SubRegSize::i64Bit, Dst.Q(), Vector.Q()); break;
case FEXCore::IR::Round_Negative_Infinity.Val: frintm(ARMEmitter::SubRegSize::i64Bit, Dst.Q(), Vector.Q()); break;
case FEXCore::IR::Round_Positive_Infinity.Val: frintp(ARMEmitter::SubRegSize::i64Bit, Dst.Q(), Vector.Q()); break;
case FEXCore::IR::Round_Towards_Zero.Val: frintz(ARMEmitter::SubRegSize::i64Bit, Dst.Q(), Vector.Q()); break;
case FEXCore::IR::Round_Host.Val: frinti(ARMEmitter::SubRegSize::i64Bit, Dst.Q(), Vector.Q()); break;
}
// Now narrow from f64 to f32.
fcvtn(ARMEmitter::SubRegSize::i32Bit, Dst.Q(), Dst.Q());
///< Convert the two F32 integrals to real integers.
fcvtzs(ARMEmitter::SubRegSize::i32Bit, Dst.D(), Dst.D());
}
}
#undef DEF_OP
} // namespace FEXCore::CPU
@@ -1,18 +0,0 @@
// SPDX-License-Identifier: MIT
/*
$info$
tags: backend|arm64
$end_info$
*/
#include "Interface/Core/JIT/Arm64/JITClass.h"
namespace FEXCore::CPU {
#define DEF_OP(x) void Arm64JITCore::Op_##x(IR::IROp_Header const* IROp, IR::NodeID Node)
DEF_OP(GetHostFlag) {
auto Op = IROp->C<IR::IROp_GetHostFlag>();
ubfx(ARMEmitter::Size::i64Bit, GetReg(Node), GetReg(Op->Value.ID()), Op->Flag, 1);
}
#undef DEF_OP
} // namespace FEXCore::CPU
+17 -20
View File
@@ -47,8 +47,8 @@ static uint64_t LUDIV(uint64_t SrcHigh, uint64_t SrcLow, uint64_t Divisor) {
return Res;
}
static int64_t LDIV(int64_t SrcHigh, int64_t SrcLow, int64_t Divisor) {
__int128_t Source = (static_cast<__int128_t>(SrcHigh) << 64) | SrcLow;
static int64_t LDIV(uint64_t SrcHigh, uint64_t SrcLow, int64_t Divisor) {
__int128_t Source = (static_cast<__uint128_t>(SrcHigh) << 64) | SrcLow;
__int128_t Res = Source / Divisor;
return Res;
}
@@ -59,8 +59,8 @@ static uint64_t LUREM(uint64_t SrcHigh, uint64_t SrcLow, uint64_t Divisor) {
return Res;
}
static int64_t LREM(int64_t SrcHigh, int64_t SrcLow, int64_t Divisor) {
__int128_t Source = (static_cast<__int128_t>(SrcHigh) << 64) | SrcLow;
static int64_t LREM(uint64_t SrcHigh, uint64_t SrcLow, int64_t Divisor) {
__int128_t Source = (static_cast<__uint128_t>(SrcHigh) << 64) | SrcLow;
__int128_t Res = Source % Divisor;
return Res;
}
@@ -496,7 +496,7 @@ static uint64_t Arm64JITCore_ExitFunctionLink(FEXCore::Core::CpuStateFrame* Fram
uintptr_t branch = (uintptr_t)(Record)-8;
auto offset = HostCode / 4 - branch / 4;
if (vixl::IsInt26(offset)) {
if (ARMEmitter::Emitter::IsInt26(offset)) {
// optimal case - can branch directly
// patch the code
ARMEmitter::Emitter emit((uint8_t*)(branch), 4);
@@ -654,12 +654,6 @@ bool Arm64JITCore::IsGPR(IR::NodeID Node) const {
return Class == IR::GPRClass || Class == IR::GPRFixedClass;
}
bool Arm64JITCore::IsGPRPair(IR::NodeID Node) const {
auto Class = GetRegClass(Node);
return Class == IR::GPRPairClass;
}
CPUBackend::CompiledCode Arm64JITCore::CompileCode(uint64_t Entry, const FEXCore::IR::IRListView* IR, FEXCore::Core::DebugData* DebugData,
const FEXCore::IR::RegisterAllocationData* RAData) {
FEXCORE_PROFILE_SCOPED("Arm64::CompileCode");
@@ -724,12 +718,22 @@ CPUBackend::CompiledCode Arm64JITCore::CompileCode(uint64_t Entry, const FEXCore
offsetof(FEXCore::Core::InternalThreadState, InterruptFaultPage) - offsetof(FEXCore::Core::InternalThreadState, BaseFrameState));
}
#ifdef _M_ARM_64EC
static constexpr uint16_t SuspendMagic {0xCAFE};
ldr(TMP2.W(), STATE_PTR(CpuStateFrame, SuspendDoorbell));
ARMEmitter::SingleUseForwardLabel l_NoSuspend;
cbz(ARMEmitter::Size::i32Bit, TMP2, &l_NoSuspend);
brk(SuspendMagic);
Bind(&l_NoSuspend);
#endif
SpillSlots = RAData->SpillSlots();
if (SpillSlots) {
const auto TotalSpillSlotsSize = SpillSlots * MaxSpillSlotSize;
if (vixl::aarch64::Assembler::IsImmAddSub(TotalSpillSlotsSize)) {
if (ARMEmitter::IsImmAddSub(TotalSpillSlotsSize)) {
sub(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::rsp, ARMEmitter::Reg::rsp, TotalSpillSlotsSize);
} else {
LoadConstant(ARMEmitter::Size::i64Bit, TMP1, TotalSpillSlotsSize);
@@ -872,7 +876,7 @@ void Arm64JITCore::ResetStack() {
const auto TotalSpillSlotsSize = SpillSlots * MaxSpillSlotSize;
if (vixl::aarch64::Assembler::IsImmAddSub(TotalSpillSlotsSize)) {
if (ARMEmitter::IsImmAddSub(TotalSpillSlotsSize)) {
add(ARMEmitter::Size::i64Bit, ARMEmitter::Reg::rsp, ARMEmitter::Reg::rsp, TotalSpillSlotsSize);
} else {
// Too big to fit in a 12bit immediate
@@ -885,11 +889,4 @@ fextl::unique_ptr<CPUBackend> CreateArm64JITCore(FEXCore::Context::ContextImpl*
return fextl::make_unique<Arm64JITCore>(ctx, Thread);
}
CPUBackendFeatures GetArm64JITBackendFeatures() {
return CPUBackendFeatures {
.SupportsFlags = true,
.SupportsVTBL2 = true,
};
}
} // namespace FEXCore::CPU
@@ -14,9 +14,6 @@ $end_info$
#include "Interface/IR/IntrusiveIRList.h"
#include "Interface/IR/RegisterAllocationData.h"
#include <aarch64/assembler-aarch64.h>
#include <aarch64/disasm-aarch64.h>
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/IR/IR.h>
#include <FEXCore/fextl/map.h>
@@ -40,25 +37,10 @@ public:
explicit Arm64JITCore(FEXCore::Context::ContextImpl* ctx, FEXCore::Core::InternalThreadState* Thread);
~Arm64JITCore() override;
[[nodiscard]]
fextl::string GetName() override {
return "JIT";
}
[[nodiscard]]
CPUBackend::CompiledCode CompileCode(uint64_t Entry, const FEXCore::IR::IRListView* IR, FEXCore::Core::DebugData* DebugData,
const FEXCore::IR::RegisterAllocationData* RAData) override;
[[nodiscard]]
void* MapRegion(void* HostPtr, uint64_t, uint64_t) override {
return HostPtr;
}
[[nodiscard]]
bool NeedsOpDispatch() override {
return true;
}
void ClearCache() override;
void ClearRelocations() override {
@@ -67,8 +49,6 @@ public:
private:
FEX_CONFIG_OPT(ParanoidTSO, PARANOIDTSO);
FEX_CONFIG_OPT(VectorTSOEnabled, VECTORTSOENABLED);
FEX_CONFIG_OPT(MemcpySetTSOEnabled, MEMCPYSETTSOENABLED);
const bool HostSupportsSVE128 {};
const bool HostSupportsSVE256 {};
@@ -114,15 +94,6 @@ private:
FEX_UNREACHABLE;
}
[[nodiscard]]
std::pair<ARMEmitter::Register, ARMEmitter::Register> GetRegPair(IR::NodeID Node) const {
const auto Reg = GetPhys(Node);
LOGMAN_THROW_AA_FMT(Reg.Class == IR::GPRPairClass.Val, "Unexpected Class: {}", Reg.Class);
return std::make_pair(GeneralRegisters[Reg.Reg], GeneralRegisters[Reg.Reg + 1]);
}
[[nodiscard]]
FEXCore::IR::RegisterClassType GetRegClass(IR::NodeID Node) const;
@@ -253,8 +224,6 @@ private:
bool IsFPR(IR::NodeID Node) const;
[[nodiscard]]
bool IsGPR(IR::NodeID Node) const;
[[nodiscard]]
bool IsGPRPair(IR::NodeID Node) const;
[[nodiscard]]
ARMEmitter::ExtendedMemOperand GenerateMemOperand(uint8_t AccessSize, ARMEmitter::Register Base, IR::OrderedNodeWrapper Offset,
@@ -345,6 +314,11 @@ private:
uint32_t SpillSlots {};
using OpType = void (Arm64JITCore::*)(const IR::IROp_Header* IROp, IR::NodeID Node);
using ScalarFMAOpCaller =
std::function<void(ARMEmitter::VRegister Dst, ARMEmitter::VRegister Src1, ARMEmitter::VRegister Src2, ARMEmitter::VRegister Src3)>;
void VFScalarFMAOperation(uint8_t OpSize, uint8_t ElementSize, ScalarFMAOpCaller ScalarEmit, ARMEmitter::VRegister Dst,
ARMEmitter::VRegister Upper, ARMEmitter::VRegister Vector1, ARMEmitter::VRegister Vector2,
ARMEmitter::VRegister Addend);
using ScalarBinaryOpCaller = std::function<void(ARMEmitter::VRegister Dst, ARMEmitter::VRegister Src1, ARMEmitter::VRegister Src2)>;
void VFScalarOperation(uint8_t OpSize, uint8_t ElementSize, bool ZeroUpperBits, ScalarBinaryOpCaller ScalarEmit,
ARMEmitter::VRegister Dst, ARMEmitter::VRegister Vector1, ARMEmitter::VRegister Vector2);
@@ -352,6 +326,10 @@ private:
void VFScalarUnaryOperation(uint8_t OpSize, uint8_t ElementSize, bool ZeroUpperBits, ScalarUnaryOpCaller ScalarEmit, ARMEmitter::VRegister Dst,
ARMEmitter::VRegister Vector1, std::variant<ARMEmitter::VRegister, ARMEmitter::Register> Vector2);
void Emulate128BitGather(size_t Size, size_t ElementSize, ARMEmitter::VRegister Dst, ARMEmitter::VRegister IncomingDst,
std::optional<ARMEmitter::Register> BaseAddr, ARMEmitter::VRegister VectorIndexLow,
std::optional<ARMEmitter::VRegister> VectorIndexHigh, ARMEmitter::VRegister MaskReg, size_t VectorIndexSize,
size_t DataElementOffsetStart, size_t IndexElementOffsetStart, uint8_t OffsetScale);
// Runtime selection;
// Load and store TSO memory style
OpType RT_LoadMemTSO;
Loaded 100 of 492 files, more files were not shown because too many files have changed in this diff. Show more