Commit Graph
178 Commits
Author SHA1 Message Date
Ryan Houdek 69013772c1 StructVerifier: Fixes CI failure
The x86 runner had unattended-upgrades accidentally still enabled. It
upgraded a bunch of development packages which broke CI.

Fix the struct verifier so it works with the new packages.
Sadly python3-clang doesn't support all the new CursorKind types so we
need to self-define some of them for now.

Once this tool gets converted over to C++ it will be a non-issue.
2022-09-16 19:45:13 -07:00
Ryan Houdek 9fac1b8105 CI: Adds support for flakes
If a test is marked as a flake then it will be tried five times before
giving up.

Works around the problem of needing to babysit CI once a PR is pushed.
As long as we have all the flake tests marked.
2022-09-15 11:44:56 -07:00
Ryan Houdek 121f0a2c6c CI: FetchRootFS More robust rootfs fetching
Permissions mean we need to delete the folder before extracting.
On error make sure to delete the image file as well to ensure it reruns
everything.
2022-09-14 13:18:24 -07:00
Ryan Houdek 8ba0312d40 Syscalls: Prefix _ to shm syscall names to avoid namespace conflict
Termux uses defines for these, so our token pasting fails, but we also
still want to use their define so we can fall down their emulation
library whenever possible.

Prefix an underscore to be able to use both our number definitions and
their defines in the same file.
2022-09-05 02:39:42 -07:00
Ryan Houdek cc7fb008fc New domain.
Needed to fix FEXRootFSFetcher from #1967
2022-09-02 10:43:07 -07:00
Ryan Houdek edad24479b unittests: Support skipping unit tests based on host feature support
For these unit tests we no longer need to put them in the disabled tests
file. Instead it will be skipped if the host doesn't support the feature
required.
2022-08-14 20:04:26 -07:00
lioncash 5e0205378b Allow skipping tests based on desired host features
Necessary for tests that depend on the state of the running context.

Since we support an SSE mode and an AVX mode, the FPR store truncate
test will fail on hosts that don't support AVX as the register offsets
are going to be different between the two. So we can conditionally
enable support for these tests.
2022-07-26 16:56:57 -04:00
Ryan Houdek 84379b5fdf CMake: Check for binfmt_misc conflicts before install
Check for qemu and box binfmt_misc file conflicts before the
`binfmt_misc` install command.

This ensures if you're building from source that you won't inadvertently
install conflicting binfmt_misc files, breaking program execution.
2022-07-01 13:42:45 -07:00
Ryan Houdek 8b35275ec1 unittests: Classify CPU based on CPU features
Instead of relying on runner features, classify based on CPU features.

This fixes an annoying issue where if running unit tests locally without
it set then you get an unexpected failure.

Fixes #1807
2022-06-30 13:55:38 -07:00
Stefanos Kornilios Misis Poiitidis 19fbc10c16 TestHarnessRunner: Don't setup a FEX Context when running host tests 2022-06-16 19:41:42 +00:00
Ryan Houdek c0a8984799 Merge pull request #1764 from Sonicadvance1/fix_xxhash
FEXRootFSFetcher: Update and fix xxhash file hashing
2022-06-10 04:57:09 -07:00
Stefanos Kornilios Misis Poiitidis 40ec910108 unittests: Add FEXLinuxTests, with a few signal and smc tests 2022-06-10 08:34:40 +03:00
Ryan Houdek bc2840e4a7 FEXRootFSFetcher: Update and fix xxhash file hashing
The final tail of the file reading was incorrect, so our hashing was
"correct" but it was using stale data from the previous block size read.

Noticed this while wiring up the CI rootfs fetching since the hashing is
a lot simpler there.

Now instead of reading a tail, just attempt to read the full block size
and use the resulting data size instead. Confirmed it matches expected
results now.

In the process we are going to need to update hyperlinks and hashes
anyway, change the hash to XXH3 so it is faster to run.
2022-06-09 20:20:43 -07:00
Ryan Houdek d2a57f6231 Scripts: Add CI rootfs fetch script
This will allow our CI system to automatically pull their rootfs.
2022-06-09 20:08:24 -07:00
Ryan Houdek 952e157770 Scripts: Allow user override on tagged version
In the case of a missing month or a minor version, need to allow user
defined overrides.
2022-06-04 18:06:28 -07:00
Ryan Houdek 82319c9deb Scripts: Updates generate syscall numbers to support renaming
Instead of manually renaming the three syscalls each time, let the
script do it automatically.
2022-04-23 11:56:33 -07:00
Ryan Houdek fba698cb74 Scripts: Updates AArch64 fit for Clang 14
Clang now supports these latest ARMv9 CPUs
2022-04-01 18:08:02 -07:00
Ryan Houdek 9911fe68d4 Scripts: Stop using deprecated Distutils
According to PEP 386: https://www.python.org/dev/peps/pep-0386/

distutils is deprecated and will be removed in an upcoming python
version.

Switch over to pkg_resources for version parsing and comparison
2022-03-01 07:10:41 -08:00
Ryan Houdek 2af23d9bec Merge pull request #1591 from Sonicadvance1/new_cpus_in_native_fit
Scripts: Updates CPU fitting script for latest CPUs
2022-02-28 07:05:45 -08:00
Ryan Houdek bb7fa84fb8 Scripts: Updates CPU fitting script for latest CPUs
Clang-13 doesn't yet understand the latest ARM CPUs so just document them
and set to the closest thing.
2022-02-26 02:14:52 -08:00
Ryan Houdek 2b8f60c108 TestHarness: Support for asm files having the option to set config options
Allows some something like the following:
"Env": {
  "FEX_MAXINST": "500"
}

Not that I would recommend overriding MAXINST in the asm tests, as
command line overrides that
2022-02-21 14:53:27 -08:00
Ryan Houdek 228aed98c7 unittests: Fixes ROOTFS needing to be defined prior to cmake
cmake will bake in the environment variable in to the build scripts.
Instead have the guest_test_runner fetch it at runtime.

This means if you forget to set ROOTFS prior to running cmake, you can
now set it afterwards and rerun with just ctest instead of a cmake
dance.

Fixes #315
2022-01-09 01:56:13 -08:00
Ryan Houdek f836ff0897 Scripts: Adds a python script that can hand hold a user through FEX install
This is definitely a bit divisive but overall this is a win.
This is a user pattern that is emerging in a bunch of projects.
Allow an officially sourced script that lets you pipe a script directly
in to python/bash and setup the environment entirely.

This only supports Ubuntu {20.04, 21.04, 21.10, 22.04} which matches
exactly what we expose in the PPA.

Once this is in the repo, and our PPA is updated to the latest release
tag you can run this script like:
`curl --silent <Direct Github raw link> | python3`

Once the PPA is updated, the README and Wiki will be updated with Quick
Start guides to use this path.

The script steps
1) Checks if ARMv8, or fail
2) Checks if supported Ubuntu, or fail
3) Checks if PPA installed
3a) Install PPA if not, or fail
4) Check if packages are installed
4a) Install non-installed packages, or fail
5) Check if RootFS is configured/exists
5a) Run through FEXRootFSFetcher to get/setup rootfs, or fail
6) Attempt to run emulated uname -a through FEX, or fail
7) Provide some examples for how to use FEX
8) Exit with success!
2022-01-03 14:20:09 -08:00
Ryan Houdek 33be4fa98d Scripts: Updates syscall definition extractor
We were missing one of the definition types which means we missed
definitions
2021-11-22 14:03:28 -08:00
Ryan Houdek aae1dd4d81 Adds new script to generate syscall enums
Lets us define our own enums for syscall mapping.
Necessary to get all the syscall IDs in a sane way
2021-11-15 14:29:49 -08:00
Ryan Houdek 0bfc1bbe70 Arm64: Don't fall back to native
In the case of Arm64, make sure not to fallback to native if we hit an
unsupported CPU.
Can cause issues depending on system configuration.
2021-10-21 20:39:49 -07:00
Ryan Houdek 2b757a9b9a Merge pull request #1284 from Sonicadvance1/fix_struct_match
StructVerifier: Fix struct match and minor fixes
2021-10-02 11:07:51 -07:00
Ryan Houdek 1b0d2bbf9f Scripts: Adds a new script for extracting function definitions
This is very useful for extracting function definitions for thunks.
Keep it in upstream to not get lost.

Sometimes it can munge a definition but it is usually fine.
2021-09-30 18:35:43 -07:00
Ryan Houdek 5b2d944886 Linux: Minor struct verifier and types fixes
fex-match annotation was actually not doing anything due to python typo.
Fixes the minor warnings that cropped up. Nothing actually broken
2021-09-30 18:08:05 -07:00
Ryan Houdek 083d3a464a StructPackVerifier: Add a couple missing defines 2021-09-03 16:46:16 -07:00
Ryan Houdek fd6d33d197 Merge pull request #996 from FEX-Emu/skmp/aotirgen
AOTIR: Offline Generation
2021-05-03 15:58:57 -07:00
Stefanos Kornilios Mitsis Poiitidis be41f14452 Fix generation script 2021-05-03 18:16:43 +03:00
Stefanos Kornilios Mitsis Poiitidis cf5bd202d4 AOTIR: Update gen script to default to global fex installation 2021-05-03 06:15:25 +03:00
Ryan Houdek 8d34222556 Add vardecl type matching for ioctl verification 2021-05-01 07:39:14 -07:00
Stefanos Kornilios Mitsis Poiitidis 9a4fc758c1 AOTIR: FEXUpdateAOTIRCache.sh now passes compilation modifiers 2021-05-01 09:40:50 +03:00
Stefanos Kornilios Mitsis Poiitidis 4bd997adaa AOT: Postfix ir files with .aotir 2021-05-01 09:40:50 +03:00
Stefanos Kornilios Mitsis Poiitidis 244ef94e9b AOTIR: Generate list of files with code, add Scripts/FEXUpdateAOTIRCache.sh to generate all files 2021-05-01 09:40:50 +03:00
Ryan Houdek b06ebf486f Work around M1 Parallels hypervisor not showing CPU type
Parallels claims that the CPU is of implementor 0x41 and part number 0.
Easy enough to work around before real numbers get exposed
2021-04-27 22:26:35 -07:00
Ryan Houdek 8adb1ca3a2 Fixes ordering problem of tag generation in release script
Wasn't actually creating an annotated tag with this order
2021-04-05 18:35:09 -07:00
Ryan Houdek 26d9ddef9f Updates release script to match what I want it to be
Generates the current and previous tag automatically.
Checks to ensure the previous tag exists and that the current tag does not
2021-04-02 11:49:19 -07:00
Stefanos Kornilios Mitsis Poiitidis 15537f8c8c Scripts: Also add unittests folder for doc outline 2021-03-30 11:58:18 +03:00
Stefanos Kornilios Mitsis Poiitidis 1fdd6fbb14 Scripts: Add a documentation comment in changelog_generator.py 2021-03-30 11:55:39 +03:00
Stefanos Kornilios Mitsis Poiitidis 458bebf598 Scripts: Add a documentation comment in doc_outline_generator.py 2021-03-30 11:53:11 +03:00
Stefanos Kornilios Mitsis Poiitidis 2b10b9792b Scripts: Fix generate_release, don't use markdown for changelogs 2021-03-30 11:23:00 +03:00
Stefanos Kornilios Mitsis Poiitidis 0dd02da57c Docs: Update outline scripts 2021-03-30 11:23:00 +03:00
Stefanos Kornilios Mitsis Poiitidis 309139e203 Scripts: Add generate_release script 2021-03-30 11:23:00 +03:00
Stefanos Kornilios Mitsis Poiitidis 350c33ada4 Docs: Update outline script 2021-03-30 11:23:00 +03:00
Stefanos Kornilios Mitsis Poiitidis bdd35e5743 docs: Update outline generator 2021-03-30 11:23:00 +03:00
Stefanos Kornilios Mitsis Poiitidis ceb7082e37 Docs/Versioning: Add Changelog Generator 2021-03-30 11:23:00 +03:00
Stefanos Kornilios Mitsis Poiitidis 2d6dc80039 Docs: Adds tag-based outline generator w/ glossary support 2021-03-30 11:22:56 +03:00