This has been a long time coming. The C interface has been a thorn in
our side for no reason for a long time.
The purpose of this step is to remove the C interface without changing
behaviour as much as possible. This means that with this commit there
are still some bad practices but the remaining issues will be solved
with followup PRs.
Primarily, we still have a `DestroyContext(CTX)` static function which calls
the Context implementation's `DestroyContext` and does a raw C++ delete.
Follow up PR will remove that, but I didn't want to touch it yet since
it'll require checking to ensure the unique_ptr changes play nice with
our allocator hooking. Which this is already a huge PR without trying to
change behaviour.
string::find() returns npos (-1) if the given character was not found, so
it can't be used to construct a string like this. Luckily, the use of
std::span allows this code to be written such that it's both correct and
simpler than before.
Somewhere during the refactoring/review process, failed to strip the
shebang prefix off of the arguments.
Causing shebang files to always fail as if the file never existed.
Fixes steam execution.
Fixes#2136
This is a fairly tricky edge case to support with FEX.
If execveat is used with AT_EMPTY_PATH then the application can pass an
FD to execve instead of a filename. This includes FDs that have been
deleted from the disk so the child process can't open it by filename
anymore.
To work around this limitation, we need to pass the FD to the new FEX
process and open it directly, similar to how binfmt_misc works with FDs.
The FD will get passed through environment variables, which the new
process will check for and then remove the variable from the
environment.
Lots of prickly edge cases to support here.
Without binfmt_misc:
- Passes the FD to FEXLoader directly.
- Requires duplicating the FD if it has O_CLOEXEC on the FD.
With binfmt_misc:
- Shebang file, pass directly to FEXLoader, just like without binfmt.
- x86 ELF Files, rely on the kernel's binfmt_misc support here.
- Unsupported ELF files, let kernel handle it through binfmt_misc
Argument handling:
- The application can pass in no arguments.
- Means our application configurations were failing to find a config
- Also various checks in the frontend were failing.
- If opened through an FD, find the symlink for that FD for the
application configuration instead.
Side note:
Fixed a performance issue in execve where when we were checking for file
format support. Either ELF or Shebang files, we were reading the /whole/
file upfront. We only need to read a header worth of ELF files, and only
257 bytes if it is potentially a shebang file. Should dramatically
reduce some application's execve times.
Arm64 doesn't have the classic getdents syscall, only getdents64.
Old glibc versions (like 2.17) don't support getdents64
This fixes 64-bit ls with a centos 7 rootfs. Likely also fixes some other very
old applications.
Packing differences between 32-bit and 64-bit getdents means we need to
template this between the two types, otherwise it's quite similar.
No known applications rely on the 32-bit getdents but worked with test
applications.
We were registering syscall handlers to the temporary working vectors for
both x86 and x86_64 regardless of which bitness we launched with.
We then only installed the syscall handlers depending on bitness.
This was burning a decent amount of time and some memory on
initialization. Instead just don't register syscall handlers on the
other bitness.
Also stop wasting memory, clear the syscall registration vector after
we've registered everything.
Saves about 28KB of memory per process.
We weren't adjusting the guest stack size when using clone3.
If the clone comes from CLONE3 then we need to offset the RSP by the
provided stack size.
This also translates to fork/vfork through clone, so make sure to adjust
stack in that case as well.
Fixes Ender Lilies crashing with Ubuntu 22.04 rootfs
clone2 stack pointer passed in points to the highest address for the
stack.
clone3 switches this around and gives us a base pointer and a size.
glibc started using clone3 for its thread cloning which finally caught
this bug. Necessary to run any application under the Ubuntu 22.04 rootfs
since that uses a new enough glibc to encounter this.
Adds a header only include utility folder that can be included from
everywhere.
Contains syscall helpers for older glibc and defines for older Linux
uapi headers missing some defines.
Migrates lingering instances of the old logger over to fmt where
applicable. This allows removing some of the old defines and functions.
The only remaining usages of the printf-based variant of the logger is
in Tests/LinuxSyscalls/Syscalls.cpp for the strace handling.
Previous we were just using an address hint to emulate MAP_32BIT.
Seemingly this behaviour has changed on AArch64 where it now isn't
guaranteed to scan up from the hint provided if exact allocation fails.
Now we pull in the full 32-bit allocator and add support for MAP_32BIT
in it. This limits the allocations there in to the first 2GB which Linux
expects.
Necessary for Mono's trampolines to work since it requires code to be in
the first 2GB on x86-64.
Ubuntu soft links a bunch of binaries in /usr/bin to softlinks that live
in /etc/alternatives/
When hitting any of these alternative softlinks execve would fail if the
host also didn't have the same softlink paths.
Allows us to correctly follow the symlinks on execve as well which fixes
launching wine directly from the wine symlink.
Alternatively you could have launched /usr/bin/wine-stable directly.
Also fixes FEX strace again.
In the case of binfmt_misc being installed, but the user was still using
FEXLoader to pass in arguments then we wouldn't pass the arguments
forward to applications passed through execve.
This resolves an issue where Wine would fail to know where the rootfs
is since Wine launches a bunch of processes.
eg: `FEXLoader -R Ubuntu_21_04 wine winecfg` would fail before
Fixes#1323
If the binfmt_misc interpreter was installed then we were running execve
directly.
This allowed shebang programs to escape and see the host architecture
when we weren't planning on it.
Fixes `FEXBash steam` from complaining about missing packages.
This isn't quite a 100% clean sweep of IWYU.
There are some false positives where clang fails.
Additionally there are still a few missed in the frontend side of things
that I didn't get to
This is very tricky to handle and it has a bunch of rough edges.
One of the major problems that we can't workaround is that if we receive a
clone flag that pthreads can't support with THREAD, then we are required to fall down
the pthreads code path.
This is because threads going down the clone path will break TLS and we don't have
a way to work around it currently.
So this adds a clone path, a clone3 path, and keeps the legacy path as well.
Which makes this fairly convoluted but it gets pressure-vessel working on x86-64 host.
It's a bit tricky to setup but it does work.
Still some work necessary to get pressure-vessel working on AArch64 host, but I'm working on that.
The SyscallHandler takes over ownership of BRK from the ELFLoader.
We were failing to deallocate the full range.
Additionally if the guest changed BRK size to something smaller than initial load then the size would underflow.
Don't print how many instructions are installed in the tables.
This isn't useful anymore
Not installing signal 32 and 33 are something we don't support right now. Stop complaining in that case.
Stop printing when a thread is starting up and shutting down. If you want to see this then gdb shows it well.
Don't print clone flags unless we are hitting a case where we are printing another log message.
Only the frontends need to deal with ELF files specifically.
The backend doesn't need to be aware of them at all.
Since the ELF handling is the frontend's responsibility, move all the code to the frontend.
Syscall entry points still have different argument orders,
Moves the arguments to the clone3 argument structure and passes to generic handler.
Also implements clone3 while doing this
Takes the host kernel version and makes sure it fits in our supported kernel range
Minimum kernel version FEX reports to the guest is 5.0
Maximum kernel version FEX reports to the guest is currently 5.12
Ryan Houdek