Commit Graph
979 Commits
Author SHA1 Message Date
Ryan Houdek 13f35f7b79 Merge pull request #1530 from Sonicadvance1/rootfs_fetcher_fixes
FEXRootFSFetcher: Fixes some edge case behaviours
2022-01-25 10:29:55 -08:00
Ryan Houdek 4823933f79 Linux: Fixes 32-bit getsockopt and setsockopt
On Set, we have four options that need to be converted.
On Get, we have two options that need to be converted.

This fixes a crash that Tomb Raider 2013 was having on launch.
2022-01-24 17:14:50 -08:00
Ryan Houdek ee04067424 FEXRootFSFetcher: Fixes some edge case behaviours
Makes curl do its continue feature to give the users the best chance of
downloading a rootfs. We don't need to restart the full file transfer on
failure. Helps people with slower connections.

On failure to download, asks the user if they want to retry the download
rather than just exiting with a weird error about hash failure.

Once the image is downloaded, now changes options depending on if
squashfuse or unsquashfs works.

Prevents the user from selecting a bad option and getting unexpected
behaviour. Ideally we would do a squashfs mount test as well for
platforms that don't have working FUSE, like termux. This is harder to
get right and its for an unsupported platform, so I'm not going to
invest more time with it.

Fixes #1525
Fixes #1526
Fixes #1527
2022-01-23 22:54:05 -08:00
Ryan Houdek e4aef26ef5 FEXRootFSFetcher: Adds helper namespace for tool checking
Location to check if curl, squashfuse, and unsquashfs are working.

unsquashfs is a bit more complex where it needs to parse the help output
to see if zstd is supported
2022-01-23 22:44:31 -08:00
Ryan Houdek a41dc8eafa FEXRootFSFetcher: Fix pipe redirecting
In the case of launching without stdout/stderr then redirection could
have these constants be a redirected FD that sits in the same fd number.

Use -2 to indicate no redirection.
Use -1 to indicate closing traditional stderr/stdout
The rest will indicate if stdout and stderr should be replaced as
normal.
Making sure not to close the incoming fds if they matched the
stdout/stderr FD numbers.
2022-01-23 22:41:39 -08:00
lioncash 2296126af3 externals: Update fmt to 8.1.1
Brings along a bunch of enhancements and ensures we always build against
the latest version.

Also fixes up a few issues that arose due to changes in fmt
2022-01-13 14:48:35 -05:00
Ryan Houdek 7768f6965a Merge pull request #1501 from Sonicadvance1/finish_siginfo_32bit
Linux: Handles the remaining 32-bit siginfo_t usage
2022-01-11 00:02:55 -08:00
Ryan Houdek 023aaaae0c Merge pull request #1499 from Sonicadvance1/resolve_rootfs_path_in_interpreter
FEXLoader: Resolve the absolute path to rootfs if possible
2022-01-11 00:02:27 -08:00
Ryan Houdek 784cbdd973 Merge pull request #1500 from Sonicadvance1/rootfsfetch_check_curl
FEXRootFSFetcher: Check if curl is installed and fail before running
2022-01-10 16:23:05 -08:00
Ryan Houdek 9eb5ba5ad1 Merge pull request #1509 from Sonicadvance1/fix_logserver_sync
SocketLogging: Fixes MsgHandler not syncing with Assert level
2022-01-10 02:12:01 -08:00
Ryan Houdek 5dda3a1599 Merge pull request #1497 from Sonicadvance1/fix_alternative_links
Linux: Fixes emulatedpath with symlink following
2022-01-10 02:10:54 -08:00
Ryan Houdek 3ba5371d36 SocketLogging: Fixes MsgHandler not syncing with Assert level
AssertHandler by default synchronizes but MsgHandler with Assert level
should also synchronize.

Fixes an issue where LogMan::Msg::AFmt wasn't syncing so the
FEXLogServer would never see the messages.
2022-01-10 01:16:53 -08:00
Ryan Houdek 250504502a Fixes some of the warnings that cropped up 2022-01-10 00:46:10 -08:00
Ryan Houdek d4e2f42f90 FEXLogServer: Stop duplicating and dropping messages
In the case that multiple messages appearing in a single packet then we
were repeating the first message and dropping any subsequent messages.

Fixes #1496
2022-01-09 00:22:12 -08:00
Ryan Houdek 3055c23365 Linux: Handles the remaining 32-bit siginfo_t usage
Just need to translate them between 32-bit and 64-bit versions.

Fixes #1254
2022-01-09 00:03:17 -08:00
Ryan Houdek 5a5a498ed6 FEXRootFSFetcher: Check if curl is installed and fail before running
Before doing anything that requires curl, actually check if it is
installed.
Then instruct the user to install curl before using.

Doesn't try installing curl itself since we don't have a clean way to
execute sudo from potentially GUI.

Fixes #1498
2022-01-08 21:35:29 -08:00
Ryan Houdek 285ed8f1e0 FEXRootFSFetcher: Adds new Exec function with stdout,stderr redirection
Just so we can test for applications without spamming terminal
2022-01-08 21:34:59 -08:00
Ryan Houdek a68da468a5 FEXRootFSFetcher: ExecAndWaitForResponse sign extend program result
Only the lower 8bits of the execve result is the program result.
Makes sure to sign extend it so -1 is a true -1 instead of 255
2022-01-08 21:33:24 -08:00
Ryan Houdek d59aa6874e FEXLoader: Resolve the absolute path to rootfs if possible
If the user passes in an absolute path then check to see if it exists in
the rootfs before executing.

Useful for launching applications directly out of the rootfs with
FEXInterpreter.

In the case that the absolute path doesn't exist in the rootfs then
fallback to the host system as usual
2022-01-07 03:42:41 -08:00
Ryan Houdek 19fd89d2bf Linux: Fixes emulatedpath with symlink following
Some syscalls support `AT_SYMLINK_NOFOLLOW` In these instances we need
to follow the symlink on a couple of syscalls.

Fixes executing wine using the basic wine path
eg:
FEXBash "wine dxcapsviewer.exe"
2022-01-07 02:26:26 -08:00
Mai M 0bf577654d Merge pull request #1491 from Sonicadvance1/fix_signal_pause
FEXCore: Don't use Run() inside RunUntilExit()
2022-01-03 06:56:43 -05:00
Ryan Houdek 6b8e6c18ee SignalDelegator: Increase signal stack size
Just to help in the case that a program will dequeue a bucketload of
signals in one go. Basic signal stack size wasn't enough.
2022-01-03 03:29:35 -08:00
Ryan Houdek 2d573f7bf9 Linux: Fixes pidfd_send_signal for 32-bit
Due to the way this syscall handles siginfo_t, we only need to shift
data through the padding.

This is due to the syscall not allowing you to send siginfo_t with
si_code being positive. Which is what the kernel uses to interpret the
data if it is reinterpreting it.
2022-01-03 03:19:12 -08:00
Ryan Houdek 9f8071ce8f RootFS: Stop trying to retry rootfs after five times
In most cases this will be fixed on the first retry, but give it a
chance at five.
Otherwise you have a chance to infinite loop.
2021-12-31 20:04:57 -08:00
Ryan Houdek 3d1cef4afe Linux: Documents msgctl syscall only supporting IPC_64
I thought this was incorrect but since it only supports IPC_64 it is
actually correct.
If a 32-bit application wants to use the old encoding then it needs to
use the ipc syscall instead.
Fixes #1253
2021-12-28 23:04:53 -08:00
Ryan Houdek 060ab98378 Linux: Implements 32-bit msgctl IPC_SET
Needed struct version conversion support.
Removes the remaining UNHANDLED defines since we support them all now.
2021-12-28 23:02:18 -08:00
Ryan Houdek 9eca823e84 Stop hardcoding /usr/bin paths
Instead of using execve, use execvpe.
This glibc helper will search PATH or `confstr(_CS_PATH)`

Fixes #1475
2021-12-28 21:40:22 -08:00
Ryan Houdek c611eb228f RootFS: Be more robust against stale lock files
In the case of having a stale lock file on the system, usually due to an
unclean shutdown, have FEX retry if the socket for that lock file is
also not available.

This resolves the exact case of:
```
FEXBash xeyes &
kill -9 `pidof FEXMountDaemon`
sudo shutdown -r now
FEXBash xeyes & <--- This command would now fail
```

The first command would do:
 - Check lock file and socket
 - Spin up FEXMountDaemon
 - Create Lock file and Socket file
 - Start executing XEyes in FEXInterpreter

The second command would do:
 - Kills the FEXMountDaemon process
 - This leaves squashfuse running
 - The lock file and socket file are now unmanaged and just files on the filesystem

The third command would do:
 - unmounts the squashfuse mount
 - Leaving a dangling folder as a mount point

The fourth command would do:
 - Check for lock file and socket
 - Attempt to use lock file, socket, and mount that is dangling
 - Fail with error

Now instead fourth command will do:
 - Check for lock file and socket
 - Check if filesystem is valid
 - Knows that lock file exists but socket isn't active at all
 - Deletes lock file
 - Spin up FEXMountDaemon
 - Create Lock File and Socket File
 - Start executing XEyes in FEXInterpreter
2021-12-28 16:48:46 -08:00
Ryan Houdek 995672921f Linux: Implements the remaining 32-bit syscalls
The main one that we can't implement is readdir. Falls in to the same
problem space as getdents.

With this, we have the full entry tables filled out for both 64-bit and
32-bit. With some holes in the implementation, we have almost all
coverage now.
2021-12-27 02:19:30 -08:00
Ryan Houdek 514b6a822f Linux: Oops, forgot to pass usig to io_pggetevents
Easy enough to fix
2021-12-27 02:17:52 -08:00
Ryan Houdek 0945c728e9 Linux: Oops, missed fsmount syscall implementation
Simple fix
2021-12-27 02:17:20 -08:00
Ryan Houdek 9c49bd3c8e FEXRootFSFetcher: Remove 0x prefix on hash
So as to not confuse users who just copy and paste the hash without
thinking
2021-12-24 20:33:23 -08:00
Ryan Houdek c691d70919 Merge pull request #1467 from Sonicadvance1/fix_really_old_ubuntu
Improves compile ability for older libraries
2021-12-24 18:50:39 -08:00
Ryan Houdek 2079f6b3c7 Improves compile ability for older libraries
Adds a header only include utility folder that can be included from
everywhere.

Contains syscall helpers for older glibc and defines for older Linux
uapi headers missing some defines.
2021-12-24 15:43:25 -08:00
Ryan Houdek 2c31080fd4 FEXRootFSFetcher: Adds a new tool to help set up a new RootFS
This tool supports both a zenity and tty interface.
TTY will be presented if available while Zenity will be used otherwise.

This tool pulls a rootfs list from https://rootfs.fex-emu.org/
It then allows you to select a rootfs from the list, download it, place
it in to the correct working folder, extract it if desired, and set it
as the current default RootFS.

This requires curl and potentially zenity installed to use.
Maybe also unsquashfs if the user chooses to extract the image.

This is an all in one tool to quickly get a new user up and running.

Additionally if you pass in a file path in to the tool, it will generate
an xxhash of the file and exit out. This is the hash used to ensure the
files are valid.
2021-12-24 13:38:20 -08:00
Ryan Houdek 09a5f8c6b5 FEX: Move common config loading to a helper 2021-12-24 13:21:24 -08:00
Ryan Houdek 8f2cd39802 Linux: Fixes build on Ubuntu 20.04 take 3
Fixes #1457
Header is necessary for x64 types as well for struct verifier
2021-12-24 13:05:32 -08:00
Ryan Houdek 5fae07f6ee Linux: Fixes build on Ubuntu 20.04 take 2
Fixes #1457

Header include order matters here. Rude.
2021-12-24 12:23:52 -08:00
Ryan Houdek 59cc5228b8 Linux: Fixes build on Ubuntu 20.04
Fixes #1457
2021-12-24 02:46:29 -08:00
Ryan Houdek 9aab7e07f4 Linux: Fixes Ubuntu 20.04 compilation on AArch64
This header was missing
2021-12-16 22:23:32 -08:00
Mai M f2fd9d9e3f Merge pull request #1451 from Sonicadvance1/fix_semid_ds
Linux: Fixes semid_ds_64 definition
2021-12-17 01:19:41 -05:00
Ryan Houdek 03cc35f341 Linux: Fixes 32-bit shmctl
We were dereferencing the shmun ptr instead of using it directly.
Resulting in an almost immediate crash

Additionally IPC_SET doesn't write back to the shmid_ds provided.

Additional still SHM_STATE/SHM_STAT_ANY/IPC_STAT wasn't writing its
result back to the guest. Resulting in invalid stat information for the
guest.

Additionally SHM_INFO doesn't follow IPC64 behaviour since there is no
shm_info 64-bit type for a 32-bit OS.
The kernel just truncates the results in this case. Could be an
oversight on the kernel dev's side?

Fixes #1252
2021-12-15 23:34:15 -08:00
Ryan Houdek 3aead5ef65 Linux: Move some 32-bit ipc types to Types.h
This way we can run the struct verifier over it
2021-12-15 23:29:08 -08:00
Ryan Houdek b9abd091d5 Linux: Fixes semid_ds_64 definition
Wasn't quite the expected definition.

Fixes #1450
2021-12-15 22:02:37 -08:00
Ryan Houdek 43dc232e5a Merge pull request #1442 from Sonicadvance1/more_aot_changes
More AOT code movement
2021-12-15 03:27:01 -08:00
Ryan Houdek ea73c9d7ea Merge pull request #1443 from Sonicadvance1/squashfs_fixes
FEXMountDaemon squashfs fixes.
2021-12-15 03:04:15 -08:00
Ryan Houdek 4a7c65b20d Linux: Adds the safe syscall unimplemented gap for x86-64
Syscalls on x86-64 have a gap in the range of [335, 424) where these are
defined as returning ENOSYS.
This was a decision that the Linux developers decided on so x86-64 can
realign its syscall numbers to a common infrastructure.

Turns out that Proton Experimental was using a 32-bit only syscall to
check if the feature existed and was listening for ENOSYS.
They did this unconditionally on both x86 and x86-64 and it hits this
gap section.

This was introduced in Proton Experimental with commit
563fb0fbe2a49734aede87bedccb20daefc553b0
Message: "ntdll: Use clock_gettime64 if supported."

This is technically valid since it falls within this gap section but
leaves a bad taste.

We can safely ignore this section so set it up with the
UnimplementedSyscallSafe handler and also assign it to Syscall MAX so it
can go down our fast handler.

Not that this is likely to be a performance critical path.

Fixes Proton Experimental crashing when run under FEX.
2021-12-14 03:45:55 -08:00
Ryan Houdek 5d0734a7f2 FEXMountDaemon: Attempt to increase FD max when close to limit
On FEXMountDaemon startup, fetch the FD limit and current number of open
files. This allows us to track the number of open FDs we have.

Once we get close the the safe threshold we then try increasing the soft
limit towards the hard limit. If we bump up to the max (Say someone
setting the hard limit to 256) then print some fairly big warning
messages.

With the previously fixed FD leak, we could very quickly hit the pipe
limit because Steam is constantly opening new processes all the time.
With the leak fixed, running Steam and idling, FEXMountDaemon only has
around 16 pipes being watched.
2021-12-12 22:51:53 -08:00
Ryan Houdek 3e0e922fef FEXMountDaemon: Fixes EPoll thread not shutting down on SIGTERM
This is another edge case where if the process was sent a SIGTERM then
it can get in to a weird state where pipe tracking is broken.

We already check for zero pipes being tracked at the end of this loop,
just check for the shutdown variable to be set instead
2021-12-12 22:51:15 -08:00
Ryan Houdek e2004f4999 FEXMountDaemon: Fix leaked pipe FDs
Even though we were removing the pipe from the epoll interest list,
we were failing to close the pipe fd. This was leaking the FD and
running in to the Linux process FD limitation over time.

This fixes a class of issues where if you were running FEXMountDaemon
that hit the max, then there were FEX processes not being watched. Which
means that FEXMountDaemon could exit while FEX was still trying to run.

Usually a non-issue since squashfuse would keep running in the
background since open files in the mount were still in use.

Could hit fun race conditions because of it though.
2021-12-12 22:45:40 -08:00