Commit Graph
77 Commits
Author SHA1 Message Date
Ryan Houdek 9c49bd3c8e FEXRootFSFetcher: Remove 0x prefix on hash
So as to not confuse users who just copy and paste the hash without
thinking
2021-12-24 20:33:23 -08:00
Ryan Houdek c691d70919 Merge pull request #1467 from Sonicadvance1/fix_really_old_ubuntu
Improves compile ability for older libraries
2021-12-24 18:50:39 -08:00
Ryan Houdek 2079f6b3c7 Improves compile ability for older libraries
Adds a header only include utility folder that can be included from
everywhere.

Contains syscall helpers for older glibc and defines for older Linux
uapi headers missing some defines.
2021-12-24 15:43:25 -08:00
Ryan Houdek 2c31080fd4 FEXRootFSFetcher: Adds a new tool to help set up a new RootFS
This tool supports both a zenity and tty interface.
TTY will be presented if available while Zenity will be used otherwise.

This tool pulls a rootfs list from https://rootfs.fex-emu.org/
It then allows you to select a rootfs from the list, download it, place
it in to the correct working folder, extract it if desired, and set it
as the current default RootFS.

This requires curl and potentially zenity installed to use.
Maybe also unsquashfs if the user chooses to extract the image.

This is an all in one tool to quickly get a new user up and running.

Additionally if you pass in a file path in to the tool, it will generate
an xxhash of the file and exit out. This is the hash used to ensure the
files are valid.
2021-12-24 13:38:20 -08:00
Ryan Houdek 43dc232e5a Merge pull request #1442 from Sonicadvance1/more_aot_changes
More AOT code movement
2021-12-15 03:27:01 -08:00
Ryan Houdek 5d0734a7f2 FEXMountDaemon: Attempt to increase FD max when close to limit
On FEXMountDaemon startup, fetch the FD limit and current number of open
files. This allows us to track the number of open FDs we have.

Once we get close the the safe threshold we then try increasing the soft
limit towards the hard limit. If we bump up to the max (Say someone
setting the hard limit to 256) then print some fairly big warning
messages.

With the previously fixed FD leak, we could very quickly hit the pipe
limit because Steam is constantly opening new processes all the time.
With the leak fixed, running Steam and idling, FEXMountDaemon only has
around 16 pipes being watched.
2021-12-12 22:51:53 -08:00
Ryan Houdek 3e0e922fef FEXMountDaemon: Fixes EPoll thread not shutting down on SIGTERM
This is another edge case where if the process was sent a SIGTERM then
it can get in to a weird state where pipe tracking is broken.

We already check for zero pipes being tracked at the end of this loop,
just check for the shutdown variable to be set instead
2021-12-12 22:51:15 -08:00
Ryan Houdek e2004f4999 FEXMountDaemon: Fix leaked pipe FDs
Even though we were removing the pipe from the epoll interest list,
we were failing to close the pipe fd. This was leaking the FD and
running in to the Linux process FD limitation over time.

This fixes a class of issues where if you were running FEXMountDaemon
that hit the max, then there were FEX processes not being watched. Which
means that FEXMountDaemon could exit while FEX was still trying to run.

Usually a non-issue since squashfuse would keep running in the
background since open files in the mount were still in use.

Could hit fun race conditions because of it though.
2021-12-12 22:45:40 -08:00
Ryan Houdek 80e66a36eb FEXConfig: Adds AOT options to the UI 2021-12-12 18:22:02 -08:00
Ryan Houdek edc31fe0a3 FEXConfig: Removes jemalloc usage
Splits out the few required dependencies to a FEXCore_Base static
library.

FEXCore then links to this directly.

Then make it so the FEX Common code links to FEXCore_Base so the
jemalloc dependency doesn't get pulled in.
2021-12-09 17:54:53 -08:00
Ryan Houdek fc858ca0a5 Adds FEXLogServer
This tool works in both gui-less and gui modes.
FEXLogServer executed alone will open a socket on `localhost:8087` and
pass log output to stderr.

If passed the `-g` argument then it will open the IMGui UI which has
some more options.
It's fairly basic right now but eventually should allow filtering by
PIDs, TIDs, and log level type
2021-11-30 14:58:06 -08:00
Ryan Houdek a8b9b351cb FEXConfig: Splits GUI setup in to a header
This will be used in another tool soon
2021-11-29 13:14:33 -08:00
Lioncash 75b2f226f6 General: Migrate over to fmt where possible
Migrates lingering instances of the old logger over to fmt where
applicable. This allows removing some of the old defines and functions.

The only remaining usages of the printf-based variant of the logger is
in Tests/LinuxSyscalls/Syscalls.cpp for the strace handling.
2021-11-23 12:51:57 -05:00
Ryan Houdek 005c3ce4b3 Tools: Strip binaries on release
lintian complains about this
2021-11-21 10:30:26 -08:00
Ryan Houdek bf5042bdf0 FEXConfig: Fixes timeout in select causing 100% CPU load
glibc 2.34 changed the select interface to update the timeout on return
to more closely match the kernel interface.
glibc 2.33 always made a copy instead of updating.
Make sure to set the timeout on each iteration of select otherwise we
will end up having a timeout of zero. Thus burning a CPU core.
2021-11-06 14:33:44 -07:00
Ryan Houdek 8cf4b285bf Adds support for setting host environment variables from config
This can be useful for edge case environment variable setting
2021-09-30 17:51:49 -07:00
Ryan Houdek acfcfa127a FEXConfig: Change shortcut for opening application profile
CTRL+A is used for selecting all in a text box.
Don't override that, it's annoying
2021-09-11 02:13:00 -07:00
Ryan Houdek 4ccd68af8a FEXMountDaemon: Fixes shutdown race conditions
This solves a problem where sometimes FEX would spin up a new process
while FEXMountDaemon was in the process of shutting down. Breaking
things on both sides.

Now the race conditions are squashed that I could see.

Also fixes one race where FEX is starting up and FEXMountDaemon is
spinning up. This case is where FEX managed to pull the lock file just
before it got deleted. Then sent the FEXMountDaemon a request to be
observed. With DGRAM sockets we would fire and forget. Use a STREAM with
a ack result so we know we can return.
2021-09-07 19:30:38 -07:00
Ryan Houdek 0fa7af15b2 FEXMountDaemon: Fixes dangling mounts problem
The FEXMountDaemon no longer uses the inotify interface for refcounting
instances of FEX.
The inotify interface fails to send close events when an application
crashes. Which is either an API oversight or intentional choice.

Now to use FEXMountDaemon the FEX process must send the daemon a pipe
fd.
The FEXMountDaemon then uses the write end of the pipe to determine if
the read end of the pipe is still open. It does this using the epoll API
and ref counting how many pipes are still active.
This is possible since epoll will tell us if pipe status has changed to
error. Signalling to the write end that the read end has closed for
whatever reason.

Now we only use the "lock" file to remove races and tell the new
instances of FEX where the rootfs is mounted
2021-09-05 00:36:44 -07:00
Ryan Houdek eefcde369a FEXMountDaemon: Fixes some minor issues
Now that we deparent FEXMountDaemon we can no longer use
PR_SET_PDEATHSIG.
Instead we rely on the ref counting and checking the pipe status to see
if the original parent has left us.
Further improvements that could be done in the future is that every user
of the mount point talks to the daemon to give it a pipe to check is
still live, since the ref counting sometimes is incorrect.
2021-09-02 15:42:55 -07:00
Ryan Houdek 0b564652d2 FEXMountDaemon: Early fork to deparent child
This allows FEXMountDaemon to remove FEXInterpreter as its parent.
Instead becoming the parent of whatever the current reaper process is.

Do it as early as possible this way FEXInterpreter won't get an
erroneous SIGCHLD.
2021-08-30 22:57:45 -07:00
Ryan Houdek 61260c2a62 FEXConfig: Load application config and advanced tab
Loading a preexisting application config was impossible through the gui.
Adds a way to do so.

Also adds an advanced tab that just displays all the items.
Allows pruning of application config options to be fairly efficient
2021-08-29 12:30:39 -07:00
Ryan Houdek f161e3bfb0 Massive amount of IWYU cleanup
This isn't quite a 100% clean sweep of IWYU.
There are some false positives where clang fails.
Additionally there are still a few missed in the frontend side of things
that I didn't get to
2021-08-28 00:32:15 -07:00
Ryan Houdek f131f07612 Config: Moves non-OS specific configuration loading to FEXCore
Puts the visibility of the main layer, application layers, and
environment in to FEXCore instead of FEX.
These layers aren't specific to FEX/FEXLoader and should live in
FEXCore.

Only the EmptyMapper remains in FEX, which should eventually move over
to FEXConfig since that is the only user.
2021-08-24 23:17:32 -07:00
Ryan Houdek c07b5e480b Adds new FEXGetConfig program
This is a simple program to get a few configuration options that are necessary
to expose for pressure-vessel
2021-08-13 20:53:30 -07:00
Ryan Houdek b586592643 Make sure our std::filesystem users use the std::error_code versions
Otherwise these crash out.
2021-08-02 23:10:23 -07:00
Ryan Houdek a6c4ad889b FEXMountDaemon: Make squashfs mounting more robust
Instead of watching to ensure our parent process is still alive. Mount the
squashfs once and use a combination of file leases and inotify to
ref count how many processes are using the rootfs.

This makes it so in the common case, the FEXMountDaemon only ever executes once
and runs until all FEX processes stop running.
In the rare edge case there is a race condition where multiple FEXMountDaemon
applications will start, but only one will end up mounting the squashfs.
In this case, one application wins and the one that failed to grab the lease
will wait until the other one completes.

With this change, squashfs should be reasonable to use now.
2021-07-29 21:45:30 -07:00
Ryan Houdek 6b90663be3 Adds cmake option ENABLE_STATIC_PIE
This option does cmake checks to determine if your system can handle static-pie.
With upstream projects static-pie only works if you use the binutils linker.
Using lld doesn't currently work because it defines __rela_iplt_{start,end} symbols.

Our cmake file will now compile a test application and check for these symbols.
Either the symbols will not exist at all or they will exist but be a null address

Once your system passes the checks then it will allow you to enable static-pie
2021-07-14 09:56:47 -07:00
Ryan Houdek 8274058895 Fixes typo on extension check for squashfs
Accidentally deleted the period on this
2021-06-15 23:03:46 -07:00
Ryan Houdek 4ff3705ff0 Update fusermount location
fuse3 package installs this to /usr/fusermount
Try both locations on failure
2021-06-15 19:50:39 -07:00
Ryan Houdek e5d3699b26 FEXConfig: Adds support for squashfs files in FEXConfig
Searches the RootFS path in the fex-emu config folder for squashfs files and displays them
Allows easy configuration of squashfs
2021-06-15 19:34:42 -07:00
Ryan Houdek d77a503510 Adds new FEXMountDaemon tool
This is a tool that communicates with FEXLoader/FEXInterpreter to automatically mount
squashfs based rootfs files on execve.

This tool will launch automatically if you have a squashfs based rootfs selected.
Once the rootfs is mounted, it will watch for the parent FEX processe to completely exit.
Once the parent FEX exits it will unmount and cleanup after itself.

This tool has a dependency on your host having FUSE, fusermount, and squashfuse applications.
If anything goes wrong in the bringup process then it propagates the erro up the chain and will
let FEX know that it couldn't mount.
2021-06-15 19:08:54 -07:00
Ryan Houdek 822a08f271 FEXConfig: Allow lower GL versions
Automatically fall back through older GL versions

This allows us to freely support GL 3.0, 2.1 and ES 2.0.
Should fix an issue where Pi devices don't support GL 3.0 in all configs

External imgui had to be updated to fix an issue with ES 2.0

Fixes #1036
2021-05-26 18:45:03 -07:00
Ryan Houdek 36fdd7f6e2 FEXConfig: Fixes INotify watcher never coming up
Inverted this check on accident
2021-05-05 19:44:31 -07:00
Ryan Houdek f6cc75de87 Fixes SDL2 interface check...again. 2021-04-30 01:55:08 -07:00
Ryan Houdek b17464d81a Oops: Missed line on FEXConfig SDL2 2021-04-27 16:51:12 -07:00
Ryan Houdek 2a1a863c58 Only use SDL2 import target if it exists
Should fix Arch and Ubuntu 20.04 together
2021-04-27 00:45:20 -07:00
Ryan Houdek 04e0baadc6 Changes LogMan assert to defines
C++ no-op functions can't optimize out the predicate arguments in all cases.
This was causing a problem where zero cost assertions weren't actually zero cost.

The only way to resolve this is to actually use macros sadly enough.

This will give a fairly hefty performance uplift with anything operating on IR.
2021-04-26 13:49:35 -07:00
Ryan Houdek e9aeebb3a1 Use cmake SDL2 target properties
Some distros include generated cmake files instead of the two cmake files from the source.
Distros that use the generated cmake files only have SDL import properties, so we are required
to use those.
2021-04-16 18:45:06 -07:00
Ryan Houdek 849b83bd02 Adds new ParanoidTSO option 2021-04-06 18:46:52 -07:00
Ryan Houdek fb94e62df1 Fixes FEXConfig filling in unnamed options
These aren't meant to be set by the user
2021-04-02 20:28:12 -07:00
Ryan Houdek f922e6420b Allow opening the first configuration file passed in
Lets us easily open any configuration file
2021-04-01 19:25:17 -07:00
Ryan Houdek 7ff06c828f Loads default config if doesn't exist
Allows us to save a folder with the default if we want it
2021-04-01 19:22:53 -07:00
Ryan Houdek be16cc1456 Fixes window ordering so popup message is visible without config open
This was a child of the config popup before.
Moves it to a child of the workspace instead so that if we had a load error the message is still visible
2021-04-01 19:16:03 -07:00
Ryan Houdek 59df3d13cc Don't set up RootFS folder or inotify thread on config load failure
Can happen in the case that the default file tries to get loaded and doens't exist
2021-04-01 19:15:08 -07:00
Ryan Houdek 1adf518e6f Don't try creating inotify thread twice
If the inotify FD is already opened then just early exit
Can happen in the case that a config tries to get opened and didn't exist
2021-04-01 19:13:43 -07:00
Ryan Houdek 68171ad9b5 Creates RootFS data folder as a user convenience
To make sure users don't mess this up, create the folder if it isn't found
2021-04-01 19:12:56 -07:00
Ryan Houdek 3294cc209f Adds support for Named RootFS folders in FEXConfig
Provides a drop down dialog of RootFS folders to select.
Will be empty if the user doesn't have any folders in place.
Just allowing directory paths to be inserted

Fixes #689
2021-04-01 00:15:29 -07:00
Ryan Houdek 1ab6726498 Moves Config directory finding to FEXCore
This will be necessary in the next commit
2021-03-31 23:46:07 -07:00
Stefanos Kornilios Mitsis Poiitidis 942b8549c6 Docs: Add tags to the source code 2021-03-30 12:21:18 +03:00