It turns out the way I set up file sharing between the front end web server and the app server created immense slowdowns on fedia. I consolidated some things (web server on app server now), which isn’t a long term solution, but will work in the very short term.
I am having to copy media from the s3 storage bucket back to a local directory - that is going to take a long time and images on the site are going to be broken until that finishes - hopefulle overnight.
There are still a few lingering error 500s and I will tackle that tomorrow evening.
I intend to spend some time debugging the reason that federation periodically just stops and has to be manually restarted, as well as the cause of the error 500’s. Sorry for the problems.
No worries! This is new and I appreciate the time and effort you are spending on it.