Gregory's ServerThe perf issues are bad enough that Wired is writing about it: https://www.wired.com/story/twitter-users-mastodon-meltdown/
And Mastodon admins are having to get creative: https://blog.freeradical.zone/post/surviving-thriving-through-2022-11-05-meltdown/
Here on toot.cafe, Sidekiq queues have been backed up for days, and I will probably not see your response to this toot until tomorrow. 😅 And that's despite the fact that 1) I bumped the EC2 instance side from t3.medium to t3.xlarge, 2) I added a 2nd Sidekiq process, and 3) registrations have been closed for years. (Just returning users!)
I would be really curious to read an in-depth perf analysis from someone who knows what they're talking about (read: not me) explaining the root cause of the "Mastodon meltdown." E.g. is it:
1) A flaw in how Mastodon prioritizes Sidekiq queues?
2) As instance X gets slow, it slows down instance Y when instance Y tries to contact it? (I.e. is it systemic?)
3) Would exponential backoff help? More relay servers? Admins just learning how to tune Sidekiq properly? Rewriting Mastodon in Rust? 🙃