We've been hit by a massive DDoS attack. The site may not work as expected. We're working on mitigating the attack. Fastly are helping us. If in doubt, check status.mastodon.social for information.
We've been hit by a massive DDoS attack. The site may not work as expected. We're working on mitigating the attack. Fastly are helping us. If in doubt, check status.mastodon.social for information. 136 comments
[DATA EXPUNGED]
@andre_601 @Gargron @airportline A bot once said that Australia isn’t real, I think this is how it felt. @Gargron wonder if another service has spent more resources on DDoS than maintaining their own site, seeing as this coincides with them being down… @Gargron thank goodness for Eugen! what would we do without him ... the world would be a much darker place @Gargron Meanwhile, the other social network DoS-ed itself... {"errors":[{"message":"Your current API plan does not include access to this endpoint, please see https://developer.twitter.com/en/docs/twitter-api for more information","code":467}]} Thoughts to you guys. It's sad that certain people want to take something good and destroy it for no good reason. ✊
It is certainly a weird coincidence that exactly when Twitter is down we suddenly get a DDoS attack. I wonder...🤔 @nagmay @Gargron Hmm it does indeed look like there was a spike recently: https://mastodon.social/@mastodonusercount/109978348827276053 @Gargron Nice that federation results in a working Mastodon experience for most not being interrupted. In theory, I suppose, you could have a secondary account on another instance following the same people and have built in personal redundancy. @FirefighterGeek @Gargron I have a secondary account. Just in case. I update followers once a month. They are totally unrelated so that gives me access even if my instance is down. @bowreality @Gargron I'd love to set that up, but I know I would never get around to keeping them in sync. Would be a cool third party plug in though. @FirefighterGeek @Gargron It’s very quick to export followers and import them on the 2nd account but I agree a tool would be nice @cautionarytale @Gargron They fear what they are unable to control. Since Mastodon is decentralized and not owned by any corpos, it's difficult if not impossible to contain. @ablackcatstail this is true, a huge number of intellectually gifted people gathered here per chance, in what could be a public globalization effort. This place enables alternatives for everyone with its diversity, and purpose. @Gargron Everything working fine for me right now, so whilst it still is... 🔶 Find #JohnMastodon 🔶 He'll know what to do. @Gargron thank you and the entire team for all the hard work keeping everything running as smoothly as possible. @Gargron This sucks, truly, and yet also shows how incredible decentralized infrastructure is for a platform. (This is an example of something I'd quote-toot, and I'm still not sure how that is "properly" done... I didn't screenshot because I know the concept is offensive to you personally. Anyway, as to the original toot, thanks for making a platform so open and resilient.) Looks like the DDoS attackers forgot to take down every other Mastodon instance... I didn't even know this until somebody else mentioned it. @Gargron It's no DDoS, it's everyone coming on to masto to post #twitterdown and upload screenshots of the json responses they were getting @lostgen Actually other instances have reported higher load too. Check the stats at https://mastodon.fediverse.observer/list : mastodon.social has 13 % of total users active in the last month, vs. e.g. 40 % in troet.cafe. So it stands to reason that returning users might create bigger swings in one case than in the other. However I have no information to doubt Gargron's assessment on the source of today's downtime. @Gargron another bene of a distributed system. They *can't* DDos all the instances. @Gargron Suspicious timing with the twitter outage, did someone time this attack to prevent another migration of users I wonder Everything is back to normal now. Yes, it was an attack, not legitimate traffic. No, we don't know who was behind it. I agree the timing with Twitter being down was unfortunate. Fastly and Datadog who both sponsor us by providing us a free service were instrumental in analyzing and mitigating the attack. Shout out to our own team as well, glad I don't have to do this alone anymore!
[DATA EXPUNGED]
@Gargron Will there be a blog post detailing what and how it all happened, and list any IOCs, etc.? I still see this via US-CA-SJ: Error 403 Forbidden Error 54113 Varnish cache server @Gargron I'm getting 403 errors trying to access the site - am I blocked then? I didn't do anything apart from try to load my timeline while you were down! @Gargron Need to work with your CDN to at least splash up a failephant when stuff like this happens. The 503 error-page otherwise has no zing. Does it waddle and quack like the RU mafia state? And they thought turning twtr into their propaganda and terrorist mush via their psycho puppet was going to be enough. |