|
9 posts total
 Reading all these articles about the high traffic from AI bots, here is my experience from a small German hoster: It's a huge problem, and it is getting worse very fast. There are bots like claudebot from Antrophic which are so aggressive that they block whole systems, outranking actual attacks for example against WordPress. And they change their AWS cloud IPs so often that blocking doesn't help for long and they also ignore robots.txt. Our on-call is regularly alerted for such issues. Fancy, mein Lieblings-Wetterdienst hat jetzt für 80 Darunter auch einige größere in Deutschland, z.B. Heidelberg. https://www.meteoblue.com/de/products/cityclimate/heatmaps/heidelberg#14/49.40808/8.68558 (Aktuell aber scheinbar nur für dieses Wochenende, trotzdem spanned.) As an experienced admin and #mastoadmin here is my opinion on you running your own instance. Don't. Do. It! I have your attention now? I will explain myself in this short thread. 🧵 👇 The above is a good and safe default, so I repeat, don't do it! At least, unless you have a good reason and know what you're doing. I'm serious, and in most cases you really don't know what you're doing. I'm currently rescuing an instance that is a mess, because I couldn't say no. And I'm sure there a many more instances like this out there. Instances that have grown too fast and are maintained by a single person. THIS could break the fediverse. Just finished my article on all the things we've done to #scale chaos.social to be the fast instance it is today. #mastoadmin
Show previous comments
@leah A really big thank you to both of you! For your extensive contribution to the community, as well as for the documentation and sharing this with us! 💪 Uiii, die #SheLikesTech Folge mit mir zum Thema Mastodon ist online. Dann muss ich sie wohl selbst mal probehören ^^ #podcast https://www.ndr.de/nachrichten/info/Troeoet-statt-Tweet-mit-Leah-Oswald,audio1258934.html If someone wants to learn more about the moderation of a mastodon instance, especially chaos.social, I can really recommend the article my co-admin @rixx has written: https://rixx.de/blog/on-running-a-mastodon-instance/ Und wo hänge ich jetzt diese mega geilen Plakate der NASA zum Artemis Programm hin? 😍 Ich hab doch gar keinen Platz. 😱 Download Link: https://www.nasa.gov/feature/artemis-i-travel-essentials-the-ultimate-personal-tour-guide-for-your-trip-to-the-moon |
Gregory's Server
@leah I've been supporting a small company with their website issues - their dev has been complaining about the database server for a long time... turns out ~80% of their traffic was Claude, and the remaining ~20% was Facebook's bot. After blocking these two based on their user agent, the CPU usage on the system has gone from "pinned at 100%" to basically idle, and response times have fallen back into the acceptable range (previously there were regular spikes to many minutes)
@leah Been seeing this issue grow too.
@leah Not to mention the attitude of "we ignore robots.txt, sue us if you want" and "Without using copyrighted content for free our business model doesnt work" parasites at its finest.