@siguza "It's causing us trouble despite using Cloudflare" they have max-age=0 so Cloudflare isn't caching anything :|

@siguza
I wonder if this may be why @Teri_Kanefield has problems after linking published blogs/opeds…? iirc
https://news.itsfoss.com/mastodon-link-problem/?cache=bust
@Gargron cc

@siguza blogspam gonna blogspam

@siguza impressive.

Screw Itsfoss

@siguza nginx/1.18.0 💀

@siguza a much bigger prefetch DOS issue is recent changes to gmail and yahoo mail that prefetch urls in emails in new alleged anti-spam measures.

This has made life for email marketeers, and those who look after their servers, very interesting over the past month or so…

@siguza It would probably help if they configured caching headers properly; currently responses include 'cache-control: public, max-age=0'. This negates the value of any CDN.

@siguza but until Google finally phase out 3rd party cookies, and the GDPR and similar legislation is the US bites more, the traffic you rightly moan about is mainly third party - so a lower cost of doing business and less impactful than first party.

It’s all competing against itself for scant user resources and self-defeating, as well as terrible for the user and the planet, but companies are loathe to put a performance budget on async requests with clear bottom line benefits.

@siguza I'mma be honest, this also kind of sounds like ad click fraud. Why would it make so many requests in 4 minutes?

@siguza@infosec.space i love posting their link >:3 https://news.itsfoss.com/mastodon-link-problem/?cache=ermwhatthesigma

@siguza ItsFOSS cares about your privacy, which is why we’d like you to take a moment to review our sharing of your data with our 1424 partners…..

@siguza I disabled uBlock for 3 minutes and this is what I got from that same article:

3500 requests
189,51 MB / 16,34 MB transferred
Finish: 2,74 min

Insanity.

@siguza or in other words, please DONT link to to itsfoss- not because it ddoses them, just because their website is full of fucking shit- both figuratively and in terms of design.

@siguza A month ago somebody at Google used brute force to guess my notification system password and used it to send spam to google users, but their attenpt failed because their own antispam system. They used a bot to try to takedown my mail server. People that reads my blog didn't notice this ddos attack because my wordpress site uses a minimalistic aproach, so it has less than 3 plugins. They wanted a new google workspace user, but they failed and tutamail has more users than before

@siguza They're using a cache=bust querystring and a max-age=0 cache control... basically, what they're moaning about is entirely their own fault.

@itsfoss

@siguza if your server can't handle 3740 requests over a 5 minute window (less than 1000req/min) from a web crawler that's not even hitting all the assets there's something wrong with your server, site, or both.

@itsfoss woman up!

@siguza
It takes about 15 minutes to configure your system to handle the HTTP requests intelligently.

@siguza damn! I've never heard of this site before, but damn they must hate people or something to inflict this on us.

@siguza @jcrabapple yeah, that waterfall is 🤯

https://www.webpagetest.org/result/240502_AiDcFR_8RQ/1/details/

I especially like using Cloudflare, disabling Cloudflare’s caching, and then complaining.

Mastodon does have some architectural efficiency issues but this is like buying an F-250 for an office commute and complaining when gas prices go up.

@siguza@infosec.space Is most of these 267.22 MB even transferred from this website's server or an ad company's server? ​:neocat_confused:​

@siguza@infosec.space
They can show you a bajillion ads but 114mb of tx is too much ​:laugh_about_it:​
We have other services that often get linked on fedi and this never resulted in even a second of downtime.

@siguza they're complaining that Cloudflare doesn't help, but if their website is not cacheable Cloudflare ain't gonna do shit

@siguza It would take the full might of my internet connection 9 full minutes to load this 267MB webpage but sure there's nothing wrong with how we make websites and applications today 🤧

@siguza I'd like to ask you something because you seem to understand this process.

Neocities sites that are linked from Mastodon accumulate an unbelievable number of views per the default hit counter. Are link previews doing this?

Like, if my new and small site that barely anyone knows exists shows 40,000 unique views, is that because a few link previews count as thousands of views each?

@siguza Just checked and … having the site open alone causes more requests than what I like to call the Fedi-Hammer on my little site (which I notice in my request statistics, but not on server resources) …

For context: A typical load of my site is 100KB with a cold cache and the html is 10 to 20KB … including a decorative banner image.

Also if one has a >1MB preview image >1000px wide … apology denied.

@siguza That's not to mention it's loaded with broken scripts that are causing 400s and 403s loading external resources, at least one redirect loop, and so much more. They seem to have at least four ad exchange providers *just from the ones erroring out*. Some Google ad thing is going absolutely nuts.

This really is an unusable abomination. They should be ashamed of it. Even if they had or have a legit problem with thundering herds (which they can solve), there's no excuse for what they are.

@siguza Wait I thought those 114 MB of traffic they talked about was the total mastodon OG image network traffic sent to all mastodon instances. They send 114MB PER REQUEST???? And these guys send 267MB for each page load???

Am I understanding this correctly?