Email or username:

Password:

Forgot your password?
All posts nixCraft's posts Post Back to profile
Top-level
nixCraft 🐧

@cquest @osm_fr any idea how to black list them? robots.txt? CIDR block?

27 April at 16:59 | Open on mastodon.social
4 comments
Christian Quest 🌍

@nixCraft @osm_fr robots.txt + user-agent rule in nginx

Few days ago that silly Claudebot made 100.000+ queries on five years old phpBB urls and got the same amount of 404 in return.
Now it is only 403 they will get whatever the url

27 April at 17:06 | Open on amicale.net
madopal

@cquest @nixCraft @osm_fr Did the same for another site, and it just happily continues to request urls, hundreds of thousands of 403s not dissuading it in the slightest. I reached out to them via LinkedIn, got a response, trying to get them to pull their heads out of their collective asses.

27 April at 18:00 | Open on mastodon.social
F4GRX Sébastien

@cquest @nixCraft @osm_fr ah so thats just a:

RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} <regex>
RewriteRule . - [R=403,L]

Nice and easy! What regex should we block? is "^.*Claude.*$" enough?

27 April at 19:59 | Open on chaos.social
Gonçalo Ribeiro

@f4grx Just checked the logs on my web server. I see they've used "ClaudeBot" and "claudebot" in my case.

27 April at 20:11 | Open on infosec.exchange
Go Up