Email or username:

Password:

Forgot your password?
Ethan Marcotte

šŸ¦Š

wrote up how iā€™m blocking ā€œartificial intelligenceā€ bots from accessing my website, with some copy-and-paste code that should (šŸ¤žšŸ») stay up-to-date whenever i update my blocklist ethanmarcotte.com/wrote/blocki

23 comments
Ethan Marcotte

i really wish i didnā€™t have to think about any of this

Eric

@beep I've also seen folks experiment with poisoning the well, off-screen text that instructs scrapers to ignore the content and/or feed it volatile information because of this.

But! To the larger point, it is entirely ridiculous to have to even consider these things.

Cory :prami_pride_demi:

@beep šŸ™ŒšŸ» me too ā€” but I'll be off shoveling away at sludge until I can stop github.com/ai-robots-txt/ai.ro šŸ«£

Ethan Marcotte

@cory Oh hey WONDERFUL, Iā€™d missed this. Thank you, Cory!

Cory :prami_pride_demi:

@beep absolutely! Going to keep on adding to it as time permits. šŸ«”

Cory :prami_pride_demi:

@ayo absolutely! I figure the more folks that adopt it the more we can chip away at it as a community.

Ethan Marcotte

@chriscunningham I have, thank you! (Itā€™s one of the sources I referenced in the post. I think itā€™s stellar.)

Mads the Mad

@beep Fixing up my site to defend against AI invaders is high on my weekend to do list, so your post is both timely and useful. Many thanks!

Ethan Marcotte

@mads Oh, youā€™re so welcome šŸ’œ Good luck with the site tweaks!

TomasHradcky

@beep Many thanks for this. I had done the robots version just for the sake of it but this is important for me. Appreciated.

Estelle

@beep I wish we didnā€™t have to think about thatā€¦

datarama

@beep More and more, I find I hate tech.

Shane Rogers

@beep This is a great thread! Will most definitely be taking advantage of some of the information that's contained here, not that my personal site has content worth stealing. šŸ¤£

Go Up