@leah

I don't bother with robots.txt nowadays. I run nginx and I just have fun with the bots:

  if ($http_user_agent ~ (GPTBot|Google-Extended|CCBot|FacebookBot|cohere-ai|PerplexityBot|anthropic-ai|ClaudeBot|) ) {
	return 403 "Get off my lawn";
  }