@matthiasott@mastodon.social I'm listing AI (and other poorly behaved crawlers) in my robots.txt while also sending them a 403 to anything else they request should they not honor that. I also had to block all traffic from China due to bots fetching endless pages sequentially. https://www.coryd.dev/posts/2026/blocking-entire-countries-because-of-scrapers