2026-11-03 23:01 UTC
I built a quick and dirty script to trap the bots which endlessly crawl my site.
I'm calling it "Blunt Banhammer Blocks Basic Bots Ballistically" - or B⁶ for short.
It has already dropped traffic from 200GB per day down to about 60GB per day.
Still nuts, of course, but slightly less nuts now.
Replies (1)
-
@Edent@mastodon.social 2026-11-04 12:18
Since yesterday, I've blocked nearly 100 IP addresses which had requested a hidden directory blocked by robots.txt - the graph already shows a dramatic fall in data transfer. B⁶ is, as the name suggests, a blunt tool. But it's effective. I'll release the source code once I've tidied it up. Some bots, like intricle-image-extractor are obvious scrapers, but most spoof common browsers. I half wish OSes would cryptographically sign their requests so you knew it came from a genuine device.