this post was submitted on 26 Jul 2025
243 points (98.4% liked)
Linux
8652 readers
472 users here now
A community for everything relating to the GNU/Linux operating system (except the memes!)
Also, check out:
Original icon base courtesy of lewing@isc.tamu.edu and The GIMP
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Good luck detecting bots...
It's actually not that hard. Most of these bots are using a predictable scheme of headless browsers with no js or minimal js rendering to scrape the web page. Fully deployed browser instances are demonstrably harder to scale and basically impossible to detect without behavioral pattern detection or sophisticated captchas that also cause friction to users.
The problem with bots has never rested solely on detectability. It's about:
A. How much you inconvenience the user to detect them
B. Impacting good or acceptable bots like archival, curl, custom search tools, and loads of other totally benign use cases.