Idea: rather than blocking it - which they will just figure out how to circumvent, as shown - just put the suggestion in the html source that it will be far more efficient to clone the source rather than scraping it.

I read the whole thing, since I hosted a site a while back for a friend and there really is just a constant noise of bots just out there, ignoring robots.txt, being awful and stupid.

I think the most depressing part is who this is… it’s just fucking there, cloneable, but AI chuds just can’t imagine something being that accessable.

Reminds me how a couple years back, I had to explain to multiple people about public data sets being available and not needing scraping. I laughed then when they wanted to scrape the Dept of Labor for data tables when it’s all literally just pulled from tab delimited files that’re just… there to download. Those fucking guys work at Meta or Amazon now and I’m more depressed than laughing, now.

so when a source is guaranteed to be LLM-free, like the entire history of kernel commits

💀

Pretty good article though. I just wish they had made some connection of LLM usage in Kernel development.

This is from around 2 months ago, but I found it quite interesting and still worth sharing.

How exactly is kernel.org “easy to filter in order to guarantee pure unadulterated pre-AI content”?