@[email protected] to [email protected]English • 9 months agoReddit changes have blocked all search engines except Google amid AI 'misuse' [U]9to5google.comexternal-linkmessage-square39fedilinkarrow-up1357
arrow-up1357external-linkReddit changes have blocked all search engines except Google amid AI 'misuse' [U]9to5google.com@[email protected] to [email protected]English • 9 months agomessage-square39fedilink
minus-square@[email protected]linkfedilinkEnglish3•9 months agoAsking for a friend… What would it take to create a domain that just acts as a proxy to Reddit but serves up its own robots.txt that allows all bots?
minus-square@[email protected]linkfedilinkEnglish7•9 months agoProbably a LOT of proxy IPs to act as different “Users” so you can overcome the rate limit that I expect they would be using to enforce such a deal
Asking for a friend…
What would it take to create a domain that just acts as a proxy to Reddit but serves up its own robots.txt that allows all bots?
Probably a LOT of proxy IPs to act as different “Users” so you can overcome the rate limit that I expect they would be using to enforce such a deal