cm0002@lemmy.world to Technology@lemmy.worldEnglish · 10 days agoOpen Source devs say AI crawlers dominate traffic, forcing blocks on entire countriesarstechnica.comexternal-linkmessage-square25fedilinkarrow-up1379arrow-down14cross-posted to: technology@lemmy.ziptechnology@lemmy.mlopensource@programming.devhackernews@lemmy.bestiver.se
arrow-up1375arrow-down1external-linkOpen Source devs say AI crawlers dominate traffic, forcing blocks on entire countriesarstechnica.comcm0002@lemmy.world to Technology@lemmy.worldEnglish · 10 days agomessage-square25fedilinkcross-posted to: technology@lemmy.ziptechnology@lemmy.mlopensource@programming.devhackernews@lemmy.bestiver.se
minus-squarehenfredemars@infosec.publinkfedilinkEnglisharrow-up26·10 days agoEven mainly text-mode sites like LWN are feeling the strain and finding it hard to support all this parasite bots.
minus-squarezonnewin@feddit.nllinkfedilinkEnglisharrow-up11·10 days agoSo don’t support them. Block them!
minus-squarePlutoniumAcid@lemmy.worldlinkfedilinkEnglisharrow-up19·10 days agoSure, but the challenge is how to block them without putting undue load on humans. In the olden days, you’d just host a webserver and be done with it. Today you need elaborate setups to trick bots. It’s a losing proposition.
minus-squarewhyNotSquirrel@sh.itjust.workslinkfedilinkEnglisharrow-up5·10 days agoIs there a tools to not have to manually block them? Like an updated list of AIshit IP?
Even mainly text-mode sites like LWN are feeling the strain and finding it hard to support all this parasite bots.
So don’t support them. Block them!
Sure, but the challenge is how to block them without putting undue load on humans.
In the olden days, you’d just host a webserver and be done with it. Today you need elaborate setups to trick bots. It’s a losing proposition.
Is there a tools to not have to manually block them? Like an updated list of AIshit IP?