AI crawlers cause Wikimedia(The umbrella organization of Wikipedia and a dozen or so other crowdsourced knowledge projects) Commons bandwidth demands to surge 50%.

Tea@programming.dev · edit-2 21 days ago

AI crawlers cause Wikimedia(The umbrella organization of Wikipedia and a dozen or so other crowdsourced knowledge projects) Commons bandwidth demands to surge 50%.

katy ✨@lemmy.blahaj.zone · 21 days ago

wikipedia should install ai mazes on their servers

KumaSudosa@feddit.dk · 20 days ago

Not in this case, to be fair. The only concern is cost - since Wiki wouldn’t be opposed to them getting their actual data - and AI mazes are designed to safeguard more sensitive data, not reducing cost

upandatom@lemmy.dbzer0.com · 20 days ago

Nice analysis. Need more smart people like you in the world

KumaSudosa@feddit.dk · 20 days ago

I agree with that assessment!

inbeesee@lemmy.world · 20 days ago

Are there alternatives besides Cloudflare’s solution?

skulblaka@sh.itjust.works · 20 days ago

Nepenthes does about the same thing but isn’t managed by a corp.

brognak@lemm.ee · 20 days ago

There’s also Anubis, but it uses proof of work not a maze.

https://anubis.techaro.lol/

AI crawlers cause Wikimedia(The umbrella organization of Wikipedia and a dozen or so other crowdsourced knowledge projects) Commons bandwidth demands to surge 50%.

AI crawlers cause Wikimedia(The umbrella organization of Wikipedia and a dozen or so other crowdsourced knowledge projects) Commons bandwidth demands to surge 50%.

How crawlers impact the operations of the Wikimedia projects – Diff