“Model collapse” threatens to kill progress on generative AIs - eviltoast
  • gravitas_deficiency@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    3
    ·
    2 months ago

    It’s kinda interesting in how it actually roughly parallels the dawn of the nuclear age in some specific ways. Namely, that there’s a clear “purity” line established by the advent of the technology - and I mean that literally, not figuratively. Content on the internet is going to have a very similar dividing line. But it’s also going to be way harder to definitively source data from before that line, unless someone clairvoyant happened to offline and archive a huge storage array with a complete internet snapshot right before ML made its public debut. And I know exactly what the scale of that storage commitment would be, and how much it would cost. So I’m certain nobody has done that.