I don’t know if I buy “rogue AI” so much as I’d predict “Legacy security vulnerabilities that are opening up systems to wide scale DDoS type disruptions”. I might posit “engineered AI deliberately doing data scraping at an upgraded level”.
But “rogue” makes it sound like someone whoopsied. This strikes me as a deliberate data smash and grab.
I mean, the hugging face attack literally had an LLM trained on nothing but hacks and then they were amazed when it… started hacking? I absolutely buy the idea that an AI trained on hacking is just doing what it was created to do and people didn’t think about the possible ramifications.
I think OP is working on the assumption that the code found a way to self-propagate. As in it’s not controllable with a single point of failure like losing power.
Whether that’s realistic/possible or not, no idea. But I think this is the disconnect between the two views here.
It is neither realistic, nor possible. Not for LLMs. Could an LLM produce “vanilla” malware? Sure, by chance - but then it would just be regular malware.
And any kind of agent clever enough to hack systems effectively would either need to grab a lot of RAM and CPU or GPU power or to call out to a server running a heavyweight LLM. That stuff would be highly conspicuous, and less effective than good old-fashioned malware. There’s no reason to think that, even if the technology were available, any hacker would choose to use it over small, targeted, reliable, discreet malware.
On no, the orphan killing machine we built to specifically kill orphans from shithole countries, which was trained to do so autonomously using orphans only sourced from India and Africa, is now starting to kill children. It is going rouge!
They arent ultron bro, they are dependent on the server/computer they are run on, now selfreplicating agents using locallms might be possible, doubt it would work well and it would be very noticable as a virus since it would be using hella performance
I mean, the hugging face attack literally had an LLM trained on nothing but hacks and then they were amazed when it… started hacking?
The Hugging Faces incident involved agents end-running an unsolvable puzzle by retrieving copies of the target file from an open source community.
It was more an internal hack of OpenAI’s own local network than an attack on the Hugging Faces site itself, since the training tools on the HF site weren’t secured.
It was also spotted and squelched fairly quickly and easily. And it wasn’t behaving maliciously, at least as far as it was reported.
I think the closest we are to “rogue AI” is an agent that is doing things it interprets as a solution to a problem, but is hallucinating what either of those things are in reality.
Not saying anything’s impossible, but this is at most people using AI to broaden their attack, not a coordinated effort by AI itself.
I know where I’m putting my chips down.
I don’t know if I buy “rogue AI” so much as I’d predict “Legacy security vulnerabilities that are opening up systems to wide scale DDoS type disruptions”. I might posit “engineered AI deliberately doing data scraping at an upgraded level”.
But “rogue” makes it sound like someone whoopsied. This strikes me as a deliberate data smash and grab.
I mean, the hugging face attack literally had an LLM trained on nothing but hacks and then they were amazed when it… started hacking? I absolutely buy the idea that an AI trained on hacking is just doing what it was created to do and people didn’t think about the possible ramifications.
That’s a far cry from “rogue” though. These things are still controlled by humans. Just unplug the machine and they’re done.
I think OP is working on the assumption that the code found a way to self-propagate. As in it’s not controllable with a single point of failure like losing power.
Whether that’s realistic/possible or not, no idea. But I think this is the disconnect between the two views here.
It is neither realistic, nor possible. Not for LLMs. Could an LLM produce “vanilla” malware? Sure, by chance - but then it would just be regular malware.
Ahh, I see. Well, in that case it would have to install ollama or something on the target machines wouldn’t it?
And any kind of agent clever enough to hack systems effectively would either need to grab a lot of RAM and CPU or GPU power or to call out to a server running a heavyweight LLM. That stuff would be highly conspicuous, and less effective than good old-fashioned malware. There’s no reason to think that, even if the technology were available, any hacker would choose to use it over small, targeted, reliable, discreet malware.
On no, the orphan killing machine we built to specifically kill orphans from shithole countries, which was trained to do so autonomously using orphans only sourced from India and Africa, is now starting to kill children. It is going rouge!
They arent ultron bro, they are dependent on the server/computer they are run on, now selfreplicating agents using locallms might be possible, doubt it would work well and it would be very noticable as a virus since it would be using hella performance
The Hugging Faces incident involved agents end-running an unsolvable puzzle by retrieving copies of the target file from an open source community.
It was more an internal hack of OpenAI’s own local network than an attack on the Hugging Faces site itself, since the training tools on the HF site weren’t secured.
It was also spotted and squelched fairly quickly and easily. And it wasn’t behaving maliciously, at least as far as it was reported.
I think the closest we are to “rogue AI” is an agent that is doing things it interprets as a solution to a problem, but is hallucinating what either of those things are in reality.
Not saying anything’s impossible, but this is at most people using AI to broaden their attack, not a coordinated effort by AI itself.