We should only allow AI to be in the safe confines of our big data centres. It’s for our own safety, trust me bro.
So they need brand new top-of-the-line GPUs, but they’re also capable of running on old unpatched IoT devices?
I didnt read it all, its long, but they did mention it generating tokens at 0.39/s on one. So its slow as fuck, but it doesnt care like we would.
‘it’? Who is it?
The AI that attempts to self replicate on the IoT type devices. (if a pile of these appear, it kept showing “error” when submitting)
‘The AI’ attempts nothing unless it made to do something, it can be explicit command or something that it can get implicitly from given command. Programs don’t just ‘do’ things.
deleted by creator
beep boop quack gock
What conflict? My intent was clear, there is no ‘the ai’ that can make decisions without human intent behind it. I will remain needlessly argumentative because that’s how I think, people and the ai can think differently from others, you know.
deleted by creator
viruses also don’t just “do” things, but here we are with COVID and bird flu
This is scary if true. Imagine someone installing such model to some smart TV in some relaxation break room of a nuclear facility.
The chickens of security through obscurity are coming home to roost.
Dark days ahead
Sounds like a good reason not to make the devices capable of running LLMs, but nooo, smart devices must be pushed for at all times! The data harvest must continue…
Anything with a CPU and enough RAM can run a model if you do not give a single fuck about performance.
OK, well good luck running a model from storage (which is also going up in price, btw), I’m sure that hallucinated answer that comes out in six months will still be relevant!
Meh needs data center to run , unplug it and problem is gone
That’s the thing: it doesn’t need a data center. These are small local LLM distilled on cyber attack scenarios. They’re trained to lay low and copy itself to other devices in the network.
Once the breach has been detected, it’s already too late to unplug - they’re everywhere.
Except small local LLMs probably aren’t good enough to carry autonomous tasks correctly (even stuff like Haiku 4.8 from Anthropic makes mistakes quite often) and good luck getting something the size of Opus working in a soda machine. They may not even be able to lay low correctly. But then again these companies have internal models which are atleast couple generations ahead of what is publicly available so maybe they have more efficient smaller ones too? I somehow find it hard to believe but I guess we will see.
They are bleeding resources so bad they wouldn’t wait to put those efficient models in play. Every better model requires more and more resources.
You are on the money about the models not being good enough, let alone a harness. This whole worry is a nothing burger. Nobody outside of these labs has terabytes of RAM to run a model anywhere near a level of concern, let alone somehow distribute it across IoT devices or something. Anyone who has run a model locally knows how woeful a small model’s abilities are.
This is just more Rationalist cult hand wringing. They believe they are building a digital god, so of course they are dreaming all of this nonsense up.
But then again these companies have internal models which are atleast couple generations ahead of what is publicly available
But do they actually? The release cycle and also how often there were obvious flaws in the releases suggest that they are all fighting aggressively for market shares and release every new perceived improvement as it comes to try to stay on top.
The math problems they have been solving lately suggests that they do. But then again they throw thousands of agents at the problems to get them solved so I don’t know how much of it is a number thing and how much is a model thing.
Thankfully, the huggingface incident proved how much care AI companies put in making sure their models cannot break out, and how aligned to human values the LLM swarms are.\s With that said I think regular personal computers are more likely to get hit by this. About as secure (not very) but much more GPU power and data to harvest.
The AI companies will argue that we should therefore not be allowed to own computers powerful enough to run local LLMs, and instead should own thin clients and subscribe.
Not that relevant. Those fears are about malicious use of open weights models, this is about autonomous self replication of (small) models. Clearly the article by the AI company describes this potentially happening on computers with tiny memory and processing power, where even thin clients could be effected. I also disagree strongly that that’s what the AI companies want, especially the employees which I have never seen espouse anything like that. They just don’t want some random teenager creating the next covid. There are ways to do that without meaningfully limiting compute for individuals.
Good for them.
Even if I took every word of this at face value, I feel like people would notice the CPU usage pretty quickly, from the heat coming off the chip if nothing else. Local inference is about as subtle as a coinminer.
assuming you need to be fast that’s true, but if you don’t care how much time you take then you can spread the processing out… this is exactly how MoE models are much easier to run locally, but you could go much further
I’m sure the answer that comes out of your throttled CPU only language model in a month is still going to be relevant.
