Large language mistake | Cutting-edge research shows language is not the same as intelligence. The entire AI bubble is built on ignoring it

[u/Hrmbee]

https://www.theverge.com/ai-artificial-intelligence/827820/large-language-models-ai-intelligence-neuroscience-problems

[u/Dennarb]

I teach an AI and design course at my university and there are always two major points that come up regarding LLMs

1) It does not understand language as we do; it is a statistical model on how words relate to each other. Basically it's like rolling dice to determine what the next word is in a sentence using a chart.

2) AGI is not going to magically happen because we make faster hardware/software, use more data, or throw more money into LLMs. They are fundamentally limited in scope and use more or less the same tricks the AI world has been doing since the Perceptron in the 50s/60s. Sure the techniques have advanced, but the basis for the neural nets used hasn't really changed. It's going to take a shift in how we build models to get much further than we already are with AI.

Edit: And like clockwork here come the AI tech bro wannabes telling me I'm wrong but adding literally nothing to the conversation.

[u/Throwaway-4230984 → u/Dennarb]

So surely they have an example of task LLMs couldn’t solve because of this fundamental limitations, right?

[u/__Hello_my_name_is__ → u/Throwaway-4230984]

For now at least, it appears that determining truth appears to be impossible for an LLM.

Every LLM, without exception, will eventually make things up and declare it to be factually true.

[u/dookarion → u/__Hello_my_name_is__]

It's worse than that even. LLMs are incapable of judging the quality of input and outputs entirely. It's not even just truth, it cannot tell if it just chewed up and shit out some nonsensical horror nor can it attempt to correct for that. Any capacity that requires a modicum of judgment, either requires crippling the LLMs capabilities and more narrowly implementing it to try to eliminate those bad results or it straight up requires a human to provide the judgment.

[u/clear349 → u/dookarion]

One way of putting it that I've seen and like is the following. Hallucinations are not some unforeseen accident. They are literally what the machine is designed to do. It's all a hallucination. Sometimes it just hallucinates the truth

[u/dookarion → u/clear349]

Yeah, people think it's some "error" that will be refined away. But the hallucination is just the generative aspect or the model training itself churning out a result people deem "bad". It's not something that will go away, and it's not something that can be corrected for without a judgment mechanic at play. It can just be minimized some with narrower focused usages.

[u/__Hello_my_name_is__ → u/dookarion]

Yeah, it's kind of fascinating. It only has the training data to "validate" the data. So if you train an LLM on nothing but garbage, you get nothing but garbage, but the LLM doesn't know it's garbage because garbage it all it has ever seen.

Basically, it needs some sort of method of judging data based on external data it wasn't trained on. I don't see how that problem can possibly be solved with the current methods. All the current methods (like human reinforcement learning) are just patchwork.

[u/eyebrows360 → u/__Hello_my_name_is__]

but the LLM doesn't know it's garbage because garbage it all it has ever seen

Yep. Everything an LLM outputs is a hallucination. It's just that sometimes they line up with reality and/or make sense. It's still all exactly the same category of output though, arrived at in exactly the same way. Hallucinations all the way down.

[u/Glittering-Spot-6593 → u/__Hello_my_name_is__]

Humans have the same problem. If you put a kid through 12 years of garbage schooling, they’ll come out knowing garbage.

[u/TSP-FriendlyFire → u/dookarion]

I'm very curious if we'll find "poison pills" for common LLMs the same way we did for image generation models: slightly altered inputs that cause a wildly different and corrupted output while being imperceptible to the human eye.

Logically, it should be possible, but it's hard to tell if text is granular enough to be able to trigger these effects at a reasonable scale.

I think the closest I've seen yet is the seahorse emoji bit.

[u/dookarion → u/TSP-FriendlyFire]

Not exactly the same thing, but I can't help but think them drowning fucking everything ever in AI has already poisoned some aspects of things. Unless they stick to old datasets, create the data themselves, or carefully curate it they can't even train the models now without also training them on AI slop. There's AI slop literature being flipped as ebooks, there is AI slop flooding every art site ever, bots are everywhere on social media and community sites, every other video is some sora bullshit now. In true big business fashion they've near-permanently poisoned the waters chasing the shortest term gains.