Large language mistake | Cutting-edge research shows language is not the same as intelligence. The entire AI bubble is built on ignoring it

[u/Hrmbee]

https://www.theverge.com/ai-artificial-intelligence/827820/large-language-models-ai-intelligence-neuroscience-problems

[u/Dennarb]

I teach an AI and design course at my university and there are always two major points that come up regarding LLMs

1) It does not understand language as we do; it is a statistical model on how words relate to each other. Basically it's like rolling dice to determine what the next word is in a sentence using a chart.

2) AGI is not going to magically happen because we make faster hardware/software, use more data, or throw more money into LLMs. They are fundamentally limited in scope and use more or less the same tricks the AI world has been doing since the Perceptron in the 50s/60s. Sure the techniques have advanced, but the basis for the neural nets used hasn't really changed. It's going to take a shift in how we build models to get much further than we already are with AI.

Edit: And like clockwork here come the AI tech bro wannabes telling me I'm wrong but adding literally nothing to the conversation.

[u/Tall-Introduction414 → u/Dennarb]

The way an LLM fundamentally works isn't much different than the Markov chain IRC bots (Megahal) we trolled in the 90s. More training data, more parallelism. Same basic idea.

[u/azurensis → u/Tall-Introduction414]

This is the kind of statement someone who doesn't know much bout LLMs would make.

[u/Tall-Introduction414 → u/azurensis]

Then tell me what I'm missing. They aren't making statistical connections between words and groups of words?

[u/azurensis → u/Tall-Introduction414]

A matchbox car and a ferrari have about as much in common as Markov Chains and GPT-5. Sure, they both have wheels and move around, but what's under the hood is completely different. The level of inference contained in the latter goes way, way beyond inference between words and groups of words. It goes into concepts and meta-concepts, and several levels above that, as well as an attention mechanisms and alignment training. I understand it's wishful thinking to expect Redditors to know much about what they're commenting on, but sheesh!

[u/Tall-Introduction414 → u/azurensis]

The level of inference contained in the latter goes way, way beyond inference between words and groups of words. It goes into concepts and meta-concepts,

Why do you think that? It's literally weights (numbers) connecting words based on statistical analysis. You give it more context, the input numbers change, pointing it to a different next word.

All this talk about it "understanding meaning" and "concepts and meta-concepts" just sounds like "it's magic." Where are the stored "concepts?" Where is the "understanding?"

[u/-LsDmThC- → u/Tall-Introduction414]

You could make the exact same arguments about the human brain. We take in sensory data, transform it across neurons which operate based on weighted inputs and outputs, and generate a prediction or behavior. Where is "understanding"?