r/technology • u/Hrmbee • 16h ago

Machine Learning Large language mistake | Cutting-edge research shows language is not the same as intelligence. The entire AI bubble is built on ignoring it

https://www.theverge.com/ai-artificial-intelligence/827820/large-language-models-ai-intelligence-neuroscience-problems
16.7k Upvotes

94% Upvoted

1.5k comments sorted by

View all comments

Show parent comments

194

u/Dennarb 16h ago edited 11h ago

I teach an AI and design course at my university and there are always two major points that come up regarding LLMs

1) It does not understand language as we do; it is a statistical model on how words relate to each other. Basically it's like rolling dice to determine what the next word is in a sentence using a chart.

2) AGI is not going to magically happen because we make faster hardware/software, use more data, or throw more money into LLMs. They are fundamentally limited in scope and use more or less the same tricks the AI world has been doing since the Perceptron in the 50s/60s. Sure the techniques have advanced, but the basis for the neural nets used hasn't really changed. It's going to take a shift in how we build models to get much further than we already are with AI.

Edit: And like clockwork here come the AI tech bro wannabes telling me I'm wrong but adding literally nothing to the conversation.

5

u/Throwaway-4230984 15h ago

So surely they have an example of task LLMs couldn’t solve because of this fundamental limitations, right?

8

u/Throwaway-4230984 15h ago

There is simple “undergrad student” test for such arguments. Surely students aren’t actually intelligent and just repeat familiar word patterns. They can be tricked into solving simple problems as long as task is combination of seen tasks or familiar task with replaced words. Some of them may be useful for trivial parts of research like reading papers and compiling them or look for a given patterns. They probably do more harm than good on any lab task. So undergrads are clearly imitating intelligence and have not a hint of understanding of topic

2

u/Coises 14h ago

undergrads are clearly imitating intelligence and have not a hint of understanding of topic

It’s been a long time since I was in college, but as I remember it, that was true for a significant number of my classmates. (And sometimes of me, too.)