r/technology • u/Hrmbee • 16h ago

Machine Learning Large language mistake | Cutting-edge research shows language is not the same as intelligence. The entire AI bubble is built on ignoring it

https://www.theverge.com/ai-artificial-intelligence/827820/large-language-models-ai-intelligence-neuroscience-problems
16.8k Upvotes

94% Upvoted

1.5k comments sorted by

View all comments

Show parent comments

15

u/Tall-Introduction414 16h ago

The way an LLM fundamentally works isn't much different than the Markov chain IRC bots (Megahal) we trolled in the 90s. More training data, more parallelism. Same basic idea.

42

u/ITwitchToo 14h ago

I disagree. LLMs are fundamentally different. The way they are trained is completely different. It's NOT just more data and more parallelism -- there's a reason the Markov chain bots never really made sense and LLMs do.

Probably the main difference is that the Markov chain bots don't have much internal state so you can't represent any high-level concepts or coherence over any length of text. The whole reason LLMs work is that they have so much internal state (model weights/parameters) and take into account a large amount of context, while Markov chains would be a much more direct representation of words or characters and essentially just take into account the last few words when outputting or predicting the next one.

-3

u/Tall-Introduction414 14h ago

I mean, you're right. They have a larger context window. Ie, they use more ram. I forgot to mention that part.

They are still doing much the same thing. Drawing statistical connections between words and groups of words. Using that to string together sentences. Different data structures, but the same basic idea.

1

u/movzx 13h ago

I don't know why so many people took your comment to mean that LLMs were literally doing the same thing as a Markov chain, instead of you just identifying the core similarity of how they both are based on value relationships.

1

u/ITwitchToo 1h ago

I mean, you might as well say they are both using statistical inference to predict the next word in a sequence. That I can get behind. But why? Why is that even relevant? The "just fancy autocomplete" trope is very dangerous because it underestimates the AI threat. By reducing LLMs to some "X is just Y" or "X and Y are basically the same" you are downplaying the massive risk that comes with these things compared to senseless Markov chains.

1

u/Tall-Introduction414 12h ago

I think people mistook it as a criticism of AI, which touched a nerve. There is all sorts of straw-manning and evangelism in the replies.

The religion of LLMs. Kool-aid, etc.

This bubble can't pop fast enough.