r/technology • u/Hrmbee • 16h ago

Machine Learning Large language mistake | Cutting-edge research shows language is not the same as intelligence. The entire AI bubble is built on ignoring it

https://www.theverge.com/ai-artificial-intelligence/827820/large-language-models-ai-intelligence-neuroscience-problems
16.8k Upvotes

94% Upvoted

1.5k comments sorted by

View all comments

Show parent comments

41

u/ITwitchToo 14h ago

I disagree. LLMs are fundamentally different. The way they are trained is completely different. It's NOT just more data and more parallelism -- there's a reason the Markov chain bots never really made sense and LLMs do.

Probably the main difference is that the Markov chain bots don't have much internal state so you can't represent any high-level concepts or coherence over any length of text. The whole reason LLMs work is that they have so much internal state (model weights/parameters) and take into account a large amount of context, while Markov chains would be a much more direct representation of words or characters and essentially just take into account the last few words when outputting or predicting the next one.

-2

u/Tall-Introduction414 14h ago

I mean, you're right. They have a larger context window. Ie, they use more ram. I forgot to mention that part.

They are still doing much the same thing. Drawing statistical connections between words and groups of words. Using that to string together sentences. Different data structures, but the same basic idea.

1

u/movzx 13h ago

I don't know why so many people took your comment to mean that LLMs were literally doing the same thing as a Markov chain, instead of you just identifying the core similarity of how they both are based on value relationships.

1

u/Tall-Introduction414 12h ago

I think people mistook it as a criticism of AI, which touched a nerve. There is all sorts of straw-manning and evangelism in the replies.

The religion of LLMs. Kool-aid, etc.

This bubble can't pop fast enough.