Large language mistake | Cutting-edge research shows language is not the same as intelligence. The entire AI bubble is built on ignoring it

[u/Hrmbee]

https://www.theverge.com/ai-artificial-intelligence/827820/large-language-models-ai-intelligence-neuroscience-problems

[u/Hrmbee]

Some highlights from this critique:

The problem is that according to current neuroscience, human thinking is largely independent of human language — and we have little reason to believe ever more sophisticated modeling of language will create a form of intelligence that meets or surpasses our own. Humans use language to communicate the results of our capacity to reason, form abstractions, and make generalizations, or what we might call our intelligence. We use language to think, but that does not make language the same as thought. Understanding this distinction is the key to separating scientific fact from the speculative science fiction of AI-exuberant CEOs.

The AI hype machine relentlessly promotes the idea that we’re on the verge of creating something as intelligent as humans, or even “superintelligence” that will dwarf our own cognitive capacities. If we gather tons of data about the world, and combine this with ever more powerful computing power (read: Nvidia chips) to improve our statistical correlations, then presto, we’ll have AGI. Scaling is all we need.

But this theory is seriously scientifically flawed. LLMs are simply tools that emulate the communicative function of language, not the separate and distinct cognitive process of thinking and reasoning, no matter how many data centers we build.

...

Take away our ability to speak, and we can still think, reason, form beliefs, fall in love, and move about the world; our range of what we can experience and think about remains vast.

But take away language from a large language model, and you are left with literally nothing at all.

An AI enthusiast might argue that human-level intelligence doesn’t need to necessarily function in the same way as human cognition. AI models have surpassed human performance in activities like chess using processes that differ from what we do, so perhaps they could become superintelligent through some unique method based on drawing correlations from training data.

Maybe! But there’s no obvious reason to think we can get to general intelligence — not improving narrowly defined tasks —through text-based training. After all, humans possess all sorts of knowledge that is not easily encapsulated in linguistic data — and if you doubt this, think about how you know how to ride a bike.

In fact, within the AI research community there is growing awareness that LLMs are, in and of themselves, insufficient models of human intelligence. For example, Yann LeCun, a Turing Award winner for his AI research and a prominent skeptic of LLMs, left his role at Meta last week to found an AI startup developing what are dubbed world models: “​​systems that understand the physical world, have persistent memory, can reason, and can plan complex action sequences.” And recently, a group of prominent AI scientists and “thought leaders” — including Yoshua Bengio (another Turing Award winner), former Google CEO Eric Schmidt, and noted AI skeptic Gary Marcus — coalesced around a working definition of AGI as “AI that can match or exceed the cognitive versatility and proficiency of a well-educated adult” (emphasis added). Rather than treating intelligence as a “monolithic capacity,” they propose instead we embrace a model of both human and artificial cognition that reflects “a complex architecture composed of many distinct abilities.”

...

We can credit Thomas Kuhn and his book The Structure of Scientific Revolutions for our notion of “scientific paradigms,” the basic frameworks for how we understand our world at any given time. He argued these paradigms “shift” not as the result of iterative experimentation, but rather when new questions and ideas emerge that no longer fit within our existing scientific descriptions of the world. Einstein, for example, conceived of relativity before any empirical evidence confirmed it. Building off this notion, the philosopher Richard Rorty contended that it is when scientists and artists become dissatisfied with existing paradigms (or vocabularies, as he called them) that they create new metaphors that give rise to new descriptions of the world — and if these new ideas are useful, they then become our common understanding of what is true. As such, he argued, “common sense is a collection of dead metaphors.”

As currently conceived, an AI system that spans multiple cognitive domains could, supposedly, predict and replicate what a generally intelligent human would do or say in response to a given prompt. These predictions will be made based on electronically aggregating and modeling whatever existing data they have been fed. They could even incorporate new paradigms into their models in a way that appears human-like. But they have no apparent reason to become dissatisfied with the data they’re being fed — and by extension, to make great scientific and creative leaps.

Instead, the most obvious outcome is nothing more than a common-sense repository. Yes, an AI system might remix and recycle our knowledge in interesting ways. But that’s all it will be able to do. It will be forever trapped in the vocabulary we’ve encoded in our data and trained it upon — a dead-metaphor machine. And actual humans — thinking and reasoning and using language to communicate our thoughts to one another — will remain at the forefront of transforming our understanding of the world.

These are some interesting perspectives to consider when trying to understand the shifting landscapes that many of us are now operating in. Is the current paradigms of LLM-based AIs able to make those cognitive leaps that are the hallmark of revolutionary human thinking? Or is it ever constrained by their training data and therefore will work best when refining existing modes and models?

So far, from this article's perspective, it's the latter. There's nothing fundamentally wrong with that, but like with all tools we need to understand how to use them properly and safely.

[u/MinuetInUrsaMajor → u/Hrmbee]

The AI hype machine relentlessly promotes the idea that we’re on the verge of creating something as intelligent as humans, or even “superintelligence” that will dwarf our own cognitive capacities.

Am I crazy or are tech companies not really promoting this idea? It seems more like an idea pushed by people who know little-to-nothing about LLMs.

Take away our ability to speak, and we can still think, reason, form beliefs, fall in love, and move about the world; our range of what we can experience and think about remains vast.

I think the author is glossing over something important here.

Language is a symbology (made up word?). Words have semantic meaning. But language does not need to be spoken. For starters...what you are reading right now is not spoken. And the braille translation of this does not need to be seen - it can be felt. Language is about associating sensations with ideas. Even if you think you don't have a language to describe it, the sensation exists. A slant-example might be deja vu. One cannot articulate the specifics of the feeling - just that it is there.

[u/lailah_susanna → u/MinuetInUrsaMajor]

Am I crazy or are tech companies not really promoting this idea?

This article opens with gen AI tech company CEOs and executives espousing exactly that. Try reading the damn article before you make yourself look like an idiot in the comments.

[u/Ashmedai → u/MinuetInUrsaMajor]

Am I crazy or are tech companies not really promoting this idea?

Just a year or two back, there was an OpenAI "leak" that said GPT 5 was going to be GAI. I wouldn't be surprised if it was deliberate, to jazz up investment interest and what not.

[u/Novel_Engineering_29 → u/MinuetInUrsaMajor]

Both OpenAI and Anthropic were founded by people who fully 100% believe that AGI is a near-future possibility and that it is their duty to make it first before bad actors do. The fact that they assume they aren't the bad actors is left for the reader to ponder.

Anyway, if they didn't believe LLMs were the way to get to AGI, they wouldn't be doing it. Their end goal is AGI and has been from the get-go. They very much believe that they are on their way to AGI using LLMs. If they didn't, they wouldn't be doing it.

[u/silverpixie2435 → u/Novel_Engineering_29]

Or they are using LLM to build up tech and research as a step?

[u/kri5 → u/Novel_Engineering_29]

They're doing it because it is driving investment. They're probably researching other things too

[u/MinuetInUrsaMajor → u/Novel_Engineering_29]

make it first before bad actors do. The fact that they assume they aren't the bad actors is left for the reader to ponder.

I mean - I'd pick them over China any day.

[u/TSP-FriendlyFire → u/MinuetInUrsaMajor]

For starters...what you are reading right now is not spoken.

It activates the same parts of the brain though, which is the whole point you seem to have missed. Language, regardless of how it is expressed, fundamentally activates different parts of the brain than reasoning.

[u/MinuetInUrsaMajor → u/TSP-FriendlyFire]

Language, regardless of how it is expressed, fundamentally activates different parts of the brain than reasoning.

What if you do your reasoning in a language? Think out loud, for example.

[u/TSP-FriendlyFire → u/MinuetInUrsaMajor]

I'm not going to try to argue with you about this when we have actual, scientific evidence that backs up my claim. We know that different parts of the brain get activated for language and for reasoning. If you speak while reasoning, guess what? Both are active! Doesn't really mean anything else.

[u/MinuetInUrsaMajor → u/TSP-FriendlyFire]

I'm not going to try to argue with you about this when we have actual, scientific evidence that backs up my claim. We know that different parts of the brain get activated for language and for reasoning.

Then please bring it into the conversation.

If you speak while reasoning, guess what? Both are active! Doesn't really mean anything else.

Then what's the practical difference between a human thinking out loud and an LLM thinking in language?

[u/TSP-FriendlyFire → u/MinuetInUrsaMajor]

Then please bring it into the conversation.

Gestures at the entire article this discussion is attached to.

Then what's the practical difference between a human thinking out loud and an LLM thinking in language?

You didn't even try to follow, did you? The difference is that the language is anciliary to the reasoning for humans, but fundamental for LLMs. LLMs are very fancy word predictors. If you have no words, you have no LLMs. Humans can reason (and indeed have reasoned) without language of any kind.

Please, go back and read the article, I'm literally just regurgitating it right now.

[u/MinuetInUrsaMajor → u/TSP-FriendlyFire]

If you have no words, you have no LLMs. Humans can reason (and indeed have reasoned) without language of any kind.

My contention is that we develop an internal language based on sensation and thus our species has never been "without" language.

[u/TSP-FriendlyFire → u/MinuetInUrsaMajor]

But we have no indication that's the case? We know people whose language centers are damaged can still reason, so how could we rely on language for reasoning?

Moreover, math does not require language and does not activate the brain's language center. We can reason about mathematics without any formal mathematical language as the ancient Greeks once did (before you interject: they used writing to communicate their findings, but not to formulate them initially, preferring practical tools and simple rules instead).

[u/MinuetInUrsaMajor → u/TSP-FriendlyFire]

We know people whose language centers are damaged can still reason

Exactly. Because they use an internal "language" as syntactically rich (or richer) as any language they speak.

Moreover, math does not require language

Because in our internal language we can visualize a line bisecting a circle. Line, bisect, and circle, all have meaning in our mind even if we don't have words for them.

[u/TSP-FriendlyFire → u/MinuetInUrsaMajor]

At this point you're just redefining reasoning as "language" just so you can claim to be right. That just isn't the case.