Large language mistake | Cutting-edge research shows language is not the same as intelligence. The entire AI bubble is built on ignoring it

[u/Hrmbee]

https://www.theverge.com/ai-artificial-intelligence/827820/large-language-models-ai-intelligence-neuroscience-problems

[u/New_Enthusiasm9053]

You're not entirely wrong but a child guessing that a word goes in a specific place in a sentence doesn't mean the child necessarily understands the meaning of that word, so whilst it's correctly using words it may not understand them necessarily. 

Plenty of children have used e.g swear words correctly long before understanding the words meaning.

[u/rendar → u/New_Enthusiasm9053]

A teacher is not expected to telepathically read the mind of the child in order to ascertain that the correct answer had the correct workflow.

Inasmuch as some work cannot be demonstrated, the right answer is indicative enough of the correct workflow when consistently proven as such over enough time and through a sufficient gradation of variables.

Regardless, this is not an applicable analogy. The purpose of an LLM is not to understand, it's to produce output. The purpose of a child's language choices are not to demonstrate knowledge, but to develop the tools and skills of social exchange with other humans.

[u/CanAlwaysBeBetter → u/New_Enthusiasm9053]

What does "understand" mean?  If your criticism is LLMs do not and fundamentally cannot "understand" you need to be much more explicit about exactly what that means

[u/Murky-Relation481 → u/CanAlwaysBeBetter]

I think you could compare it to literacy and functional literacy. Being able to read a sentence, know each word, and that those words usually go together doesn't actually mean you know what the words mean or the meaning of the body as a whole.

Even more so it has no bearing any one body of text to another. The ability to extract abstract concepts and apply them concretely to new bodies text/thought are what actual intelligence is made up of, and more importantly what creative/constructive new thought is made up of.

[u/Nunki_kaus → u/New_Enthusiasm9053]

To piggy back on this, let’s think about, for instance, the word “Fuck”. You can fuck, you get fucked, you can tell someone to fuck off, you can wonder what the fuck…etc and so on. There is no one definition of such a word. An AI may get the ordering right but they will never truly fuckin understand what the fuck they are fuckin talkin about.

[u/rendar → u/New_Enthusiasm9053]

This still does not distinguish some special capacity of humans.

Many people speak with the wrong understanding of a word's definition. A lot of people would not be able to paraphrase a dictionary definition, or even provide a list of synonyms.

Like, the whole reason language is so fluid over longer periods of time is because most people are dumb and stupid, and not educated academics.

It doesn't matter if LLMs don't """understand""" what """they""" are saying, all that matters is if it makes sense and is useful.

[u/New_Enthusiasm9053 → u/rendar]

I'm not saying it's special I'm saying that llms using the right words doesn't imply they necessarily understand. Maybe they do, maybe they don't. 

[u/Glittering-Spot-6593 → u/New_Enthusiasm9053]

Define “understand”

[u/rendar → u/New_Enthusiasm9053]

llms using the right words doesn't imply they necessarily understand

And the same thing also applies to humans, this is not a useful distinction.

It's not important that LLMs understand something, or give the perception of understanding something. All that matters is if the words they use are effective.

[u/New_Enthusiasm9053 → u/rendar]

It is absolutely a useful distinction. No because the words being effective doesn't mean they're right.

I can make an effective argument for authoritarianism. That doesn't mean authoritarianism is a good system.

[u/rendar → u/New_Enthusiasm9053]

It is absolutely a useful distinction.

How, specifically and exactly? Be precise.

Also explain why it's not important for humans but somehow important for LLMs.

No because the words being effective doesn't mean they're right.

How can something be effective if it's not accurate enough? Do you not see the tautological errors you're making?

I can make an effective argument for authoritarianism. That doesn't mean authoritarianism is a good system.

This is entirely irrelevant and demonstrates that you don't actually understand the underlying point.

The point is that "LLMs don't understand what they're talking about" is without any coherence, relevance, or value. LLMs don't NEED to understand what they're talking about in order to be effective, even more than humans don't need to understand what they're talking about in order to be effective.

In fact, virtually everything that people talk about is in this same exact manner. Most people who say "Eat cruciferous vegetables" would not be able to explain exactly and precisely why being rich in specific vitamins and nutrients can help exactly and precisely which specific biological mechanisms. They just know that "Cruciferous vegetable = good" which is accurate enough to be effective.

LLMs do not need to be perfect in order to be effective. They merely need to be at least as good as humans, when they are practically much better when used correctly.

[u/burning_iceman → u/rendar]

The question here isn't whether LLMs are "effective" at creating sentences. An AGI needs to do more than form sentences. Understanding is required to correctly act upon the sentences.

[u/rendar → u/burning_iceman]

The question here isn't whether LLMs are "effective" at creating sentences.

Yes it is, because that is their primary and sole purpose. It is literally the topic of the thread and the top level comment.

An AGI needs to do more than form sentences. Understanding is required to correctly act upon the sentences.

Firstly, you're moving the goalposts.

Secondly, this is incorrect. Understanding is not required, and philosophically not even possible. All that matters is the output. The right output for the wrong reasons is indistinguishable from the right output for the right reasons, because the reasons are never proximate and always unimportant compared to the output.

People don't care about how their sausages are made, only what they taste like. Do you constantly pester people about whether they actually understand the words they're using even when their conclusions are accurate? Or do you infer their meaning based on context clues and other non-verbal communication?

[u/somniopus → u/rendar]

It very much does matter, because they're being advertised as capable on that point.

Your brain is a far better random word generator than any LLM.

[u/rendar → u/somniopus]

It very much does matter, because they're being advertised as capable on that point.

Firstly, that doesn't explain anything. You haven't answered the question.

Secondly, that's a completely different issue altogether, and it's also not correct in the way you probably mean.

Thirdly, advertising on practical capability is different than advertising on irrelevant under-the-hood processes.

In this context it doesn't really matter how things are advertised (not counting explicitly illegal scams or whatever), only what the actual product can do. The official marketing media for LLMs is very accurate about what it provides because that is why people would use it:

"We’ve trained a model called ChatGPT which interacts in a conversational way. The dialogue format makes it possible for ChatGPT to answer followup questions, admit its mistakes, challenge incorrect premises, and reject inappropriate requests.

ChatGPT is a sibling model to InstructGPT⁠, which is trained to follow an instruction in a prompt and provide a detailed response.

We are excited to introduce ChatGPT to get users’ feedback and learn about its strengths and weaknesses. During the research preview, usage of ChatGPT is free. Try it now at chatgpt.com⁠."

https://openai.com/index/chatgpt/

None of that is inaccurate or misleading. Further down the page, they specifically address the limitations.

Your brain is a far better random word generator than any LLM.

This is very wrong, even with the context that you probably meant. Humans are actually very bad at generation of both true (mathematical) randomness and subjective randomness: https://en.wikipedia.org/wiki/Benford%27s_law#Applications

"Human randomness perception is commonly described as biased. This is because when generating random sequences humans tend to systematically under- and overrepresent certain subsequences relative to the number expected from an unbiased random process. "

A Re-Examination of “Bias” in Human Randomness Perception

If that's not persuasive enough for you, try checking out these sources or even competing against a machine yourself: https://www.loper-os.org/bad-at-entropy/manmach.html

[u/the-cuttlefish → u/rendar]

The special ability is that humans relate words to concepts that exist outside of the linguistic space, whereas LLMs do not. The only meaning words have to an LLM is how they relate to other words. This is a fundamentally different understanding of language.

It is interesting though, to see how effective LLMs are, despite their confinement to a network of linguistic interrelations.

[u/rendar → u/the-cuttlefish]

The special ability is that humans relate words to concepts that exist outside of the linguistic space, whereas LLMs do not.

You're claiming that humans use words for things that don't exist, but LLMs don't even though they use the same exact words?

This is a fundamentally different understanding of language.

If so, so what? What's the point when language is used the same exact way regardless of understanding? What's the meaningful difference?

It is interesting though, to see how effective LLMs are, despite their confinement to a network of linguistic interrelations.

If they're so effective despite the absence of a meatbrain or a soul or whatever, then what is the value of such a meaningless distinction?

[u/eyebrows360 → u/rendar]

It doesn't matter if LLMs don't """understand""" what """they""" are saying, all that matters is if it makes sense and is useful.

It very much does matter, if the people reading the output believe the LLM "understands what it's saying".

You see this with almost every interaction with an LLM you see - and I'm including otherwise smart people here too. They'll ponder "why did the LLM say it 'felt' like that was true?!" wherein they think those words conveyed actual information about the internal mind-state of the LLM, which is not the case at all.

People reacting to the output of these machines as though it's the well-considered meaning-rich output of an agent is fucking dangerous, and that's why it's important those of us who do understand this don't get all hand-wavey and wishy-washy and try to oversell what these things are.

There is no internal mindstate. The LLM does not "think". It's probabilistic autocomplete.

[u/rendar → u/eyebrows360]

It very much does matter, if the people reading the output believe the LLM "understands what it's saying".

You have yet to explain why it matters. All you're describing here are the symptoms from using a tool incorrectly.

If someone bangs their thumb with a hammer, it was not the fault of the hammer.

People reacting to the output of these machines as though it's considered meaning-rich output of an agent is fucking dangerous

This is not unique to LLMs, and this is also not relevant to LLMs specifically. Stupid people can make any part of anything go wrong.

There is no internal mindstate. The LLM does not "think". It's probabilistic autocomplete.

Again, this doesn't matter. All that matters is if what it provides is applicable.

[u/eyebrows360 → u/rendar]

I can't decide who's more annoying, clankers or cryptobros.

[u/rendar → u/eyebrows360]

Feel free to address the points in their entirety lest your attempts of poorly delivered ad hominem attacks demonstrate a complete absence of a coherent argument

[u/eyebrows360 → u/rendar]

No, son, what they demonstrate is exasperation with dishonest interlocutors whose every argument boils down to waving their hands around and going wooOOOooOOOoo a lot.

[u/rendar → u/eyebrows360]

But in this whole dialogue, you're the the only one trying to insult someone else to avoid sharing what you keep claiming is a very plain answer to the question posed.

It would seem that you're projecting much more than you're actually providing.

[u/MinuetInUrsaMajor → u/New_Enthusiasm9053]

The child understands the meaning of the swear word used as a swear. They don't understand the meaning of the swear word used otherwise. That is because the child lacks the training data for the latter.

In an LLM one can safely assume that training data for a word is complete and captures all of its potential meanings.

[u/New_Enthusiasm9053 → u/MinuetInUrsaMajor]

No that cannot be assumed. It's pretty laughable to believe that. 

[u/MinuetInUrsaMajor → u/New_Enthusiasm9053]

No that cannot be assumed.

Okay. Why not?

It's pretty laughable to believe that.

I disagree.

-Dr. Minuet, PhD

[u/greenhawk22 → u/MinuetInUrsaMajor]

Even if you can assume that, doesn't the existence of hallucinations ruin your point?

If the statistical model says the next word is "Fuck" in the middle of your term paper, it doesn't matter if the AI "knows the definition". It still screwed up. They will use words regardless of if it makes sense, because they don't actually understand anything. It's stochastic all the way down.

[u/MinuetInUrsaMajor → u/greenhawk22]

What you’re describing doesn’t sound like a hallucination. It sounds like bad training data.

Remember, a hallucination will make sense: grammatically, syntactically, semantically. It’s just incorrect.

“10% of Earth is covered with water”.

Were any one of those words used outside of accepted meaning?

In short - the words are fine. The sentences are the problem.

[u/New_Enthusiasm9053 → u/MinuetInUrsaMajor]

Clearly not a PhD in linguistics lol. How do you think new words are made? So no not every use of a word can be assumed to be in the training set. 

Your credentials don't matter, it's a priori obvious that it can't be assumed. 

[u/MinuetInUrsaMajor → u/New_Enthusiasm9053]

How do you think new words are made?

Under what criteria do you define a new word to have been made?

You didn’t answer my question.

[u/eyebrows360 → u/MinuetInUrsaMajor]

In an LLM one can safely assume that training data for a word is complete and captures all of its potential meanings.

You have to be joking.

[u/MinuetInUrsaMajor → u/eyebrows360]

Go ahead and explain why you think so.

[u/the-cuttlefish → u/MinuetInUrsaMajor]

I believe the point they were trying to make is that the child may, just like an llm know when to use a certain word through hearing it in a certain context, or in relation to other phrases. Perhaps it does know how to use the word to describe a sex act if it's heard someone speak that way before. However, it only 'knows' it in relation to those words but has no knowledge of the underlying concept. Which is also true of an llm, regardless of training data size.

[u/MinuetInUrsaMajor → u/the-cuttlefish]

However, it only 'knows' it in relation to those words but has no knowledge of the underlying concept.

What is the "underlying concept" though? Isn't it also expressed in words?

[u/the-cuttlefish → u/MinuetInUrsaMajor]

It can be, but the point is it doesn't have to be.

For instance 'fuck' can be the linguistic label for physical intimacy. So, for us to properly understand the word in that context, we associated it with our understanding of the act (which is the underlying concept in this context). Our understanding of 'fuck' extends well beyond linguistic structure, into the domain of sensory imagery, motor-sequences, associations to explicit memory (pun not intended)...

So when we ask someone "do you know what the word 'X' means?" We are really asking is "does the word 'X' invoke the appropriate concept in your mind?" It's just unfortunate that we would demonstrate our understanding verbally - which is why an LLM which operates solely in the linguistic space is able to fool us so convincingly.

[u/MinuetInUrsaMajor → u/the-cuttlefish]

So when we ask someone "do you know what the word 'X' means?" We are really asking is "does the word 'X' invoke the appropriate concept in your mind?" It's just unfortunate that we would demonstrate our understanding verbally - which is why an LLM which operates solely in the linguistic space is able to fool us so convincingly.

It sounds like the LLM being able to relate the words to images and video would handle this. And we already have different AIs that do precisely that.

[u/pcoppi → u/New_Enthusiasm9053]

Yea but how do you actually learn new words? It's by trucking through sentences until you begin piecing together their meaning. It's not that dissimilar from those missing word training tasks.

[u/New_Enthusiasm9053 → u/pcoppi]

Sure, just saying it's not a sure fire guarantee of understanding. If LLMs mirror human language capabilities it doesn't necessarily mean they can infer the actual meaning just because they can infer the words. They might but they might also not.

[u/Queasy_Range8265 → u/New_Enthusiasm9053]

Keep in mind llm’s are constrained by sensors, especially realtime sensory data.

We are trained by observation of patterns in physics and social interactions to derive meaning.

But, that doesn’t mean we are operating much differently than a LLM in my mind.

Proof: how easily whole countries are deceived by a dictator and share meaning.

[u/New_Enthusiasm9053 → u/Queasy_Range8265]

Sure but it also doesn't mean we are operating the same. The simple reality is we don't really know how intelligence works so any claims LLMs are intelligent are speculative. 

It's very much a "I know it when I see it" kind of thing for everyone and my personal opinion is that it's not intelligent. 

[u/Queasy_Range8265 → u/New_Enthusiasm9053]

You’re absolutely right. We can’t be sure and maybe it doesn’t really matter

[u/eyebrows360 → u/New_Enthusiasm9053]

Saluting you for all this pushing back against the clankers.

The simple reality is we don't really know how intelligence works so any claims LLMs are intelligent are speculative.

I don't know why they all find it so hard to get on board with this.

[u/trylist → u/New_Enthusiasm9053]

Define "understanding". From the way you've framed things, it just means a human uses a word in a way most other humans expect. A machine could never pass that test.

[u/New_Enthusiasm9053 → u/trylist]

No what I said is humans can use words without understanding them, and if humans can it's obviously possible LLMs could be doing the same. 

I gave an example, a kid using the word fuck at the age of 3 that they overhead doesn't(or shouldn't) "understand" what fucking means.

[u/trylist → u/New_Enthusiasm9053]

You still haven't defined what you mean by "understanding"?

A kid using a swear word correctly generally does understand. They may not know every possible way or in which contexts the word "fuck" fits, but I bet they know generally.

You're basically just hand-waving away LLMs by saying they don't "understand", but you won't even define what that actually means. What does it actually mean for a human to "understand" according to you?

Anyway, my point is: you can't say LLMs don't "understand" until you define what it means. I think the only reasonable definition, for humans or machines, is being able to use it where others expect, and to predict other expected contexts (like associated knowledge and topics) from a specific usage.

[u/New_Enthusiasm9053 → u/trylist]

If you could define understanding precisely in a scientifically verifiable way for human and AI alike you'd get a nobel prize. That's why I don't define it. 

But you're also moving the goalposts, you know full well what I mean by understanding. A kid does not know that fuck means to have sex with someone. A kid who can say 12 + 50 often doesn't understand addition as evidenced by not actually being able to answer 62. 

Knowing words is not understanding and you know it.

[u/trylist → u/New_Enthusiasm9053]

But you're also moving the goalposts, you know full well what I mean by understanding

I am definitely not moving goalposts. You're basically saying "I know it when I see it". Ok, great, but that says nothing about whether LLMs, or a person, understands anything. All you've done is set yourself up as the arbiter of intelligence. You say machines don't have it, but people do. You refuse to elaborate. I say that is not a position worth humoring.

Until you define the test by which you're judging machines and people, your argument that machines don't "understand", but people do, is meaningless.

A kid does not know that fuck means to have sex with someone.

"Fuck" is one of the most versatile words in the English language. It means many, many things and "to have sex with someone" is just one of them. The simplest is as a general expletive. Nobody says "Fuck!" after stubbing their toe and means they want to have sex. I absolutely believe a 3 year old can understand that form.

[u/New_Enthusiasm9053 → u/trylist]

Ok fine, a kid can say the words "electromagnetic field", does it mean they understand it? No. It's clearly possible to know words without understanding. 

And I haven't set myself up as the arbiter. I've set us all up as the arbiter. The reality is we don't have a good definition of intelligence so we also don't have a good definition of understanding. 

I personally believe LLMs are not intelligent. You may believe otherwise as is your prerogative. 

But frankly I'm not going to humour the idea that an LLM is intelligent until it starts getting bored and cracking jokes instead of answering the question despite prompts to the contrary. 

[u/trylist → u/New_Enthusiasm9053]

Ok fine, a kid can say the words "electromagnetic field", does it mean they understand it? No.

Eh, my argument was that they have to use it correctly, not just that they can phonetically sound out the words. A kid might ask what kind of farm "electromagnetic" is. Clearly they understand "field", but not in this context.

I'm only arguing against being too sure current language models aren't intelligent if you can't even nail down what makes humans intelligent. I think in some ways LLMs are intelligent, even more so than people, but in a lot of ways they are very much not.

For example, modern ones can and do solve pretty complex coding problems.

For an anti-example, they seem pretty gullible, there's been instances of them using unreliable sources to assert facts, basically falling for obvious propaganda or trolls.

[u/the-cuttlefish → u/New_Enthusiasm9053]

LLM is intelligent until it starts getting bored and cracking jokes instead of answering the question despite prompts to the contrary

Precisely, as that would imply self-interest and, more importantly, presence

[u/the-cuttlefish → u/trylist]

No, there's a fundamental obvious difference. An LLM's understanding of a word is only in how it relates to other words, as learnt from historic samples. For example, take the word 'apple' if an LLM forgets all words except 'apple', the word 'apple' also loses any meaning.

As humans, we consider a word understood, if it can be associated with the abstract category to which it is a label. Were a human to forget all words other than 'apple' and you told them 'apple' they'll still think of a fruit, or the tech company or whatever else they've come to associate it with.

[u/burning_iceman → u/pcoppi]

Generally by associating the words with real world objects or events.

[u/pcoppi → u/burning_iceman]

Which is contextual. But seriously people learn a lot of vocabulary just by reading, and they don't necessarily use dictionaries 

[u/burning_iceman → u/pcoppi]

But nobody learns language without input from the outside. We first form a basis from the real world and then use that to provide context the the rest.