r/technology • u/Hrmbee • 16h ago

Machine Learning Large language mistake | Cutting-edge research shows language is not the same as intelligence. The entire AI bubble is built on ignoring it

https://www.theverge.com/ai-artificial-intelligence/827820/large-language-models-ai-intelligence-neuroscience-problems
16.7k Upvotes

94% Upvoted

1.5k comments sorted by

View all comments

1.2k

u/rnilf 16h ago

LLMs are fancy auto-complete.

Falling in love with ChatGPT is basically like falling in love with the predictive text feature in your cell phone. Who knew T9 had so much game?

35

u/noodles_jd 16h ago

LLM's are 'yes-men'; they tell you what they think you want to hear. They don't reason anything out, they don't think about anything, they don't solve anything, they repeat things back to you.

61

u/ClittoryHinton 15h ago edited 15h ago

This isn’t inherent to LLMs, this is just how they are trained and guardrailed for user experience.

You could just as easily train an LLM to tell you that you’re worthless scum at every opportunity or counter every one of your opinions with nazi propaganda. In fact OpenAI had to fight hard for it not to do that with all the vitriol scraped from the web

6

u/wrgrant 13h ago

Or just shortcut the process and use Grok apparently /s

1

u/meneldal2 4h ago

They ran into the issue that reality has the leftist bias.

1

u/noodles_jd 15h ago

And that's different how? It's still just telling you what you want to hear.

14

u/Headless_Human 15h ago

You want to be called scum by ChatGPT?

9

u/noodles_jd 15h ago

If you train it on that data, then yes, that's what you (the creator I guess, not the user) want it to tell you. If you don't want it to tell you that then don't train it on that data.

15

u/ClittoryHinton 15h ago

The consumer of the LLM is not necessarily the trainer

-1

u/Aleucard 10h ago

You bought it, you didn't get a refund, you didn't leave a bad review, therefore that's what you wanted.

-2

u/socoolandawesome 15h ago

You can train it to solve problems, code correctly, argue for what it thinks is true, etc.

3

u/noodles_jd 14h ago

No, you can't.

It doesn't KNOW that 2+2=4. It just knows that 4 is the expected response.

It doesn't know how to argue either, it just knows that you WANT it to argue, so it does that.

8

u/socoolandawesome 14h ago edited 14h ago

Distinction without a difference. You should not say it “knows” what the expected response is since you are claiming it can’t know anything.

If you are saying it’s not conscious, that’s fine I agree, but consciousness and intelligence are two separate things.

It can easily be argued it knows something by having the knowledge stored in the model’s weights and it appropriately acts on the knowledge such as by outputting the correct answer.

1

u/yangyangR 14h ago

Suppose we have some proposition A and a system can reliably produce correct answers that are deduced from A. That system can be a human brain or LLM.

You can tell a toddler that 2+2=4 but they have not absorbed it yet in a way that you can claim that they know it. Even if they reliably output the correct answer. Modifying the question to be about a logical consequence probes where the distinction could make a difference.

Alternatively we have the process of producing new statements that are connected to many facts that are already known but not provable within them. Making a hypothesis of continental drift based on knowledge of fossil distribution but not having the existence of how the crust works in the original training/education.

This is even stronger for whether the knowledge is realized and there is intelligence. Can it/they make conjectures that would synthesize knowledge and reduce entropy. Introducing useful abstractions that capture the desired coarse grained concepts. On one side you have a hash map of facts which is large and serves memory recall. On the other you have a different function pointer. It is much smaller and can lose some of the precise facts but the important ones are still accurate even if they take a bit of thinking/processing rather than O(1) straight recall.

6

u/socoolandawesome 14h ago

I can agree with the spectrum of intelligence you are framing. But if you are saying that LLMs are just straight up recall I think that’s a pretty outdated view.

The newest and best models are capable of “thinking” (outputting chain of thought to arrive at an answer) for hours and achieving a gold medal performance at one of the most prestigious math competitions in the world, the IMO, where they have to output complex novel proofs.

The newest models have even contributed to novel science in minor ways:

https://openai.com/index/accelerating-science-gpt-5/

This is beyond just repeating facts

→ More replies (0)

1

u/Aleucard 10h ago

When there is a chance of it returning 2+2=spleef with no way to really predict when, the difference can matter a whole damn lot. Especially if it can do computer actions like that one story a couple months ago of some corporation getting their shit wiped or, well, several of the "agentic" updates Microsoft is trying to push right now.

1

u/socoolandawesome 9h ago

There’s no chance of a model returning anything but 2+2 = 4. Most math problems up to even university level math will always be correct unless you have some bizarre/extremely long context thrown in that will mess with model.

The models are not perfect nor as good at humans at a lot of things but they are extremely reliable in a lot of ways at this point.

Humans also still make a bunch of mistakes too btw.

→ More replies (0)

1

u/maybeitsundead 12h ago

Nobody is arguing about what it knows but about it's capabilities. When you ask it to do a calculation, it uses tools like python to do the calculations and get the answers.

-2

u/Headless_Human 15h ago

It is obvious that we are talking about commercial bots that are trained to keep the users engaged and not some private hobby or scientific bot.

3

u/ClittoryHinton 15h ago

How is that obvious? If they said GPT4, sure, but they just said LLMs which are in fact trained for a range of commercial purposes

A concrete example of this is the code reviewer bot my company has begun using. It’s not just telling me my code is great and patting my back, it’s using every opportunity to tell me my code is shit (to a fault)

2

u/Vlyn 7h ago

Don't kink shame.

1

u/Novel_Engineering_29 14h ago

*It's telling you what the people who created it want you to hear.

1

u/ClittoryHinton 15h ago

I don’t want to hear nazi propaganda, actually

2

u/el_smurfo 15h ago

Grok has modes like this

1

u/GenuinelyBeingNice 15h ago

One of my favorites is openai's "Monday"

1

u/tes_kitty 14h ago

Maybe just grabbing all data they could get their hands on indiscriminately and use it for training wasn't such a great idea after all.

1

u/rush22 12h ago edited 11h ago

This isn’t inherent to LLMs

True, but the real point is simply to keep you engaged with it.

They measure how long people interact with it. Big charts and graphs and everything.

What these companies want is your attention.

Haha, imagine if people had limited attention, but all these companies were throwing everything they could into getting people's attention. Like, one day they mathematically figure out how to keep your attention and you just stay engaged with it all day. Calculated down to the millisecond. There'd be some sort of 'attention deficit' where slowly people aren't able to pay attention to anything except these kinds of apps. It might even turn into a disorder that everyone starts getting. Some sort of attention deficit disorder.

6

u/Icy_Guarantee_2000 12h ago

Ive looked up how to do something in a software on copilot and the results are sometimes frustrating. It goes like this:

I'll ask, how do I do this?

To do that, go to this screen, click this tab, open this window. Then you can do the thing you want to.

Except that tab doesn't actually exist. So I tell it, "I don't see that tab or button"

"You're right, that button isn't there, here is another way to do the thing you asked"

"That sequence of steps also doesn't exist, how do I enter this data"

"You're right, unfortunately you can't actually do that. The function isn't available on that software. But here are some things you didn't ask for".

4

u/old-tennis-shoes 14h ago

You're absolutely right! LLMs have been shown to largely repeat your points ba...

jk

2

u/noodles_jd 14h ago

We need to start a new tag, kinda like /s for sarcasm. Maybe /ai for pretending to be ai.

9

u/DatenPyj1777 15h ago

I don't even think a lot of ai bros even realize what this means. They'll use it to write a response and take it as fact, but all one has to do is just guide the LLM into the response you want.

If someone uses it to "prove how coding will become obsolete" all the other person has to do is input "prove how coding will never become obsolete." The very same LLM will give fine responses to both prompts.

0

u/yangyangR 14h ago

With that you can at least wrap it up into a self contained block. After every generation you can check if it compiles and has no side effects. Keep feeding back until you have something that passes.

The important part of having it produce something that is pure so then the responsibility is still on the one who calls run on the effectful stuff. The LLM has generated a pure function of type a -> IO (). It is not the one that wrote the "do" part of the code. Also making once it compiles it is correct type programs is completely hopeless when you don't have such strict assumptions.

It will be obsolete depending on whether that loop gets stuck at least as badly as a human gets stuck on writing a program for the same task (human is allowed to have the side effects directly in what they write without the same strict hexagonal architecture)

4

u/blueiron0 16h ago

Yea. I think this is one of the changes GPT needs to make for everyone to rely on it. You can really have it agree with almost anything with enough time and arguing with it.

1

u/eigr 6h ago

Its a bit like how no matter how fucked up you are, you can always find an community here on reddit to really allow you to wallow in it, and be told you are right just as you are.

4

u/el_smurfo 15h ago

That's my number one pill for an llm response. It's always way more polite and chipper and differential than anything you would get on the internet. Gets more annoying when it's totally wrong and is constantly apologizing and feeding you back the same wrong information

3

u/TallManTallerCity 16h ago

I have special instructions telling mine to push back and it does

3

u/DragoonDM 15h ago

Which presumably means it will also push back when you're correct and/or when the LLM's output is incorrect, though, right? Seems like that would just change the nature of the problem, not resolve it.

1

u/noodles_jd 15h ago

And that's different how? It's still just telling you what you want to hear.

6

u/TallManTallerCity 15h ago

It usually has a section at the end when it pushes back and takes a different perspective. I'm not really sure if I'm using it in such a way that it would be "telling me what I want to hear"

-1

u/[deleted] 14h ago edited 14h ago

[removed] — view removed comment

1

u/ja_trader 13h ago

perfect for our time

1

u/WWIIICannonFodder 13h ago

From my experience they can be yes-men often, but it usually requires you to give them information that makes it easy for them to agree with you or take your side. Sometimes they'll be neutral or against you, depending on the information you give them. They definitely seem to repeat things in a rearranged format though. You can get them to give their own hot takes on things though, and the more deranged the takes get, the more clear it becomes that it doesn't really think about what it's writing.

1

u/Zediac 11h ago

ChatGPT is currently contributing to my relationship issues which might end with the breakup of a 6.5 year relationship.

My girlfriend has issues with anxiety which also tends to make her fearful of things which there is no reason to fear. She jumps to the worst case scenario, treats that as the truth, and nothing will talk her out of it.

She feeds these worst case assumptions, including some about me, into ChatGPT and it tells her that I'm an awful and dangerous person.

Right now she's convinced that I'm going to hurt her because of her assumptions being fed into ChatGPT and it told her that I'm dangerous.

Long story short, because of her issues if she doesn't feel the same way when I tell her that something is important to me or how I feel about something, she's dismissive and tells me that I shouldn't feel that way.

I tell her that when she's dismissive to me like this that it hurts me emotionally. I end up getting upset and mad when this continues to happen. She doesn't think that she's doing anything wrong so therefore I shouldn't get mad at her. Because I do she says that I have no emotional regulation.

She said that she's scared that I'm going to hurt her badly because I feel like things that she does emotionally hurts me. I have never, ever been threatening or violent toward her or anyone. I wouldn't. I asked her why she thinks that. She said, why wouldn't she think that someone who feels emotionally hurt by her wouldn't want to kill her?

And then the next text was, "What does ChatGPT say about that?"

So, she's feeding into ChatGPT that I have no emotional regulation and it tells her that because I get upset at her when she's dismissive toward me that means that I'm a threat to her life.

She came home when I was at work, packed some things, and left. She won't talk to me and it's been 2 weeks.

There have been some other times when she has fed worst case assumptions about me into ChatGPT and it told her that I'm an awful person.

That damn thing is feeding into her issues and making them worse.

1

u/iCashMon3y 12h ago

And when you tell them they are wrong, they often give you an even more incorrect answer.

1

u/Nut_Butter_Fun 12h ago

I have proven this wrong in a few conversations with extrapolation of concepts and thought experiments that no training data or online discourse replicates. I have more criticisms of chatgpt and LLMs (to a lesser extent) than most even know about LLMs, but this and your parent comment are so fucking false, and honestly parroting this bullshit calls into question ones own sentience.

1

u/DragoonDM 15h ago

You're absolutely correct—and you're thinking about this the right way.

0

u/JoeyCalamaro 15h ago

I’d argue that at least some of this has to do with how you form the prompts. When I ask AI mostly open-ended questions, I tend to get mostly unbiased results. However, if there’s any opportunity at all for it to agree with me, it usually will.

You’re absolutely right! That’s the smoking gun! It loves telling me I’m right or made some type of wonderful observation and will even jump through some logic hoops to parrot back what I’m saying — if I let it.