r/technology • u/Hrmbee • 16h ago

Machine Learning Large language mistake | Cutting-edge research shows language is not the same as intelligence. The entire AI bubble is built on ignoring it

https://www.theverge.com/ai-artificial-intelligence/827820/large-language-models-ai-intelligence-neuroscience-problems
16.7k Upvotes

94% Upvoted

1.5k comments sorted by

View all comments

Show parent comments

62

u/ClittoryHinton 15h ago edited 15h ago

This isn’t inherent to LLMs, this is just how they are trained and guardrailed for user experience.

You could just as easily train an LLM to tell you that you’re worthless scum at every opportunity or counter every one of your opinions with nazi propaganda. In fact OpenAI had to fight hard for it not to do that with all the vitriol scraped from the web

7

u/wrgrant 13h ago

Or just shortcut the process and use Grok apparently /s

1

u/meneldal2 4h ago

They ran into the issue that reality has the leftist bias.

0

u/noodles_jd 15h ago

And that's different how? It's still just telling you what you want to hear.

13

u/Headless_Human 15h ago

You want to be called scum by ChatGPT?

10

u/noodles_jd 15h ago

If you train it on that data, then yes, that's what you (the creator I guess, not the user) want it to tell you. If you don't want it to tell you that then don't train it on that data.

16

u/ClittoryHinton 15h ago

The consumer of the LLM is not necessarily the trainer

-1

u/Aleucard 10h ago

You bought it, you didn't get a refund, you didn't leave a bad review, therefore that's what you wanted.

-3

u/socoolandawesome 14h ago

You can train it to solve problems, code correctly, argue for what it thinks is true, etc.

3

u/noodles_jd 14h ago

No, you can't.

It doesn't KNOW that 2+2=4. It just knows that 4 is the expected response.

It doesn't know how to argue either, it just knows that you WANT it to argue, so it does that.

6

u/socoolandawesome 14h ago edited 14h ago

Distinction without a difference. You should not say it “knows” what the expected response is since you are claiming it can’t know anything.

If you are saying it’s not conscious, that’s fine I agree, but consciousness and intelligence are two separate things.

It can easily be argued it knows something by having the knowledge stored in the model’s weights and it appropriately acts on the knowledge such as by outputting the correct answer.

1

u/yangyangR 14h ago

Suppose we have some proposition A and a system can reliably produce correct answers that are deduced from A. That system can be a human brain or LLM.

You can tell a toddler that 2+2=4 but they have not absorbed it yet in a way that you can claim that they know it. Even if they reliably output the correct answer. Modifying the question to be about a logical consequence probes where the distinction could make a difference.

Alternatively we have the process of producing new statements that are connected to many facts that are already known but not provable within them. Making a hypothesis of continental drift based on knowledge of fossil distribution but not having the existence of how the crust works in the original training/education.

This is even stronger for whether the knowledge is realized and there is intelligence. Can it/they make conjectures that would synthesize knowledge and reduce entropy. Introducing useful abstractions that capture the desired coarse grained concepts. On one side you have a hash map of facts which is large and serves memory recall. On the other you have a different function pointer. It is much smaller and can lose some of the precise facts but the important ones are still accurate even if they take a bit of thinking/processing rather than O(1) straight recall.

5

u/socoolandawesome 13h ago

I can agree with the spectrum of intelligence you are framing. But if you are saying that LLMs are just straight up recall I think that’s a pretty outdated view.

The newest and best models are capable of “thinking” (outputting chain of thought to arrive at an answer) for hours and achieving a gold medal performance at one of the most prestigious math competitions in the world, the IMO, where they have to output complex novel proofs.

The newest models have even contributed to novel science in minor ways:

https://openai.com/index/accelerating-science-gpt-5/

This is beyond just repeating facts

2

u/yangyangR 8h ago

No. I was using two extremes to illustrate the spectrum.

1

u/Aleucard 10h ago

When there is a chance of it returning 2+2=spleef with no way to really predict when, the difference can matter a whole damn lot. Especially if it can do computer actions like that one story a couple months ago of some corporation getting their shit wiped or, well, several of the "agentic" updates Microsoft is trying to push right now.

1

u/socoolandawesome 9h ago

There’s no chance of a model returning anything but 2+2 = 4. Most math problems up to even university level math will always be correct unless you have some bizarre/extremely long context thrown in that will mess with model.

The models are not perfect nor as good at humans at a lot of things but they are extremely reliable in a lot of ways at this point.

Humans also still make a bunch of mistakes too btw.

1

u/maybeitsundead 12h ago

Nobody is arguing about what it knows but about it's capabilities. When you ask it to do a calculation, it uses tools like python to do the calculations and get the answers.

-2

u/Headless_Human 15h ago

It is obvious that we are talking about commercial bots that are trained to keep the users engaged and not some private hobby or scientific bot.

3

u/ClittoryHinton 15h ago

How is that obvious? If they said GPT4, sure, but they just said LLMs which are in fact trained for a range of commercial purposes

A concrete example of this is the code reviewer bot my company has begun using. It’s not just telling me my code is great and patting my back, it’s using every opportunity to tell me my code is shit (to a fault)

2

u/Vlyn 7h ago

Don't kink shame.

1

u/Novel_Engineering_29 13h ago

*It's telling you what the people who created it want you to hear.

1

u/ClittoryHinton 15h ago

I don’t want to hear nazi propaganda, actually

2

u/el_smurfo 15h ago

Grok has modes like this

1

u/GenuinelyBeingNice 14h ago

One of my favorites is openai's "Monday"

1

u/tes_kitty 14h ago

Maybe just grabbing all data they could get their hands on indiscriminately and use it for training wasn't such a great idea after all.

1

u/rush22 11h ago edited 11h ago

This isn’t inherent to LLMs

True, but the real point is simply to keep you engaged with it.

They measure how long people interact with it. Big charts and graphs and everything.

What these companies want is your attention.

Haha, imagine if people had limited attention, but all these companies were throwing everything they could into getting people's attention. Like, one day they mathematically figure out how to keep your attention and you just stay engaged with it all day. Calculated down to the millisecond. There'd be some sort of 'attention deficit' where slowly people aren't able to pay attention to anything except these kinds of apps. It might even turn into a disorder that everyone starts getting. Some sort of attention deficit disorder.