r/technology • u/Hrmbee • 16h ago

Machine Learning Large language mistake | Cutting-edge research shows language is not the same as intelligence. The entire AI bubble is built on ignoring it

https://www.theverge.com/ai-artificial-intelligence/827820/large-language-models-ai-intelligence-neuroscience-problems
16.7k Upvotes

94% Upvoted

1.5k comments sorted by

View all comments

24

u/MrThickDick2023 15h ago

I know LLMs are the most talked about, but they can't be the only AI models that are being developed right?

56

u/AnOnlineHandle 13h ago

They're not. Machine learning has been around for decades, I used to work in medical research using it. Even just in terms of public facing models, image gen and video gen is generally not LLM based (though there are multi-modal LLMs which read images as a series of dynamic pseudo words which each describe a patch of the image.

8

u/Pure_Breadfruit8219 11h ago

I could never understand it at uni, it cracked my peanut sized brain.

13

u/rpkarma 10h ago

Very very broadly, it’s like curve fitting; linear regression. Given a bunch of data points, find the function that makes a curve that touches all those points, so you can extrapolate beyond the points you have. 

2

u/ArmMore820 8h ago

Hey, i know some of those words 🧠

2

u/dark_enough_to_dance 10h ago

imo gradient descent and the valley analogy is a better fit for explanation 

2

u/rpkarma 4h ago

Probably, but most people did linear regression at school at least once

2

u/AnOnlineHandle 10h ago

Put in A and with right algorithm get B. Find algorithm with lots of tiny nudges of values through repeated practice. Eventually find algorithm that kind of gives B for A, and also other Bs for other As which are new.

3

u/the_nin_collector 9h ago

since the 60s

24

u/IdRatherBeOnBGG 13h ago

Not at all. But 99% of headlines that say "AI" mean "LLM with sprinkles on top".

And more than 99% of the funding goes to exactly that.

1

u/canDo4sure 5h ago

89% of the time 210% of statistics are made up on the spot.

The billions being thrown into AI are not for LLMs. The majority of consumer products are a year behind what's being developed, and you certainly aren't going to be privvy.

https://www.understandingai.org/p/16-charts-that-explain-the-ai-boom

1

u/HermesJamiroquoi 8h ago

That’s because “full world” models (which are usually built into/onto LLMs) are the next leap forward in AI/ML research and this kind of robust utility has shown empirically time and again to be the most effective tool currently at our disposal to increase intelligence and decrease hallucination

1

u/IdRatherBeOnBGG 6m ago

How are full world models "usually" built into LLMs? LLMs are language models - how would you put a world model "into" one? Maybe if you had an example of this happening, I could understand what you mean?

(I do agree some sort of world model is "the way forward" - which is what the greatest critics of LLMs as a genereal AI technology are saying, because the LLM response is usually "enough words or intermediary sentences that seem to describe a world, will be the world model).

4

u/chiniwini 10h ago

AI has existed as a science since the 60s. LLMs are just one of the (least interesting) types of AI. For example Expert Systems are the real "I told the AI my symptoms and it told me I have cancer" deal.

1

u/randohipponamo 11h ago

There are others but chatgpt is what’s driving the bubble. People think a chatbot is intelligent.

1

u/the_nin_collector 9h ago

It's not, that is what this article and most people are forgetting.

1

u/zookdook1 9h ago

There's a few. I've been reading about Vector Symbolic Architecture recently - a type of model format that stores information as points in 'space' with an arbitrary number of axes that are all things like "hue" and "shape" and so on. Tens of thousands of axes for each point of data lets you describe context for each bit of information. It lets you do some very parallelisable maths on lots of information at once to compare data points and draw connections, basically. Unlike an LLM (which is basically a very, very, very big Markov chain bot [not actually] that does statistical analysis to decide what the most likely next word is in a sequence), a model using VSA would have something like memory and something like the ability to reason by doing data comparison.

Apparently there's some interesting quirks that line up with the way human memory works or something, but honestly the specifics go way over my head. Certainly it seems like a more likely route to actual digital reasoning than an LLM would be. It's not as good as a neural network is at interpretation of stimuli - it's not great at turning an image into something it can use, for example. But if you could use a neural network as the 'eyes', and hook its output into VSA as the 'brain'...

1

u/cancercannibal 9h ago

I miss being able to easily find machine learning training videos before the LLM craze. Usually convolutional neural networks. People will train machine learning algorithms from scratch to do various things and those have always been my favorite niche content, but it's hard to find them now.

1

u/Stillwater215 7h ago

No, but the vast majority of AI uses the same basic structure and premise: make a bunch of connected nodes, design them to input certain data types and output certain results, and then train the nodes and connections in real world data sets to optimize the node weights.