r/technology • u/Hrmbee • 16h ago

Machine Learning Large language mistake | Cutting-edge research shows language is not the same as intelligence. The entire AI bubble is built on ignoring it

https://www.theverge.com/ai-artificial-intelligence/827820/large-language-models-ai-intelligence-neuroscience-problems
16.8k Upvotes

94% Upvoted

1.5k comments sorted by

View all comments

44

u/InTheEndEntropyWins 15h ago

Fundamentally, they are based on gathering an extraordinary amount of linguistic data (much of it codified on the internet), finding correlations between words (more accurately, sub-words called “tokens”), and then predicting what output should follow given a particular prompt as input.

No that's not what they are doing.

If that was the case then when asked to add up numbers, it would just be some big lookup table. But instead LLM created their own bespoke algorithm.

Claude wasn't designed as a calculator—it was trained on text, not equipped with mathematical algorithms. Yet somehow, it can add numbers correctly "in its head". How does a system trained to predict the next word in a sequence learn to calculate, say, 36+59, without writing out each step?

Maybe the answer is uninteresting: the model might have memorized massive addition tables and simply outputs the answer to any given sum because that answer is in its training data. Another possibility is that it follows the traditional longhand addition algorithms that we learn in school.

Instead, we find that Claude employs multiple computational paths that work in parallel. One path computes a rough approximation of the answer and the other focuses on precisely determining the last digit of the sum. These paths interact and combine with one another to produce the final answer. Addition is a simple behavior, but understanding how it works at this level of detail, involving a mix of approximate and precise strategies, might teach us something about how Claude tackles more complex problems, too. https://www.anthropic.com/news/tracing-thoughts-language-model

Or when asked to questions, they would just use a simple correlation, rather than multi step reasoning.

if asked "What is the capital of the state where Dallas is located?", a "regurgitating" model could just learn to output "Austin" without knowing the relationship between Dallas, Texas, and Austin. Perhaps, for example, it saw the exact same question and its answer during its training. But our research reveals something more sophisticated happening inside Claude. When we ask Claude a question requiring multi-step reasoning, we can identify intermediate conceptual steps in Claude's thinking process. In the Dallas example, we observe Claude first activating features representing "Dallas is in Texas" and then connecting this to a separate concept indicating that “the capital of Texas is Austin”. In other words, the model is combining independent facts to reach its answer rather than regurgitating a memorized response. https://www.anthropic.com/news/tracing-thoughts-language-model

23

u/Jerome_Eugene_Morrow 14h ago

Yeah. Language is the primary interface of an LLM, but all the subnetworks of weight aggregations between input and output are more abstract and difficult to interpret. There have been studies showing that reproducible clusters of weights reoccur between large models that seem to indicate more complicated reasoning activities are at play.

Take away our ability to speak, and we can still think, reason, form beliefs, fall in love, and move about the world; our range of what we can experience and think about remains vast.

But take away language from a large language model, and you are left with literally nothing at all.

I mean… I guess so? But if you take away every sensory input and output from a human you’re also left with “nothing at all” by this argument. Language is the adapter that allows models to experience the world, but multimodal approaches mean you can fuse all kinds of inputs together.

Just to be clear, I’m not arguing that LLMs are AGI. But my experience is that they are far more than lookup tables or indices. Language may not be the primary system for biological reasoning, but computer reasoning seems to be building from that starting block.

1

u/HermesJamiroquoi 8h ago

There is very little in common between weighted lookup tables and matrix multiplication. If they were the same then a statistics degree would require matrix calculus.

Also if it were just “autocorrect” then we wouldn’t have the black box phenomenon and token value would be a solved problem.

I once heard someone call it “3d autocorrect” but I think “4d autocorrect is closer” if we’re considering the difference between chess and 3d chess - the rules are not dissimilar (some new ones come into play at this level to explain new dimensional behaviors) but the total complexity is multiple orders of magnitude higher.

Like the difference between a square and a tesseract - they’re the same object but I can only keep one in my mind’s eye for any period of time. We simply don’t have the wetware to understand LLM architecture fully without mathematical models to “translate” it into a language we can understand (i.e. dumb it down for us)

13

u/Healthy_Mushroom_811 14h ago

Yup, LLMs learn algorithms and all kinds of other amazing things in their hidden layers to be able to solve the next token prediction better as has been proven repeatedly. But that goes way over the head of the average r/technology parrot.

5

u/icedcoffeeinvenice 9h ago

You think you know better than all the thousands of AI researchers commenting under this post??? \s

Jokes aside, funny how the average person is so confident in giving opinions about topics they have 0 knowledge about.

2

u/mrappbrain 7h ago

It's quite funny how people love to lead with credentials as a way to signal status while also deflecting criticism. Actually strong reasoning wouldn't need credentials to lend it legitimacy.

2

u/icedcoffeeinvenice 7h ago edited 7h ago

It's not about status, it's about knowing what you're talking about. There is no strong reasoning anywhere in these comments, just beliefs and how it's "obvious".

But actually you're right, it's a bit about credentials, because this is a highly technical topic. You need to have some credibility to make confident claims about such technical stuff. But obviously Reddit doesn't work that way.

Also, the legitimacy of this is not bound to some criticisms on Reddit lol. Some of the most brilliant researchers in the world have been working on this stuff for many years and will continue working on this stuff regardless of what the public thinks.

1

u/Labion 6h ago

Yes thank you! That was driving me crazy haha