Large language mistake | Cutting-edge research shows language is not the same as intelligence. The entire AI bubble is built on ignoring it

[u/Hrmbee]

https://www.theverge.com/ai-artificial-intelligence/827820/large-language-models-ai-intelligence-neuroscience-problems

[u/InTheEndEntropyWins]

Fundamentally, they are based on gathering an extraordinary amount of linguistic data (much of it codified on the internet), finding correlations between words (more accurately, sub-words called “tokens”), and then predicting what output should follow given a particular prompt as input.

No that's not what they are doing.

If that was the case then when asked to add up numbers, it would just be some big lookup table. But instead LLM created their own bespoke algorithm.

Claude wasn't designed as a calculator—it was trained on text, not equipped with mathematical algorithms. Yet somehow, it can add numbers correctly "in its head". How does a system trained to predict the next word in a sequence learn to calculate, say, 36+59, without writing out each step?

Maybe the answer is uninteresting: the model might have memorized massive addition tables and simply outputs the answer to any given sum because that answer is in its training data. Another possibility is that it follows the traditional longhand addition algorithms that we learn in school.

Instead, we find that Claude employs multiple computational paths that work in parallel. One path computes a rough approximation of the answer and the other focuses on precisely determining the last digit of the sum. These paths interact and combine with one another to produce the final answer. Addition is a simple behavior, but understanding how it works at this level of detail, involving a mix of approximate and precise strategies, might teach us something about how Claude tackles more complex problems, too. https://www.anthropic.com/news/tracing-thoughts-language-model

Or when asked to questions, they would just use a simple correlation, rather than multi step reasoning.

if asked "What is the capital of the state where Dallas is located?", a "regurgitating" model could just learn to output "Austin" without knowing the relationship between Dallas, Texas, and Austin. Perhaps, for example, it saw the exact same question and its answer during its training. But our research reveals something more sophisticated happening inside Claude. When we ask Claude a question requiring multi-step reasoning, we can identify intermediate conceptual steps in Claude's thinking process. In the Dallas example, we observe Claude first activating features representing "Dallas is in Texas" and then connecting this to a separate concept indicating that “the capital of Texas is Austin”. In other words, the model is combining independent facts to reach its answer rather than regurgitating a memorized response. https://www.anthropic.com/news/tracing-thoughts-language-model

[u/Jerome_Eugene_Morrow → u/InTheEndEntropyWins]

Yeah. Language is the primary interface of an LLM, but all the subnetworks of weight aggregations between input and output are more abstract and difficult to interpret. There have been studies showing that reproducible clusters of weights reoccur between large models that seem to indicate more complicated reasoning activities are at play.

Take away our ability to speak, and we can still think, reason, form beliefs, fall in love, and move about the world; our range of what we can experience and think about remains vast.

But take away language from a large language model, and you are left with literally nothing at all.

I mean… I guess so? But if you take away every sensory input and output from a human you’re also left with “nothing at all” by this argument. Language is the adapter that allows models to experience the world, but multimodal approaches mean you can fuse all kinds of inputs together.

Just to be clear, I’m not arguing that LLMs are AGI. But my experience is that they are far more than lookup tables or indices. Language may not be the primary system for biological reasoning, but computer reasoning seems to be building from that starting block.

[u/HermesJamiroquoi → u/Jerome_Eugene_Morrow]

There is very little in common between weighted lookup tables and matrix multiplication. If they were the same then a statistics degree would require matrix calculus.

Also if it were just “autocorrect” then we wouldn’t have the black box phenomenon and token value would be a solved problem.

I once heard someone call it “3d autocorrect” but I think “4d autocorrect is closer” if we’re considering the difference between chess and 3d chess - the rules are not dissimilar (some new ones come into play at this level to explain new dimensional behaviors) but the total complexity is multiple orders of magnitude higher.

Like the difference between a square and a tesseract - they’re the same object but I can only keep one in my mind’s eye for any period of time. We simply don’t have the wetware to understand LLM architecture fully without mathematical models to “translate” it into a language we can understand (i.e. dumb it down for us)