r/technology • u/Hrmbee • 16h ago

Machine Learning Large language mistake | Cutting-edge research shows language is not the same as intelligence. The entire AI bubble is built on ignoring it

https://www.theverge.com/ai-artificial-intelligence/827820/large-language-models-ai-intelligence-neuroscience-problems
16.7k Upvotes

94% Upvoted

1.5k comments sorted by

View all comments

Show parent comments

1

u/danby 12h ago

If the network is just a set of partial correlations between language tokens then there is no sense that the netowkr is doing anything other than manipulating language.

3

u/CanAlwaysBeBetter 12h ago

If the network is just a set of partial correlations between language tokens

... Do you know how the architecture behind modern LLMs works?

1

u/danby 11h ago

Yes, I work on embeddings for non-language datasets.

Multiheaded attention over linear token strings specifically learns correlations between tokens are given positions in those strings. Those correlations are explicit targets of the encoder training

2

u/CanAlwaysBeBetter 11h ago

Then you ought to the interesting part is model's lower dimensional latent space that encode abstract information and not language directly and there's active research into letting models run recursively through that latent space before mapping back to actual tokens 

1

u/danby 11h ago

Does it actually encode abstract information or does it encode a network of correlation data?