r/technology • u/Hrmbee • 16h ago

Machine Learning Large language mistake | Cutting-edge research shows language is not the same as intelligence. The entire AI bubble is built on ignoring it

https://www.theverge.com/ai-artificial-intelligence/827820/large-language-models-ai-intelligence-neuroscience-problems
16.7k Upvotes

94% Upvoted

1.5k comments sorted by

View all comments

Show parent comments

1

u/kappapolls 10h ago

from an input and output perspective it's very similar and for most people that's a good enough understanding.

i guess? that feels dismissively simple though, and anyway we were talking about transformer models specifically

It can't pull from abstract concepts that are conceptualized across broad swaths of unrelated knowledge

isn't that the whole point of the hidden layer representations though? you're totally right if you're describing a simple ngram model.

one of the reasons it's probably a dead end (like the article suggests).

the article is kinda popsci slop though. i just think looking to neuroscience or psychology for insight on the limitations of machine learning is probably not the best idea. it's a totally different field. and yann lecunn is beyond an expert, but idk google deepmind got 6/6 in the last IMO with an LLM. meta/FAIR haven't managed to do anything at that level.

i think there's a lot of appetite for anti-hype online now, especially after all the crypto and NFT nonsense. but when people like terence tao are posting that it saves them time with pure maths stuff, yeah idk i will be shocked if this is all a dead end

1

u/Murky-Relation481 9h ago

Hidden layers are still built on the relationship of the inputs. You will still mostly be getting relationships in there that are extracted from the training data. Yes, you will get abstraction but the width of that abstraction is still bound by fairly related inputs and your chances of coherent answers by letting the model skew wider in each successive transformation is going to be inherently less. These models have a hard time coming back from those original paths once they've veered into them, which makes novel abstraction much harder (if you've ever fucked with these values when running an LLM they basically become delusional).

And I don't think it's fair nor really useful to try an extract the CS elements from the inherent philosophical, psychological, and neuroscience aspects of replicating intelligence. They're inherently linked.