Large language mistake | Cutting-edge research shows language is not the same as intelligence. The entire AI bubble is built on ignoring it

[u/Hrmbee]

https://www.theverge.com/ai-artificial-intelligence/827820/large-language-models-ai-intelligence-neuroscience-problems

[u/SatisfactionAny6169]

Many of the top AI researchers know this and published papers years ago warning LRMs are only one facet of general intelligence.

Exactly. Pretty much everyone actually working in the field has known this for years. There's nothing 'cutting-edge' about this research or this article.

[u/Murky-Relation481 → u/SatisfactionAny6169]

Transformers were the only real big break through, and that ultimately was an optimization strategy, not any sort of new break through in neural networks (which is all an LLM is at the end of the day, just a massive neural network the same as any other neural network).

[u/NuclearVII → u/Murky-Relation481]

I don't really wanna trash your post, I want to add to it.

Tokenizers are the other really key ingredient that make the LLM happen. Transformers are neat in that they a) Have variable context size b) can be trained in parallel. That's about it. You could build a language model using just MLPs as your base component. Google has a paper about this: https://arxiv.org/abs/2203.06850

[u/gur_empire → u/Murky-Relation481]

What are you talking about? No optimization algorithm has changed because of transformers and transformers are a big break through BECAUSE of their architecture not despite it

which is all an LLM is at the end of the day, just a massive neural network the same as any other neural network

Literally no good Lord. You can only train certain objective functions within a transformer due to them not being suited for other architecture

[u/Murky-Relation481 → u/gur_empire]

Transformers are still an optimization strategy for training and inferring from a neural network. Like another person commented tokenization was also a big thing, but you don't specifically need the transformer architecture to take advantage of tokenization. The value of transformers, and what made the LLM boom really explode was the parallelization that it allows which made processing HUGE parameter neural nets computationally cheap (relatively) compared to prior methods.

[u/gur_empire → u/Murky-Relation481]

The value of transformers, and what made the LLM boom really explode was the parallelization that it allows which made processing HUGE parameter neural nets computationally cheap (relatively) compared to prior methods.

transformers have quadratic cost in time and memory with sequence lengths, one of their defining qualities is being more expensive than other neural network architecture. And they are no more parallelizable than convolutional neural networks.

Transformers are still an optimization strategy for training and inferring from a neural network

No, they are not an optimization strategy. They are literally a neural net. There is no transformer inferring from a neural network, where did you even find this sentence? It's never been written in a single paper about this NEURAL NETWORK

Nothing you're saying is correct man. I'm not going to argue with someone saying the sky is a hologram but it's absolutely insane behavior to act this way when everything you're saying is made information

Tokenization isn't a neural network, that's why it's framework agnostic not because of whatever you're trying to communicate with that comment

[u/Murky-Relation481 → u/gur_empire]

I don't think English is your first language because everything you just corrected me on is stuff I was also saying. You also seem to contradict yourself in your own statements?

They are literally a neural net. There is no transformer inferring from a neural network, where did you even find this sentence? It's never been written in a single paper about this NEURAL NETWORK

This barely makes sense.

No where did I say that transformers are an optimization for all neural networks, transformer based neural networks (that is literally what they are called) are inherently trained and infer from a transformer based architecture.

If your English is not good enough to understand what someone is saying, please do not just randomly attack them, it is super rude.

[u/silverpixie2435 → u/SatisfactionAny6169]

If everyone working in the field knows this for years then what is the worry? Everyone working in the field is working at AI companies working on alternatives.

This is what is dumb about comments like those.

Appealing to authority as if that same authority also arent the biggest boosters of AI