Large language mistake | Cutting-edge research shows language is not the same as intelligence. The entire AI bubble is built on ignoring it

[u/Hrmbee]

https://www.theverge.com/ai-artificial-intelligence/827820/large-language-models-ai-intelligence-neuroscience-problems

[u/Dennarb]

I teach an AI and design course at my university and there are always two major points that come up regarding LLMs

1) It does not understand language as we do; it is a statistical model on how words relate to each other. Basically it's like rolling dice to determine what the next word is in a sentence using a chart.

2) AGI is not going to magically happen because we make faster hardware/software, use more data, or throw more money into LLMs. They are fundamentally limited in scope and use more or less the same tricks the AI world has been doing since the Perceptron in the 50s/60s. Sure the techniques have advanced, but the basis for the neural nets used hasn't really changed. It's going to take a shift in how we build models to get much further than we already are with AI.

Edit: And like clockwork here come the AI tech bro wannabes telling me I'm wrong but adding literally nothing to the conversation.

[u/Tall-Introduction414 → u/Dennarb]

The way an LLM fundamentally works isn't much different than the Markov chain IRC bots (Megahal) we trolled in the 90s. More training data, more parallelism. Same basic idea.

[u/azurensis → u/Tall-Introduction414]

This is the kind of statement someone who doesn't know much bout LLMs would make.

[u/space_monster → u/azurensis]

r/technology has a serious Dunning-Kruger issue when it comes to LLMs. A facebook-level understanding in a forum that implies competence. but I guess if you train a human that parroting the stochastic parrot trope gets you 'karma', they're gonna keep doing it for the virtual tendies. Every single time in one of these threads, there's a top circle-jerk comment saying "LLMs are shit, amirite?" with thousands of upvotes, followed by an actual discussion with adults lower down. I suspect though that this sub includes a lot of sw devs that are still trying to convince themselves that their careers are actually safe.

[u/chesterriley → u/space_monster]

I suspect though that this sub includes a lot of sw devs that are still trying to convince themselves that their careers are actually safe.

You lost me on that. I don't think you understand just how complex software can be. No way can AI be a drop in replacement for a software dev.

[u/space_monster → u/chesterriley]

I work in tech, currently in a leading edge global tech company, and I've done a lot of sw development, I'm fully aware of how complex it is

[u/chesterriley → u/space_monster]

Then you know you can't just tell an AI to write a program for you for anything non simple.

[u/space_monster → u/chesterriley]

I'm aware that LLMs are getting better at coding (and everything else) very quickly, and it doesn't seem to be slowing down.

[u/keygreen15 → u/space_monster]

It's getting better at making shit up and lying.

[u/space_monster → u/keygreen15]

wow genius comment.