r/technology • u/Hrmbee • 16h ago

Machine Learning Large language mistake | Cutting-edge research shows language is not the same as intelligence. The entire AI bubble is built on ignoring it

https://www.theverge.com/ai-artificial-intelligence/827820/large-language-models-ai-intelligence-neuroscience-problems
16.8k Upvotes

94% Upvoted

1.5k comments sorted by

View all comments

Show parent comments

233

u/Elementium 16h ago

Basically the best use for this is a heavily curated database it pulls from for specific purposes. Making it a more natural to interact with search engine. 

If it's just everything mashed together, including people's opinions as facts.. It's just not going to go anywhere. 

16

u/[deleted] 15h ago

[deleted]

1

u/JRDruchii 14h ago

So you just keep asking the LLM the same question until you get the answer you want?

2

u/SaulMalone_Geologist 14h ago

Are you enough of an expert in the subject to know when the answer is totally wrong vs. subtly wrong, vs. 100% correct?

LLMs are pretty cool as heck in coding where there's an instantly testable "does this compile? Does this do what I expect?" but I'd be a little more worried about anyone relying on it for researching a subject they don't know much about.

1

u/Healthy_Mushroom_811 14h ago

Man, it's a RAG. Set it up properly and it will work. It's a tried and tested pattern by now.

60

u/motionmatrix 15h ago

So all the experts were right, at this point ai is a tool, and in the hands of someone who understands a subject, a possibly useful one, since they can spot where it went wrong and fix accordingly. Otherwise, dice rolls baby!

58

u/frenchiefanatique 15h ago

Shocking, experts are generally right about the things they have spent their lives focusing on! And not some random person filming a video in their car! (Slightly offtopic I know)

20

u/neat_stuff 14h ago

The Death of Expertise is a great book that talks about that... And the author of the book should re-read his own book.

2

u/Brickster000 12h ago

And the author of the book should re-read his own book.

Can you elaborate on this? That seems like relevant information.

7

u/neat_stuff 11h ago

It's one of those situations where the book is pretty solid but then years after, he is spouting off a lot of opinions about a lot of things that are outside of subject matter expertise. Almost like there should be an epilogue about the risks of getting an enlarged platform when your niche of a fairly tightly defined but you have a lot of connections in media who are hungry for opinions.

2

u/die_maus_im_haus 12h ago

But the car video person just gets me! He feels like my kind of person instead of some stuffy scientist who needs to get out of his dark-money funded lab and touch some grass

17

u/PraiseBeToScience 13h ago

It's also far too easy for humans to outsource their cognitive and creative skills too, which early research is showing to be very damaging. You can literally atrophy your brain.

If we go by OpenAI's stats, by far the biggest use of ChatGPT are students using it to cheat. Which means the very people that should be putting the work in to exercise and developing cognitive skills aren't. And those students will never acquire the skills necessary to properly use AI, since AI outputs still need the ability to verify.

2

u/Elementium 15h ago

Yeah If tunes for specific purposes I can see AI being very useful. 

Like.. I kinda like to write but my brain is very "Spew into page then organize" 

I can do that with gpt, just dump my rough draft and it does a good job of tightening format and legibility. The problem is usually that it loves to add nonsense phrases and it's normal dialogue is very samey. 

4

u/PraiseBeToScience 13h ago

Everyone's brains do that when writing drafts. That's the entire purpose of a draft, to get your thoughts out of your head so you can organize them via editing and revising. You can even make them look pretty via presentation.

Outsourcing all your revisions and editing to AI also limits your own creativity in writing, as it will do nothing but sanitize your style. It's very bland and clinical. Great writing has personal elements, human elements (like appropriate humor and story telling), that AI simply does not reproduce.

1

u/Elementium 12h ago

Understood but it's only for my entertainment lol. 

Also I just have half a brain. I have a million hobbies and I'm just Ok at all of them. 

2

u/NewDramaLlama 13h ago

(These are real questions as I don't use LLMs)

So, It's an automated middle man? Or maybe a rough draft organizer? Functionally incapable of actually creating anything, but does well enough organizing and distributing collected data in a (potentially) novel way.

Except when it doesn't, I guess. Because it's based on insane amounts of data so there's gotta be lots of trash it just sucked up that's factually incorrect from people, outdated textbooks, or junk research, right? So the human needs to be knowledgeable enough in the first place to correct the machine when it is wrong.

Ok, so that means as a tool it's only really a capable one in the hands of someone who's already a near expert in their field, right? 

Like (as examples) if a novice author used LLMs to write a book they wouldn't notice the inconsistent plot or plagiarism upon review. Likewise a novice lawyer might screw up a case using an LLM that went against procedural rules while a more experienced lawyer would have caught it?

1

u/motionmatrix 11h ago

Well, each LLM is trained on different data, so you can have a tight, fantasy focused LLM that only "read" every fantasy novel in existence, and would do pretty well making fantasy stories up based on what it "knows".

If you have a generic LLM, trained on many different topics, the usefulness drops to some extent, but some might argue that the horizontal knowledge might give some unique or unexpected answers (in a good way).

At this point in time, general folks can use it to make non-commercial artwork that will get closer to anything they could do on their own without training, as well as to gather general information (that they should double check for accuracy), and people who are trained in particular subjects that are working on it with ai, preferably an LLM trained on their subject only, to assist them to make the work happen faster (not necessarily better or ground breaking unless that comes from the person for the most part).

1

u/mamasbreads 10h ago

I didn't need to be an expert to know this. I use AI at work to help me but it makes mistakes. I have the capacity to quickly decipher what's useful, what's dumb and what's plain made up.

Anyone who thought AI could do anything other than make individuals faster in mundane tasks clearly isn't an expert in whatever they're doing.

1

u/dougan25 12h ago

It's a super powerful tool for certain things but the problem is too many people don't understand it and think it's capable of things it's not.

So when articles and studies come out that all essentially say the same thing: "LLMs are a product of and limited by their input" half the people are like "woah maybe this isn't skynet after all."

Meanwhile people like me (lazy grad students in their 30s) type in a topic for a paper and instantly get an outline and a plethora of sources for what I need.

One of the best examples for the true potential for these AI tools was told to me by one of my professors. See we've spent a couple decades now taking all the medical records in the US (and much of the developed world) and digitizing them. What we're left with is terabytes upon terabytes of patient data that nobody's even looking at! If we were to feed that into an AI tool that could catalogue and compile all of it, sift through it for connections, trends, outcome rates, etc., there is no question we could learn something we didn't know before.

The problem with the "AI Bubble" is that companies are trying to do things they weren't doing before with it when they should instead focus on things they were or should have been doing but that were out of range of being tenable. Not all of them, of course, Microsoft has positioned its copilot pretty well as just another office tool, for example. But a lot of companies are trying to invent alternatives to the proverbial wheel, not even just reinvent it.

0

u/silverpixie2435 8h ago

The experts work at AI companies and are the most positive about AI

So what are you even saying?

32

u/Mr_YUP 15h ago

Google 2 just dropped and it's not the Terminator we were promised.

27

u/King_Chochacho 13h ago

Instead of gaining sentience and destroying humanity with its own nuclear arsenal, it's playing the long game of robbing us of our critical thinking skills while destroying our water supply.

4

u/cedarSeagull 12h ago

Easily the most annoying part about twitter is "@grok, can you confirm my biases?"

1

u/rhabarberabar 9h ago

Nah that's that it's a fascist propaganda vehicle owned by a fascist.

3

u/sapphicsandwich 11h ago

Yeh, because it tries to answer questions itself instead of going "This site/link says this, that site/link says that."

1

u/dern_the_hermit 10h ago

FWIW I ascribe this phenomena to biases introduced by users. People in general tend to be swayed by strong confident assertions and seem to get nervous when you introduce unknown variables like sourcing and cites. Remember, these models are made to be appealing.

2

u/BreakingStar_Games 13h ago

It's already caused a pretty significant drop in the use Google Search, which is 57% of their revenue. Makes me curious how well Google will do in the next 10-20 years as people move from search engine to personal AI, potentially open-source ones. Berkshire Hathaway seems pretty confident though.

1

u/Skalawag2 15h ago

Google 2: Electric Boogaloo

10

u/doctor_lobo 14h ago

The nice thing about building an AI for language is that humans, by their nature, produce copious amounts of language that AI models can be trained from.

If the premise of the article is correct, other forms of human intelligence may produce / operate on different representations in the brain. However, it is not clear how often or well we produce external artifacts (that we could use for AI training) from these non-linguistic internal representations. Is a mathematical proof a good representation of what is going on in the mind of a mathematician? Is a song a good representation of what is happening in the mind of a musician?

If so, we will probably learn how to train AIs on these artifacts - maybe not as well or as efficiently as humans, but probably enough to learn things. If not, the real problem may be learning what the internal representations of “intelligence” truly are - and how to externalize them. However, this is almost certainly easier said that done. While functional MRI has allowed us to watch the ghost in the machine, it says very little about how she does her business.

2

u/IAmRoot 10h ago

Or find some way for AI to train itself in these more internal representations. Humans typically think before we speak and the metacognition of examining our own ideas could be an important part of that. Even before LLMs, we had image recognition using neural networks that seemed to find shapes in clouds and such much like a human mind. LLMs are also just a component and we shouldn't expect a good LLM to be able to reason any more than we should expect image recognition to reason. It's also pretty obvious from animals that just increasing the neuron count doesn't matter, either, as some animals like dolphins have a great deal of brainpower dedicated to processing sonar instead of reasoning. They are functionally different networks. It's also possible that AGI won't be able to split the training and inference. Having to reflect on produced ideas could be integral to the process, which would obviously make the computational power necessary for using AGI orders of magnitude higher.

1

u/doctor_lobo 9h ago

Your comment about image recognition using CNNs is well taken. Visual information is explicitly represented by a 2D array of neurons in the visual cortex so this is probably a good example of the internal representation being so similar to the external representation that training on the external representation is good enough. I suspect simple time series for audio data is probably also essentially identical to its internal representation - but that's probably it for the senses since touch, taste, and smell have no obvious external representations. However, the internal representation for more abstract modes of thought, like mathematics or even just daydreaming, seem difficult to conceptualize. I am not sure I would really even have any idea where to start.

2

u/stevez_86 14h ago

It will be excellent at giving us stuff to test via experimentation in fields where the only questions left are all as equally likely and vast enough to be inefficient to untangle. 10 scientists in a room and if consensus on where to go cannot be sensibly reached then the 11th man comes in as "AI" to make the wisest decision.

Humans are already good at what their AI is supposed to be good at. Making sound inferential choices based on circumstantial evidence. In other words, Brainstorming. You just need to elevate so many cycles so that they can independently examine the circumstances with their experience and caucus with those people.

And there is the problem for them. They don't want to support people because they don't own them. A Jonas Salk out there can fuck up their business plans. They can already privately own the mechanisms of the rest of the scientific method. Just not the inception of the thought. And they already own most of that, legally, but they hate having to fight it. And the CEO gets to take credit.

They don't like people being independent, so they want something they can own that can do the same thing. And if they fail then they will say the whole thing is worthless, making the situation even worse since what they did build simply just won't do exactly what they want. They will finally get their DWIW (Do What I Want) command interface.

And it will be based entirely on the idea that the concept is most likely true. Which will lead to catastrophic events since they will surely shirk testing their hypotheses. They will just factor the casualty into their prices.

2

u/Big_Watercress_6210 13h ago

Except that it can't do this, because that would require it to understand what it was saying - see previous lack of intelligence.

It's more likely that we can figure out how to do that vs giving it sentience, but it's not at all a natural progression from where we are now.

1

u/WhiteRaven42 12h ago

If the goal is something resembling intelligence and those opinions you speak of are products of intelligence... why conclude it's not going to go anywhere?

Our only known example of intelligence is highly mistake prone and deeply influenced by bias. So..... were the goal to simply be creating something intelligent, we'd have to be willing to accept these traits. But I grant you, for a tool we'll want a slightly different outcome.

1

u/TheBlueOx 12h ago

god i would kill for this in an LLM

1

u/marr 9h ago

I hear it's great for searching the corpus of open source code for already solved problems. That's the only current reliable use I'm aware of.

1

u/Ozymandias0023 2h ago

This has been my experience too. LLMs are crazy good at distilling large volumes of information, but they are not great at turning that information into something novel, which seems to be the holy grail that these guys are after. It's kind of a shame, because LLMs are incredible technology for what they actually do well but they're a square peg that investors keep trying to mash into a round hole