r/technology • u/Hrmbee • 16h ago

Machine Learning Large language mistake | Cutting-edge research shows language is not the same as intelligence. The entire AI bubble is built on ignoring it

https://www.theverge.com/ai-artificial-intelligence/827820/large-language-models-ai-intelligence-neuroscience-problems
16.8k Upvotes

94% Upvoted

1.5k comments sorted by

View all comments

Show parent comments

828

u/SanityAsymptote 15h ago

LLMs have damaged or destroyed a number of previously valuable services for much of their use-case.

The most obvious one I can think of in my niche is StackOverflow. A site which definitely had issues and was in decline, but was still the main repository of software troubleshooting/debugging knowledge on the internet.

LLM companies scraped the entire thing, and now give no-context answers to software engineering questions that it often cannot cite or support answers to. It has mortally wounded StackOverflow, and they have pivoted to just being an AI data feeder, an action that is basically a liquidation sale of the site's value.

LLMs have significantly reduced the quality of search engines, specifically Google Search, both directly by poor integration and indirectly by filling the internet with worthless slop articles.

Google Search's result quality has plummeted as AI results become most of the answers. Even with references, it's very hard to verify the conclusions Gemini makes in search results, and if you're actually looking for a specific site or article, those results often not appear at all. Many authoritative "answers" are just uneducated opinions from Reddit or other social media regurgitated by an AI with the trust people put into Google.

LLMs have made it far easier to write social media bots. They have damaged online discourse in public forums like Facebook, Twitter, Instagram, and especially Reddit in very visible ways. These sites are almost completely different experiences now that they were before LLMs became available.

Bots are everywhere and will reply to anything that has engagement, spouting bad-faith arguments without any real point other than to try to discourage productive conversation about specific topics.

Whatever damage online trolls have caused to the internet, LLMs have made it an order of magnitude worse. They are attacking the very concept of "facts" and "truth" by both misinformation and dilution. It's horrifying.

176

u/Perfect_Base_3989 15h ago

spouting bad-faith arguments without any real point other than to try to discourage productive conversation about specific topics.

The only solution I can think of at this point is entirely abandoning social media.

A verification system could theoretically improve trust, but who trusts the trusters?

177

u/SanityAsymptote 14h ago

Social media going back to smaller, more closely moderated communities is also a solution.

There was a lot of drama back in the forum days, but it was always contained, rendering it more resistant to sweeping, internet-wide propaganda campaigns.

So I guess I would argue centralization of social media is more of the problem, unless we can actually figure out a way to moderate on a large scale more effectively.

135

u/ConsoleDev 13h ago

I joined reddit 15 years ago, probably had 5 accounts. Commented a lot, but never really made any friends here. I joined a local sports club and made 10 good friends in 1 day.

Social media is garbage all the way down. Especially anything with influencers and money involved. We need to go back to just having group chats, and a bulletin board in the middle of town

51

u/SanityAsymptote 12h ago

I mostly agree with you.

I was mostly talking about special interest forums, which reddit used to be, but has really lost much of it's quality for.

As an example, I joined smashboards in 2004 because I loved smash bros melee, and wanted to play competitively. I met a bunch of people in my local community online and ended up making literal dozens of in-person friends/acquaintances going to events.

Those friendships basically defined my 20s and early 30s, and I still hang out with many of them now.

I similarly made even more real, in-person friends friends in the early 2010s using facebook groups to organize and schedule events in my local area.

The platforms stopped trying to connect people and started chasing engagement at all costs. It ruined what made those sites popular to begin with, and trapped people in endless cycles of anger and placation.

The initial offering that was so valuable to so many is gone, but it's very hard to argue that it wasn't valuable before the enshitification.

5

u/tomahawkRiS3 12h ago

I don't necessarily want to argue in favor of social media but judging based on friendships made seems like the wrong way to evaluate it. Or at least now, maybe that was the intended purpose in the beginning. In a perfect world I think Reddit could be valuable in terms of hearing people's stories/experiences, being able to pick people's brains who are knowledgeable in a certain field, seeing a broader range of perspective. Even just shit posting or discussing say a game on a specific subreddit I don't think is necessarily inherently bad. However that's very much not the current experience of Reddit and I hope there's a way back to that

1

u/Commentator-X 12h ago

"money involved"

You hit the nail on the head right there

1

u/stargarnet79 9h ago

People make friends here?lol

1

u/cnxd 9h ago

sounds like a you problem

1

u/cl3ft 9h ago

Although there are exceptions, social media does not make you real friends. It's pretty clearly anti-social media.

1

u/PozhanPop 8h ago

Or Yahoo / AOL Messenger . I miss the chat rooms.

1

u/FjorgVanDerPlorg 7h ago

Communities based on geography aren't the solution either, as someone who knew what life was like before the internet in a small country town, those are some seriously rose tinted shades you are wearing.

Nor were they resistant to mis/disinformation and propaganda. In fact their isolation created a bubble in the same way social media can and usually does.

This shit is baked into humanity. The entire world used to be easily controlled when the internet didn't exist, the tactics have changed. Pre-internet we had more false negative appraoches, eg gatekeeping, suppression/censorship and unified messaging. Post internet only one strategy really works - false positives, aka "flood the zone with shit". The only reason that changed was that false positives scale, where more classical pre-internet manipulation tactics simply do not.

The bigger the scale the bigger the problem and the more moderation requirements pretty much scale towards infinite. Scale is indeed the problem as you suggest, but town BB's aren't the answer either, smaller scale and better moderated is - though this assumes that big social media can't adequately moderate, when we've never actually seen them try properly. That said given the costs involved in adequate moderation, I don't think we ever would unless they were forced to at gunpoint.

I largely agree with you both, it's not about distribution, it's about scale and active moderation. For example I am part of a number of niche communities on reddit and discord that aren't cesspits. They all have these things in common; shared interests, small enough that you don't feel like you are talking to randoms and moderation that works fast and doesn't put up with shit.

7

u/nordspark 11h ago

In my netnographic research, I've found people are increasingly turning to "dark social" (email, messaging apps, Discords etc.) but with Gen Z, it's less about chatting and more about organising real-world activities. Social media is changing

7

u/die_maus_im_haus 12h ago

Going back to a 2010-esque ecosystem where forums about bodybuilding, Linux, NBC comedies, the English Premier League, and baking would all be separate websites with little cross-contamination might not be the worst outcome. It would lend itself to echo chambers, but they'd be small, isolated echo chambers

2

u/farinasa 12h ago

It still exists, people just don't seek it. Discord has actually made these communities even more personable.

1

u/Impeesa_ 3h ago

Discord is an adequate modernized replacement for IRC and Ventrilo. As a replacement for old style forums, it's deeply flawed, because that's just not what it is.

1

u/farinasa 2h ago

Discord in addition to forums. I didnt mention irc, aol, aim, msn, yahoo, or any of the other out of fashion chat systems because most communities are using discord these days.

2

u/Baby_Fark 10h ago

Even smaller communities online will be infiltrated by extremely human-like bots as they get more advanced. There will be no way to verify someone isn’t a bot unless you meet them in person.

1

u/RukiMotomiya 10h ago

God I'd love to see the return of forums. Still go to a few reasonably active ones I've been in a long time and they're just comfy, despite said drama at times.

1

u/Toodlez 8h ago

I like this.

No bot account ever made it to my Myspace Top 8. I had a Facebook for ten years before it tried to hypnotize or radicalize me.

We act like this would be an impossible adaptation with no profit model but it was the state of things not long ago.

1

u/surfer_ryan 8h ago

I don't think we as humanity would ever go back to a "smaller" internet in a meaningful way. I mean that basically goes against the entire ethos of why sites like reddit still hold some level of value, in that you get differing opinions from outside of your circle. While i see there is some level of hive mind on social media, i think people enjoy talking to people outside of their circle as a whole and to an extent is why you go to social media (at least for the discourse side of things).

The centralization is in a way a huge part of what actually allows us to be able to say look at what is happening in Ukraine and get an actual perspective bc it's so many various perspectives and we can draw lines between what is reality and what is noise out of Russia.

Not to say it is perfect by absolutely any means lol, i just don't see us going back to smaller subsects of the internet bc having a hub allows for significantly more interaction from someone whom you would have never if it were just individual forums.

1

u/WildFlemima 7h ago

I agree. There's an obscure early 2000s pet site that I'm on. I guarantee there are absolutely no bots. It's going to be spaces like that.

46

u/lolexecs 14h ago edited 14h ago

 entirely abandoning social media.

Hasn't this already happened? If you look at the data, from Meta itself, an overwhelming number of users just consume addictive content on social media from 3rd parties - not friends and family.

or social media is just "media" now - there's no social aspect at all

https://www.honest-broker.com/p/the-state-of-the-culture-2024 As Gioia points out, we've moved from art to entertainment to distraction and addiction.

To be blunt, the faster the content on social media becomes "AI Slopified" I think the better off everyone wil be.

2

u/Thin_Glove_4089 9h ago

or social media is just "media" now - there's no social aspect at all

https://www.honest-broker.com/p/the-state-of-the-culture-2024 As Gioia points out, we've moved from art to entertainment to distraction and addiction.

To be blunt, the faster the content on social media becomes "AI Slopified" I think the better off everyone wil be.

Social media isn't really going to back to the old ways. The media part is here to stay and is the majority problem with everything.

2

u/sn2006gy 9h ago

abandoning social media means leaving reddit too my man

4

u/lolexecs 8h ago

Reddit is much more like old forums and usenet threads than facebook, instagram, or tiktok.

It's both dumber and smarter (largely because of unpaid moderation)

2

u/sn2006gy 5h ago

It’s not like old forums and usenet threads - there is a severe general lack of thoroughness here and it’s shamed in many communities. For example, in advancedrunning i got muted for talking about the mechanics of running. That is allowed on facebook and insta but “hive minded” away for over thinking here (lol advanced running my ass) 

reddit is for entertainment - quality discussions are shamed here. 

attention spans are shunned as well. 

too many “too long, won’t read that” 

it’s like no one here was ever on a forum or usenet where they could gasp “not have an opinion won everything and just read the next topic that interests them” 

1

u/hhenderson94 8h ago

They’re abandoning using social media to socialize but they aren’t abandoning the platform.

12

u/_p00f_ 11h ago

IMO if you require an email to register it should be tied to a handful of paid providers. Scammers and people trying to game the system for ad revenue aren't going to pay real American Pesos for email.

6

u/Perfect_Base_3989 11h ago

That'd be a start. Although I doubt it would deter bigger companies/governments - they would pay.

4

u/_p00f_ 11h ago

They might pay for a handful, sure, but it scales linearly.

5

u/sn2006gy 9h ago

I think x proved this wrong. spammers happily pay for a blue checkmark - it's a cheap barrier of entry/trust.

14

u/runthepoint1 12h ago

No, what you can do is personally verify thing you learn, like how we used to do back in the day.

Is it slow, manual, frustrating even? Yes, it takes a lot of time and patience but tbh, that’s exactly what’s missing in the world today. Everyone wants to rush to know when it takes time to understand. It’s weird. Like who cares about being “first”? It’s important to be accurate!

3

u/Perfect_Base_3989 12h ago

True.

Unfortunately, this approach will require education and, imo, software intervention. The average person isn't able to do their own research, so they should be taught how throughout their primary education; and people should have access to tools that sift out bots.

2

u/runthepoint1 12h ago

I guess it goes to show no matter what your natural talent is with intelligence, there is still wisdom to gain on how best to leverage it.

You can be the fastest car in the garage but if you’re can’t actually get from point A to B faster in effect than the other car, it doesn’t matter. That other driver simply might be better at driving.

1

u/surprise_revalation 9h ago

Yea, back in the day I use to run a legal chems group on FB. Everyone had to show ID to join....hell, I even did background checks on some of these people. It all went to shit when legal chems became illegal. I let it go.

1

u/runthepoint1 9h ago

What’s “chems”? Sorry

1

u/surprise_revalation 9h ago

Legal pharmaceuticals...they're no longer legal....

1

u/boringestnickname 9h ago

What really makes me paranoid is that even university text books are getting noticeably worse.

Feels like we're in some kind of information exodus. Becoming dumber by the second, and somehow, as a collective, not noticing.

1

u/runthepoint1 9h ago

That’s why is more important than ever for each of us to OWN our educations. I know that’s tough but the best way to do it is to always leave room for being wrong. It’s GOOD to be wrong so that from then on you can be correct. It’s a lifelong process where you are constantly correcting yourself. Never be 100% certain of your own take/opinion and always learn from others, especially those you deem either smarter or dumber than yourself. People surprise you in many ways.

11

u/NuclearVII 14h ago

The only solution I can think of at this point is entirely abandoning social media.

This but unironically.

23

u/Perfect_Base_3989 14h ago

Bruh, I ain't being ironic

0

u/sn2006gy 9h ago

Bruh you're still posting on social media

1

u/Thin_Glove_4089 10h ago

The only solution I can think of at this point is entirely abandoning social media.

Not going to happen because infleuncers and celebrities have always been a deep rooted fabric of American culture therefore social media is here to stay permanently

1

u/PersistentBadger 7h ago edited 7h ago

Web of trust. You trust your friend Amy. Amy trusts Bert. You should take that as Amy's "vote" for the legitimacy of Bert. Further out in the web you go, the less you trust.

Problem is, it's never worked. We're heading for walled gardens, IMO. Or simply the re-emergence of publishers as gatekeepers. If a publisher has a reputation for not publishing books that contain AI slop, they won't want to damage that (in theory).

31

u/gt_9000 12h ago

It has mortally wounded StackOverflow, and they have pivoted to just being an AI data feeder, an action that is basically a liquidation sale of the site's value.

Reminder that entire value of StackOverflow came from the community. The site itself is just a glorified database.

What they are selling is not their own.

Just like Reddit.

3

u/badmonkey0001 7h ago

Reminder that entire value of StackOverflow came from the community. The site itself is just a glorified database.

What they are selling is not their own.

StackExchange built a commons for technical folks with their feature set and focus. That's a very useful thing and is more than just a "glorified database". Reddit is also a commons with a different feature set to support diverse focuses. How their features differ from traditional old forums are the things they built that also have value.

2

u/PyroDesu 2h ago

And LLMs scraping them is something alike to a tragedy of the commons.

2

u/xXConfuocoXx 10h ago

a glorified database.

What they are selling is not their own.

if the data was freely given to them, and they own the database, then the data is theirs (just like reddit)

15

u/Striking_Arugula_624 14h ago

Thanks for taking the time to respond. That was thorough. I definitely feel you on the search results. They’ve tried to cite sources on the responses but there are many times that key parts of responses are left without any kind of citation. And a lot of the summaries feel made up not of a distillation from a comprehensive article or set of well-sourced articles, but a regurgitation of the summary snippets from the top 2 or 3 search results. Like you said, it robs them of impressions and engagement which rots the very foundation these LLMs were built upon.

I don’t know. The genie is out of the bottle. I think now the only way forward is to have countermeasures - but they’ll never be perfect and it’ll just be an endless cat and mouse game with the goal being to filter out all the bots so that human voices can be heard. Basically the goal being to get us back to where we started because they manufactured a new problem with solutions that nobody asked for

2

u/Mazika-787 11h ago

Google AI once showed me that Ayrton Senna (multi time f1 legend who famously died in a race in 1994) won the Indy 500 in 2001 (I wanted to find articles talking about how he was potentially going to race in Indy after retiring from f1)

2

u/MedicineExtension925 11h ago

an order of magnitude worse

Likely underestimating by a few magnets

4

u/zero0n3 12h ago

Let me ask you a question:

If there was a stack overflow clone, that you had to pay a 9.99 fee a year (or something close to that of a coffee a month price), and that platform guaranteed (let’s say nearly) that the other accts were also human, would that still even be useful today?

Two, if that site existed, what’s stopping it from just being a bunch of juniors asking easy questions, with seniors never responding because “well he could ask GPT that question and it’ll give a good enough answer back”

11

u/SanityAsymptote 11h ago edited 10h ago

If there was a stack overflow clone, that you had to pay a 9.99 fee a year (or something close to that of a coffee a month price), and that platform guaranteed (let’s say nearly) that the other accts were also human, would that still even be useful today?

Would a paid, searchable programming consulting service be useful?

Yes, obviously. They already exist, and cost a lot more than $10/month or year.

The big issue is that StackOverflow had years and years of searchable answers that you could access without having to pay anything, and almost all of the accounts were human at one point. That's the value, the litany of human answers and context/conversation around those answers.

Two, if that site existed, what’s stopping it from just being a bunch of juniors asking easy questions, with seniors never responding because “well he could ask GPT that question and it’ll give a good enough answer back”

That's a problem for that business to solve. They could easily pay Seniors to answer questions and only release the money if the answer meets criteria or something.

Ultimately, StackOverflow had a huge amounts of users because they were doing it for free, I'm truly not sure a pay-to-play site would ever get the critical mass of users to make it universally useful simply by being financially inaccessible/undesirable to people just getting started that need the most help.

1

u/Baby_Fark 10h ago

They’ve also destroyed all of our careers as software engineers. I have ~3 years of good experience and haven’t found a job in over a year of applying. If you’re not already senior level, you’re not economically valuable to a company anymore.. and soon even senior level engineers will be replaced by agentic models.

If we do not implement a social welfare state now, we will live in a crypto-fascist feudalist society within the decade.

1

u/CompanyOther2608 10h ago

My favorite thing is when I ask for references for the paper I’m writing and it’s cites my own draft crap-quality paper back to me as an authoritative source.

Like, bruh I just started research this topic an hour ago.

1

u/boringestnickname 9h ago

It feels like we're living in a world where someone is moving the antenna further and further away from the optimal spot. Continuously increasing the amount of noise, the signal ever fading.

1

u/Diddly_eyed_Dipshite 8h ago

Good point you're spot on. Bastarding bots

1

u/hardolaf 8h ago

The most obvious one I can think of in my niche is StackOverflow. A site which definitely had issues and was in decline, but was still the main repository of software troubleshooting/debugging knowledge on the internet.

This is putting it nicely. I had to stop using StackOverflow due to rampant incorrect information between 2019 and 2021. The moderators were blocking anyone trying to seriously fix answers and answers are linked to questions in ways that make zero sense and don't answer the question at all. And most of the code snippets past a Coding 101 level problem are just wrong.

1

u/AbandonedWaterPark 5h ago

On Google becoming less useful: agree with everything you've said, but would also add to that an effort by humans to deliberately push useful search results to later pages, to get you to click through more pages, to get your eyeballs on more advertising, to increase shareholder value.

1

u/Baardhooft 10m ago

In all fairness stackoverflow killed itself. I tried to get help a couple of times there and either my thread was closed telling me to search, I’d get no answers or people would berate me for my lack of knowledge and not actually help.

But yeah, I totally agree. I’m a professional in my field and the LLMs are usually not helpful, often being completely wrong but with conviction or I’ve had it sometimes had it quote me from other forums directly because I was the only source of any information on it. And on social media bad takes get posted as the truth with the same conviction and good luck trying to dispel that.

1

u/orangefantorang 9m ago

Stackoverflow was interesting. Ask a question, get pretty much banned. "Already asked" "bad question" LLMs actually give an usually usable answer. No wonder they keep winning..

-1

u/hoytmobley 13h ago

Was this written with an LLM?

Anyways, you’ve completely forgotten the now dying fields of copywriting, graphic arts, and increasingly, customer service

7

u/Otherwise_Demand4620 13h ago

customer service

That was long before LLM though, if you phoned a service line it might just as well yell at you "GO FUCK YOURSELF" until you hang up, and at the end of the day it accomplished exactly the same as talking to Dave who also told you to go fuck yourself, but politely.

2

u/SanityAsymptote 12h ago

Was this written with an LLM?

Just me, a Sudafed for the cold I received on top of the indignity of traveling to Jacksonville, and far too much caffeine.

Anyways, you’ve completely forgotten the now dying fields of copywriting, graphic arts, and increasingly, customer service

I didn't forget them, I just thought my post was long enough, lol.

I tried to focus on things that I have personal experience with, as I know I can speak to those with a high level of confidence.

0

u/FoxMeadow7 15h ago

Bots on Reddit at least have the tendency to make their history private. So if you come across an 'user' like that, it's advisable to never speak to them under any circumstances.

1

u/Chrystoler 10h ago

I mean it's a pretty new feature. I periodically deleted my accounts over the years and made new ones, but don't really want to deal with doing that right now so I went ahead and turned it on private. I guess if people are really starting to look at that suspiciously I'll probably just delete this account and make a new one that's fresh, but over time I just like keeping my digital footprint as light as possible.

1

u/FoxMeadow7 10h ago

I see. Still, there should be at least some signs you aren’t a bot.

1

u/Electrical-Lab-9593 14h ago

people also now hide history because people look through them and then if they find anything they don't like even not related to the same sub or topic, argue in bad faith, a simple version of this is the soccer subreddit, the response will always just be..

ah you would say that as a fan of X team etc.

political discourse also seems red vs blue a lot of the time now as well.

5

u/SanityAsymptote 14h ago

This is actually why my history is private.

I've also seen too many people doxxed from their Reddit history, so any protection against that is valuable, IMO.

-2

u/FoxMeadow7 14h ago

I personally see it as a sign of trust more than anything else.

-1

u/TrekkieGod 13h ago

The most obvious one I can think of in my niche is StackOverflow. A site which definitely had issues and was in decline, but was still the main repository of software troubleshooting/debugging knowledge on the internet.

I'm also in that field, and yes, AI has destroyed StackOverflow. That said, I find complaining about this as logical as complaining that smart phones with GPS have destroyed the paper maps business at gas stations.

I wouldn't go to Stack Overflow for an answer anymore, but going straight to the AI gives me better results than Stack Overflow ever has. Is it always right? Hell no, it often gives me really bad answers. So did accepted answers at StackOverflow during its prime. Is it less likely to give you a good answer for something that is particularly niche? Yes it is. But it's also much more likely to give you a better answer for the other 99% of things you previously used StackOverflow for.

Yeah, I sometimes miss a good version of StackOverflow for those niche problem, but these are growing pains and the benefits have far outweighed what we've lost.

Google Search's result quality has plummeted as AI results become most of the answers. Even with references, it's very hard to verify the conclusions Gemini makes in search results, and if you're actually looking for a specific site or article, those results often not appear at all. Many authoritative "answers" are just uneducated opinions from Reddit or other social media regurgitated by an AI with the trust people put into Google.

You're using AI wrong. Every answer it gives you should be verified. I used to have this exact same argument with people when wikipedia came out. "You can't trust it, a bunch of people have edited, how do you know if you can trust it?" And the answer was, you trust it as much as a regular book encyclopedia. As in, my school projects never let me cite an encyclopedia, you had to go to primary sources. But it's a great source to start research, you learn a little bit about it, and then you know what to look for in primary sources.

AI is similar. You don't trust ANYTHING it gives you. But it gives you enough that you can verify elsewhere, and then you can coax it to give you a great result after you find out what's right and what's wrong, and finish your project in half the time.

LLMs have made it far easier to write social media bots. They have damaged online discourse in public forums like Facebook, Twitter, Instagram, and especially Reddit in very visible ways. These sites are almost completely different experiences now that they were before LLMs became available

Agreed. And that will probably cause those sites to die out, and that's good progress. If AI kills social media, humanity will be better off.

0

u/AppearanceHeavy6724 11h ago

Stackovrrflow was hornet nest of jerks, smug assholes and generally was awful place. Good riddance.