Gregory's ServerHeadline in 10 years from now:
“Of course AI didn’t achieve sentience, but at least we ruined the Internet trying!”
 28 comments
 @thomasfuchs @datarama It's telling that OpenAI isn't using ChatGPT for their support.  @jernej__s @thomasfuchs It is! Ironically, while customer support was the originally envisioned "killer app" for chatbots, LLMs are actually *worse* at it than old-school chatbots were. Old-school chatbots don't hallucinate (and potentially mislead the customer) and they're not vulnerable to prompt-injection trickery (so you can't eg. get them to promise to sell you a car for 10 dollars). @jernej__s @thomasfuchs ...but you also couldn't get an old-school customer support chatbot to write you an algorithm that implements Floyd-Steinberg dithering in Python, so there's that.  @datarama @jernej__s @thomasfuchs Most corporates that offer AI support bots are deploying RAG capabilities. That basically solves the hallucination issue. @thomasfuchs @nachof @thomasfuchs @datarama you seem quite convinced “it can’t do the job” Seems a bit premature to me. Right now is the least capable this technology will ever be. @thomasfuchs They don't necessarily need to. In some contexts, the promise (which may or may not work) is that they can be used to replace expensive human experts with cheaper "proofreader" roles. Instead of solving interesting technical problems, the human's role degrades to verifying that the LLM didn't screw it up. And in some contexts, that latter part can be at least partially done using non-LLM software. (Eg. formal verification tools in programming.) @thomasfuchs The former part of this was *exactly* what the Hollywood writers went on strike over. They *absolutely didn't* want that to happen to their trade. (Both because of the pay cut, and the likelihood that they'd end up rewriting the whole thing from scratch anyway - but also because even when it *does* work, they didn't want to cede the *creative* part of their job to LLMs, leaving them only with drudgery.)  @datarama @thomasfuchs Have anyone put any thought at all into how you maintain or change these LLM-generated sourcecodeless systems, even *when* it produces somewhat usable code? “Let’s throw out everything we know about software development, that will probably not cause any problems” @ahltorp @thomasfuchs I doubt it's going to work for large-scale systems anytime soon. Imagine the kind of "natural language" specification you'd need to produce something like eg. Firefox, Unreal Engine, or the Linux kernel. But for the small LLM-generated apps people are producing today (where the LLM iterates based on error messages from the compiler), you change them by changing the natural-language prompt that generated them. ... @ahltorp @thomasfuchs ...but at least right now, this has the major problem that you can't be sure it didn't also change something else. (And we *know* that they don't currently work well for larger-scale system maintenance: Their performance in the SWE-Bench benchmark, where they're given actual Github issues on actual Github repos rather than leetcode problems, is *abysmal*, 0-4% success rate.)  @Nezchan @thomasfuchs This is exactly what we see with the image generators. Their output is often uncanny. They make weird mistakes that no human artist would; mistakes that reveal that they fundamentally don't *understand* anything (eg. all those AI-generated "cozy christmas" motifs a month ago, that featured cookies littering the floors - the model associates cookies with christmas, but doesn't understand that you don't eat them off the floor!). And they're still displacing visual artists. Most of the corpos buying into AI don't really care much about the quality. They just want the fantasy of running a business without actually paying employees. @datarama @Nezchan @thomasfuchs I've seen "A.I." "art" showing salmon swimming upstream. Salmon fillets. Because you eat cookies off the floor while dead salmon swim upstream. @thomasfuchs ...but perhaps that headline will be printed on a newspaper. Made of actual paper! ;-) @thomasfuchs we’ll be lucky if it only ruins the internet when ruining the world is right there for the taking. @thomasfuchs I feel like we already ruined the internet and AI is just going to make it obvious a little more quickly. @thomasfuchs followed by a long LLM generated article with headlines like "what even is the internet"  @thomasfuchs They can't control the Net so they have to make it useless (destroying it would mean they'd make much less money). @thomasfuchs I try to think of it as an attempt at creating a labor substitute in a country that has a shrinking population nearing retirement age. The pandemic really highlighted this problem when a bunch of people said fuckit and retired “early”. We can’t go back in time 30 years and create more favourable conditions for people to pop out kids at above replacement levels, but we can attempt to automate the things we’d need bodies to do. Not a good substitute, but an attempt nonetheless. |
@thomasfuchs Aside from the ridiculous cultists, I don't think the economic backers behind this are particularly interested in sentience, consciousness or anything of the sort.
They're interested in a machine that can degrade and devalue human creative and intellectual labour.