Email or username:

Password:

Forgot your password?
Eniko | Kitsune Tails out now!

I hate how all AI hype is predicated on "if we can just make this not be broken then it would be an amazing product"

And because AI produced things look kinda close to the real deal people buy it. Cause it feels like it just needs a small improvement, even though its flaws are a fundamental part of the technology

Just don't draw the weird 6th finger. Just don't make up things when you don't have a real answer. Just don't change the environment in an AI generated game entirely if the player turns around 180 degrees

These things *feel* like they're small, solvable problems to people who don't know better. We could easily fix those things if humans were doing the work!

But AI can't. It will never be able to. It can't because not doing those things means it couldn't do anything else either. Like self-driving cars, the solution to these issues will always be 2 years out

103 comments
mkj

@eniko

"If we can just get our LLM to stop hallucinating, then we could <whatever>..."

"<whoever>, do you have any idea how a LLM works?"

Yeah. And I get it, it's an easy trap to fall into. Generative AI certainly has a lot of properties that make it an easy trap to fall into. I might have fallen into that trap at some point. Then I spent some time reading an article on how generative AI (specifically LLMs in that case) work.

Eniko | Kitsune Tails out now!

TL;DR "if we can just make the make-shit-up machine stop making up the parts we don't like then it'd be perfect!" is not a compelling sales pitch

Jon A. Cruz

@eniko reminds me of what my one cartooning professor would say about illustrations, including mice

Jon A. Cruz

@eniko the main gist was that as an illustrator, animator, etc. you shouldn't give the client what they asked for. Instead it was important to give your interpretation of what the customer expected/needed.

A mouse was a good example. Don't draw a realistic mouse. Draw an *impression* of a mouse that conveys the client's needs. Scary shape if for an exterminator, cute if for a children's book. And for a commercial...? 50's housewife scared of nothing.

AI just doesn't get the artist's eye.

JustAFrog

@eniko I think what amplifies this idea that LLMs are general AI is that for the most common questions, they give the same answers as all common resources would.

But once you stray outside of such well-attested things, the LLM flails about, trying to randomly find an answer-shaped response.

So if people only ever ask it simple shit, it seems omniscient.

Anyone who has more difficult questions is in the minority, and thus their complaints are kinda voted out.

Frank Hightower

@justafrog @eniko To be fair, this was true of the chatbots of 20 years ago, too (people claiming "we've done it, we've passed the turing test!" and then a real turing test was performed and it was marked failed because it couldn't answer "how are you feeling today?")

shine

@eniko I love how probably the only "hotfix" to this is literally to create AGI that reasons about all of that and fact-checks the LLM output, which is a completely different problem to solve, LLM doesn't bring us any closer to it, and if it's even possible to get there, we won't need LLM in a first place.

To continue with the fusion analogy, it's simple to solve nuclear fusion right now, we just need a really powerful infinite power source to power it.

david_chisnall

@eniko You can get more than 50% accuracy predicting the weather by predicting whatever the weather was yesterday. You can get more than 50% accuracy predicting the weather by predicting whatever the weather was this day last year. Getting to be 80% accurate, in contrast, is really hard and requires actually modelling how the atmosphere works.

Probabilistic techniques are great for rapidly getting to a kind-of okay plateau.

Sky Leite

@david_chisnall @eniko Interesting, that’s also how rollback netcode for online games works. When you don’t have the other players inputs for a given frame due to network conditions you just assume they’re holding the same inputs as last frame and a lot of the time that turns out to be correct

Kim Spence-Jones 🇬🇧😷

@david_chisnall @eniko My grandfather was a meteorologist, and he often quoted that fact. In meteorology, it even has a name: “persistence of type”.

K~

@eniko Perhaps you'll appreciate this exchange I just had with ChatGPT on a whim. (I went there inspired by your post.)

chatgpt.com/share/67288b89-303

It's a series of blind stabs in the dark, delivered with full confidence, but with no guidance from reality.

Russell S. Pfau

@karadoc And yet a regular Google search easily found this line. LLMs do not search for information.

B Kahn

@karadoc @eniko

Awesome! I'm laughing but can't help feeling a little creeped out, too. Thank you.

Farbs

@eniko It's kinda amazing that everyone nods sagely at the Eno quote about the beauty of a medium being what happens when you push it beyond its limits and it breaks apart, and we all _saw_ it 10 years ago in the DeepDream dog pics, and yet all we see from AI art now is stuff trying its darnedest not to break apart, and not to be interesting.

Eniko | Kitsune Tails out now!

Also can I just point out how fucking hilarious it is that they've only been able to do this with DOOM and Minecraft, the two games that have more hours of gameplay footage just lying around the internet than any other thing which has footage ever created

And it still sucks

And somehow, this is going to "revolutionize" game development, when it can only do the two most successful (in gameplay footage) games in history and it STILL CAN'T DO THEM RIGHT

JustAFrog

@eniko Once people are presented with 100% LLM-generated games, they'll like them just as much as they like the books that produces.

Cogito ergo mecagoendios

@eniko There was a scifi writing prompt on tumblr where they invent a transporter with lossy compression and people become more and more like each other over the decades due to average regression. I keep telling people hyping genAI to go read it. Of course the ingest-millions-of-examples-and-extract-the-most-predominant-features-machine can make DOOM-looking sorta-ok-looking screenshots! That's literally the easyest ask! Now try get it to make something niche like Microsoft Flight Simulator

Lunar 🛸 ♾

@elrohir @eniko Genuinely think it wouldn't even handle Space Invaders

Frank Hightower

@lunarloony @elrohir @eniko Well, I mean, this is kind of like the argument that AI can't make pixel art because when each "pixel" needs to be a 16×16 square, it fails to line them up properly, but has anyone tried to have it make actual 16×16 sprites and scaled them up themselves? Artificial Intelligence is just a force multiplier for stupidity, as Mark Stanley says

Lunar 🛸 ♾

@FrankHghTwr @elrohir @eniko I'd be tempted to try it for science if I wasn't vehemently opposed to doing so!

Frank Hightower

@elrohir @eniko be careful what you ask for. It could probably make A flight simulator. It could probably replicate the leaked code for some version of Microsoft's. But if what you're asking is "make a model of planet earth to fly over" it'll probably pull a Douglas Adams by consuming the planet, and outputting 42

Kristian Sivonen

@eniko This so much. Every time they're pushing a new iteration of a product, when they're showcasing its output with HAND CURATED best case picks, it's blatantly, obviously dogshit if you pay any attention.

It will always come short. With exponentially larger and larger indiscriminately pilfered datasets they define the exact contour of falling short with ever increasing precision.

gkrnours

@eniko and they first need to make the game to get the footage to train their IA on it.

fingerless

@eniko stable diffusion has limitless potential for games! you just have to:
1) make a normal game
2) film millions of hours of said game

shine

@fingerless @eniko

1. Release the first game as early access
2. be successful enough to attract streamers
3. throw it all away and release unplayable AI generated BS from stolen footage
4. ?????
5. Seriously, ?????????????
6. No profit, just ?

sibaku

@eniko absolutely. Also like most ML things I don't even find the tech interesting. It's just a big optimizer and if you add some billion parameters to a model, that might just cover like any surface level data variation by sheer size alone. And it immediately breaks down with something like games or videos because the additional temporal dimension just would add impossible memory requirements. So tired of this wave of blackbox brute-force "programming" approaches these last years

Arma Bellum

@eniko A coworker was just showing me a couple videos of the AI minecraft and I gotta say that IF (a big if) we disregard all the issues (ethical and otherwise) with genAI, it does look like a fun experiment to toy with. I could see some artsy atmospheric game implementing mechanics such as these for bizarre oniric segments.
I'd love a game where sometimes the terrain was able to hallucinate itself procedurally that way.

I don't believe for a moment genAI is going to MAKE that game, tho.

@eniko A coworker was just showing me a couple videos of the AI minecraft and I gotta say that IF (a big if) we disregard all the issues (ethical and otherwise) with genAI, it does look like a fun experiment to toy with. I could see some artsy atmospheric game implementing mechanics such as these for bizarre oniric segments.
I'd love a game where sometimes the terrain was able to hallucinate itself procedurally that way.

Eniko | Kitsune Tails out now!

@shaperOfDefiance sure, if you ignore the environmental and ethical aspects it'd be great for Fever Dream Simulator 2024 but I feel like that's a pretty niche use case

Arma Bellum

@eniko Yeah, that's the thing, I think this _looks_ very interesting and it would be incredibly cool if achieved by carefully (surgically, even) applying procedural generation to terrain, with properly herded generation algorithms, in a very limited fashion, for a particular level, with the intention of capturing a specific experience during a concrete moment of the game.

This is not it and this is never going to be it.

Arma Bellum

@eniko This is in general my posture with genAI, tbh. IF we were to look past all the layers of issues (and I want to be clear: we should NOT look past all the layers of issues) at its core it's an interesting tech demo and that's all there is.

Azuaron

@eniko My favorite part about the Minecraft one is that they were crowing so hard about it being "playable". But "I can technically input controls and get a response" has never been what "playable" means in gaming. If a game from a normal publisher accidentally forgot the whole environment and erased all your work when you turned around, literally every review for it would have the word "unplayable" in it.

Mikhail 💛💙

@eniko I'm surprised that AI doesnt hallucinate twitch chat with random nonsense, considering if was probably trained on twitch VODs.

PixelPodium

@eniko Unfortunately, "the make-shit-up machine can get close enough that if you push it on the public until they put up with it, you can make a bunch of your employees redundant, totally, trust us" speaks volumes to shareholders.

Leandro (Cerberus1746)

@PixelPodium @eniko It's fine, they just need to screw up once in the right way, like crowdstrike.

But considering it is AI, they will screw up a lot in smaller ways until they add up.

Alexander The 1st

@eniko I suspect the people most likely to be okay with this sales pitch think of it like how games at E3 often used vertical slices that sometimes ran on much beefier PCs than the target consoles.

Like, that latter *can* be useful, but... it's one of the reasons I prefer live demos to recorded demos - you can prove what has been done, and what hasn't, and what can't be done... especially in the context of "In-game footage.".

mort

@eniko This hits the nail on the head IMO. Too many people think of LLMs as "limited artificial intelligence" (mostly due to marketing, including how everyone calls them "AI"), not enough think about them as "plausible text generators"... there *is* no internal state which reflects whether or not it "knows" a particular fact

mort

@eniko See also: the scam artist who has been telling everyone that "full self driving" is one year away every year since 2016 but whose engineers hasn't been able to, and will never be able to, "iron out the kinks" which make fully autonomous vehicles death machines in everything but the most ideal conditions

mkj

@mort Some of the responses I've got from AI proponents have been... interesting.

In the Chinese curse sense.

@eniko

enoch_exe_inc

@mkj @mort @eniko There is no such curse in Chinese.

However, full self-driving cars are a thing that will always be decades ahead of Tesla.

mkj

@enoch_exe_inc Dang. I learned something new today, then! Thank you!

@mort @eniko

enoch_exe_inc

@mkj @enoch_exe_inc @mort @eniko It would be redundant anyway. The history of China is as *fascinating* as it is long and unending. The times have and shall always be interesting.

UkeBLCatboy

@enoch_exe_inc @mkj @mort @eniko ah damn it, I liked that curse! It'd also be perfect for the us as well for the next few years if the horcrux wins tomorrow 😂

enoch_exe_inc

@UkeBLCatboy @enoch_exe_inc @mkj @mort @eniko Fun fact: I tweeted this saying at Hillary Clinton after she “lost” the 2016 election.

enoch_exe_inc

@UkeBLCatboy @enoch_exe_inc @mkj @mort @eniko Canadian here. Ain’t much I can do about it except watch…and threaten to invade if things do not go as well as they should.

UkeBLCatboy

@enoch_exe_inc @mkj @mort @eniko with all due respect, I don't think Canada could invade the US much D:

Jean-Baptiste "JBQ" Quéru

@eniko I don't remember where I read this, and it could have been from you: we can't have AI advocates at the same time claim that we're steps away from AGI and dismiss anything that's grossly wrong with today's systems as "that's not what that system was built for".

Jon A. Cruz

@eniko Exactly. It's not actually artificial intelligence, it is just "spicy autocomplete"

I think a video I just saw on the 'AI minecraft' really sums it up for games

youtu.be/1Rs2rMdU5w0

MW1CFN

@eniko Not to mention the energy and resource use they demand.

Dumb people believe any old crap. AI can't even distinguish between the UK and Australia when I ask it a question about 'Carnarvon' and early wireless. It simply looks for any old 'Carnarvon' and dumps out verbose nonsense. The nonsense dumb people will accept and regurgitate.

Ben Meier

@eniko it's great for the chip manufacturers when the industry believes that if they just throw a few more chips at it, it might work.

Xandra Granade 🏳️‍⚧️

@eniko Agreed, yeah. I'd even argue it's way worse than fusion: at least there's a conceivable but extremely unlikely path to making fusion a thing, and correspondingly, a list of issues that can be concretely working down.

With AI, it's just flat-out fundamentally impossible to close that gap with LLMs or GANs. To really solve the sixth-finger or 180-degree view problems, you'd need such an incredibly different approach, it wouldn't be in the same general category any more.

Tom O'Brien

@eniko
The first time I saw something from Open AI, I thought it was a line from Bill and Ted’s Excellent Adventure. It hasn’t improved since then.

Xironimous Wu

@eniko

I keep finding people who believe that sthe technology will somehow improve in a couple of years.

I try and explain that LLMs are just statistical word salad generators that are never going to learn how to reason, or explain where they got an answer from.

Eniko | Kitsune Tails out now!

@xironwu i mean, that's what i was saying in my post: it's easy to believe that. it requires some actual understanding of the fundamentals of the technology to understand that these problems are intractable

Brandon

@xironwu @eniko been there…the only way I can try to explain it on most days is “can’t reach the moon by climbing progressively taller trees”

Andreas Grois

@eniko As a former physicist: That's unfair. Fusion is theoretically possible. The problems of the current hype "AI" are fundamental though.

Eniko | Kitsune Tails out now!

@soulsource that's fair. i guess it's more like self-driving cars always being just 2 years out

Zero Tachikoma

@eniko I object! Fusion at least as working examples in the world! They're called stars!

Hugs4friends ♾🇺🇦 🇵🇸😷

@eniko Sooner or later, they have to give up on the octagonal wheels that keep threatening to fall off. But, will it be too late? Is it already too late?

Graham Spookyland🎃/Polynomial

@eniko you're right except for the fusion comparison. CCFE finished the break-even research a while back with the MAST upgrade. the world's first energy production fusion plant is being built an hour from my house, right now (I had hoped they'd build it at Ratcliffe, but the shutdown came a couple months too late). so genML doesn't even have that going for it.

kravietz 🦇

@gsuberland

Also, those who say “fusion is always 20 years away” are mistaking objectives of various projects - for example, ITER which is the largest fusion reactor, was never intended to be a economically viable electricity production facility - it’s a 100% research project. Its results feed plenty of basic science, engineering, material research and many other sectors. But MAST and START are intended to be not only doing fusion (we know how to for a long time) but also do it in an economically profitable way, which is yet another and entirely separate challenge.

P.S. ITER goes “first plasma” in 2025 if I’m not mistaken, so it’s not really “always 20 years away” :)

@eniko

@gsuberland

Also, those who say “fusion is always 20 years away” are mistaking objectives of various projects - for example, ITER which is the largest fusion reactor, was never intended to be a economically viable electricity production facility - it’s a 100% research project. Its results feed plenty of basic science, engineering, material research and many other sectors. But MAST and START are intended to be not only doing fusion (we know how to for a long time) but also do it in an economically...

manchuck 🇬🇧

@eniko this is why I’m not worried about AI taking my job. It can’t make what’s in your head real since it can’t read you mind. Developers will always have a jibe becaue managers can barely tell a person what they want.

Bredroll

@eniko it's all just randomness and averages!!! Aaaaaaa

(Matthew)=> return 🏳‍🌈🇿🇦🎮💻📖

@eniko I think what's weird to me is to a lot of general consumers, this stuff is still a selling point? Was watching one of those consumer electronic reviewers, they were billing AI features as a reason to get one device over another, and it was... Kinda maddening. The first thing I look for in a device is whether or not it meets my needs, long battery life or adequate processing power or whatever, what are other people doing with AI???

Mobius Goddess

@eniko Well, it's still a rather dangerous tech that automates patterns. 6th finger problem was already fixed in commercial models, just a really hard pattern they brute forced along with scientists work. Human won't be able to tell away good (not Stable Diffusion, proprietary) AI gen by patterns unless they are experts, you need AI recognition tool (proprietary). I think it's very important part for people to realize.

It, however, won't be able to draw you an electronic schematic that you can recreate and work, unless super common. And that, too, is important part. AI gen info, whether text or visual, is not able to provide additional authenticy to information that isn't copying, and more likely to lose it.

@eniko Well, it's still a rather dangerous tech that automates patterns. 6th finger problem was already fixed in commercial models, just a really hard pattern they brute forced along with scientists work. Human won't be able to tell away good (not Stable Diffusion, proprietary) AI gen by patterns unless they are experts, you need AI recognition tool (proprietary). I think it's very important part for people to realize.

Shane Celis

@eniko It’s crazy to me that these big established companies are risking their brand and small fortunes on a technology that is unQA-able. How do you ensure any level of quality control?

Frank Hightower

@eniko okay, a lot of thoughts.

1. Yes, AI art is wrong.
2. "it has fundamental flaws" was the argument made against Flash and I still think "let's make everything ever made with it inaccessible to future generations" was a leap too far, so I am very wary of that kind of arguments.
3. Though you can theoretically hardwire that "don't draw the 6th finger", frustratingly no one does that! The big guys who say they've fixed it, do it in post!
4. 'are you making this up' really IS a hard problem. "There is no algorithm for the truth" as Tom Scott says
5. "remember the environment" is actually pretty easy to solve, but all the solutions require much more computing power and I'd argue they're already using too much. Let's not give them ideas

@eniko okay, a lot of thoughts.

1. Yes, AI art is wrong.
2. "it has fundamental flaws" was the argument made against Flash and I still think "let's make everything ever made with it inaccessible to future generations" was a leap too far, so I am very wary of that kind of arguments.
3. Though you can theoretically hardwire that "don't draw the 6th finger", frustratingly no one does that! The big guys who say they've fixed it, do it in post!
4. 'are you making this up' really IS a hard problem. "There...

Willow "Wolveric" Catkin

@eniko ""correction"" the solution to self-driving cars is over two centuries past: They're called trains- :blobcatlul:

TanekRune

@eniko
I cannot remember the last time a company put in their own money or effort to fix anything instead of passing the struggle on to the consumers. At least, not without blowback from the consumers.

caranea

@eniko
Feel the perfect illustration of how easily language can fill us is a study from last year. A team ran a Turing test pitting a group of volunteers against chatGPT 3.5 and GPT4. On a whim they added ELIZA into the mix at the last minute -- it beat chatGPT 3.5. A program from the mid-1960s...

Enjoying Kink

@eniko I think AI can be a really good thing. But the LLM hype at the moment is completely missing the point of why it fails. LLMs are not meant for understanding, just for autocorrection.

Dylan

@eniko Some of the trouble here is that slinging code (edit: for a significant, but _non-comprehensive_, set of applications) is one of the areas where the current generation of LLMs actually is a productivity boost. Combine that with the classic “well I’m smart and your field can’t be that hard” attitude and you get xkcd.com/1831/ but s/algorithms/ai/

I am Jack's Lost 404

@eniko

Self driving cars will never work because the problem is intractably complex.

The trolley problem is just propaganda because it assumes you're at the decision stage and completely glosses over the (accurate) realtime sensing & situation recognition problem which is equally if not more complex (and intractable) than the decision making part...

No amount of captchas will ever solve this.

moth bitch

@eniko I feel it’s less “it will never be able to” and more “it will take A LOT LONGER than expected to do it”… generative AI will be perpetually not ready for the next half decade and it will be so much fun watching this very not ready technology get inserted into every critical system in our lives :blabfoxterrified:

Demian

@eniko this is an insightful point. It’s like so many early product demos done well before release. The perpetual vaporware machine

Sven Slootweg

@eniko This framing reminds me a lot of xkcd.com/1425/, except it being weaponized to sell people bullshit hype... 😐

mirabilos

@eniko it’s soulless and broken from the inside, no add-on patches going to fix it.

Same as with the security industry putting (insecure, it turns out) add-on solutions in front of insecure services… for decades. sigh…

James 🦉 #FBPE :europe:

@eniko It's really no different than the off-the-shelf argument - "it'll be quicker and cheaper to buy this existing system that does 90% of what we need, *and then we'll just tweak it.*"

Uh no you won't, and even if you can it definitely won't be quicker or cheaper.


@eniko

1. Solution to AIs. Neuro-sama, make it entertainment. She’s an AI and she’s really funny. Because she’s broken. Recommended (Vedal is her human creator and Evil-sama her twin).

2. Solution to self driving cars. Trains. I bet the human in them is pretty close to token by now. And if not having one person per I don’t know how many passengers or tons of cargo, wow, effective!

(When I found out self driving cars are using cameras I was baffled. That can’t be smart.)

Go Up