TL;DR "if we can just make the make-shit-up machine stop making up the parts we don't like then it'd be perfect!" is not a compelling sales pitch
Top-level
TL;DR "if we can just make the make-shit-up machine stop making up the parts we don't like then it'd be perfect!" is not a compelling sales pitch 48 comments
@eniko the main gist was that as an illustrator, animator, etc. you shouldn't give the client what they asked for. Instead it was important to give your interpretation of what the customer expected/needed. A mouse was a good example. Don't draw a realistic mouse. Draw an *impression* of a mouse that conveys the client's needs. Scary shape if for an exterminator, cute if for a children's book. And for a commercial...? 50's housewife scared of nothing. AI just doesn't get the artist's eye. @eniko I think what amplifies this idea that LLMs are general AI is that for the most common questions, they give the same answers as all common resources would. But once you stray outside of such well-attested things, the LLM flails about, trying to randomly find an answer-shaped response. So if people only ever ask it simple shit, it seems omniscient. Anyone who has more difficult questions is in the minority, and thus their complaints are kinda voted out. @justafrog @eniko To be fair, this was true of the chatbots of 20 years ago, too (people claiming "we've done it, we've passed the turing test!" and then a real turing test was performed and it was marked failed because it couldn't answer "how are you feeling today?") @eniko I love how probably the only "hotfix" to this is literally to create AGI that reasons about all of that and fact-checks the LLM output, which is a completely different problem to solve, LLM doesn't bring us any closer to it, and if it's even possible to get there, we won't need LLM in a first place. To continue with the fusion analogy, it's simple to solve nuclear fusion right now, we just need a really powerful infinite power source to power it. @eniko You can get more than 50% accuracy predicting the weather by predicting whatever the weather was yesterday. You can get more than 50% accuracy predicting the weather by predicting whatever the weather was this day last year. Getting to be 80% accurate, in contrast, is really hard and requires actually modelling how the atmosphere works. Probabilistic techniques are great for rapidly getting to a kind-of okay plateau. @david_chisnall @eniko Interesting, that’s also how rollback netcode for online games works. When you don’t have the other players inputs for a given frame due to network conditions you just assume they’re holding the same inputs as last frame and a lot of the time that turns out to be correct @locksmithprime Yeah. When the prediction fails is when the game has to re-simulate the frames since then, to figure out the correct state of the world, which is the rollback. @david_chisnall @eniko My grandfather was a meteorologist, and he often quoted that fact. In meteorology, it even has a name: “persistence of type”. @david_chisnall @eniko this tradeoff is why I think the best use in the near future for LLMs will turn out to be voice controls a la "star trek: the next generation". The captain can ask for simple, commonplace high level commands and be confident the computer will do it mostly correctly. And that's a huge time saver. But for anything that requires any amount of precision, or technical knowledge, you better believe he's asking Geordi La Forge (or maybe a real AGI like Data). @eniko Perhaps you'll appreciate this exchange I just had with ChatGPT on a whim. (I went there inspired by your post.) https://chatgpt.com/share/67288b89-303c-8000-b714-e4a05125caff It's a series of blind stabs in the dark, delivered with full confidence, but with no guidance from reality. @karadoc And yet a regular Google search easily found this line. LLMs do not search for information. @karadoc it also loves to claim that you can do things that you actually can't, like connect an Xbox controller to a Switch without an adapter. I accidentally ran into this nonsense when looking that question up, as expected it turns out it's not outside the shit AI-generated articles that just claim it's possible because, well, the algorithm was told to write a guide on how to do the impossible thing, and of course the thing it assumes people want to hear is how to do that, reality be damned @eniko It's kinda amazing that everyone nods sagely at the Eno quote about the beauty of a medium being what happens when you push it beyond its limits and it breaks apart, and we all _saw_ it 10 years ago in the DeepDream dog pics, and yet all we see from AI art now is stuff trying its darnedest not to break apart, and not to be interesting. Also can I just point out how fucking hilarious it is that they've only been able to do this with DOOM and Minecraft, the two games that have more hours of gameplay footage just lying around the internet than any other thing which has footage ever created And it still sucks And somehow, this is going to "revolutionize" game development, when it can only do the two most successful (in gameplay footage) games in history and it STILL CAN'T DO THEM RIGHT @eniko There was a scifi writing prompt on tumblr where they invent a transporter with lossy compression and people become more and more like each other over the decades due to average regression. I keep telling people hyping genAI to go read it. Of course the ingest-millions-of-examples-and-extract-the-most-predominant-features-machine can make DOOM-looking sorta-ok-looking screenshots! That's literally the easyest ask! Now try get it to make something niche like Microsoft Flight Simulator @lunarloony @elrohir @eniko Well, I mean, this is kind of like the argument that AI can't make pixel art because when each "pixel" needs to be a 16×16 square, it fails to line them up properly, but has anyone tried to have it make actual 16×16 sprites and scaled them up themselves? Artificial Intelligence is just a force multiplier for stupidity, as Mark Stanley says @FrankHghTwr @elrohir @eniko I'd be tempted to try it for science if I wasn't vehemently opposed to doing so! @FrankHghTwr @lunarloony @elrohir @eniko Many decades ago I taught remedial summer school math for middle schoolers. My students were not happy to be there. One of the first questions they asked was if they could use calculators. I don't know if I knew the phrase "force multiplier" back then, but I told them that if they didn't know what they were doing, a calculator would just give them "bigger wrong answers." AI is the same on super steroids. @elrohir @eniko be careful what you ask for. It could probably make A flight simulator. It could probably replicate the leaked code for some version of Microsoft's. But if what you're asking is "make a model of planet earth to fly over" it'll probably pull a Douglas Adams by consuming the planet, and outputting 42 @eniko This so much. Every time they're pushing a new iteration of a product, when they're showcasing its output with HAND CURATED best case picks, it's blatantly, obviously dogshit if you pay any attention. It will always come short. With exponentially larger and larger indiscriminately pilfered datasets they define the exact contour of falling short with ever increasing precision. @gkrnours @eniko they can just do what capitalism has been doing for the last 15 years (or more, that's just around the point it's noticeable to me!) and rehash the same thing over and over and over and over again forever! "Western" culture has been doing a 1980s revival for longer than the real 1980s have lasted @hazelnot @eniko I think full AI game will remain a niche because the quality will be too low for people to keep buying. Game are a feedback loop of progress and reward. If you traverse a corridor, look behind yourself and it's a whole new thing, it's not a place where you can progress, it's a fever dream. Which sounds interesting but not battlefield interesting. @gkrnours@mastodon.gamedev.plac Everything is enshittifying without losing out on profits, I'm not convinced the same thing won't happen with all kinds of art, from music to movies to video games 🥲 @eniko stable diffusion has limitless potential for games! you just have to: 1. Release the first game as early access @fingerless @eniko just make a vertical slice demo, record millions of hours of those 15 minutes, and tell the AI to expand it to a whole game, duh! /s @eniko absolutely. Also like most ML things I don't even find the tech interesting. It's just a big optimizer and if you add some billion parameters to a model, that might just cover like any surface level data variation by sheer size alone. And it immediately breaks down with something like games or videos because the additional temporal dimension just would add impossible memory requirements. So tired of this wave of blackbox brute-force "programming" approaches these last years @shaperOfDefiance sure, if you ignore the environmental and ethical aspects it'd be great for Fever Dream Simulator 2024 but I feel like that's a pretty niche use case @eniko Yeah, that's the thing, I think this _looks_ very interesting and it would be incredibly cool if achieved by carefully (surgically, even) applying procedural generation to terrain, with properly herded generation algorithms, in a very limited fashion, for a particular level, with the intention of capturing a specific experience during a concrete moment of the game. This is not it and this is never going to be it. @eniko This is in general my posture with genAI, tbh. IF we were to look past all the layers of issues (and I want to be clear: we should NOT look past all the layers of issues) at its core it's an interesting tech demo and that's all there is. @eniko My favorite part about the Minecraft one is that they were crowing so hard about it being "playable". But "I can technically input controls and get a response" has never been what "playable" means in gaming. If a game from a normal publisher accidentally forgot the whole environment and erased all your work when you turned around, literally every review for it would have the word "unplayable" in it. @eniko I'm surprised that AI doesnt hallucinate twitch chat with random nonsense, considering if was probably trained on twitch VODs. @eniko what makes the whole "AI will always be bad" much worse is the environmental effect it is having, just to provide something that approximates an approximation poorly based on stolen data @eniko Unfortunately, "the make-shit-up machine can get close enough that if you push it on the public until they put up with it, you can make a bunch of your employees redundant, totally, trust us" speaks volumes to shareholders. @PixelPodium @eniko It's fine, they just need to screw up once in the right way, like crowdstrike. But considering it is AI, they will screw up a lot in smaller ways until they add up. @eniko I suspect the people most likely to be okay with this sales pitch think of it like how games at E3 often used vertical slices that sometimes ran on much beefier PCs than the target consoles. Like, that latter *can* be useful, but... it's one of the reasons I prefer live demos to recorded demos - you can prove what has been done, and what hasn't, and what can't be done... especially in the context of "In-game footage.". |
@eniko reminds me of what my one cartooning professor would say about illustrations, including mice