Gregory's Server@dalias allergic to domain expertise!!!! this is why copilot uses a fucking ENGLISH tokenizer for PROGRAM CODE which we have fucking PARSERS for!!!!!
|
18 comments
 @hipsterelectron @dalias On a related note, are there any language-specific models out there which take a parsed intermediate representation as input and confine themselves to valid output?  @hipsterelectron @mattb @dalias this here. Hand waving some of the infra improvements and some reasoning capabilities: LLMs are just Markov chains with all their pre-computed in lookup tables and loaded into memory. This is why they will never beat expert systems at reasoning - because that's not what next token prediction is. Side note: I love the idea of building a AST-based model to query than a token based one.  @hipsterelectron @mattb @dalias This has been explored in the past (known variously as e.g. "Evolutionary Programming") where you take a bunch of randomly-generated programs in their parsed format to copy/transform/combine them & assess the ones which best solve the problem to semi-randomly go into the next round. But Neural Nets seem to be the only Machine Learning tactic which gets any attention... @hipsterelectron @mattb @dalias I the problem is getting a mapping from a textual program description to the intermediate representation. Tge LLMs do their coding tricks by associating code to accompanying discussion and comments. What they want to do is let you use text to describe your problem then have the system shot out plausible code. I think you’re idea implicitly requires the model to actually have some understanding of what it’s doing.  @lain_7 @mattb @dalias to paraphrase @emilymbender, it's just acting as a much worse search engine at that point. erasing copyright/attribution is a positive for the monied interests pushing these machines over ones incorporating any level of domain expertise @mattb @hipsterelectron @dalias Yes, there is a long tradition of parsing into semantic representations, and even work on generating from them. If you look at it that way, you immediately see that generation of grammatical strings alone isn't really enough. You need to have a way to connect the semantic representations to some model of the world, and determine what valid things you want to say. @mattb @hipsterelectron @dalias One of the issues with LLMs is that they provide apparent fluency on unlimited topics, making it seem like you don't need to do the extremely difficult world modeling work on those topics... @emilymbender @mattb @hipsterelectron @dalias LLMs are just Ricardian models of the world (it's clear the people [outside acidemia] who make them think they will just infinitely grow in knowledge perfectly) @emilymbender @mattb @hipsterelectron @dalias I make this as my own observation, not as an explanation. Obviously you know more in the academic field, but I also observe on the practitioner space where we are looking at putting them in front of people and I'm not so sure. Not every message has the intent you seem to be alluding to @tanepiper Please feel free to make your observations outside of my mentions, then. As it stands, you have addressed this comment to me, in response to my post, without any connective text indicating how it is supposed to relate. It reads as if you felt that I needed to be enlightened. @tanepiper Also, in case you missed it, mansplaining is never about intent. @emilymbender no, apologies if it came off that way - reading it with a tinge of sarcasm and deadpan humour helps (but of course that does not come across in text). Many sales teams of products promise infinite productivity gains and it's exhausting. I've clarified it's this hopefully. FWIW I was already in this particular thread, just a different branch 🤷🏼♀️  |
@dalias i'm NOT fucking writing it for them they can stew in their own fucking mediocrity