 Forrest Brazeal: “I think that AI has killed, or is about to kill, pretty much every single modifier we want to put in front of the word “developer.” “.NET developer”? Meaningless. Copilot, Cursor, etc can get anyone conversant enough with .NET to be productive in an afternoon … as long as you’ve done enough other programming that you know what to prompt.” From https://newsletter.goodtechthings.com/p/the-death-of-the-modified-developer 41 comments
@simon There’s kinda a difference between tinkering where ambition is good, and writing production software, no? What that articles predicts is a wave of janky, poorly-understood, and unidiomatic code that will eventually collapse under its own weight. I like LLMs as an assistant to learning, but man a world where people “learn” .NET in an afternoon and start churning out “production” code is positively dystopian to me @hynek I thought that too, but the more work I get done with LLMs myself the less worried I am about that I have a Go project I wrote from scratch in production now, despite not being remotely fluent in Go. It has comprehensive test coverage and even implements continuous integration and continuous deployment, which is why I’m confident it’s not a spectacularly bad idea Would other people YOLO something like that to production without tests? Maybe, and that would definitely be a bad idea! @simon @hynek there’s IMHO still a significant difference between writing some code that passes happy path tests and operating a service in production when something goes wrong the first time. More projects falling apart when looking at them the wrong way and no one around understanding the tooling is IMHO not the solution. That being said it’s obviously easier with a few decades experience knowing what exactly to look for and which question to ask. But this extrapolates poorly to most devs. @simon Yeah but that’s exactly gonna happen once you work under economic constraints and middle managers pining for promotions. My point is exactly what you’re accidentally implying: they’re amazing for tinkering but a time bomb in prod envs. 🤷♂️ @hynek I certainly won’t deny that there are an incredible new array of footguns now available to anyone who wants them  @simon @hynek re: rate of adoption for new programming practices at the office It will be 22 years before there is widespread encouragement of AI aided coding. My current client bans AI through the entire org for all purposes. We're all talking about scifi futures for most people.  @simon @shauna @hynek - only frontier models routinely find bugs with unit tests. 3.5 wrote vacuous tests in comparison to 4 or 4o @simon @hynek Thus far Copilot has made me more likely to write tests -- I've always found them tedious (even though necessary!) and so have been lazy about it, but now it feels rewarding, and once the scaffolding is there, it's not too bad to add extra cases either by hand or with the LLM. I don't think I would have taken it seriously as an option without your posts Simon. @simon For many years I haven't liked calling myself an X-language developer anyway, because when working on a whole product as a solo developer, I have to work in whatever set of languages is most practical for getting the job done, and learning different flavors of imperative language isn't that hard. Plus, I don't want to tie my identity to a particular language (as I possibly did with Python early in my career), because then I'd be more resistant to using new languages when appropriate. @garyfleming @fatrat I think the most important skill in AI-assisted programming is code review and QA: being able to take code and actively test it to confirm that it does what it’s supposed to, including exercising weird edge-cases It’s a difficult thing to get good at! @simon agreed, and to do that accurately, repeatably, and in a way that is automatable (e.g. unit tests) still requires knowledge of the language. That kind of testing and review focuses on known-knowns (“this does what I expected it to do”) but largely ignores known-unknowns (“this does things I didn’t expect it to”) and unknown-unknown (“this does things I didn’t expect and I can’t see”). Those require language and tooling knowledge AFAICT.  @simon @garyfleming @fatrat I completely agree - and those are skills which only come with time and experience. That is why I remain convinced that there is going to be good money to be made in 3-4 years time if LLM-assisted coding becomes widespread - in sorting out the horrible messes that companies have got themselves into by using junior / mid-level devs who have only ever known LLM-assistent development.
[DATA EXPUNGED]
 @llimllib I don’t believe in the “model collapse” idea personally, AI models have been deliberately training on “synthetic data” for the last 12 months with increasingly impressive results How quickly models can pick up new tech is definitely an interesting question - I’ve been pasting dozens of pages of documentation directly into them with good results, eg this example https://gist.github.com/simonw/97e29b86540fcc627da4984daf5b7f9f @llimllib the idea of “model collapse” is almost irresistible, because it’s a story of LLMs being brought down by their dual sins of polluting the web and then training on unverified and unlicensed scraped data If AI labs continued to train indiscriminately it might be a problem, but those researchers are smarter than that: their whole game is about sourcing (and often deliberately generating) high quality training data  @simon maybe. But maybe there are levels of specialization particular to each language that A.I. simply can't dewelve into. I am such an specialist for Python and I can't imagine me trusting LLM to do some of the more subtle things. Adding extra methods to a namedtuple? Show me AI code -deciding- to do that instead of just using a dataclass (which would imply extra conversion steps I don't want) @gwidion that’s exactly how I work with LLMs - a lot of my time is spent saying things like “rewrite that to use a namedtuple, not a dataclass” Kind of like working with an infinitely patient intern who never gets frustrated at constant demands for tweaks and changes @simon yes - but if you put me to create .net code using the LLM, I won't know enough of C#, not by 1/10th, to make requests equivalent to "use a named tuple here".    @simon I don’t buy that at all. It’s true that many technical developer skills transfer well to other programming languages, but I’ve also seen and experienced for decades now that truly understanding an ecosystem (like learning to speak a foreign language near-native, including social, political, cultural aspects) is a long and slow process. AI might help, but it won’t make you proficient over night. @ujay68 there’s fluent, but there’s a level below that where you can build and ship small (not large) projects with confidence despite not knowing the language inside out - that’s where I am now with AI-assisted development for Go and jq and Bash and Dockerfile and AppleScript  @simon It rather reminds me of "automatic programming" ... and the new language it provided us, FORTRAN. https://archive.computerhistory.org/resources/text/Fortran/102663114.05.01.acc.pdf  @simon interesting to me, given ive always phrased it like “im a developer, and i currently work in Laravel and React” @tbortels I think I agree with everything you just said Becoming an effective, responsible developer who can reliably produce quality software is a journey I’m excited that LLM-assistance, applied in the right way, might help accelerate people on that journey - and can help a massive boost for people who have managed to develop those core skills @simon so maybe subject-matter modifiers get promoted - there are very different trade-offs and required domain knowledge and soft skills between "systems engineer", "product engineer", "Database engineer", "Data engineer", "Mobile games engineer", ... |
Gregory's Server
This meshes with my more optimistic take on AI-assisted programming from last year: AI-enhanced development makes me more ambitious with my projects https://simonwillison.net/2023/Mar/27/ai-enhanced-development/