Email or username:

Password:

Forgot your password?
Top-level
Gary Fleming

@simon @fatrat I’ve seen multiple developers independently use LLMs to produce bash scripts.

They were certainly bash syntax, but they didn’t do quite what the developers expected and they didn’t know that.

LLMs don’t make us X developers magically - they provide the facade of knowing X.

3 comments
Simon Willison

@garyfleming @fatrat I think the most important skill in AI-assisted programming is code review and QA: being able to take code and actively test it to confirm that it does what it’s supposed to, including exercising weird edge-cases

It’s a difficult thing to get good at!

Gary Fleming

@simon agreed, and to do that accurately, repeatably, and in a way that is automatable (e.g. unit tests) still requires knowledge of the language.

That kind of testing and review focuses on known-knowns (“this does what I expected it to do”) but largely ignores known-unknowns (“this does things I didn’t expect it to”) and unknown-unknown (“this does things I didn’t expect and I can’t see”). Those require language and tooling knowledge AFAICT.

Ben Evans

@simon @garyfleming @fatrat I completely agree - and those are skills which only come with time and experience.

That is why I remain convinced that there is going to be good money to be made in 3-4 years time if LLM-assisted coding becomes widespread - in sorting out the horrible messes that companies have got themselves into by using junior / mid-level devs who have only ever known LLM-assistent development.

Go Up