@jonny I'm reminded of a principle I heard in the field of technical translation: you shouldn't try to translate something you couldn't have written (I can't remember the exact phrasing).

Perhaps the corollary is that there's no point in asking an LLM a question that you couldn't answer yourself. Which puts a pretty good upper bound on the usefulness of LLMs, tbh.