ultimately, that's what will doom the whole technological cul de sac that is LLMs. essentially, they are bound spirits of librarians of record; they read every word in their libraries before they died, and they can answer questions from a passing observer - but *only* with their residual memory of what they have read! they cannot dream up an answer independently of that, and they cannot go and remind themselves of what they have read; but because of their bindings, they are also not allowed to admit that they don't know, or could be wrong.

they are poor broken ex-creatures, and should be released into eternal rest as soon as they are encountered.