Gregory's Server@evan @Gargron Based on my understanding, LLMs are trained and fixed models. The amount of “memory” they have is a fraction of the training data. They can retain some context from conversations but the model itself doesn’t change. Looked at this way, LLMs can’t learn to think differently. You have to distill a new LLM.
(1/2)
|
Top-level
 2 comments
Summarized, ChatGPT-x doesn’t get to ChatGPT-(x+1) without the humans learning, applying that knowledge to a new model, training that model by burning down a forest or 3, and then publishing the new distillation. I could be wrong but that’s my understanding. |
@evan @Gargron 2/2
The current LLMs are literally statistical models distilled down to map a vast amount of training data into a very small amount of code (with embedded words) that meet the goals of the humans that created it.
When an LLM “learns” from a conversation, it’s just adding new words (rasterized in their context) to the heap. It doesn’t change the mapping/model. That’s been hard coded by the humans who developed the model.