I'm not an AI researcher but my understanding is that LLMs work by modeling the relationship between words in a high dimensional space. Evidently, they're doing a really good job at this! AI forms good sentences!
To all the techbros trying to replace the internet and all of knowledge with ChatGPT, I say "instead of taking the average of well classified words, use the connections between words to find words written by humans!"
We should use embeddings for search. Good AI. No grift. Please.