Gregory's Server@prem_k that's VERY feasible - I've considered trying to build something like that myself in the past. https://chat.webllm.ai/ runs Llama 3.2 1B very neatly in the browser already
|
Top-level
 @prem_k that's VERY feasible - I've considered trying to build something like that myself in the past. https://chat.webllm.ai/ runs Llama 3.2 1B very neatly in the browser already 1 comment
|
@simon yes, I'm a fan of it. It's just so feature rich already. Just need a way to make them use embeddings of content & API for RAG & #LAM. Imagine if they could be exposed by websites, somewhat like robot.txt.
Also on similar lines to WebLLM chat is the code of of chromeai.co & its fork, chromeai.org both of which use the Gemini Nano that comes within Chrome 128+ (though it's a bit of a hassle to get that model downloaded right now).