 LLM routing aims to reduce costs by directing simple queries to less capable, cheaper models, and complex queries to stronger models. RouteLLM is a framework that uses preference data and data augmentation to train routers that decide which LLM to use, achieving significant cost savings without compromising quality. https://lmsys.org/blog/2024-07-01-routellm/ 3 comments
@stuartyeates in practice it's a heuristic, just the same way a human brain uses a heuristic to decide whether something is simple or not.  Here is an AI for you to put in front of your other AI so you can save on running the other AI by passing your data through this AI so it might redirect it to yet another AI helping you to dumbify the response in even more unexpected ways.
|
Gregory's Server
@yogthos
Excellent!
Detecting "simple queries" is clearly halting-complete, so someone's solved the halting problem at last!