StackMap
Subscribe
Explore / LLMRouter
ulab-uiuc

LLMRouter

UIUC's LLM routing library: 16+ trainable routers (KNN, MLP, matrix factorisation, Elo, graph, BERT) pick the best model per query, with xRouteBench and a train/serve CLI.

2,776 297 Python MITupdated 16 days ago
gatewaytraining
View on GitHubDispute this mapping →
Curator's take

Reach for LLMRouter when you have many models behind one API and want a *learned* policy deciding which one answers — by task complexity, cost and quality — rather than a hand-written fallback chain. It's a research library from the xRouteBench paper: you train a router on your own traffic (there's a data-generation pipeline over 11 benchmarks), then serve it. NOT a gateway — it doesn't call providers, hold keys or retry; put it in front of LiteLLM or a proxy. And NOT for a two-model setup: below a handful of candidates a rule beats a trained router.

Mapped by ShipWithAI editors · links verified

Continue your stack

What teams reach for next — and why each earns a place beside LLMRouter. Ranked by curator confidence.

pairs wellpairs wellalternativelitellmOmniRouteplanoLLMRouter
pairs wellalternativebuilt withpick a node for the why · open it from the panel
Weekly digest
README.md1 min read
LLMRouter Logo

🚀 LLMRouter: An Open-Source Library for LLM Routing

Python PRs Slack WeChat Docs Paper Twitter xRouteBenchxRouteBench Dataset License

✨ Introduction

LLMRouter Overview

LLMRouter is an intelligent routing system designed to optimize LLM inference by dynamically selecting the most suitable model for each query. To achieve intelligent routing, it defines:

  1. 🚀 Smart Routing: Automatically routes queries to the optimal LLM based on task complexity, cost, and performance requirements.
  2. 📊 Multiple Router Models: Support for over 16 routing models, organized into five major categories—single-round routers, multi-round routers, multimodal routers, agentic routers, and personalized routers—covering a wide range of strategies such as KNN, SVM, MLP, Matrix Factorization, Elo Rating, graph-based routing, BERT-based routing, hybrid probabilistic methods, transformed-score routers, and more.
  3. 🛠️ Unified CLI: Complete command-line interface for training, inference, and interactive chat with Gradio-based UI.
  4. 📈 Data Generation Pipeline: Complete pipeline for generating training data from 11 benchmark datasets with automatic API calling and evaluation.

📰 News

  • 🔥 [2026-08]: We are honored to have LLMRouter featured among the top papers on 🤗 Hugging Face Daily Papers.

  • 🚀 [2026-08]: LLMRouter - We've released LLMRouter, a unified infrastructure for developing, evaluating, and deploying LLM routers! LLMRouter formulates routing as a unified sequential decision process spanning single-turn, multi-turn, and personalized scenarios, and provides a modular framework with 16+ representative routing methods. It also introduces xRouteBench, a comprehensive benchmark covering generic LLM, memory-augmented, vision, time-series, and personalized routing, with automated supervision construction and joint evaluation of response quality and inference cost. Experiments show that learned routers outperform the strongest fixed-model baseline by 14.6% relatively, wh