litellm vs router
Open-source AI gateway: call 100+ LLM providers in OpenAI format via a Python SDK or self-hosted proxy — with cost tracking, virtual keys, guardrails, load balancing and logging. — versus — Weave's Go model router: drop-in proxy speaking Anthropic, OpenAI and Gemini APIs that scores each request with an on-box ONNX embedder (Avengers-Pro clusters) and routes to the cheapest fit model.
litellm is the general-purpose gateway with 100+ providers and simple routing strategies; weave router trades breadth for quality-aware per-action model selection.
| litellm | router | |
|---|---|---|
| Stars | 58k | 4.2k |
| Forks | 11k | 116 |
| Language | Python | Go |
| License | NOASSERTION | NOASSERTION |
| Last activity | yesterday | yesterday |
| Topics | gateway | gateway |
| Curated connections | 21 | 5 |
litellm — the curator's take
The default answer to "one API for every LLM." Reach for it when app code or an agent fleet must hit many providers without per-SDK glue, or when a team needs a central proxy with spend caps, virtual keys and logging. The SDK is a thin drop-in; the proxy is the real value (dashboard, budgets, rate limits). NOT an inference engine — it routes to backends like vLLM/Ollama, doesn't run models. Overkill if you only ever call one provider. Guardrails/evals exist but are lighter than dedicated tools.
router — the curator's take
Choose it when the goal is *smart* routing, not just multi-provider fan-out: an in-process embedder classifies each action and picks a model per request in under 50ms, with a `/v1/route` endpoint to inspect the decision and OTLP traces out of the box. `npx @weave-os/router` wires Claude Code, Codex, opencode or pi in one step; self-hosting needs Postgres (and Pub/Sub for multi-replica). Don't reach for it if you want a plain gateway (litellm), free-tier pooling (omniroute), or to route through subscription CLIs (cliproxyapi) — and note the Elastic License v2, which is not OSI open source, and that the frictionless quickstart is the hosted service.