litellm vs plano
Open-source AI gateway: call 100+ LLM providers in OpenAI format via a Python SDK or self-hosted proxy — with cost tracking, virtual keys, guardrails, load balancing and logging. — versus — AI-native Envoy-based proxy for agentic apps: agent orchestration via a 4B routing model, smart LLM routing, guardrail filter chains and zero-code OTEL traces. Rust, framework-agnostic.
Both sit between your app and 100+ LLM providers as a self-hosted gateway; LiteLLM is the lighter Python proxy for provider unification and cost control, Plano adds agent orchestration, guardrail filter chains and signals on an Envoy data plane.
| litellm | plano | |
|---|---|---|
| Stars | 54k | 6.9k |
| Forks | 9.9k | 469 |
| Language | Python | Rust |
| License | NOASSERTION | Apache-2.0 |
| Last activity | today | 3 days ago |
| Topics | gateway | gateway, orchestration |
| Curated connections | 10 | 4 |
litellm — the curator's take
The default answer to "one API for every LLM." Reach for it when app code or an agent fleet must hit many providers without per-SDK glue, or when a team needs a central proxy with spend caps, virtual keys and logging. The SDK is a thin drop-in; the proxy is the real value (dashboard, budgets, rate limits). NOT an inference engine — it routes to backends like vLLM/Ollama, doesn't run models. Overkill if you only ever call one provider. Guardrails/evals exist but are lighter than dedicated tools.
plano — the curator's take
Reach for it when multi-agent code is drowning in hidden middleware — intent routing, provider quirks, guardrail hooks, tracing glue. Plano moves all of that out-of-process: agents are plain OpenAI-compatible HTTP servers in any language, orchestration is YAML plus a purpose-built 4B router model. NOT for a quick single-agent demo (it adds an infra hop and Envoy operational surface), and note the catch: the hosted Plano-Orchestrator LLM is free-tier only — production means running the routing models yourself or getting API keys. If you only need provider unification, LiteLLM is lighter.