BerriAI

litellm

Open-source AI gateway: call 100+ LLM providers in OpenAI format via a Python SDK or self-hosted proxy — with cost tracking, virtual keys, guardrails, load balancing and logging.

57,669 11,040 Python NOASSERTIONupdated 2 days ago

gateway

View on GitHubDispute this mapping →

Curator's take

The default answer to "one API for every LLM." Reach for it when app code or an agent fleet must hit many providers without per-SDK glue, or when a team needs a central proxy with spend caps, virtual keys and logging. The SDK is a thin drop-in; the proxy is the real value (dashboard, budgets, rate limits). NOT an inference engine — it routes to backends like vLLM/Ollama, doesn't run models. Overkill if you only ever call one provider. Guardrails/evals exist but are lighter than dedicated tools.

Mapped by ShipWithAI editors · links verified

README.md

🚅 LiteLLM

LiteLLM AI Gateway

Open Source AI Gateway for 100+ LLMs. Self-hosted. Enterprise-ready. Call any LLM in OpenAI format.

LiteLLM Proxy Server (AI Gateway) | Hosted Proxy | Enterprise Tier | Website

What is LiteLLM

LiteLLM is an open source AI Gateway that gives you a single, unified interface to call 100+ LLM providers — OpenAI, Anthropic, Gemini, Bedrock, Azure, and more — using the OpenAI format.

Use it as a Python SDK for direct library integration, or deploy the AI Gateway (Proxy Server) as a centralized service for your team or organization.

Jump to LiteLLM Proxy (LLM Gateway) Docs
Jump to Supported LLM Providers

Why LiteLLM

Managing LLM calls across providers gets complicated fast — different SDKs, auth patterns, request formats, and error types for every model. LiteLLM removes that friction:

Unified API — one interface for 100+ LLMs, no provider-specific SDK juggling
**Drop-in

Continue your stack

What teams reach for next — and why each earns a place beside litellm. Ranked by curator confidence.

Pairs well with

vLLM

High-throughput, memory-efficient inference and serving engine for LLMs.

Why it fitsLiteLLM fronts a self-hosted vLLM server as one more OpenAI-compatible backend — vLLM does the serving, LiteLLM does routing, keys and spend tracking.

exxperts

Local-first agentic runtime with persistent AI rooms and approval-gated memory: every memory write needs your OK; rooms, KB and artifacts are plain files on disk.

Why it fitsexxperts' AI setup has a first-class 'OpenAI-compatible gateway' path and names a company LiteLLM proxy as the canonical example — run rooms against your org's gateway instead of raw provider keys.

mcp-context-forge

IBM's registry + proxy federating MCP, A2A and REST/gRPC into one governed endpoint: discovery, auth, rate limits, guardrails, OTel tracing, 40+ plugins — scales to K8s with Redis federation.

Why it fitsLiteLLM federates the model plane, ContextForge the tool/agent plane — an enterprise stack typically needs exactly one of each.

Alternative to See all + compare →

plano

AI-native Envoy-based proxy for agentic apps: agent orchestration via a 4B routing model, smart LLM routing, guardrail filter chains and zero-code OTEL traces. Rust, framework-agnostic.

Why it fitsBoth sit between your app and 100+ LLM providers as a self-hosted gateway; LiteLLM is the lighter Python proxy for provider unification and cost control, Plano adds agent orchestration, guardrail filter chains and signals on an Envoy data plane.

freellmapi

OpenAI-compatible proxy that stacks free tiers of 18 LLM providers (~1.7B tokens/mo) behind one /v1 — smart routing, failover, per-key quota tracking; Claude Code and Codex shims included.

Why it fitsBoth expose many providers behind one OpenAI-compatible /v1 endpoint; freellmapi stacks free tiers, LiteLLM targets production apps and teams.

9router

Self-hosted gateway pointing any AI coding CLI (Claude Code, Codex, Cursor, Cline) at 40+ providers, with subscription→cheap→free auto-fallback and tool_result compression for 20-40% token savings.

Why it fitsBoth are self-hosted gateways routing to many providers with fallback; 9router aims at coding CLIs, LiteLLM at app/SDK developers.

Built with / on

data-formulator

Microsoft's AI data-visualization workbench: connect files, DBs or Databricks, ask in plain language, and agents write the transforms behind 30+ chart types you branch and restyle.

Why it fitsLiteLLM is a hard dependency, so one install talks to OpenAI, Anthropic, Gemini, Ollama or any of LiteLLM's 100+ providers on your own key — swap models without touching the app.

docling-graph

Documents to validated knowledge graphs: Docling parses, an LLM or VLM fills Pydantic schemas, and you get a directed NetworkX graph with provenance, Cypher/CSV export and HTML views.

Why it fitsLLM extraction is routed through LiteLLM, so the same config reaches vLLM and Ollama locally or OpenAI, Gemini, watsonx and Mistral as APIs — swap the extraction model without touching the pipeline.

agentic-context-engine

Learning loop for any agent: reflect on failures, distill strategies into a Skillbook, inject them next run — 2x consistency on Tau2, 49% token cuts. LiteLLM-based, 100+ providers.

Why it fitsACELiteLLM is the primary entry point — the '100+ supported providers' run through LiteLLM's unified API.