BerriAI

litellm

Open-source AI gateway: call 100+ LLM providers in OpenAI format via a Python SDK or self-hosted proxy — with cost tracking, virtual keys, guardrails, load balancing and logging.

53,994 9,882 Python NOASSERTIONupdated today

gateway

View on GitHubDispute this mapping →

Curator's take

The default answer to "one API for every LLM." Reach for it when app code or an agent fleet must hit many providers without per-SDK glue, or when a team needs a central proxy with spend caps, virtual keys and logging. The SDK is a thin drop-in; the proxy is the real value (dashboard, budgets, rate limits). NOT an inference engine — it routes to backends like vLLM/Ollama, doesn't run models. Overkill if you only ever call one provider. Guardrails/evals exist but are lighter than dedicated tools.

Mapped by ShipWithAI editors · links verified

README.md

🚅 LiteLLM

LiteLLM AI Gateway

Open Source AI Gateway for 100+ LLMs. Self-hosted. Enterprise-ready. Call any LLM in OpenAI format.

LiteLLM Proxy Server (AI Gateway) | Hosted Proxy | Enterprise Tier | Website

What is LiteLLM

LiteLLM is an open source AI Gateway that gives you a single, unified interface to call 100+ LLM providers — OpenAI, Anthropic, Gemini, Bedrock, Azure, and more — using the OpenAI format.

Use it as a Python SDK for direct library integration, or deploy the AI Gateway (Proxy Server) as a centralized service for your team or organization.

Jump to LiteLLM Proxy (LLM Gateway) Docs
Jump to Supported LLM Providers

Why LiteLLM

Managing LLM calls across providers gets complicated fast — different SDKs, auth patterns, request formats, and error types for every model. LiteLLM removes that friction:

Unified API — one interface for 100+ LLMs, no provider-specific SDK juggling
**Drop-in

Continue your stack

What teams reach for next — and why each earns a place beside litellm. Ranked by curator confidence.

Pairs well with

vLLM

High-throughput, memory-efficient inference and serving engine for LLMs.

Why it fitsLiteLLM fronts a self-hosted vLLM server as one more OpenAI-compatible backend — vLLM does the serving, LiteLLM does routing, keys and spend tracking.

exxperts

Local-first agentic runtime with persistent AI rooms and approval-gated memory: every memory write needs your OK; rooms, KB and artifacts are plain files on disk.

Why it fitsexxperts' AI setup has a first-class 'OpenAI-compatible gateway' path and names a company LiteLLM proxy as the canonical example — run rooms against your org's gateway instead of raw provider keys.

Ollama

Run Llama, Mistral and other open models locally with a single command and a clean API.

Why it fitsNative Ollama provider lets local models be called through the same unified interface as cloud APIs.

Alternative to See all + compare →

plano

AI-native Envoy-based proxy for agentic apps: agent orchestration via a 4B routing model, smart LLM routing, guardrail filter chains and zero-code OTEL traces. Rust, framework-agnostic.

Why it fitsBoth sit between your app and 100+ LLM providers as a self-hosted gateway; LiteLLM is the lighter Python proxy for provider unification and cost control, Plano adds agent orchestration, guardrail filter chains and signals on an Envoy data plane.

freellmapi

OpenAI-compatible proxy that stacks free tiers of 18 LLM providers (~1.7B tokens/mo) behind one /v1 — smart routing, failover, per-key quota tracking; Claude Code and Codex shims included.

Why it fitsBoth expose many providers behind one OpenAI-compatible /v1 endpoint; freellmapi stacks free tiers, LiteLLM targets production apps and teams.

9router

Self-hosted gateway pointing any AI coding CLI (Claude Code, Codex, Cursor, Cline) at 40+ providers, with subscription→cheap→free auto-fallback and tool_result compression for 20-40% token savings.

Why it fitsBoth are self-hosted gateways routing to many providers with fallback; 9router aims at coding CLIs, LiteLLM at app/SDK developers.