freellmapi vs litellm
OpenAI-compatible proxy that stacks free tiers of 18 LLM providers (~1.7B tokens/mo) behind one /v1 — smart routing, failover, per-key quota tracking; Claude Code and Codex shims included. — versus — Open-source AI gateway: call 100+ LLM providers in OpenAI format via a Python SDK or self-hosted proxy — with cost tracking, virtual keys, guardrails, load balancing and logging.
Both expose many providers behind one OpenAI-compatible /v1 endpoint; freellmapi stacks free tiers, LiteLLM targets production apps and teams.
| freellmapi | litellm | |
|---|---|---|
| Stars | 16k | 54k |
| Forks | 2.4k | 9.9k |
| Language | TypeScript | Python |
| License | MIT | NOASSERTION |
| Last activity | 4 days ago | today |
| Topics | gateway | gateway |
| Curated connections | 4 | 10 |
freellmapi — the curator's take
Use it for personal experimentation when you want real working inference for $0: free tiers stacked across Google/Groq/Cerebras/Mistral/etc., a Thompson-sampling router with 20-attempt failover, and Anthropic Messages + Responses API shims so Claude Code and Codex CLI run straight against the free pool. Also does embeddings, images, TTS, and an MCP server agents can query for live model availability. NOT for production or teams — single-user by design, no billing, and several provider free tiers are ToS-gray (NVIDIA's is eval-only; the repo itself says personal experimentation only). Freemium catch: new models reach free installs 30 days late unless you pay $19/yr for the live catalog.
litellm — the curator's take
The default answer to "one API for every LLM." Reach for it when app code or an agent fleet must hit many providers without per-SDK glue, or when a team needs a central proxy with spend caps, virtual keys and logging. The SDK is a thin drop-in; the proxy is the real value (dashboard, budgets, rate limits). NOT an inference engine — it routes to backends like vLLM/Ollama, doesn't run models. Overkill if you only ever call one provider. Guardrails/evals exist but are lighter than dedicated tools.