StackMap
Subscribe

otari vs respan-gateway

Mozilla.ai's self-hosted AI gateway: one OpenAI/Anthropic-compatible endpoint over 40+ providers via any-llm, with virtual keys, pre-dispatch budgets, routing policies and a usage dashboard. — versus — Rust LLM gateway in one binary: any OpenAI/Anthropic/Gemini client to any model, cross-vendor realtime voice, MCP tools inside chat calls, hedged requests and a live cost log.

The curated verdict

Both self-hosted multi-provider gateways; Otari centres on virtual keys and budgets for teams, Respan on protocol translation, MCP tooling and raw performance.

otarirespan-gateway
Stars953954
Forks138145
LanguagePythonRust
LicenseApache-2.0Apache-2.0
Last activitytodaytoday
Topicsgatewaygateway
Curated connections34

otari — the curator's take

Pick it when a team needs to hand out revocable, scoped model keys and enforce spend limits before requests go out — Docker-run with zero config, Postgres when you outgrow SQLite, plus Claude Code hooks for per-developer tracking. It's young (1.0) next to LiteLLM's provider breadth and ecosystem; for a solo dev just pointing a coding CLI at free tiers, a lighter local router is less to operate.

respan-gateway — the curator's take

Pick it when gateway overhead matters (sub-0.1ms, tiny memory) or you need protocol translation beyond chat — realtime voice across vendors, MCP tools usable by any model, a plugin trait for custom guardrails. Young and tied to the Respan tracing product for deep observability; if you need team keys, budgets and a mature provider list today, LiteLLM or Otari are safer.