sie vs vLLM
Self-hosted inference cluster for everything agents call besides the big LLM: embeddings, rerankers, OCR, NER, guardrails and small LLMs — 100+ models, one OpenAI-compatible API, K8s stack included. — versus — High-throughput, memory-efficient inference and serving engine for LLMs.
Both self-hosted OpenAI-compatible inference servers, opposite shapes: vLLM maximizes throughput for one big LLM; SIE serves breadth — 100+ heterogeneous task models (embedders, rerankers, OCR, NER, guards) loaded on demand across a cluster.
| sie | vLLM | |
|---|---|---|
| Stars | 2.2k | 86k |
| Forks | 200 | 19k |
| Language | Python | Python |
| License | Apache-2.0 | Apache-2.0 |
| Last activity | yesterday | today |
| Topics | local, rag | local |
| Curated connections | 3 | 11 |
sie — the curator's take
Fills the gap between 'vLLM serves my big model' and the pile of task models agents actually need: retrieval (bge-m3, SPLADE, ColBERT), doc-to-markdown (MinerU, GLM-OCR, docling), structured extraction (GLiNER), safety verdicts (Granite Guardian) — one cluster, on-demand loading with LRU eviction instead of a server per model. The production story is unusually complete and all Apache-2.0: load-balancing gateway, KEDA scale-to-zero, Grafana dashboards, Terraform for GKE/EKS/AKS; the MCP edge offloads document work from Claude-class clients to your cluster. NOT for max-throughput single-LLM serving — that stays vLLM/SGLang territory (SIE's own generation image wraps SGLang). Telemetry is on by default, opt-out env var.
vLLM — the curator's take
High-throughput, memory-efficient inference and serving engine for LLMs.