SIE: Superlinked Inference Engine
Self-hosted inference for agents. Every open model your agents call, served from one cluster in your cloud.
Docs | Quickstart | API Reference | Models
About
SIE is an open-source inference engine that runs the models behind every agent task through one API: search and retrieval, document-to-markdown conversion, structured output, content safety, and the agent loop itself. It replaces the patchwork of a separate model server per task with one system that serves 100+ models, loading each on demand.
- OpenAI-compatible API for drop-in migration:
/v1/embeddings,/v1/chat/completions,/v1/completions,/v1/responses - Pre-configured model catalog: Stella, SPLADE, Qwen3, GLiNER, SigLIP, and more; embedding and retrieval models benchmarked on MTEB
- Serves multiple models simultaneously with on-demand loading and LRU eviction
- Ships the full production stack: load-balancing gateway, KEDA autoscaling, Grafana dashboards, Terraform for GKE, EKS, and AKS
- Integrates with LangChain, LlamaIndex, Haystack, DSPy, CrewAI, Chroma, Qdrant, Weaviate, and LanceDB
Tasks
One SIE cluster runs the inference behind a whole agent. Each task is a handful of swappable models; browse packages/sie_server/models/ for the full set.
| Task | What it does | Models |
|---|---|---|
| Search | Embed, match, and rerank to retrieve the right context. | bge-m3, splade-v3, colbertv2, qwen3-reranker |
| Document to markdown | PDFs, Office files, and scans become clean markdown. | glm-ocr, mineru, paddleocr-vl, docling |
| Structured output | Schema-valid JSON, extracted or generated. | gliner2, nuner-zero, qwen3.6-27b |
| Guard content | A safety verdict with a probability you threshold. | granite-guardian-2b |
| Run the agent loop | Plan steps and call tools with an open LLM, streaming included. | qwen3.6-27b |
Quickstart
Prefer a notebook? examples/quickstart.ipynb runs this same flow, on your machine or a free Colab GPU.
1. Start the server
# macOS (Apple Silicon) or Linux, native (requires Python 3.12)
pip install "sie-server[local]" && sie-server serve
# Linux, NVIDIA GPU
docker run --gpus all -p 8080:8080 \
-v sie-hf-cache:/app/.cache/huggingface \
ghcr.io/superlinked/sie-server:latest-cuda12-default
# Linux, CPU
docker run -p 8080:8080 \
-v sie-hf-cache:/app/.cache/huggingface \
ghcr.io/superlinked/sie-server:latest-cpu-default
# in a second terminal
curl http://localhost:8080/readyz # expect: ok
The server speaks the OpenAI API out of the box, embeddings and generation alike (the cluster gateway serves /v1/chat/completions, /v1/completions, and /v1/responses). Your first call needs nothing but curl:
curl http://localhost:8080/v1/embeddings \
-H 'Content-Type: application/json' \
-d '{"model": "sentence-transformers/al