StackMap
Subscribe
Explore / agentic-api
vllm-project

agentic-api

Rust gateway that gives vLLM a stateful, OpenAI-compatible Responses API: server-side conversation state, server-side tool loops, SSE and WebSocket streaming, background runs — Codex-ready.

167 46 Rust Apache-2.0updated today
gatewaylocal
View on GitHubDispute this mapping →
Curator's take

Install it when your client code has become the agent loop: instead of replaying transcripts and executing tools yourself, one POST /v1/responses with previous_response_id and the server hydrates state, runs the tool chain (explicit gateway/client/provider ownership), streams, and continues. Point Codex at it and drive Codex with open models on your own GPUs — no OpenAI account. Validated against the Open Responses suite with replay cassettes. NOT a multi-provider router — vLLM is the backend today — and the Anthropic-style Messages API and higher-level Interactions API are still planned; 167 stars, early.

Mapped by ShipWithAI editors · links verified

Continue your stack

What teams reach for next — and why each earns a place beside agentic-api. Ranked by curator confidence.

pairs wellpairs wellpairs wellalternativebuilt withopeninterpreterllm-dkvcachedplanovLLMagentic-api
pairs wellalternativebuilt withpick a node for the why · open it from the panel
Weekly digest
README.md2 min read
Agentic API

The stateful, agentic API layer for vLLM, written in Rust 🦀

Run OpenAI-grade agentic workloads (Responses API, server-side tools, Codex) on your own GPUs.

License Rust CI pre-commit

🧠 Overview

vLLM gives you state-of-the-art inference throughput. But real agentic applications need more than raw tokens: they need conversation state, tool-call loops, and multi-turn orchestration. Today, all of that complexity lives in your client code.

Agentic API moves it server-side. It is a Rust-native gateway that sits in front of vLLM and owns the stateful agentic APIs, starting with an OpenAI-compatible Responses API. vLLM is one supported backend, not part of the Agentic API product name. Your application makes one API call and the server handles the rest: state hydration, tool execution, streaming, and continuation.

flowchart LR
    C(["🧑‍💻 Client<br/>Codex · SDKs · curl"]) -->|"📮 <code>POST /v1/responses</code><br/>🌐 HTTP&nbsp;&nbsp;📡 SSE&nbsp;&nbsp;🔌 WebSocket"| A
    subgraph A ["⚡ Agentic API (Rust 🦀)"]
        direction TB
        S["🔄 State hydration<br/><code>previous_response_id</code>"]
        T["🛠️ Server-side tools<br/>web search · functions"]
        P["💾 Persistence<br/>SQLite response store"]
    end
    A -->|"🚀 <code>POST /v1/responses</code><br/>⚙️ stateless&nbsp;&nbsp;🤝 OpenAI-compatible"| V(["🚀 vLLM core<br/>inference engine"])

    classDef client fill:#FFE8B3,stroke:#F59E0B,stroke-width:2px,color:#7C2D12
    classDef inner fill:#E0E7FF,stroke:#6366F1,stroke-width:2px,color:#312E81
    classDef engine fill:#DCFCE7,stroke:#22C55E,stroke-width:2px,color:#14532D

    class C client
    class S,T,P inner
    class V engine
    style A fill:#F5F3FF,stroke:#8B5CF6,stroke-width:2px,color:#5B21B6
    linkStyle 0 stroke:#F59E0B,stroke-width:2px
    linkStyle 1 stroke:#22C55E,stroke-width:2px

[!TIP] Point OpenAI Codex at Agentic API and drive it entirely with open models served by vLLM. No OpenAI account required.

✨ Key Features

  • 🔄 Stateful conversations: the server manages history via previous_response_id. No client-side message tracking, no replaying full transcripts.
  • 🛠️ Server-side tool execution: an explicit tool-ownership model (gateway / client / provider) decides exactly what runs where. Web search ships today via You.com, and the model executes multi-step tool chains automatically.
  • 📡 Every transport: non-streaming HTTP, server-sent events for token streaming, and full WebSocket support for interactive clients.
  • 🧰 Codex-ready: accepts Codex-shaped Responses traffic out of the box, preserving the tool declarations and response item shapes Codex depends on.
  • 🏃 Background execution: fire-and-forget requests that keep processing server-side.
  • ✅ Compatibility tested: validated against the Open Responses compatibility suite, with replay-cassette tests for real OpenAI and vLLM traffic.

🧭 API Surface