StackMap
Subscribe
Explore / reef
Human-Agent-Society

reef

Continual-learning infra for self-improving agents: serve traffic (OpenAI/Anthropic-compatible), match feedback to runs, train weights via Slime/SGLang or evolve the harness, ship versioned updates.

911 57 Python Apache-2.0updated yesterday
trainingagents
View on GitHubDispute this mapping →
Curator's take

Reef is the missing layer between an inference engine and an RL trainer: it serves live requests, ties feedback back to recorded interactions, runs a recipe (weight training via Slime, or prompt/rule/skill optimisation with no GPUs), evaluates candidates, and commits accepted versions with git-lfs history — all while staying online. Use it when you have a real feedback signal and want the agent to keep improving in production rather than in a one-off training run. Not for you if you just need a gateway or an eval harness, or if your model is a closed API and you have no harness surface to evolve. Weight-training path needs a supported GPU stack; harness-only path runs against any endpoint.

Mapped by ShipWithAI editors · links verified

Continue your stack

What teams reach for next — and why each earns a place beside reef. Ranked by curator confidence.

pairs wellpairs wellalternativealternativebuilt withverlvLLMsiaagentic-context-engineslimereef
pairs wellalternativebuilt withpick a node for the why · open it from the panel
Weekly digest
README.md2 min read
Reef

Continual learning infra for self-improving agents

CI PyPI package: reef-infra Python License

English | 中文

Reef is the first open-source infrastructure for continual self-improving agents. It connects agent inference, feedback, learning, and versioned delivery. Use it to train model weights with Slime and SGLang, or improve an agent's harness, including its prompts, rules, and skills.

Get started | Roadmap | Launch post | Join Discord | Join WeChat Group

When to use Reef

Use Reef when you want your agent to keep improving simply by learning from how you interact with your agent.

Your goal Learning path What you need
Keep getting stronger model designed for you Model weight training A trainable model, a supported GPU stack, and feedback your recipe can use
Get your harness to self-improve Harness optimization A model endpoint, representative tasks, and an evaluator; no local training GPUs
Scientific discoveries Test-time training An execution environment, a correctness checker, and a measurable objective

How Reef fits your stack

Ability Inference engine (vLLM, SGLang, …) RL training framework (Slime, veRL, AReaL, …) Reef
Serves live traffic ✅ ❌ ✅
Trains weights ❌ ✅ ✅
Version management ❌ ❌ ✅
Stays live through updates ❌ ❌ ✅
Evolves beyond weights (skills, harness) ❌ ❌ ✅

How it works

Reef serves requests, records feedback, produces updates, and commits accepted updates to a version history.

Reef processes each learning cycle in four steps. The table also shows which modules implement each step.

Step What happens Where it lives
1 · Serve Serve agent requests and record interactions. service/ — agent requests and interaction records
runtime/ — inference and artifact updates
2 · Observe Match feedback to recorded interactions. records.py — stored interactions and feedback
train/processors/ — feedback matching and eligibility
3 · Grow Produce an update from eligible records. recipe/ — recipe integration
train/ — batches and update jobs
4 · Commit Apply the configured selection policy and publish accepted updates. train/evaluation/ — candidate evaluation
artifact/ — version history
surface/ — artifact delivery

Installation

💡 Note

Reef's artifact and checkpoint functionality requires the git-lfs system package. Reef initializes Git LFS locally for its artifact repositories.

We recommend [uv](https://docs.ast