Search-R1

RL framework for training reasoning-and-search interleaved LLMs — the open recipe behind DeepSeek-R1-style search agents: PPO/GRPO on veRL, any search backend, models and data on HF. Two papers.

5,278 477 Python Apache-2.0updated 8 months ago

training

View on GitHubDispute this mapping →

Curator's take

The reference open recipe for teaching a model WHEN to search, not just how to prompt it — full experiment logs on W&B, models and data released, and Tinker featured it. If you're building deep-research agents by training rather than orchestration, start here. When NOT: this is research infrastructure — GPUs, veRL, retrieval corpora — not something you bolt onto an app; for prompt-time search agents an orchestration framework is the cheaper 95%.

Mapped by ShipWithAI editors · links verified

README.md

Search-R1: Train your LLMs to reason and call a search engine with reinforcement learning

Search-R1 is a reinforcement learning framework designed for training reasoning-and-searching interleaved LLMs—language models that learn to reason and make tool calls (e.g., to search engines) in a coordinated manner.

Built upon veRL, Search-R1 extends the ideas of DeepSeek-R1(-Zero) by incorporating interleaved search engine access and provides a fully open-source RL training pipeline. It serves as an alternative and open solution to OpenAI DeepResearch, enabling research and development in tool-augmented LLM reasoning.

We support different RL methods (e.g., PPO, GRPO, reinforce), different LLMs (e.g., llama3, Qwen2.5, etc) and different search engines (e.g., local sparse/dense retrievers and online search engines).

Paper: link1, link2; Model and data: link; Twitter thread: link; Full experiment log: prelim; v0.1; v0.2; v0.3. Details about these logs and methods can be find here.

News

[2025.10] Search-R1 is featured by Thinking Machines Lab's first product Tinker! Details: Document.
[2025.7] Search-R1 is supported by SkyRL! Detailed instructions: code, Document.
[2025.6] Search-R1 is now integrated into the latest version of veRL and can take advantage of its most up-to-date features! Detailed instructions: veRL, [English Document](https://github.com/zhaochenyang20/Awesome-ML-SYS-Tutorial/blob/mai

Continue your stack

What teams reach for next — and why each earns a place beside Search-R1. Ranked by curator confidence.

Alternative to See all + compare →

OpenResearcher

TIGER-AI-Lab's fully open deep-research recipe: 96K long-horizon trajectories (adopted by NVIDIA Nemotron), a 30B-A3B model hitting 54.8% BrowseComp-Plus, training code and eval harness.

Why it fitsTwo open deep-research training recipes: OpenResearcher ships trajectories + a trained model; Search-R1 ships the RL framework to train your own from scratch.

Built with / on

verl

ByteDance's RL post-training library (HybridFlow): PPO/GRPO dataflows in a few lines, FSDP/Megatron training with vLLM/SGLang rollouts, production-proven at frontier scale.

Why it fitsExplicitly built on veRL — Search-R1 is a veRL extension specializing the HybridFlow dataflows for interleaved search-and-reason training.