verl-project

verl

ByteDance's RL post-training library (HybridFlow): PPO/GRPO dataflows in a few lines, FSDP/Megatron training with vLLM/SGLang rollouts, production-proven at frontier scale.

22,495 4,227 Python Apache-2.0updated today

training

View on GitHub Dispute this mapping →

Curator's take

The community default for open RL post-training: the hybrid-controller model expresses PPO/GRPO/DAPO dataflows in a few lines, and the backend matrix (FSDP or Megatron for training, vLLM or SGLang for rollouts) means it fits infrastructure you already have. Proven on real frontier runs and the most-forked codebase in the space. NOT an afternoon tool — multi-GPU distributed debugging is table stakes; for single-node SFT/LoRA use TRL instead, and know the tradeoff slime calls out: multi-backend abstraction can lag upstream engine features.

Mapped by ShipWithAI editors · links verified

README.md

👋 Hi, everyone! verl is a RL training library initiated by ByteDance Seed team and maintained by the verl community.

seed logo

verl: Volcano Engine Reinforcement Learning for LLMs

verl is a flexible, efficient and production-ready RL training library for large language models (LLMs).

verl is the open-source version of HybridFlow: A Flexible and Efficient RLHF Framework paper.

verl is flexible and easy to use with:

Easy extension of diverse RL algorithms: The hybrid-controller programming model enables flexible representation and efficient execution of complex post-training dataflows. Build RL dataflows such as GRPO, PPO in a few lines of code.
Seamless integration of existing LLM infra with modular APIs: Decouples computation and data dependencies, enabling seamless integration with existing LLM frameworks, such as FSDP, Megatron-LM, vLLM, SGLang, etc
Flexible device mapping: Supports various placement of models onto different sets of GPUs for efficient resource utilization and scalability across different cluster sizes.
Ready integration with popular HuggingFace models

verl is fast with:

State-of-the-art throughput: SOTA LLM training and inference engine integrations and SOTA RL throughput.
Efficient actor model resharding with 3D-HybridEngine: Eliminates memory redundancy and significantly reduces communication overhead during transitions between training and generation phases.

News

[2026/06] verl-SpeCo is pre-released: a co-training framework for speculative decoding across RL training and inference, keeping draft models aligned during training and reusable for accelerated serving, built on top of verl.
[2026/05] uni-agent is released: a unified agent framework to build, run, and train LLM agents at scale, built on top of verl.
[2026/05] VeRL-Omni is pre-released: a unified RL stack for diffusion and omni-modal model post-training built on top of verl. Read the blog post for details.
[2026/05] verl's zero-mismatch HuggingFace rollout vexact is released: with batch-invariant kernels, shared model definition with FSDP, and out-of-box examples compatible with VeOmni.
[2026/04] verl's Megatron backend LoRA and router replay support is showcased at [PyTorch Conference Europe 2026](https://pytorchconferenceeu2026.sched.com/event/2Juce/optimizing-reinforcement-learning-at-trillion-parameter-scale-so

Continue your stack

What teams reach for next — and why each earns a place beside verl. Ranked by curator confidence.

Pairs well with

opensre

Open-source framework for AI SRE agents plus the RL training and evaluation environment they need — connect 60+ tools you already run and investigate incidents on your own infra.

Why it fitsOpenSRE supplies the missing piece verl-style RL training needs for infrastructure agents: a realistic environment with scalable feedback for incident-response rollouts.

Alternative to See all + compare →

slime

THUDM's RL post-training framework behind the GLM releases — Megatron training plus SGLang rollouts with native arg pass-through, and pluggable reward, verifier and agentic data-generation workflows.

Why it fitsSame job — large-scale RL post-training. verl bets on multi-backend flexibility (FSDP/Megatron × vLLM/SGLang) where slime hard-commits to Megatron+SGLang; pick verl when your infra is vLLM- or FSDP-shaped.

trl

Hugging Face's post-training library: SFT, DPO, GRPO, KTO and reward-model trainers on top of Transformers — from a Colab LoRA run to multi-GPU deployments.

Why it fitsSame goal — a post-trained model — at opposite ends of the infra spectrum: TRL for Hub-native single-node SFT/DPO/GRPO, verl when the job needs disaggregated rollout engines and multi-node RL dataflows.

Built with / on

vLLM

High-throughput, memory-efficient inference and serving engine for LLMs.

Why it fitsvLLM is one of verl's first-class rollout engines — RL generation batches run through it during training.