labs-molt vs verl
Molt: NVIDIA's agentic-first RL framework in ~9K lines — Ray for placement, vLLM for rollout, AutoModel + FSDP2 for training — fully async, multimodal, multi-turn, scaling to 1T-class MoE. — versus — ByteDance's RL post-training library (HybridFlow): PPO/GRPO dataflows in a few lines, FSDP/Megatron training with vLLM/SGLang rollouts, production-proven at frontier scale.
Both are open RL post-training frameworks with vLLM rollouts and FSDP/Megatron training. verl (HybridFlow) optimises for breadth of algorithms and dataflows; Molt optimises for an agentic-first, single-actor design you can read in an afternoon.
| labs-molt | verl | |
|---|---|---|
| Stars | 1.0k | 23k |
| Forks | 100 | 4.5k |
| Language | Python | Python |
| License | Apache-2.0 | Apache-2.0 |
| Last activity | yesterday | 2 days ago |
| Topics | training | training |
| Curated connections | 5 | 6 |
labs-molt — the curator's take
The RL stack to read end-to-end: the agent is the program (one Gymnasium-aligned Env.step() or ChatAgent.run(), reward is any Python you write), the trainer is a single actor with an optional KL reference, and Ray owns the async queues between rollout, training and weight sync. Token-first contract keeps ids, logprobs, action ranges, rewards and multimodal tensors aligned so multi-turn tool use and VLM environments share one format. Same script trains 8B and DeepSeek-V3-class MoE with expert parallelism. NOT for breadth — it deliberately has fewer algorithms and integrations than verl or TRL — and NOT for SFT-only work; it's for research groups running agentic RL who want every gradient one file away.
verl — the curator's take
The community default for open RL post-training: the hybrid-controller model expresses PPO/GRPO/DAPO dataflows in a few lines, and the backend matrix (FSDP or Megatron for training, vLLM or SGLang for rollouts) means it fits infrastructure you already have. Proven on real frontier runs and the most-forked codebase in the space. NOT an afternoon tool — multi-GPU distributed debugging is table stakes; for single-node SFT/LoRA use TRL instead, and know the tradeoff slime calls out: multi-backend abstraction can lag upstream engine features.