train-llm-from-scratch vs trl
The full LLM pipeline hand-written in plain PyTorch — tokens, transformer, pretraining, then SFT, reward model, PPO, DPO, GRPO. No trl, no peft: read every algorithm, train on one GPU. — versus — Hugging Face's post-training library: SFT, DPO, GRPO, KTO and reward-model trainers on top of Transformers — from a Colab LoRA run to multi-GPU deployments.
Same post-training algorithms, opposite purposes: TRL is the production library you call; this re-implements SFT/PPO/DPO/GRPO by hand so you can read them. Learn here, ship with TRL.
| train-llm-from-scratch | trl | |
|---|---|---|
| Stars | 8.5k | 19k |
| Forks | 1.2k | 2.9k |
| Language | Python | Python |
| License | MIT | Apache-2.0 |
| Last activity | 28 days ago | 2 days ago |
| Topics | training | training |
| Curated connections | 1 | 6 |
train-llm-from-scratch — the curator's take
The textbook that runs: every stage from raw text to an aligned reasoning-style model, each algorithm implemented by hand in readable PyTorch — including the post-training alphabet (SFT → reward model → PPO/DPO → GRPO) that most tutorials wave at. If you want to understand what TRL actually does under its trainer classes, this is the fastest honest path, and it fits on a single GPU at the 13M–1B scale. NOT for production: hand-rolled training code at toy scale is the point, not the product — when you're done learning, ship with TRL or LLaMA-Factory. From the author of all-agentic-architectures, same runnable-textbook DNA.
trl — the curator's take
The on-ramp for post-training: if your model is on the Hub and your job fits SFT/DPO/GRPO/KTO, a Trainer class gets you a running job in an afternoon — and nothing else scales down to a free Colab as gracefully. PEFT/LoRA, quantized training and accelerate multi-GPU come along for free. NOT for frontier-scale RL dataflows (that's verl/slime territory — no Megatron, no disaggregated rollout engines), and the trainer abstraction that makes it easy also hides the loss mechanics: when results surprise you, read the trainer source before blaming the data.