LlamaFactory vs Soup
The unified fine-tuning framework: 100+ LLMs and VLMs via LoRA/QLoRA/full-parameter, config-driven or through the LlamaBoard GUI. ACL 2024, 1000+ citations, 73k stars. — versus — Fine-tune and post-train LLMs from one YAML: QLoRA, DPO/ORPO/SimPO/KTO — layer streaming trains an 8B model in 3.3GB VRAM on a laptop GPU, bit-exact against a resident run.
Same job, opposite ends: LlamaFactory covers 100+ models with cluster-grade breadth; Soup bets everything on one-YAML simplicity and making 4GB laptops viable.
| LlamaFactory | Soup | |
|---|---|---|
| Stars | 74k | 396 |
| Forks | 9.0k | 47 |
| Language | Python | Python |
| License | Apache-2.0 | Apache-2.0 |
| Last activity | 3 days ago | 2 days ago |
| Topics | training | training |
| Curated connections | 7 | 2 |
LlamaFactory — the curator's take
The default answer to 'how do I fine-tune model X': whatever the architecture (Llama, Qwen, Mistral, VLMs…), whatever the method (LoRA, QLoRA, DPO, PPO, full), one YAML config or the LlamaBoard GUI runs it — with the broadest model-coverage matrix in open source and academic citation weight behind it. If TRL is the library you code against, LlamaFactory is the trainer you configure. NOT for frontier-scale RL dataflows (verl/slime territory), and the kitchen-sink coverage means version bumps occasionally break niche model+method combos — pin versions for anything long-running.
Soup — the curator's take
The layer-streaming trick is the story: the frozen base never sits in VRAM, so preference tuning that normally needs a second model copy runs on a 4GB laptop — and every release ships bit-exactness proofs plus a Zenodo paper, which is more rigor than most tuning tools at ★396. When NOT: multi-GPU serious runs and exotic architectures belong to llamafactory; streaming is still BETA and single-GPU-minded, and the project is one release cycle old.