Training
Post-training and RL infrastructure for open models — fine-tuning, alignment, reward loops.
HumanSignal 1.6k · 12 days agoHumanSignal's autonomous data-labeling agent framework: define a skill, give it ground truth, and the agent iterates — learn, apply, reflect — until it hits your accuracy threshold.
thu-nics 726 · 6 days agoCache-to-Cache (ICLR'26): LLMs talk by projecting and fusing each other's KV-caches instead of generating text — higher accuracy than text relay at ~2x lower latency. Fuser weights on HF.
OpenDCAI 8.2k · 2 days agoOperator-based system for LLM data prep — 100+ operators composed into pipelines that generate, clean, evaluate and filter pretraining/SFT/RL data, with a WebUI and a pipeline-building agent.
THUDM 348 · 4 months agoTHUDM recipe for deep-search agents: synthesize hard multi-hop QA from knowledge-graph random walks, then multi-turn GRPO RL — DeepDive-32B hits 14.8% BrowseComp; data feeds GLM-4.5/4.6.
marcelroed 4.1k · 1 months agoTokenization at GB/s: ~1000x faster than HuggingFace tokenizers with drop-in compatibility modes for HF and tiktoken — Rust reading your files directly. pip install gigatoken.
InternScience 1.2k · 1 months agoKG-guided synthetic SFT data: builds a knowledge graph from source text, finds the LLM's knowledge gaps via calibration error, and generates targeted long-tail QA pairs. ACL-published.
h2oai 5.2k · 9 days agoH2O's no-code GUI and framework for fine-tuning LLMs — LoRA, 8-bit, DPO and experiment tracking behind a web UI, with CLI and Docker paths for the same configs.
bingreeky 554 · 1 months agoJIT-Agent: a meta-agent that writes a task-specific harness on the fly — memory, planning, action and capability modules emitted as code — and revises it from traces at test time.
jaredpalmer 9.2k · yesterdayOpen reproduction of TypeSafe's Jev: a LoRA + readout head on Qwen that answers many typed questions about one document in a single prefill pass, returning calibrated probabilities.
NVIDIA-NeMo 1.2k · todayMolt: NVIDIA's agentic-first RL framework in ~9K lines — Ray for placement, vLLM for rollout, AutoModel + FSDP2 for training — fully async, multimodal, multi-turn, scaling to 1T-class MoE.
hiyouga 75.5k · 5 days agoThe unified fine-tuning framework: 100+ LLMs and VLMs via LoRA/QLoRA/full-parameter, config-driven or through the LlamaBoard GUI. ACL 2024, 1000+ citations, 73k stars.
ulab-uiuc 3k · 6 days agoUIUC's LLM routing library: 16+ trainable routers (KNN, MLP, matrix factorisation, Elo, graph, BERT) pick the best model per query, with xRouteBench and a train/serve CLI.
OpenBMB 387 · 5 days agoOpenBMB's RL training framework where every role — inference, training, rollout — is an independent service talking over one TransferQueue data plane. Built on SGLang and torchtitan.
volotat 1.2k · yesterdayContinual-learning byte-level LM trained from scratch on one 8 GB GPU: weights page in from disk, capacity grows and prunes itself while it keeps reading a single data stream.
Goekdeniz-Guelmez 278 · 2 days agoNative Mac app for on-device LLM fine-tuning via mlx-lm-lora: pick a model, choose SFT/LoRA/DPO-family algorithms, watch loss fall live, push to Hugging Face. No cloud, no code.
NVIDIA 5.3k · todayNVIDIA's model-compression library: quantization (PTQ/QAT, FP8/NVFP4), pruning, distillation, NAS and speculative decoding over HF/PyTorch/ONNX, exported to TensorRT-LLM, vLLM and SGLang.
cactus-compute 14k · todayA 45M-parameter tool-calling model shipped as one 14MB binary that runs a full session in ~28MB RAM — grammar-constrained JSON, calibrated confidence, tool retrieval, LoRA fine-tuning.
TIGER-AI-Lab 1.3k · 4 months agoTIGER-AI-Lab's fully open deep-research recipe: 96K long-horizon trajectories (adopted by NVIDIA Nemotron), a 30B-A3B model hitting 54.8% BrowseComp-Plus, training code and eval harness.
neural-maze 426 · yesterdayRunnable course: a production OCR pipeline on Kubernetes — Rust ingestion, Qwen 3.5 (4B) served by vLLM at 1.86 pages/s, Redis queues, KEDA autoscaling. Deploy it on AKS/GKE, not a notebook.
Human-Agent-Society 7.6k · todayContinual-learning infra for self-improving agents: serve traffic (OpenAI/Anthropic-compatible), match feedback to runs, train weights via Slime/SGLang or evolve the harness, ship versioned updates.
google-research 1.5k · 22 days agoGoogle Research's Regularized Recursive Self-Improvement: evolves an agent harness (prompts, tools, memory, sub-agents) with constraints on the search so gains hold out of distribution.
PeterGriffinJin 5.5k · 11 months agoRL framework for training reasoning-and-search interleaved LLMs — the open recipe behind DeepSeek-R1-style search agents: PPO/GRPO on veRL, any search backend, models and data on HF. Two papers.
hexo-ai 2.2k · 1 months agoSelf-improving loop from the SIA paper: Meta, Target and Feedback agents evolve a task agent's harness AND weights against a benchmark — #1 on MLE-Bench Hard, 14x kernel speedups.
microsoft 18.4k · 9 days agoMicrosoft's text-space optimizer that trains a frozen agent's skill document like weights — rollouts, bounded edits, validation-gated updates — and ships a compact best_skill.md.
THUDM 8.7k · 3 days agoTHUDM's RL post-training framework behind the GLM releases — Megatron training plus SGLang rollouts with native arg pass-through, and pluggable reward, verifier and agentic data-generation workflows.
MakazhanAlpamys 9.1k · todayFine-tune and post-train LLMs from one YAML: QLoRA, DPO/ORPO/SimPO/KTO — layer streaming trains an 8B model in 3.3GB VRAM on a laptop GPU, bit-exact against a resident run.
FareedKhan-dev 12.1k · 13 days agoThe full LLM pipeline hand-written in plain PyTorch — tokens, transformer, pretraining, then SFT, reward model, PPO, DPO, GRPO. No trl, no peft: read every algorithm, train on one GPU.
huggingface 19.5k · todayHugging Face's post-training library: SFT, DPO, GRPO, KTO and reward-model trainers on top of Transformers — from a Colab LoRA run to multi-GPU deployments.
verl-project 23.9k · todayByteDance's RL post-training library (HybridFlow): PPO/GRPO dataflows in a few lines, FSDP/Megatron training with vLLM/SGLang rollouts, production-proven at frontier scale.