Operator-based system for LLM data prep — 100+ operators composed into pipelines that generate, clean, evaluate and filter pretraining/SFT/RL data, with a WebUI and a pipeline-building agent.
Pick it when the model isn't the problem, the data is. Ready pipelines cover text/math/code synthesis, large-scale PDF→QA, Text2SQL and knowledge-base cleaning, all in a PyTorch-like Pipeline→Operator→Prompt hierarchy that makes data governance reproducible and shareable; the DataFlow-Agent assembles pipelines from a task description, and the WebUI gives non-coders drag-and-drop. Versus Data-Juicer/Nemo-Curator the differentiator is synthesis, with peer-reviewed pedigree (ICDE/KDD acceptances, arXiv report). NOT runtime data plumbing — this is offline training-data preparation, and budget for real LLM API burn since most interesting operators call models (vLLM/SGLang backends supported for local).
Generate, Clean, and Prepare LLM Data, All-in-One
Visual, low-code pipelines with flexible orchestration across domains and use cases.💪
Turn raw data into high-quality LLM training datasets.🔧
🎉 Get smarter LLMs cheaply — give us a star ⭐ on GitHub for the latest update.
Beginner-friendly learning resources (continuously updated): [🎬 Video Tutorials] [📚 Written Tutorials]
简体中文 | English
What teams reach for next — and why each earns a place beside DataFlow. Ranked by curator confidence.
