Adala vs GraphGen
HumanSignal's autonomous data-labeling agent framework: define a skill, give it ground truth, and the agent iterates — learn, apply, reflect — until it hits your accuracy threshold. — versus — KG-guided synthetic SFT data: builds a knowledge graph from source text, finds the LLM's knowledge gaps via calibration error, and generates targeted long-tail QA pairs. ACL-published.
Two roads to training data without annotators: adala points labeling agents at real data and checks them against ground truth, graphgen synthesizes multi-hop QA from a knowledge graph aimed at the model's knowledge gaps.
| Adala | GraphGen | |
|---|---|---|
| Stars | 1.6k | 1.2k |
| Forks | 157 | 97 |
| Language | Python | Python |
| License | Apache-2.0 | Apache-2.0 |
| Last activity | 15 days ago | 7 days ago |
| Topics | agents, training | training |
| Curated connections | 3 | 4 |
Adala — the curator's take
From the Label Studio company, and the ground-truth-first design is the differentiator: instead of prompt-tuning a classifier by hand, you hand the agent labeled examples and `agent.learn()` iterates against them (student/teacher runtimes) until accuracy clears your bar — then you run it on the unlabeled pile. Skills cover classification, summarization, QA, translation, and compose into sequences; any OpenAI-compatible endpoint works (OpenRouter for Claude/Gemini). Use it for scaled labeling and dataset bootstrapping where you already have some ground truth. NOT a general agent framework despite the name — it's specialized for data processing, and note the trailing Python 3.8–3.11 support window: check activity before adopting for something new.
GraphGen — the curator's take
The interesting bit isn't "synthetic QA" — it's targeting: GraphGen measures the student model's expected calibration error to find what it *doesn't* know, then samples multi-hop KG neighborhoods to generate exactly those long-tail QA pairs. Use it when domain-adapting a model and naive synthetic data keeps rehearsing what the model already knows. Output feeds straight into LLaMA-Factory or xtuner. NOT a general data-prep system — it makes QA-style SFT/pretrain data from documents, nothing else; for cleaning, filtering and pipeline plumbing you still want an operator system like DataFlow.