evo vs sia
Karpathy-style autoresearch on any codebase: /evo:discover instruments the benchmark, /evo:optimize runs tree search with parallel subagents in worktrees. Plugin for Claude Code, Codex & co. — versus — Self-improving loop from the SIA paper: Meta, Target and Feedback agents evolve a task agent's harness AND weights against a benchmark — #1 on MLE-Bench Hard, 14x kernel speedups.
Both run autonomous improve-against-a-benchmark loops with paper-adjacent rigor. evo points coding agents at YOUR codebase (tree search over commits); SIA generates a task-specific agent and evolves the agent itself — harness and weights.
| evo | sia | |
|---|---|---|
| Stars | 1.3k | 2.1k |
| Forks | 101 | 244 |
| Language | Python | Python |
| License | Apache-2.0 | MIT |
| Last activity | 6 days ago | 21 days ago |
| Topics | coding, skills | agents, training |
| Curated connections | 3 | 2 |
evo — the curator's take
Where the autoresearch skill is a greedy hill-climb on one branch, evo is the industrialized version: tree search over committed nodes, parallel subagents in isolated git worktrees sharing failure traces, regression gates that discard bad experiments, frontier strategies (argmax to GEPA-inspired pareto), and a dashboard. Benchmark discovery is the sleeper feature — it figures out what to measure before optimizing. Runs on 8 agent hosts, local or remote sandboxes (Modal, E2B, Daytona, AWS). NOT for codebases without a mechanical metric — no benchmark, no loop — and unattended parallel agents editing your repo demand real gates; the hosted 'evo platform' beta hints where the business model lands.
sia — the curator's take
The most ambitious of the self-improvement loops on the map: where others mutate prompts or code, SIA's Feedback agent rewrites the task agent's harness and updates its weights, with paper-grade receipts (70.1% on LawBench vs 45% prior SOTA, 14x on an AlphaFold-3 Triton kernel, #1 on MLE-Bench Hard). Reach for it when your problem IS a benchmark: a scoreable task the loop can grind against. NOT for tasks without a mechanical metric, and not a polished product — it's the official research implementation, so expect to build the task harness yourself and budget real GPU time for the weight-update path.