rrsi vs sia
Google Research's Regularized Recursive Self-Improvement: evolves an agent harness (prompts, tools, memory, sub-agents) with constraints on the search so gains hold out of distribution. — versus — Self-improving loop from the SIA paper: Meta, Target and Feedback agents evolve a task agent's harness AND weights against a benchmark — #1 on MLE-Bench Hard, 14x kernel speedups.
Both run a meta-loop that evolves a task agent's harness against a benchmark; SIA also trains weights, RRSI keeps the model frozen and regularizes the search so gains survive out of distribution.
| rrsi | sia | |
|---|---|---|
| Stars | 1.5k | 2.2k |
| Forks | 141 | 258 |
| Language | Python | Python |
| License | Apache-2.0 | MIT |
| Last activity | 18 days ago | 1 months ago |
| Topics | agents, training | agents, training |
| Curated connections | 3 | 5 |
rrsi — the curator's take
Read and run it if you evolve harnesses against a benchmark and keep watching in-distribution gains evaporate — annealed edit budgets, a leakage critic, a noise-adjusted acceptance floor and a cost rule are the fix, each mapped from paper to code, with every candidate a git worktree. Not a product or a plug-in: it's paper code wired to Claude Opus on Vertex AI and three specific benchmarks (Terminal-Bench, Harvey LAB, EngDesign), and a run costs real compute.
sia — the curator's take
The most ambitious of the self-improvement loops on the map: where others mutate prompts or code, SIA's Feedback agent rewrites the task agent's harness and updates its weights, with paper-grade receipts (70.1% on LawBench vs 45% prior SOTA, 14x on an AlphaFold-3 Triton kernel, #1 on MLE-Bench Hard). Reach for it when your problem IS a benchmark: a scoreable task the loop can grind against. NOT for tasks without a mechanical metric, and not a polished product — it's the official research implementation, so expect to build the task harness yourself and budget real GPU time for the weight-update path.