RRSI: Regularized Recursive Self-Improvement of Agent Harnesses
Check out our paper and project page for more details.
🔥 Updates
- [09/21/2026] Our paper is out! Check it out! [project page]
🧬 Overview
An LLM agent's capability is largely set by its harness: the prompts, control flow, tools, memory and context management around a frozen model. Evolving the harness against a fixed evolve set is effective but overfits: the harness memorizes the training tasks, and large in-distribution gains shrink or vanish out of distribution. RRSI keeps the harness edit space open and regularizes the search trajectory through it instead.
On the proposal side, an annealed budget caps how many independent edits one candidate may bundle, the proposer is conditioned on the full edit history so a falsified hypothesis is not redrawn, and a stalled run is redirected toward components it has never exercised. On the selection side, a critic screens every candidate for suite-specific logic before it is evaluated, a noise-adjusted floor blocks gains within evaluation variance, a cost rule requires added inference tokens to be paid for by measured gain, and components that stop helping are pruned.
✨ Key features
- Open edit space, regularized search. Prompts, control flow, configuration, context management, tools, skills, memory and sub-agents may all be modified; the constraints act on how the search moves, not on what the harness may contain.
- One method, three instances. The same loop drives a terminal agent (Terminal-Bench 2.1), a document-work agent (Harvey LAB) and an engineering-design agent (EngDesign); each instance is a
Domainadapter plus its starting harness. - Candidates in git worktrees. Every candidate harness is drafted, screened and evaluated in its own worktree on a branch off
evolve/<domain>; accepting one fast-forwards the branch, so the incumbent is always a commit. - Evidence you can audit. The edit history records, per edit, the component, the hypothesis, the measured score and cost change and the verdict; the prompts the proposer, analyst and critic receive are plain files in
domains/<name>/.
🧩 Method to code
| Paper | Code |
|---|---|
| Empirical score and cost estimate | rrsi/evaluate.py: aggregate (weighted per-trial rewards; a missing trial counts 0 with the full denominator) |
| Annealed edit budget b_t | rrsi/schedule.py: edit_budget, enforced in the proposer's done() |
| Edit history L_t, tried set T_t, recent yield g_t | rrsi/history.py: History (one JSONL record per edit; a = 1 only for the edits of the candidate that became H_{t+1}) |
| Stall flag, untried components, exploration directives | rrsi/history.py: stall_flag, exploration; reserved slots enforced in rrsi/propose.py |
| Analyze(H_t, D) | rrsi/analyst.py dispatching rrsi/digester.py |
| Proposer with (component, hypothesis, diff) tags | rrsi/propose.py; tags validated against the diff by rrsi/components.py |
| Critic (leakage screen before evaluation) | rrsi/critic.py (domain regex denylist plus LLM review, bounded repair) |
| Evaluate in parallel | rrsi/evaluate.py, Run.round thread pool |
| Noise-adjusted floor, cost rule, within-band rule, argmax | rrsi/selection.py |
| Novelty nu_t (structural component types never in a winning edit) | rrsi/components.py: novelty over K_str = client_tool, skill, memory, subagent |
| Prune set B_t | History.prune_set, handed to the proposer with the accepted machinery to remove |
| Noise band delta | fixed per instance in rrsi.json (0.017 / 0.004 / 0.020); rrsi/calibrate.py re-estimates it when delta is null (bootstrap over trials of the b |