StackMap
Subscribe
Explore / JIT
bingreeky

JIT

JIT-Agent: a meta-agent that writes a task-specific harness on the fly — memory, planning, action and capability modules emitted as code — and revises it from traces at test time.

381 43 Python NOASSERTIONupdated 9 days ago
agentstraining
View on GitHubDispute this mapping →
Curator's take

Research code for one sharp idea: the scaffold around an LLM is a trainable, transferable axis of intelligence, separate from the base model. Given a task spec, a protocol and a tool registry, JIT-Agent-27B emits an executable harness (four modules over HarnessFactory interfaces, so it's structured code, not a free-form program), then repairs it as feedback arrives while the generator stays frozen. Reproduce the paper with the runners and benchmark adapters. NOT a framework you build on — no licence file, conda env, benchmark-shaped — and it assumes you serve the meta model yourself (vLLM/SGLang over HTTP).

Mapped by ShipWithAI editors · links verified

Continue your stack

What teams reach for next — and why each earns a place beside JIT. Ranked by curator confidence.

alternativealternativesiaagentic-context-engineJIT
pairs wellalternativebuilt withpick a node for the why · open it from the panel
Weekly digest
README.md2 min read
JIT-Agent

Scaling Harness Intelligence via Just-in-Time Harness Evolution

GitHub arXiv Hugging Face License

What is JIT-Agent?

JIT-Agent is a compact meta-agent that writes your agent harness on the fly. Instead of precompiling one general-purpose scaffold and hoping it transfers, JIT-Agent takes a task spec, a protocol, a tool/skill registry, and a few retrieved prior harnesses, and emits an executable, task-specific harness that wraps any off-the-shelf agentic LLM — Model-as-a-Harness.

Overview of JIT-Agent
Overview of JIT-Agent. Given a task, JIT-Agent composes a problem-specific agent harness by selecting and instantiating four modules: memory, planning, action, and capability. Different task structures therefore induce distinct executable protocols and state organizations.

Every harness is factored into four modules — memory, planning, action, capability orchestration — implemented against the shared interfaces in HarnessFactory, so generation means emitting structured code rather than free-form agent programs. As traces and feedback come back, JIT-Agent revises the harness and updates the archive: harnesses keep improving at test time while the generator itself stays frozen.

JIT-Agent leaderboard across four representative agent benchmarks

Results. The resulting JIT-Agent-27B lifts a wide range of backbone agents across deep research, daily work, planning, and workspace tasks.

Building the scaffold turns out to be a trainable, transferable axis of agent intelligence — orthogonal to scaling the base model.

Repository layout

Directory What it holds
jit/ The meta agent: generation / repair prompts, best-of-N selection
scripts/ The agent kernel, tools, models, evaluation engine, and the two runners
harness_factory/ Hand-written harness implementations and their design write-ups
benchmark/ One adapter, config and evaluator per benchmark
dataset/ The benchmark data itself

Each directory has its own README with the details.

Setup

1. Clone the repository

git clone https://github.com/bingreeky/JIT.git
cd JIT

2. Environment (Python 3.11)

conda env create -f environment.yml && conda activate jit

or, in an existing environment: pip install -r requirements.txt. Serving a local meta model (vLLM/SGLang + torch) is deliberately not included — the pipeline only ever talks HTTP to it.

3. Credentials

cp .env.example .env   # then fill it in

Anything already exported in the shell wins over .env, and every model role can also be overridden per run on the command line.

Group Keys Used for
Execution model OPENAI_API_BASE, OPENAI_API_KEY, EXEC_MODEL runs the generated harness's agent loop
Judge model JUDGE_MODEL, optional JUDGE_API_* grades produced artifacts (falls back to the execution endpoint)
Meta model META_MODEL, META_API_BASE, META_API_KEY, META_TOKENIZER writes the harness (JIT pipeline only)
Tools SERPER_API_KEY, JINA_API_KEY `web_s