Searchbox

Given a query, a .zip dataroom, and a token budget. The local Qwen3.6-35B-A3B in a minimal Pi harness explores the dataroom with local tools such as bash, grep, embeddings, rerankers in an airgapped loop, then answers.

Live demo → hanxiao.io/searchbox

Why

Everyone who knows me knows I'm super test-time-com"pilled". In my view, search is test-time compute (TTC): you wire trained embeddings, rerankers, multi-vector retrievers, query expanders into a pipeline at test-time to squeeze out relevancy. Don't scale TTC, say a keyword search hands you the answer, and it's probably not good enough. Scale it, say add embedding search then filter with a reranker, and you most likely get a better one. So I built searchbox as a testbed to explore a few questions on TTC:

• Model preferences: which tool does it reach for in agentic search?
• Is grep really all you need, i.e. where does a dense retriever add nothing to search quality?
• Does scaling test-time compute via token budget forcing give better answers, especially on the hard questions?

To make this work, I prebuilt a few projects to pave the road for searchbox: dataroom, which does agentic crawling and spits out a zip; and knowledge-graph, which extracts entity relations and walks the longest path to find non-trivial questions to test searchbox with. Feel free to dig into those too.

Finally, I made searchbox an airgapped harness, because I don't want the model cheating with web information. I want to lock search in the box and it should exhaustively and exclusively use what's in the box (which is a knowledge dump .zip from the web via dataroom, but not at the searchbox step).

How it works

You submit a prompt, an optional dataroom .zip (when not given the built-in jina-corpus.zip is then used), and a turn budget from the homepage:

server/run_searchbox.py then drives a pi --mode rpc session:

1. The dataroom is unzipped to dataroom/ (read-only; a single wrapper dir is stripped). The sidecar (server/dataroom_service.py) indexes nothing at boot - retrieval scopes are embedded lazily on first use, and embeddings are cached per (text, role) for the life of the job process so identical text is never embedded twice (across turns or tools).
2. The task is appended to Pi's system prompt (--append-system-prompt): answer from dataroom/, no network, use any tools or build your own, write the answer to ANSWER.md. It is present every turn and never compacted, so the task stays stable for the whole budget. The question is sent once as the first user message. No skill.
3. Pi runs its own loop and compaction, untouched. The only thing added over vanilla Pi: while the budget is unspent and Pi goes idle, send a bare Continue.. As a backstop the harness also captures the model's final non-thinking message to ANSWER.md each turn, so there is an answer even if the model never wrote the file itself (it never clobbers a model-written one).
4. Force-budget OFF (default): stop after the first turn (turn=1 probe - the common case for measuring single-shot answer quality). ON: run until the turn budget is used (one turn = one Continue. -> agent works -> idle). run_meta.json records stop reason, turns, per-turn token breakdown, tool calls, and config. (We stop at a turn boundary because a run cannot be cleanly interrupted mid-turn anyway, and turns are the user-legible unit.)

The per-job dashboard streams the live run: context window, token usage, turn budget, tool-call distributi