[{"data":1,"prerenderedAt":4},["ShallowReactive",2],{"readme:spec-ptc":3},"\u003Ch1>Speculative Programmatic Tool Calling\u003C\u002Fh1>\n\u003Cp>Speculative programmatic tool-calling (\u003Cstrong>sPTC\u003C\u002Fstrong>) is a technique for harnesses that use tools like sub-agents \u002F sub-calls in code. While the LLM is streaming tokens to generate a REPL call, \u003Cstrong>sPTC\u003C\u002Fstrong> speculates and queues up tool calls in the partially-generated code that act as Futures when the actual code is executed.\u003C\u002Fp>\n\u003Cp>Learn more in \u003Ca href=\"https:\u002F\u002Falexzhang13.github.io\u002Fblog\u002F2026\u002Fspec-ptc\u002F\" rel=\"nofollow ugc noopener\">\u003Cstrong>the blogpost here\u003C\u002Fstrong>\u003C\u002Fa>.\u003C\u002Fp>\n\u003Cimg src=\"https:\u002F\u002Fraw.githubusercontent.com\u002Falexzhang13\u002Fspec-ptc\u002FHEAD\u002Fmedia\u002Fcomparison.gif\" alt=\"sPTC vs serial comparison\" width=\"640\" \u002F>\u003Cp>Many harness designs like \u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2512.24601\" rel=\"nofollow ugc noopener\">Recursive Language Models (RLMs)\u003C\u002Fa> and \u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2402.01030\" rel=\"nofollow ugc noopener\">CodeAct\u003C\u002Fa> rely on \u003Ca href=\"https:\u002F\u002Fplatform.claude.com\u002Fdocs\u002Fen\u002Fagents-and-tools\u002Ftool-use\u002Fprogrammatic-tool-calling\" rel=\"nofollow ugc noopener\">programmatic tool-calling (PTC)\u003C\u002Fa>, where all tools are embedded as functions inside a single code REPL tool that is generated per turn. For RLMs in particular, sub-LLM and sub-RLM calls are expensive, often blocking tools in code that take up a majority of the runtime. sPTC is the general technique of speculating tool and sub-LLM calls that will happen as the root LLM is generating the codeblock, allowing the RLM to batch and asynchronously compute these expensive calls while the full codeblock is still being generated to overlap these calls with the logic of the code REPL.\u003C\u002Fp>\n\u003Cpre>\u003Ccode>baseline   tokens──────────────────▶ exec: call₁──▶call₂──▶…──▶callₙ──▶ answer\nspec-ptc   tokens──────────────────▶ exec: claim·claim·claim ──▶ answer\n                 ╲ call₁ ▶▶▶ done ╱\n                  ╲ call₂ ▶▶▶ done╱     (calls run inside generation time)\n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>This repository is a simple library and demo for this technique.\u003C\u002Fp>\n\u003Ch2>Getting Started\u003C\u002Fh2>\n\u003Cp>You can either clone this repository (uses \u003Ccode>uv\u003C\u002Fcode>), or install with:\u003C\u002Fp>\n\u003Cpre>\u003Ccode class=\"language-bash\">pip install spec-ptc\n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>The \u003Ccode>Speculator\u003C\u002Fcode> object is used to track and store tools to be speculated, as well as the shadow REPL that is used to speculate. You can add tools with the \u003Ccode>spec.tool\u003C\u002Fcode> decorator and control whether you want them to be speculated or not.\u003C\u002Fp>\n\u003Cp>The simplest example is to install tool hooks into the REPL you already have, feed tokens as they stream and feed them to the speculator, then \u003Ccode>exec\u003C\u002Fcode> as usual when finished:\u003C\u002Fp>\n\u003Cpre>\u003Ccode class=\"language-python\">from spec_ptc import Speculator\n\nspec = Speculator()\n\n\n# tools can also be async\n@spec.tool(speculatable=True, pure=True)  # add as tool to be speculated\ndef llm_query(prompt: str) -> str:\n    return sub_lm.complete(prompt)\n\n\n@spec.tool()  # side effects: never speculated\ndef send_report(text: str) -> str:\n    return mail.send(text)\n\n\nns.update(spec.hooks())  # same names, claim-or-run\n\ncode = \"\"\nwith spec.turn(repl_locals=ns) as t:  # snapshot → discarded shadow fork\n    for delta in model_stream:\n        code += delta\n        t.feed(delta)  # closed stmts launch calls now\nexec(code, ns)  # hits return immediately\n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>For the \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Falexzhang13\u002Frlm\" rel=\"nofollow ugc noopener\">RLM\u003C\u002Fa> this is one line: \u003Ccode>from demo.rlm import patch_rlm; patch_rlm()\u003C\u002Fcode>.\u003C\u002Fp>\n\u003Cp>\u003Ccode>example.py\u003C\u002Fcode> is an example you can start with for looking how this is done for the RLM.\u003C\u002Fp>\n\u003Cp>For arbitrary harness, we provide a simple daemon \u003Ccode>spec-ptc-daemon\u003C\u002Fcode> that runs the same\nshadow + store out of process (default socket \u003Ccode>\u002Ftmp\u002Fspec-ptc.sock\u003C\u002Fcode>) with four JSON-lines messages:\u003C\u002Fp>\n\u003Cpre>\u003Ccode>turn_begin {vars}      snapshot REPL variables into the shadow\nfeed {delta}           stream tokens; the daemon launches calls\nresolve {tool, args}   → hit{result} | miss   (miss: run the tool yourself)\nturn_end               evict leftovers, return hit\u002Fmiss counts\n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cpre>\u003Ccode class=\"language-python\">from plugins.client import SpecClient  # ~60 lines, stdlib only — copy it\n\nc = SpecClient()\nc.turn_begin({\"context\": doc})\nc.feed(delta)  # per streamed token\nhit = c.resolve(\"llm_query\", [prompt])  # result, or None → call it yourself\nc.turn_end()\n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>Wrappers in \u003Ccode>plugins\u002F\u003C\u002Fcode>: Claude Code (\u003Ccode>PreToolUse\u003C\u002Fcode>), OpenCode, Pi-mono.\u003C\u002Fp>\n",1787958257625]