README.md2 min read

kimi-k3-in-c

A 2.78-trillion-parameter model. One CPU. 8 GB of RAM.

Kimi K3 inference in portable C99.
No BLAS. No framework. No GPU.

2.78T
_parameters

1.56 TB
_{checkpoint on disk}

8.24 GB
_{peak RSS, measured}

176 KB
_{the whole engine}

0
_GPUs

The same 2.78-trillion-parameter model, the same answer, on whatever machine you own.
More memory only buys speed:

the machine you have	RAM	time per token	what is going on
an ordinary laptop	8 GB	26.5 s	the whole model streams off the disk on every step
a high-end laptop	32 GB	24.2 s	some of the model now sits in memory
a desktop	64 GB	19.8 s	more of it sits in memory
a heavy workstation	128 GB+	5.6 s	the model fits entirely in memory, the disk wait is gone

_{Same short prompt at every size, and the output is byte-identical from the smallest machine to the largest; only the clock changes. One machine, 124 cores, fast NVMe drive: the first three rows still read the model from disk each step, so a slower drive is slower there, while the 128 GB+ row keeps everything in memory and no longer waits on the disk. On that same machine v1.0.0 made the math per token about 8× lighter, a follow-up question in a chat 3.9× faster, and long prompts about half as costly. (A token is roughly a short word-piece; the two runnable demos below are the original captures on a slower drive, so their clock reads a little higher.) Full data in docs/data/.}

I am open to AI research roles and PhD positions. CV.

$ ./bin/k3 ~/k3model --trunk ~/k3trunk --preset laptop \
           --tok ~/k3model --prompt "The capital of France is" --gen 8 --incremental

--- generated text ---
 Paris.",
+            "The Eiffel
----------------------
8 tokens in 261.5 s, 32.69 s/token average
PEAK RSS for the whole run: 8.24 GB

Slow, and answering correctly, in 8.24 GB, from a checkpoint of 1.56 TB. This particular batch command deliberately asks for a raw continuation. The official Kimi K3 checkpoint is also chat-capable; use the XTML REPL below when you want answers and multi-turn history. Give the same batch request more memory and the answer does not change, only the clock:

$ ./bin/k3 ~/

kimi-k3-in-c

Continue your stack

kimi-k3-in-c

A 2.78-trillion-parameter model. One CPU. 8 GB of RAM.