colibri vs FreeToken
Pure-C, zero-dep MoE runtime that runs GLM-5.2 (744B) on a 25GB-RAM consumer box by streaming experts from disk — VRAM/RAM/NVMe as one tiered hierarchy, never touching precision. — versus — Edge-native MoE serving engine: bandwidth-adaptive CPU-GPU co-execution, global LRU expert caching and elastic VRAM run 290B+ frontier MoE models on a gaming PC at interactive speed.
Same trick — stream MoE experts so the model can exceed your memory — different engineering: colibri is pure C with zero dependencies treating NVMe/RAM/VRAM as one hierarchy, FreeToken a Python/CUDA server tuned for RTX GPUs with elastic VRAM and OpenAI-compatible APIs.
| colibri | FreeToken | |
|---|---|---|
| Stars | 26k | 4.3k |
| Forks | 2.8k | 388 |
| Language | C | Python |
| License | Apache-2.0 | Apache-2.0 |
| Last activity | today | today |
| Topics | local | local |
| Curated connections | 5 | 8 |
colibri — the curator's take
Use it when you want a frontier-scale open MoE (GLM-5.2, 744B) on hardware that can't hold it, and you care about fidelity: forward pass is token-exact against the transformers oracle, and placement only ever changes speed, never semantics. The learning cache pins the experts YOUR workload actually routes, so it genuinely gets faster with use. NOT a general runner — it's a one-model engine; for everyday multi-model local use pick ollama. And expect 0.05–6.8 tok/s depending on tier residency: this is patience-ware for correctness fanatics, not latency-sensitive serving.
FreeToken — the curator's take
The research-grade answer to 'frontier MoE on one consumer box' — Berkeley/MIT-flavored author list, an arXiv paper behind it, and the two ideas that matter for agents: a q* policy that splits work across GPU, CPU and host memory by available bandwidth, and semantic anchor checkpoints so tool calls and thinking-block edits don't force full context recompute. Ships Anthropic/OpenAI-compatible APIs, so Claude Code, Codex or OpenCode can point straight at it, plus a desktop GUI. Constraints are real: NVIDIA RTX 30/40/50 on Windows or Linux, MoE architectures only, and a paper-fresh engine that moves fast — pin versions. Serving a dense 8B on a proper GPU? This is the wrong tool.