FreeToken vs Strata
Edge-native MoE serving engine: bandwidth-adaptive CPU-GPU co-execution, global LRU expert caching and elastic VRAM run 290B+ frontier MoE models on a gaming PC at interactive speed. — versus — Hybrid GPU/CPU MoE engine that runs Qwen3.8-Flash-Next and DeepSeek V4.1 on consumer cards: hot experts cached in VRAM, rest computed in RAM, MTP drafting, local OpenAI/Anthropic API.
Same trick for the same job: CPU-GPU co-execution with an adaptive expert cache in VRAM to run 290B-class MoE on a gaming PC. Strata ships it as a one-click app for specific models; FreeToken is the more general serving engine.
| FreeToken | Strata | |
|---|---|---|
| Stars | 17k | 41k |
| Forks | 1.6k | 4.4k |
| Language | Python | C++ |
| License | Apache-2.0 | MIT |
| Last activity | yesterday | yesterday |
| Topics | local | local, coding |
| Curated connections | 9 | 5 |
FreeToken — the curator's take
The research-grade answer to 'frontier MoE on one consumer box' — Berkeley/MIT-flavored author list, an arXiv paper behind it, and the two ideas that matter for agents: a q* policy that splits work across GPU, CPU and host memory by available bandwidth, and semantic anchor checkpoints so tool calls and thinking-block edits don't force full context recompute. Ships Anthropic/OpenAI-compatible APIs, so Claude Code, Codex or OpenCode can point straight at it, plus a desktop GUI. Constraints are real: NVIDIA RTX 30/40/50 on Windows or Linux, MoE architectures only, and a paper-fresh engine that moves fast — pin versions. Serving a dense 8B on a proper GPU? This is the wrong tool.
Strata — the curator's take
Reach for it when you own a 12-24 GB GPU (NVIDIA, AMD, Apple Silicon) and want a frontier-class MoE behind Claude Code, Codex or OpenCode with no cloud — one installer, `strata connect` wires the tools. It is a purpose-built engine for a handful of model families, not a general runner: for arbitrary GGUFs or multi-user serving use Ollama/vLLM, and on low RAM or small disks the DeepSeek packs (200+ GB) will not fit.