colibriPure-C, zero-dep MoE runtime that runs GLM-5.2 (744B) on a 25GB-RAM consumer box by streaming experts from disk — VRAM/RAM/NVMe as one tiered hierarchy, never touching precision.
Why switchThe same trick aimed at different hardware: Colibri is a pure-C MoE runtime streaming experts from NVMe to fit a 744B model on a 25 GB desktop; BigMoeOnEdge targets phones on plain CPU and rides llama.cpp's format support instead of its own runtime.
Full comparison → FreeTokenEdge-native MoE serving engine: bandwidth-adaptive CPU-GPU co-execution, global LRU expert caching and elastic VRAM run 290B+ frontier MoE models on a gaming PC at interactive speed.
Why switchbigmoeonedge keeps always-needed weights resident and streams each token's experts for byte-identical CPU-only output, even on phones; FreeToken assumes a consumer GPU and trades that purity for interactive throughput.
Full comparison → airllmLayer-by-layer inference that runs 70B models on a 4GB GPU — no quantization required; 405B on 8GB, DeepSeek-V3 671B on ~12GB. One AutoModel line for most open model families.
Why switchTwo ways to run a model that doesn't fit: AirLLM streams transformer layers sequentially to squeeze a 70B onto a 4 GB GPU and works on dense models; BigMoeOnEdge streams only the experts a token selects, which is faster but MoE-only.
Full comparison → kimi-k3-in-cKimi K3 (2.78T-parameter MoE) inference in portable C99 on one CPU: the dense trunk stays in RAM, 4-bit experts stream from disk, and output is byte-identical from 8 GB to 224 GB.
Why switchSame insight, resident trunk plus experts streamed from flash with byte-identical output; bigmoeonedge builds on llama.cpp so many models work, kimi-k3-in-c is a from-scratch engine for one model.
Full comparison → needleA 45M-parameter tool-calling model shipped as one 14MB binary that runs a full session in ~28MB RAM — grammar-constrained JSON, calibrated confidence, tool retrieval, LoRA fine-tuning.
Why switchOpposite answers to the same on-device memory ceiling. BigMoeOnEdge streams a 284B MoE's experts from flash so a phone can run a frontier model; Needle ships a 45M model purpose-built for tool calling that fits in ~28MB and never touches the network. Capability versus footprint.
Full comparison → mesh-llmDistributed LLM inference in Rust: pool GPUs across machines into one OpenAI-compatible endpoint — local fit first, mesh routing, and stage splits for models too large for any single box.
Why switchWhen the model exceeds the box you have two moves — borrow more memory or stream from storage. mesh-llm pools GPUs across machines into one endpoint; BigMoeOnEdge assumes you have exactly one device and its flash chip.
Full comparison → h3.cantirez's C/Metal engine running MiniMax-H3 video and audio generation natively on Apple Silicon — interactive prompting, 4-step schedules, SSD block streaming down to ~2 GB DiT residency.
Why switchThe same memory trick aimed at different weights: BigMoeOnEdge streams an MoE's per-token experts from flash so a phone can run a 284B model; h3.c streams DiT transformer blocks from SSD so a Mac can run video diffusion. Read either to understand the other's tradeoff.
Full comparison →