airllmLayer-by-layer inference that runs 70B models on a 4GB GPU — no quantization required; 405B on 8GB, DeepSeek-V3 671B on ~12GB. One AutoModel line for most open model families.
Why switchSame job — giant models on tiny hardware — opposite technique: AirLLM streams dense layers through a 4GB GPU from Python/HF, colibrì streams MoE experts from NVMe in pure C with a workload-learning pin cache.
Full comparison → FreeTokenEdge-native MoE serving engine: bandwidth-adaptive CPU-GPU co-execution, global LRU expert caching and elastic VRAM run 290B+ frontier MoE models on a gaming PC at interactive speed.
Why switchSame trick — stream MoE experts so the model can exceed your memory — different engineering: colibri is pure C with zero dependencies treating NVMe/RAM/VRAM as one hierarchy, FreeToken a Python/CUDA server tuned for RTX GPUs with elastic VRAM and OpenAI-compatible APIs.
Full comparison → BigMoeOnEdgeRun MoE models bigger than your RAM: keep the always-needed weights resident and stream each token's experts from flash — a 284B model on a 12 GB phone, CPU only, byte-identical output.
Why switchThe same trick aimed at different hardware: Colibri is a pure-C MoE runtime streaming experts from NVMe to fit a 744B model on a 25 GB desktop; BigMoeOnEdge targets phones on plain CPU and rides llama.cpp's format support instead of its own runtime.
Full comparison → OllamaRun Llama, Mistral and other open models locally with a single command and a clean API.
Why switchBoth run open models locally. Ollama is the multi-model daily driver for models that fit; colibrì is a single-model specialist that makes a 744B MoE fit where nothing else will.
Full comparison → mesh-llmDistributed LLM inference in Rust: pool GPUs across machines into one OpenAI-compatible endpoint — local fit first, mesh routing, and stage splits for models too large for any single box.
Why switchBoth attack 'model bigger than your box': mesh-llm scales OUT by pooling GPUs across machines into one endpoint, colibrì scales DOWN by deepening one machine's memory hierarchy to disk.
Full comparison →