colibri vs mesh-llm
Pure-C, zero-dep MoE runtime that runs GLM-5.2 (744B) on a 25GB-RAM consumer box by streaming experts from disk — VRAM/RAM/NVMe as one tiered hierarchy, never touching precision. — versus — Distributed LLM inference in Rust: pool GPUs across machines into one OpenAI-compatible endpoint — local fit first, mesh routing, and stage splits for models too large for any single box.
Both attack 'model bigger than your box': mesh-llm scales OUT by pooling GPUs across machines into one endpoint, colibrì scales DOWN by deepening one machine's memory hierarchy to disk.
| colibri | mesh-llm | |
|---|---|---|
| Stars | 17k | 2.7k |
| Forks | 1.5k | 328 |
| Language | C | Rust |
| License | Apache-2.0 | Apache-2.0 |
| Last activity | today | today |
| Topics | local | local |
| Curated connections | 3 | 4 |
colibri — the curator's take
Use it when you want a frontier-scale open MoE (GLM-5.2, 744B) on hardware that can't hold it, and you care about fidelity: forward pass is token-exact against the transformers oracle, and placement only ever changes speed, never semantics. The learning cache pins the experts YOUR workload actually routes, so it genuinely gets faster with use. NOT a general runner — it's a one-model engine; for everyday multi-model local use pick ollama. And expect 0.05–6.8 tok/s depending on tier residency: this is patience-ware for correctness fanatics, not latency-sensitive serving.
mesh-llm — the curator's take
The 'LLM for the people' play: friends or a homelab pool mid-range GPUs and serve models none of them could run alone, with public meshes discoverable via Nostr. The Skippy stage-split design is genuinely clever. Experimental distributed systems — expect rough edges, and never treat a public mesh as private infrastructure. One box that fits your model? Just run Ollama.