mesh-llm vs Ollama
Distributed LLM inference in Rust: pool GPUs across machines into one OpenAI-compatible endpoint — local fit first, mesh routing, and stage splits for models too large for any single box. — versus — Run Llama, Mistral and other open models locally with a single command and a clean API.
Both give you a local OpenAI-compatible endpoint for GGUF-family models; Ollama serves from one machine, mesh-llm pools many and routes to whoever can serve.
| mesh-llm | Ollama | |
|---|---|---|
| Stars | 2.7k | 176k |
| Forks | 322 | 17k |
| Language | Rust | Go |
| License | Apache-2.0 | MIT |
| Last activity | today | 3 days ago |
| Topics | local | local |
| Curated connections | 4 | 19 |
mesh-llm — the curator's take
The 'LLM for the people' play: friends or a homelab pool mid-range GPUs and serve models none of them could run alone, with public meshes discoverable via Nostr. The Skippy stage-split design is genuinely clever. Experimental distributed systems — expect rough edges, and never treat a public mesh as private infrastructure. One box that fits your model? Just run Ollama.
Ollama — the curator's take
Run Llama, Mistral and other open models locally with a single command and a clean API.