mesh-llm vs Ollama
Distributed LLM inference in Rust: pool GPUs across machines into one OpenAI-compatible endpoint — local fit first, mesh routing, and stage splits for models too large for any single box. — versus — Run Llama, Mistral and other open models locally with a single command and a clean API.
Both give you a local OpenAI-compatible endpoint for GGUF-family models; Ollama serves from one machine, mesh-llm pools many and routes to whoever can serve.
| mesh-llm | Ollama | |
|---|---|---|
| Stars | 3.3k | 180k |
| Forks | 404 | 18k |
| Language | Rust | Go |
| License | Apache-2.0 | MIT |
| Last activity | 2 days ago | 4 days ago |
| Topics | local | local |
| Curated connections | 6 | 24 |
mesh-llm — the curator's take
The 'LLM for the people' play: friends or a homelab pool mid-range GPUs and serve models none of them could run alone, with public meshes discoverable via Nostr. The Skippy stage-split design is genuinely clever. Experimental distributed systems — expect rough edges, and never treat a public mesh as private infrastructure. One box that fits your model? Just run Ollama.
Ollama — the curator's take
Run Llama, Mistral and other open models locally with a single command and a clean API.