turbovec

Rust vector index with Python bindings built on Google's TurboQuant: no training step, online ingest, hand-written SIMD kernels — 10M x 1536 vectors in ~4 GB, with allowlist-filtered search.

16,587 1,436 Rust MITupdated 10 days ago

rag local storage

View on GitHubDispute this mapping →

Curator's take

Use it when the vector index is a component, not a product: you already have Postgres or BM25 doing the filtering and just need dense rerank that fits in RAM and accepts writes without a rebuild. The data-oblivious quantizer is the real ergonomic win — no train step, no parameter tuning, no reindex as the corpus grows, which is exactly the FAISS ritual people hate. It is an index, not a database: no metadata, no replication, no server, no hybrid scoring, bring your own storage and ID mapping. And it is quantized, so measure recall on your own corpus before you trust it over exact search.

Mapped by ShipWithAI editors · links verified

README.md

turbovec — Google's TurboQuant for vector search

A 10 million document corpus takes 31 GB of RAM as float32. turbovec fits it in 4 GB - and searches it faster than FAISS.

turbovec is a Rust vector index with Python bindings, built on Google Research's TurboQuant algorithm — a data-oblivious quantizer with near-optimal distortion and no separate training phase.

Online ingest. Add vectors, they're indexed — no train step, no parameter tuning, no rebuilds as the corpus grows.
Fast SIMD search. Hand-written kernels — NEON SDOT/SMMLA on ARM, AVX-512 VNNI and vpermb on x86, with AVX2 and scalar fallbacks — beat FAISS IndexPQFastScan in every measured config, averaging 3.4× at 4-bit and 23% at 2-bit across the eight cells of each width, on both architectures.
Incremental saves. sync(path) persists just what changed since the last sync — one fsync per call, crash-safe at any byte, and a removal or a small append costs milliseconds however large the index. write/load stay for whole-file snapshots.
Filter at search time. Pass an id allowlist (or a slot bitmask) to search() and the kernel honours it directly. You always get up to k results from the allowed set — no over-fetching, no recall hit on selective filters.
Pure local. No managed service, no data leaving your machine or VPC. Pair with any open-source embedding model for a fully air-gapped RAG stack.

Building RAG where privacy, memory, or latency matters? You're in the right place.

Python

pip install turbovec

from turbovec import TurboQuantIndex

index = TurboQuantIndex(dim=1536, bit_width=4)
index.add(vectors)
index.add(more_vectors)

scores, indices = index.search(query, k=10)

index.write("my_index.tv")
loaded = TurboQuantIndex.load("my_index.tv")

index.sync("my_index.tv")   # after more changes: durable incremental save

vectors and query are 2-D float32 arrays of shape (n, dim) — other dtypes are rejected rather than silently converted, so cast with np.asarray(x, dtype=np.float32) first if needed.

Need stable ids that survive deletes? Use IdMapIndex:

import numpy as np
from turbovec import IdMapIndex

index = IdMapIndex(dim=1536, bit_width=4)
index.add_with_ids(vectors, np.array([1001, 1002, 1003], dtype=np.uint64))

scores, ids = index.search(query, k=10)   # ids are your uint64 external ids
index.remove(1002)                         # O(1) by id

index.write("my_index.tvim")
loaded = IdMapIndex.load("my_index.tvim")

index.sync("my_index.tvim")   # durable incremental save, ids included

Hybrid retrieval (filtered search)

Restrict results to a candidate set produced by another system (SQL, BM25, ACL, time window, …):

import numpy as np
from turbovec import IdMapIndex

idx = IdMapIndex(dim=1536, bit_width=4)
idx.add_with_ids(vectors, ids)

# Stage 1: external system narrows to candidate ids.
allowed = np.array(db.execute("SELECT id FROM docs WHERE tenant=?", (t,)).fetchall(),
                   dtype=np.uint64)

# Stage 2: dense rerank within the candidate set.
scores, ids = idx.search(query, k=10, allowlist=allowed)

Filtering happens inside the SIMD kernel at 32-vector block granula

Continue your stack

What teams reach for next — and why each earns a place beside turbovec. Ranked by curator confidence.

Pairs well with

LlamaIndex

Data framework for connecting custom data sources to LLMs — ingestion, indexing, retrieval.

Why it fitsLlamaIndex owns ingestion, chunking and query orchestration but delegates the vector store. turbovec is a local drop-in for that slot when a hosted DB is overkill and RAM is the binding constraint.

swiftide

Rust framework for LLM apps: an agent harness, compile-time-typed task graphs, and streaming RAG pipelines — MCP toolboxes, human-in-the-loop approval, tracing with Langfuse support.

Why it fitsSwiftide is the Rust RAG and agent pipeline; turbovec is a Rust index that slots underneath it without leaving the language or shelling out to a Python vector service.

Alternative to See all + compare →

Chroma

Open-source embedding database for building AI apps with retrieval.

Why it fitsBoth give you dense retrieval in-process. Chroma is a batteries-included embedding database with metadata, collections and persistence; turbovec is only the index — dramatically smaller and faster, but you supply everything around it.

latticedb

Embedded single-file graph database in Zig: graph traversal, HNSW vector search and BM25 full-text in one query language, plus a durable event log — built for Graph RAG and local agent memory.

Why it fitsBoth embed search with no server to run. LatticeDB fuses graph traversal, HNSW and BM25 into one query language in a single file; turbovec does one thing — quantized dense search — and expects another system to handle the rest.

PageIndex

Vectorless, reasoning-based RAG — builds a hierarchical tree index from long documents so an LLM retrieves by relevance instead of similarity. No chunking, no embeddings, no vector DB.

Why it fitsOpposite bets on retrieval. PageIndex removes vectors entirely and has the model reason over a tree index; turbovec makes vectors cheap enough that keeping them is the easy call. Choose by whether your corpus rewards similarity or structure.