TurboVec Fits 10M Vectors in 4 GB and Still Beats FAISS
A vector index that stores ten million documents in four gigabytes of RAM and still searches faster than FAISS sounds like a benchmark someone tuned to flatter itself. So I read the repo, the paper, and the comments. The numbers mostly hold up. Here is what is real and where it falls short.
turbovec is a Rust vector index with Python bindings, built on Google Research's TurboQuant, an ICLR 2026 paper on online vector quantization. It hit the front page of Hacker News this week at 126 points, and the reaction is telling. Nobody is debating whether it works, they are asking what to do with it: codebase indexing, docs and wikis, or a privacy-first local search stack where nothing leaves the machine.
Why no codebook changes everything
The default way to compress embeddings is product quantization, and FAISS is the reference. You train a set of codebooks on a sample of your data, snap every vector to the nearest entries, and search through lookup tables. It works, but the training step is the friction. The codebooks only describe the data you sampled. Change the distribution and you retrain. Grow the corpus and you rebuild. For a search index that is being appended to constantly, that rebuild loop is where the real cost lives.
TurboQuant sidesteps it. It is data-oblivious. It randomly rotates every vector, which pushes the coordinates into a concentrated Beta distribution, then applies a near-optimal scalar quantizer to each coordinate on its own. For the inner products that nearest-neighbor search actually cares about, it adds a 1-bit quantized JL transform on the residual so the estimate stays unbiased. The paper proves the distortion sits within roughly a 2.7x factor of the information-theoretic lower bound, which is a tight guarantee for a quantizer. No training, no tuning, no rebuild.
That is the core payoff: quantization happens as vectors arrive, so the index is always current.
The numbers, and the two that do not impress
On a 10 million document corpus of 1536-dimensional embeddings, float32 costs 31 GB. turbovec at 4-bit fits it in 4 GB. That is a 7.75x cut in RAM, the number that made the thread light up. One commenter noted that removal latency sits on a log scale, the quiet detail that matters for a live index.
Speed is where it actually wins. The kernels are hand written, NEON SDOT and SMMLA on ARM, AVX-512 VNNI and a vector permute on x86, with AVX2 and a scalar fallback behind them. Against FAISS IndexPQFastScan it averages 3.4x faster at 4-bit and about 23% to 26% faster at 2-bit, on both architectures. The 4-bit margin is the interesting one. It comes from scoring the vector-major layout directly in the dot-product kernel rather than chasing a lookup table.
Now the two numbers that should make you slow down. Recall at 4-bit is solid. Calibrated TurboQuant beats FAISS at R@1 on most cells and both hit 1.0 by k=8. At 2-bit the gap closes, and FAISS keeps a thin edge at deeper k. And at low dimensions, where TurboQuant leans on the near-independence of coordinates, the assumption weakens. On GloVe at d=200 it is ahead at 4-bit but trails at 2-bit. If your embeddings are short and dense, do not assume the headline numbers carry over.
The parts that survive contact with production
The incremental save is the feature I would actually reach for. Calling sync persists only what changed since the last sync, one fsync per call, crash-safe at any byte. A single removal or a small append costs milliseconds no matter how large the index has grown. That is the opposite of the rebuild-everything model, and it is why the log-scale removal got a callout.
Filtering happens inside the SIMD kernel at 32-vector block granularity. You pass an id allowlist, blocks with no allowed slots short-circuit before any lookup, and non-allowed slots inside scored blocks are dropped at insert. A selective filter avoids most of the cost instead of paying it and throwing the result away. Output length is the minimum of k and the number of distinct allowed vectors, so you get back exactly what is permitted, never padded.
It rejects non-float32 arrays instead of silently converting them, the kind of strictness you appreciate when a wrong dtype would have been a silent recall bug. There are also drop-in replacements for the in-memory stores in LangChain, LlamaIndex, Haystack, and Agno, so you swap one import and keep the pipeline.
Where it still earns a shrug
The best comment on the thread is also the most honest. One person said the README reads like it was written by a model, and a reply from an Anthropic employee did not help. Fair. The benchmarks are real and reproducible, the scripts are in the repo, but the marketing copy needs a human pass before a team trusts the tool.
The repo is young, and it has quirks. The author list includes a co-author literally named t with the email t@t. It is approximate search, so you are trading exactness for speed and memory, and the recall edge, where it exists, is thin. Two-bit is the weak point, and low-dimensional embeddings are the regime where the theory is loosest.
When would I reach for it? When a RAG stack is being held back by RAM or latency, when the corpus is live and the rebuild loop is painful, and when privacy means the vectors never leave the box. If you are already happy with FAISS and your embeddings are high-dimensional, the gap is real but not dramatic. What actually matters is not the speed win over FAISS. It is the removal of the training step, and that is the part that quietly changes how you operate an index.
Comments