Repository navigation
Description
What problem does this solve?
As code graphs and cross-repository indexes grow, the semantic-search path could benefit from a lower-memory vector index without weakening retrieval quality or incremental indexing.
TurboVec implements Google Research's TurboQuant in Rust and advertises online ingestion, 2/4-bit compression, SIMD search, persistence, stable IDs, deletion, and allowlist filtering:
https://github.com/RyanCodrai/turbovec
The allowlist filtering is especially interesting for codebase-memory-mcp: structural/BM25/graph signals could narrow candidates, then dense retrieval could rerank only the allowed symbol IDs.
This is a recommendation to evaluate, not a request to replace the current implementation based on upstream benchmarks alone.
Proposed solution
Create a bounded benchmark or design spike comparing the current semantic-search implementation against TurboVec/TurboQuant on representative code embeddings and repository sizes.
Measure:
- recall@k and ranking quality on code-symbol queries
- memory usage and persisted index size
- indexing and incremental-add latency
- deletion/update behavior
- single-query and batch-query latency
- filtered retrieval using graph-derived candidate IDs
- portability across Windows, Linux, macOS, x86, and ARM
Preserve these constraints:
- the pure-C, single-static-binary distribution remains the default
- AST, SQLite graph storage, BM25, and graph relationships remain unchanged
- no mandatory Rust runtime or external service
- no adoption unless project-owned benchmarks show a meaningful end-to-end gain
If results justify further work, possible paths include:
- Port the relevant TurboQuant indexing/search algorithm to C.
- Keep an optional experimental external backend behind a stable internal interface.
- Adopt only the quantization or filtered-search ideas, not TurboVec itself.
Alternatives considered
- Keep the current semantic-search implementation unchanged; this remains the correct baseline.
- FAISS adds a much larger native dependency and conflicts with the zero-dependency distribution goal.
- Integrating TurboVec directly would introduce Rust build and supply-chain complexity.
- Replacing SQLite or the graph model is unnecessary; this proposal concerns only dense vector retrieval.
Confirmations
- I searched existing issues and this is not a duplicate.
Activity
DeusData commentedon Jul 25, 2026
Thanks for this — and specifically for writing it as an evaluation proposal with explicit non-goals rather than a "please adopt X" request. The constraints you list (pure-C single static binary stays the default, no mandatory Rust runtime or external service, AST/SQLite/BM25/graph untouched, no adoption without project-owned benchmarks) are the projects actual constraints, so this lands as a coherent proposal rather than something transplanted from a different architecture.
For grounding, the surface you are proposing against does exist today. Semantic embeddings live in
src/semantic/(semantic.c/semantic.h, plusrotsq.candast_profile.c), with vector storage in the graph buffer (src/graph_buffer/graph_buffer.h:149, "Vector storage (for semantic embeddings)"). The dimension is 768 to match nomic-embed-code (src/semantic/semantic.h:35), and the whole path is opt-in behindCBM_SEMANTIC_ENABLED=1(semantic.h:87). It currently producesSEMANTICALLY_RELATEDedges as a pre-dump pass (src/pipeline/pipeline_internal.h:603).Two observations that matter for how such an evaluation would have to be designed:
The allowlist-filtering angle you highlight is the genuinely interesting part, and you are right that it fits this project unusually well — narrowing candidates by structural/BM25/graph signals and reranking only allowed symbol IDs matches how the graph is already used. That is worth measuring independently of any quantization result, because it is a retrieval-architecture question rather than a vector-index question.
The portability line in your measurement list is not a formality here; it is close to decisive. The distribution model is a single static C binary across Windows, Linux and macOS on both x86 and ARM, and a Rust dependency is a build- and release-pipeline change well before it is a performance question. Of your three follow-on paths, that asymmetry means path 1 (port the algorithm to C) and path 2 (optional backend behind a stable internal interface) carry very different costs, and the benchmark would need to be structured to inform that choice rather than just producing recall/latency numbers.
Whether to spend the bench cycles on this at all is a roadmap call rather than a triage one, so I am recording it rather than scheduling it, and leaving it at
priority/backlogwhere you filed it. Flagging one thing for whoever picks it up: the current implementation is algorithmic-embedding based and opt-in, so a fair comparison needs a baseline measured withCBM_SEMANTIC_ENABLED=1on representative code embeddings, not against the default build where the path is inert.Thanks for the link and for the disciplined framing — recommendations that state their own falsification criteria are rare and much appreciated.
nwn900 commentedon Aug 19, 2026
A concrete integration direction that seems worth benchmarking is to use TurboVec as a query-time semantic vector index, while leaving the rest of codebase-memory-mcp intact.
The goal would be enhancement, not replacement: preserve CBM’s tree-sitter/LSP knowledge graph, SQLite graph storage, BM25/FTS, structural queries, call tracing, impact analysis, architecture tooling, and existing semantic signal generation. TurboVec would replace only the current dense-vector candidate-search hot path.
Suggested architecture
Source -> CBM ------+
+-- TurboVec IdMapIndex
768 dimensions
node_id -> stable u64 id
4-bit TurboQuant / TQ+
For semantic_query, the flow could be:
This keeps TurboVec behind an internal retrieval abstraction rather than making it the system of record.
Why this boundary looks promising
CBM’s query-time semantic vectors are 768-dimensional int8 vectors, and the current search path scans them using a SQLite cosine function before reranking candidates.
TurboVec supports dimensions that are multiples of 8, stable external IDs through IdMapIndex, O(1) deletion, incremental sync(), and allowlist filtering, so it maps naturally onto the existing node-vector workload.
At the raw vector-payload level, 768-dimensional storage is approximately:
TurboVec has additional index metadata, so these are not total-index-size claims.
I would start evaluation with 4-bit TQ+, not 2-bit, and treat retrieval quality as a hard acceptance criterion.
Preserve CBM’s strengths
I would explicitly not use TurboVec to replace:
TurboVec should only accelerate and compress dense candidate retrieval.
Hybrid retrieval opportunity
TurboVec’s allowlist search could also enable a stronger hybrid path:
This may be more valuable than simply replacing one full scan with another because CBM’s structural knowledge can directly reduce vector-search work.
Engineering concerns
A few areas would need explicit design:
Suggested benchmark
Compare the current CBM semantic-search implementation with TurboVec-backed candidate retrieval on representative repositories and several symbol counts.
Measure:
If the results are favorable, this could make semantic retrieval faster and more memory-efficient without sacrificing any of the structural code-intelligence functionality that makes CBM valuable.
DeusData commentedon Oct 19, 2026
Thank you for the additional query-time design and generation-consistency considerations. A benchmark and any dependency/FFI choice still require a maintainer roadmap decision. No evaluation has been scheduled or backend approved in this triage pass.
DeusData commentedon Oct 25, 2026
Thank you for the query-time proposal and consistency considerations. Benchmark scope and any backend/dependency choice still need a maintainer roadmap decision. Sorry for the delay; this is waiting on maintainer follow-up, and I’ll follow up soon.