Skip to content

Commit 8b4d5e8

Browse files
committed
perf(store): in-memory adjacency layer for trace_path and degree filters
On a kernel-scale index (8.5M nodes, 16.9M edges) trace_path's recursive CTE and search_graph's per-row degree subqueries spend seconds in B-tree seeks. graph_mem reads the edges once, in one sequential scan of the covering UNIQUE index, into a typed CSR (node id -> out/in neighbours with interned edge types) and answers both from RAM: - store_bfs (shortest-path mode): BFS over the layer with the CTE's exact semantics (minimal hop per node, the start only via a cycle back to it, (hop, id) order, probe row, JOIN-skips for ids without a node row); node rows and edge collection still come from SQL. Trail mode is unchanged. - cbm_store_search with min/max_degree: the two correlated COUNT(*) subqueries become cbm_adj_degree(); filter, order, page and total are the same SQL. Layers live in a process-wide registry (the daemon closes its stores after every request), refcounted across request threads, allocated through the memory core (CBM_MEM_CLASS_STORE) and bounded: all layers stay under a quarter of the budget, idle ones are evicted oldest-first, and a layer that does not fit is not built (SQL path, never a partial layer). A layer is keyed by db path + store generation + the rollback-journal file state (the header change counter every rollback commit bumps), read in one read transaction with the scan; only read-only query stores of sealed DBs qualify. The file state matters: the cross-repo pass writes CROSS_* edges into other projects' published DBs in place without a new generation; such a DB is in WAL mode and keeps the SQL path until the next publish. Kernel, macOS, same index, fresh daemon: trace_path schedule 7.5 s -> 21 ms warm (1.23 s first call incl. a 1.2 s build), search_graph Function min_degree 50 4.2 s -> 170 ms warm (1.35 s first). Layer: 16,890,273 edges, 237 MB. Responses byte-identical to the baseline on django, redis and the kernel (tools, Cypher and extra BFS/degree cases). Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
1 parent 312fb36 commit 8b4d5e8

6 files changed

Lines changed: 1749 additions & 63 deletions

File tree

‎Makefile.cbm‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -449,7 +449,7 @@ PREPROCESSOR_SRC = $(CBM_DIR)/preprocessor.cpp
449449
SQLITE_WRITER_SRC = $(CBM_DIR)/sqlite_writer.c
450450

451451
# Store module (new)
452-
STORE_SRCS = src/store/store.c src/store/architecture_projection.c src/store/store_impact.c src/store/store_graph_digest.c src/store/store_content_digest.c
452+
STORE_SRCS = src/store/store.c src/store/graph_mem.c src/store/architecture_projection.c src/store/store_impact.c src/store/store_graph_digest.c src/store/store_content_digest.c
453453

454454
# Cypher module (new)
455455
CYPHER_SRCS = src/cypher/cypher.c
@@ -744,6 +744,7 @@ TEST_STORE_SRCS = \
744744
tests/test_store_impact.c \
745745
tests/test_store_scope.c \
746746
tests/test_store_graph_digest.c \
747+
tests/test_graph_mem.c \
747748
tests/test_store_search.c \
748749
tests/test_store_arch.c \
749750
tests/test_architecture_projection.c \

0 commit comments

Comments
 (0)