Repository navigation
Commit 8b4d5e8
committed
perf(store): in-memory adjacency layer for trace_path and degree filters
On a kernel-scale index (8.5M nodes, 16.9M edges) trace_path's recursive
CTE and search_graph's per-row degree subqueries spend seconds in B-tree
seeks. graph_mem reads the edges once, in one sequential scan of the
covering UNIQUE index, into a typed CSR (node id -> out/in neighbours with
interned edge types) and answers both from RAM:
- store_bfs (shortest-path mode): BFS over the layer with the CTE's exact
semantics (minimal hop per node, the start only via a cycle back to it,
(hop, id) order, probe row, JOIN-skips for ids without a node row); node
rows and edge collection still come from SQL. Trail mode is unchanged.
- cbm_store_search with min/max_degree: the two correlated COUNT(*)
subqueries become cbm_adj_degree(); filter, order, page and total are the
same SQL.
Layers live in a process-wide registry (the daemon closes its stores after
every request), refcounted across request threads, allocated through the
memory core (CBM_MEM_CLASS_STORE) and bounded: all layers stay under a
quarter of the budget, idle ones are evicted oldest-first, and a layer that
does not fit is not built (SQL path, never a partial layer). A layer is
keyed by db path + store generation + the rollback-journal file state (the
header change counter every rollback commit bumps), read in one read
transaction with the scan; only read-only query stores of sealed DBs
qualify. The file state matters: the cross-repo pass writes CROSS_* edges
into other projects' published DBs in place without a new generation; such
a DB is in WAL mode and keeps the SQL path until the next publish.
Kernel, macOS, same index, fresh daemon: trace_path schedule 7.5 s -> 21 ms
warm (1.23 s first call incl. a 1.2 s build), search_graph Function
min_degree 50 4.2 s -> 170 ms warm (1.35 s first). Layer: 16,890,273 edges,
237 MB. Responses byte-identical to the baseline on django, redis and the
kernel (tools, Cypher and extra BFS/degree cases).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>1 parent 312fb36 commit 8b4d5e8
6 files changed
Lines changed: 1749 additions & 63 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
449 | 449 | | |
450 | 450 | | |
451 | 451 | | |
452 | | - | |
| 452 | + | |
453 | 453 | | |
454 | 454 | | |
455 | 455 | | |
| |||
744 | 744 | | |
745 | 745 | | |
746 | 746 | | |
| 747 | + | |
747 | 748 | | |
748 | 749 | | |
749 | 750 | | |
| |||
0 commit comments