Skip to content

Commit 3b79e3d

Browse files
Merge branch 'main' into perf/py-cross-registry-index
Signed-off-by: Nelson Spence <nelson@projectnavi.ai>
2 parents 65acd38 + 268a9d8 commit 3b79e3d

155 files changed

Lines changed: 1118952 additions & 678651 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/_lint.yml‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,11 @@ permissions:
1010

1111
jobs:
1212
lint:
13-
runs-on: ubuntu-latest
13+
# Pinned, not ubuntu-latest: the LLVM install below adds an
14+
# apt.llvm.org llvm-toolchain-NOBLE repo, which is 24.04 by name. When
15+
# GitHub moves ubuntu-latest to the next LTS this job breaks on a day
16+
# nobody chose. Pin the runner to the release the repo line already names.
17+
runs-on: ubuntu-24.04
1418
# 15 min proved too tight on a slow runner day and a timed-out lint reads
1519
# as "cancelled", which the release graph must treat as a hard stop; keep
1620
# a bound, but one only a genuine hang can hit (normal runtime ~5 min).
@@ -60,7 +64,8 @@ jobs:
6064
# provability (never suppressed — the NOLINT ban applies here too).
6165
# Vendored-tree diagnostics are path-filtered, mirroring .cppcheck.
6266
lint-mem:
63-
runs-on: ubuntu-latest
67+
# Pinned for the same reason as `lint`: llvm-toolchain-noble-21 below.
68+
runs-on: ubuntu-24.04
6469
timeout-minutes: 60
6570
steps:
6671
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

‎.github/workflows/_test.yml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -245,7 +245,9 @@ jobs:
245245
# parallel suite — catches what an older compiler-rt can miss. Same
246246
# canonical scripts/test.sh wave as every other leg.
247247
test-diag:
248-
runs-on: ubuntu-latest
248+
# Pinned for the same reason as the lint jobs: the pinned diagnostic
249+
# toolchain below installs from llvm-toolchain-noble-21.
250+
runs-on: ubuntu-24.04
249251
timeout-minutes: 120
250252
steps:
251253
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

‎.github/workflows/codeql.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ jobs:
2929
run: sudo apt-get update && sudo apt-get install -y zlib1g-dev
3030

3131
- name: Initialize CodeQL
32-
uses: github/codeql-action/init@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v4.38.0
32+
uses: github/codeql-action/init@1c5b675653bb5c22dbe9b12b556ec555138e09fd # v4.38.1
3333
with:
3434
languages: c-cpp
3535
build-mode: manual
@@ -38,6 +38,6 @@ jobs:
3838
run: scripts/build.sh
3939

4040
- name: Perform CodeQL Analysis
41-
uses: github/codeql-action/analyze@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v4.38.0
41+
uses: github/codeql-action/analyze@1c5b675653bb5c22dbe9b12b556ec555138e09fd # v4.38.1
4242
with:
4343
category: "/language:c-cpp"

‎.github/workflows/release.yml‎

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -236,8 +236,20 @@ jobs:
236236
- name: Generate SBOM
237237
run: python3 scripts/ci/generate-sbom.py "${{ inputs.version }}"
238238

239+
# actions/attest-sbom is deprecated. What it does now is emit
240+
# "::warning::actions/attest-sbom has been deprecated" and forward to
241+
# actions/attest with NODE_OPTIONS set, passing sbom-path straight
242+
# through. Calling attest directly produces the same attestation without
243+
# the wrapper, and sbom-path is a first-class input there (verified
244+
# against the action definition at this exact SHA, which is tag v4.1.0).
245+
#
246+
# This step deliberately carries no continue-on-error: a release must
247+
# fail if its SBOM cannot be attested. An unattested SBOM is worse than
248+
# an absent one, because the artifact still ships looking complete.
239249
- name: Attest SBOM
240-
uses: actions/attest-sbom@c604332985a26aa8cf1bdc465b92731239ec6b9e # v4.1.0
250+
uses: actions/attest@59d89421af93a897026c735860bf21b6eb4f7b26 # v4.1.0
251+
env:
252+
NODE_OPTIONS: '--max-http-header-size=32768'
241253
with:
242254
subject-path: '*.tar.gz'
243255
sbom-path: 'sbom.json'

‎.github/workflows/scorecard.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,6 @@ jobs:
2727
publish_results: true
2828

2929
- name: Upload SARIF results
30-
uses: github/codeql-action/upload-sarif@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v4
30+
uses: github/codeql-action/upload-sarif@1c5b675653bb5c22dbe9b12b556ec555138e09fd # v4
3131
with:
3232
sarif_file: results.sarif

‎Makefile.cbm‎

Lines changed: 30 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -338,6 +338,7 @@ FOUNDATION_SRCS = \
338338
EXTRACTION_SRCS = \
339339
$(CBM_DIR)/cbm.c \
340340
$(CBM_DIR)/extract_defs.c \
341+
$(CBM_DIR)/callable_sig.c \
341342
$(CBM_DIR)/extract_calls.c \
342343
$(CBM_DIR)/extract_imports.c \
343344
$(CBM_DIR)/extract_usages.c \
@@ -436,6 +437,7 @@ PIPELINE_SRCS = \
436437
src/pipeline/lsp_surface.c \
437438
src/pipeline/pipeline_delta.c \
438439
src/pipeline/path_alias.c \
440+
src/pipeline/laravel_routing.c \
439441
src/pipeline/registry.c \
440442
src/pipeline/pipeline.c \
441443
src/pipeline/pipeline_incremental.c \
@@ -633,6 +635,7 @@ TEST_FOUNDATION_SRCS = \
633635

634636
TEST_EXTRACTION_SRCS = \
635637
tests/test_extraction.c \
638+
tests/test_callable_sig.c \
636639
tests/test_extraction_inheritance.c \
637640
tests/test_extraction_imports.c \
638641
tests/test_parse_coverage.c \
@@ -832,8 +835,14 @@ BUILD_DIR = build/c
832835
# ── Object file compilation (grammars need relaxed warnings) ─────
833836

834837
# Grammar + tree-sitter runtime: compiled without -Werror (upstream code has warnings)
838+
# $(TEST_SEAM_DEFINE): these prod objects (grammars, ts_runtime, lsp_all) are
839+
# linked into the same binary as the CFLAGS_PROD sources, so both halves must
840+
# agree on the seam switch. lsp_all holds seam code of its own (ts_lsp.c's
841+
# budget probes, any seam counter a resolver defines): without the define a
842+
# TEST_SEAMS=1 binary silently lacks those seams, and a seam symbol that cbm.c
843+
# references fails to link. GRAMMAR_CFLAGS_TEST always carries the define.
835844
GRAMMAR_CFLAGS = -std=c11 -D_DEFAULT_SOURCE -O2 -w -I$(CBM_DIR) -I$(TS_INCLUDE) -I$(TS_SRC) \
836-
$(MEMWASTE_VENDOR_CFLAGS)
845+
$(TEST_SEAM_DEFINE) $(MEMWASTE_VENDOR_CFLAGS)
837846
GRAMMAR_CFLAGS_TEST = -std=c11 -D_DEFAULT_SOURCE -g -O1 -w -I$(CBM_DIR) -I$(TS_INCLUDE) -I$(TS_SRC) \
838847
-DCBM_ENABLE_TEST_SEAMS=1 $(SANITIZED_DEFINE) $(SANITIZE)
839848
GRAMMAR_CFLAGS_TSAN = -std=c11 -D_DEFAULT_SOURCE -g -O1 -w -I$(CBM_DIR) -I$(TS_INCLUDE) -I$(TS_SRC) \
@@ -941,7 +950,9 @@ $(BUILD_DIR)/sqlite3.o: $(SQLITE3_SRC) | $(BUILD_DIR)
941950
$(BUILD_DIR)/tsan_sqlite3.o: $(SQLITE3_SRC) | $(BUILD_DIR)
942951
$(CC) $(SQLITE3_CFLAGS_TEST) $(TSAN_SANITIZE) -c -o $@ $<
943952

944-
$(BUILD_DIR)/prod_sqlite3.o: $(SQLITE3_SRC) Makefile.cbm | $(BUILD_DIR)
953+
# .build-config: SQLITE3_CFLAGS carries MEMWASTE_VENDOR_CFLAGS (see the stamp
954+
# under "Production binary").
955+
$(BUILD_DIR)/prod_sqlite3.o: $(SQLITE3_SRC) Makefile.cbm $(BUILD_DIR)/.build-config | $(BUILD_DIR)
945956
$(CC) $(SQLITE3_CFLAGS) -c -o $@ $<
946957

947958
# TRE regex (only compiled on Windows — POSIX uses system <regex.h>)
@@ -971,7 +982,7 @@ $(BUILD_DIR)/tre.o: $(TRE_SRC) | $(BUILD_DIR)
971982
$(BUILD_DIR)/tsan_tre.o: $(TRE_SRC) | $(BUILD_DIR)
972983
$(CC) $(TRE_CFLAGS) $(TSAN_SANITIZE) -c -o $@ $<
973984

974-
$(BUILD_DIR)/prod_tre.o: $(TRE_SRC) | $(BUILD_DIR)
985+
$(BUILD_DIR)/prod_tre.o: $(TRE_SRC) $(BUILD_DIR)/.build-config | $(BUILD_DIR)
975986
$(CC) $(TRE_CFLAGS) -O2 $(MEMWASTE_VENDOR_CFLAGS) -c -o $@ $<
976987
endif
977988

@@ -1125,28 +1136,35 @@ TS_RUNTIME_OBJ_PROD = $(BUILD_DIR)/prod_ts_runtime.o
11251136
LSP_OBJ_PROD = $(BUILD_DIR)/prod_lsp_all.o
11261137
PP_OBJ_PROD = $(BUILD_DIR)/prod_preprocessor.o
11271138

1128-
$(BUILD_DIR)/prod_%.o: $(CBM_DIR)/%.c $(GRAMMAR_UNITY_DEPS) | $(BUILD_DIR)
1139+
# Every prod object whose flags carry a BUILD_CONFIG_SIG variable
1140+
# (TEST_SEAM_DEFINE via GRAMMAR_CFLAGS, MEMWASTE_VENDOR_CFLAGS) depends on the
1141+
# $(BUILD_DIR)/.build-config stamp. These objects are compiled once and linked
1142+
# into every later binary, so without the stamp a TEST_SEAMS or MEMWASTE flip
1143+
# relinks the PREVIOUS flavour: a release binary keeps seam code compiled into
1144+
# lsp_all, or a seams binary misses it. prod_mimalloc.o and the unixcoder blob
1145+
# take no config variable and stay off the stamp.
1146+
$(BUILD_DIR)/prod_%.o: $(CBM_DIR)/%.c $(GRAMMAR_UNITY_DEPS) $(BUILD_DIR)/.build-config | $(BUILD_DIR)
11291147
$(CC) $(GRAMMAR_CFLAGS) -c -o $@ $<
11301148

1131-
$(BUILD_DIR)/prod_ts_runtime.o: $(TS_RUNTIME_DEPS) | $(BUILD_DIR)
1149+
$(BUILD_DIR)/prod_ts_runtime.o: $(TS_RUNTIME_DEPS) $(BUILD_DIR)/.build-config | $(BUILD_DIR)
11321150
$(CC) $(GRAMMAR_CFLAGS) -c -o $@ $<
11331151

1134-
$(BUILD_DIR)/prod_lsp_all.o: $(LSP_UNITY_DEPS) | $(BUILD_DIR)
1152+
$(BUILD_DIR)/prod_lsp_all.o: $(LSP_UNITY_DEPS) $(BUILD_DIR)/.build-config | $(BUILD_DIR)
11351153
$(CC) $(GRAMMAR_CFLAGS) -c -o $@ $<
11361154

1137-
$(BUILD_DIR)/prod_preprocessor.o: $(CBM_DIR)/preprocessor.cpp | $(BUILD_DIR)
1155+
$(BUILD_DIR)/prod_preprocessor.o: $(CBM_DIR)/preprocessor.cpp $(BUILD_DIR)/.build-config | $(BUILD_DIR)
11381156
$(CXX) $(CXXFLAGS_PROD) -w -I$(CBM_DIR)/vendored -c -o $@ $<
11391157

11401158
# Vendored LZ4 (compiled separately, not unity-built via lz4_store.c)
11411159
LZ4_OBJ_PROD = $(BUILD_DIR)/prod_lz4.o $(BUILD_DIR)/prod_lz4hc.o
1142-
$(BUILD_DIR)/prod_lz4.o: $(CBM_DIR)/vendored/lz4/lz4.c | $(BUILD_DIR)
1160+
$(BUILD_DIR)/prod_lz4.o: $(CBM_DIR)/vendored/lz4/lz4.c $(BUILD_DIR)/.build-config | $(BUILD_DIR)
11431161
$(CC) -std=c11 -D_DEFAULT_SOURCE -O2 -w $(MEMWASTE_VENDOR_CFLAGS) -I$(CBM_DIR) -c -o $@ $<
1144-
$(BUILD_DIR)/prod_lz4hc.o: $(CBM_DIR)/vendored/lz4/lz4hc.c | $(BUILD_DIR)
1162+
$(BUILD_DIR)/prod_lz4hc.o: $(CBM_DIR)/vendored/lz4/lz4hc.c $(BUILD_DIR)/.build-config | $(BUILD_DIR)
11451163
$(CC) -std=c11 -D_DEFAULT_SOURCE -O2 -w $(MEMWASTE_VENDOR_CFLAGS) -I$(CBM_DIR)/vendored/lz4 -c -o $@ $<
11461164

11471165
# Vendored zstd (compiled separately, not unity-built via zstd_store.c)
11481166
ZSTD_OBJ_PROD = $(BUILD_DIR)/prod_zstd.o
1149-
$(BUILD_DIR)/prod_zstd.o: $(CBM_DIR)/vendored/zstd/zstd.c | $(BUILD_DIR)
1167+
$(BUILD_DIR)/prod_zstd.o: $(CBM_DIR)/vendored/zstd/zstd.c $(BUILD_DIR)/.build-config | $(BUILD_DIR)
11501168
$(CC) -std=c11 -D_DEFAULT_SOURCE -O2 -w $(MEMWASTE_VENDOR_CFLAGS) -I$(CBM_DIR)/vendored/zstd -c -o $@ $<
11511169

11521170
OBJS_VENDORED_PROD = $(MIMALLOC_OBJ_PROD) $(SQLITE3_OBJ_PROD) $(TRE_OBJ_PROD) $(GRAMMAR_OBJS_PROD) $(TS_RUNTIME_OBJ_PROD) $(LSP_OBJ_PROD) $(PP_OBJ_PROD) $(LZ4_OBJ_PROD) $(ZSTD_OBJ_PROD) $(UNIXCODER_OBJ)
@@ -1162,7 +1180,8 @@ MAIN_SRC = src/main.c
11621180
# configuration and it is rewritten only when that content changes, so ordinary
11631181
# incremental builds are unaffected. (Single-quoted: the signature may contain
11641182
# the escaped quotes of -DCBM_VERSION, which are literal inside '', but must not
1165-
# contain a literal single quote.)
1183+
# contain a literal single quote.) The config-flavoured prod objects above
1184+
# depend on the stamp for the same reason as the binary.
11661185
# SANITIZE belongs here: it changes the instrumentation compiled into every
11671186
# object, so a build that only changes it MUST rebuild. Its absence made
11681187
# `scripts/test.sh SANITIZE= ...` -- the documented way to get a plain build

‎README.md‎

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -129,7 +129,7 @@ The native `install`, `update`, and `uninstall` commands are the deliberate exce
129129

130130
Package-manager setup (npm, PyPI, or Go) verifies and publishes a coherent private cached runtime set. Sidecars are replaced before the executable with per-file atomic renames; an interrupted multi-file publication is detected and repaired on the next launch rather than being described as one crash-atomic filesystem transaction. It does not replace the active native installation and therefore does not stop running CBM sessions. When that cached binary is executed, it still enters the same exact-build admission barrier. The shell and PowerShell installers invoke the verified candidate's native `install` command, so they do receive the full account-wide activation guarantee.
131131

132-
The ordinary `cli` mode is intentionally separate: it runs one command locally and never starts or connects to the coordination daemon, registers a daemon session, or starts watchers/UI. Its only shared state is the OS admission barrier plus per-project locks for graph mutations. While the command is running, a temporary monitor lets activation cancel that operation and its supervised worker safely; the monitor exits with the command and never becomes a standing daemon. See [CLI Mode](#cli-mode) for details.
132+
The ordinary `cli` mode runs one command locally, but it is not disconnected from the daemon above: it connects to the shared coordination daemon — starting one if none is already running — for that same admission barrier plus per-project locks on graph mutations. Its connection is a `cli_session`: it never registers with the background watcher and never starts a UI, and it holds the admission lease only for the command's lifetime. If the command's own connection is what started the daemon, that daemon exits again once the command closes and no other session is attached; if a daemon was already running, the command simply joins and leaves it exactly as found. While the command is running, a temporary monitor separately lets activation cancel that operation and its supervised worker safely; the monitor exits with the command and never becomes a standing daemon. See [CLI Mode](#cli-mode) for details.
133133

134134
### Graph Visualization UI
135135

@@ -155,6 +155,8 @@ Watcher registration is controlled separately by `auto_watch` (default `true`).
155155

156156
To turn the watcher off entirely, set `config set watcher_enabled false` (default `true`): the background poll thread never starts and no project is registered, while `auto_index` and manual `index_repository` keep working. Unlike `auto_watch` — which is consulted per session — `watcher_enabled` is read once when the background daemon starts, so run `codebase-memory-mcp daemon stop` after changing it; reconnecting your MCP client alone will not restart the daemon. See [docs/CONFIGURATION.md](docs/CONFIGURATION.md#2-cli-managed-runtime-settings).
157157

158+
The watcher follows **git** projects only by default: a project indexed from a directory that is not a git repository is not watched and stays at its last index until you run `index_repository` again. To have such roots polled too, set `config set watch_non_git true` (default `false`, read when the daemon starts): they are then checked on the same cadence with a file-tree scan that uses the indexer's own ignore rules.
159+
158160
### Keeping Up to Date
159161

160162
**Updates run from the install script on every platform, not from inside the running binary.** `codebase-memory-mcp update` validates your flags and then prints the exact command to run:
@@ -183,7 +185,14 @@ Installed through **npm or pip**? Update with your package manager on every plat
183185
codebase-memory-mcp uninstall
184186
```
185187

186-
Removes owned agent config entries, skills, hooks, instructions, and the installed binary. Existing graph indexes are listed and deleted only after confirmation.
188+
Removes owned agent config entries, skills, hooks, instructions, and the installed binary. Existing graph indexes are listed and **kept by default**. `-y`/`--yes`, `-n`/`--no`, and noninteractive input keep them unless `--delete-indexes` is explicitly given. An interactive terminal without that flag is asked separately; the default answer is to keep them.
189+
190+
```bash
191+
codebase-memory-mcp uninstall -y --delete-indexes # also delete every project index
192+
codebase-memory-mcp uninstall --dry-run --delete-indexes # preview; change nothing
193+
```
194+
195+
`--delete-indexes` is explicit consent and takes precedence over `--no`; `--dry-run` always preserves the files. When indexes are kept, uninstall prints their cache directory (`${CBM_CACHE_DIR:-~/.cache/codebase-memory-mcp}`) and how to remove them.
187196

188197
The install script placed beside the binary is **reported, not deleted** — uninstall prints its path and the `rm` command for it. It is left alone on purpose: it may be your own copy, a symlink into a checkout, or managed by a package manager, and an uninstaller should not delete a file it cannot prove it owns.
189198

@@ -630,7 +639,7 @@ no longer a suitable automatic global target.
630639

631640
## CLI Mode
632641

633-
Every MCP tool can be invoked as a local, one-shot command. CLI tools neither start nor connect to the coordination daemon and leave no standing process behind. They hold a crash-safe exact-build admission lease only for the command lifetime. `index_repository` is the only exception internally: it starts a temporary, exact-build supervised worker for the index, then stops that worker before the CLI command exits; the worker holds its own lease until exit.
642+
Every MCP tool can be invoked as a local, one-shot command. CLI tools connect to the shared coordination daemon — starting one if none is already running — to get a crash-safe exact-build admission lease for the command's lifetime; they never register with the background watcher, and a daemon a command had to start itself exits again once the command's connection closes and no other session is using it. `index_repository` is the only exception internally: it starts a temporary, exact-build supervised worker for the index, then stops that worker before the CLI command exits; the worker holds its own lease until exit.
634643

635644
Commands that mutate graph data use shared OS-backed, per-project locks. This serializes conflicting work from CLI and MCP sessions on the same project while allowing unrelated projects to proceed independently.
636645

@@ -763,6 +772,7 @@ codebase-memory-mcp config set auto_index true # auto-index on session
763772
codebase-memory-mcp config set auto_index_limit 50000 # max files for auto-index
764773
codebase-memory-mcp config set auto_watch false # don't register background git watcher (default: true)
765774
codebase-memory-mcp config set watcher_enabled false # stop the watcher thread entirely (default: true)
775+
codebase-memory-mcp config set watch_non_git true # also poll non-git project roots (default: false)
766776
codebase-memory-mcp config set index_max_files 250000 # optional per-index source-file limit
767777
codebase-memory-mcp config set index_max_source_mb 16384 # optional per-index source-size limit
768778
codebase-memory-mcp config reset auto_index # reset to default
@@ -785,7 +795,7 @@ is preserved. See [Index resource limits](docs/INDEX_RESOURCE_LIMITS.md).
785795
| `CBM_MEM_BUDGET_MB` | *(detected)* | Override the in-memory graph budget with an explicit cap in MiB, taking precedence over the `ram_fraction × total_RAM` default. Useful on bare-metal hosts without a cgroup limit, or to pin a budget *below* the cgroup limit so headroom is left for sibling processes. Must be a positive integer; it is clamped to detected total RAM (logged as `mem.budget.clamped`), and non-numeric or non-positive values are ignored with a warning (`mem.budget.env.invalid`). |
786796
| `CBM_DUMP_VERIFY_MIN_RATIO` | `0.5` | After indexing, compare persisted SQLite node count to the in-memory dump count. When persisted nodes fall below this fraction of committed nodes (and committed > 50), `index_repository` returns `status:"degraded"` instead of silent `indexed`. Range 0–1; set `0` to disable. Invalid values are ignored with a warning. |
787797

788-
Environment used by daemon-owned components—such as diagnostics, daemon logging, and process-wide indexing resource limits—is captured from the first daemon-backed session that starts the daemon. Later sessions join that process and cannot replace those values. To change them, close all daemon-backed sessions, update the relevant agent configurations consistently, and restart a session. `CBM_ALLOWED_ROOT` remains session-specific, a conflicting `CBM_CACHE_DIR` is rejected, and one-shot CLI commands read their own environment without starting the daemon.
798+
Environment used by daemon-owned components—such as diagnostics, daemon logging, and process-wide indexing resource limits—is captured from the first daemon-backed session that starts the daemon. Later sessions join that process and cannot replace those values. To change them, close all daemon-backed sessions, update the relevant agent configurations consistently, and restart a session. `CBM_ALLOWED_ROOT` remains session-specific, a conflicting `CBM_CACHE_DIR` is rejected, and a one-shot CLI command is not exempt from the rule above: it connects to the coordination daemon like any other session, starting one if none is running, so its own environment becomes the captured daemon-owned environment only when its invocation is the one that starts the daemon — joining an already-running daemon, it inherits that daemon's already-captured values instead.
789799

790800
```bash
791801
# Store indexes in a custom directory

0 commit comments

Comments
 (0)