Skip to content

Commit 8dca7d1

Browse files
committed
Merge upstream DeusData/codebase-memory-mcp through 96c3f41 (step 4)
Finish the true upstream merge begun from sync/upstream-step3 at 3eff09c. Resolve all 39 remaining unmerged paths without aborting or flattening history. Ported upstream behavior from the deleted MCP surface into CLI operations: - index --async starts/joins a daemon-owned job which outlives the request; index --status reports queued/running/succeeded/failed/cancelled state. Strict booleans, exclusive modes, and cross-repo refusal remain enforced. - Daemon admission refuses lone one-shot async callers of temporary daemons; a permanent daemon or another live session supplies a durable host. Cut-short synchronous calls carry --async advice into later replies. - Unnamed re-index and status polls reuse the existing project owning a root; the path-derived owner wins, one custom owner is reused, and ambiguous ownership requires --name. Worker arguments carry the resolved owner and omit async/status wait-mode flags. - Bare non-ASCII project selectors use the same encoding as project names. - Status reports failed node/edge reads as error, clamps displayed counts, and names the unreadable tables instead of claiming an empty graph. - detect_changes uses project-relative diff/hunk paths and translates Git-root porcelain paths for subdirectory projects, excluding unrelated siblings. - Git command children clear repository-local environment variables. - Cross-repo mode reports zero resolved targets explicitly, preserving edges. - Artifact-export failures name the post-publication export phase. - Trace qualified-name fallback releases the empty name-lookup allocation. - Supervisor artifact creation and daemon recovery retain failure diagnostics. MCP-only drops: JSON-RPC/stdio transport, tool annotations, client MCP plugin registration/generation (including the OpenCode V2 adapter), and bundle seams. Deleted src/mcp, client_adapter and MCP tests remain deleted. Native-operation async tests replace MCP packets and test the equivalent live-session host. No newly deferred upstream operation behavior. Earlier output-budget/compact encoding and graph-UI /rpc follow-ups remain outside this merge's new delta. Preserve fork CLI naming, README, Codex managed instruction markers, release and installer flow, and LLVM 21 sanitizer image. Adopt upstream wording where both sides implement the same fix. Retain fork regression cases through their upstream equivalents, including the fork grammar's coalesced macro-error regions. Remove the duplicated definitions-walk failure block; inspect shared paths for duplicate declarations, functions, tests and platform branches. Vendored content was not edited; the incoming upstream grammar files and checksums verify. Regenerate graph-ui lock with npm install --package-lock-only --ignore-scripts; fflate 0.8.3, browserslist 4.29.3 and baseline-browser-mapping 2.11.27 are unchanged. npm audit: zero vulnerabilities. Validation: scripts/build-dev.sh and make -f Makefile.cbm -j32 build/c/test-runner compile with -Werror. clang-format-20 and cppcheck pass. The full lint entry still requires the requested follow-up memory-core baseline integration/refresh for fork files predating the gate. No full test suite was run; Linux compile evidence does not establish Windows/macOS runtime behavior. Graph probe used the exact worktree with persistence=false and left Git porcelain unchanged; 107 parse-partial files were checked via direct source where relevant. The graph became stale after source edits and is discovery only.
2 parents 3eff09c + 96c3f41 commit 8dca7d1

171 files changed

Lines changed: 655557 additions & 188483 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/_lint.yml‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,11 @@ permissions:
1010

1111
jobs:
1212
lint:
13-
runs-on: ubuntu-latest
13+
# Pinned, not ubuntu-latest: the LLVM install below adds an
14+
# apt.llvm.org llvm-toolchain-NOBLE repo, which is 24.04 by name. When
15+
# GitHub moves ubuntu-latest to the next LTS this job breaks on a day
16+
# nobody chose. Pin the runner to the release the repo line already names.
17+
runs-on: ubuntu-24.04
1418
# 15 min proved too tight on a slow runner day and a timed-out lint reads
1519
# as "cancelled", which the release graph must treat as a hard stop; keep
1620
# a bound, but one only a genuine hang can hit (normal runtime ~5 min).
@@ -60,7 +64,8 @@ jobs:
6064
# provability (never suppressed — the NOLINT ban applies here too).
6165
# Vendored-tree diagnostics are path-filtered, mirroring .cppcheck.
6266
lint-mem:
63-
runs-on: ubuntu-latest
67+
# Pinned for the same reason as `lint`: llvm-toolchain-noble-21 below.
68+
runs-on: ubuntu-24.04
6469
timeout-minutes: 60
6570
steps:
6671
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

‎.github/workflows/_test.yml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -257,7 +257,9 @@ jobs:
257257
# parallel suite — catches what an older compiler-rt can miss. Same
258258
# canonical scripts/test.sh wave as every other leg.
259259
test-diag:
260-
runs-on: ubuntu-latest
260+
# Pinned for the same reason as the lint jobs: the pinned diagnostic
261+
# toolchain below installs from llvm-toolchain-noble-21.
262+
runs-on: ubuntu-24.04
261263
timeout-minutes: 120
262264
steps:
263265
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

‎.github/workflows/release.yml‎

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -260,8 +260,20 @@ jobs:
260260
- name: Generate SBOM
261261
run: python3 scripts/ci/generate-sbom.py "${{ inputs.version }}"
262262

263+
# actions/attest-sbom is deprecated. What it does now is emit
264+
# "::warning::actions/attest-sbom has been deprecated" and forward to
265+
# actions/attest with NODE_OPTIONS set, passing sbom-path straight
266+
# through. Calling attest directly produces the same attestation without
267+
# the wrapper, and sbom-path is a first-class input there (verified
268+
# against the action definition at this exact SHA, which is tag v4.1.0).
269+
#
270+
# This step deliberately carries no continue-on-error: a release must
271+
# fail if its SBOM cannot be attested. An unattested SBOM is worse than
272+
# an absent one, because the artifact still ships looking complete.
263273
- name: Attest SBOM
264-
uses: actions/attest-sbom@c604332985a26aa8cf1bdc465b92731239ec6b9e # v4.1.0
274+
uses: actions/attest@59d89421af93a897026c735860bf21b6eb4f7b26 # v4.1.0
275+
env:
276+
NODE_OPTIONS: '--max-http-header-size=32768'
265277
with:
266278
subject-path: '*.tar.gz'
267279
sbom-path: 'sbom.json'

‎Makefile.cbm‎

Lines changed: 31 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -329,6 +329,7 @@ FOUNDATION_SRCS = \
329329
src/foundation/limits.c \
330330
src/foundation/index_policy.c \
331331
src/foundation/subprocess.c \
332+
src/foundation/git_env.c \
332333
src/foundation/sha256.c \
333334
src/foundation/secure_random.c \
334335
src/foundation/macos_acl.c \
@@ -351,6 +352,7 @@ EXTRACTION_SRCS = \
351352
$(CBM_DIR)/extract_channels.c \
352353
$(CBM_DIR)/extract_k8s.c \
353354
$(CBM_DIR)/extract_dbt.c \
355+
$(CBM_DIR)/sql_values.c \
354356
$(CBM_DIR)/helpers.c \
355357
$(CBM_DIR)/result_compact.c \
356358
$(CBM_DIR)/result_spill.c \
@@ -451,6 +453,7 @@ PIPELINE_SRCS = \
451453
src/pipeline/pass_gitdiff.c \
452454
src/pipeline/pass_configures.c \
453455
src/pipeline/pass_configlink.c \
456+
src/pipeline/pass_doclinks.c \
454457
src/pipeline/pass_route_nodes.c \
455458
src/pipeline/pass_enrichment.c \
456459
src/pipeline/pass_envscan.c \
@@ -677,7 +680,7 @@ TEST_DISCOVER_SRCS = \
677680

678681
TEST_GRAPH_BUFFER_SRCS = tests/test_graph_buffer.c
679682

680-
TEST_PIPELINE_SRCS = tests/test_registry.c tests/test_pipeline.c tests/test_importance.c tests/test_cross_repo.c tests/test_fqn.c tests/test_route_canon.c tests/test_path_alias.c tests/test_configlink.c tests/test_infrascan.c tests/test_worker_pool.c tests/test_parallel.c tests/test_index_resilience.c tests/test_index_format.c tests/test_call_reference_contract.c tests/repro/repro_call_scope_usages.c tests/repro/repro_call_argument_usages.c tests/repro/repro_reference_precision.c tests/repro/repro_lexical_binding_precision.c tests/repro/repro_call_argument_matrix_a.c tests/repro/repro_call_argument_matrix_b.c tests/repro/repro_call_node_behaviors.c tests/repro/repro_language_registry.c tests/repro/repro_call_node_manifest.c tests/repro/repro_lsp_ordered_signatures.c tests/repro/repro_lsp_ordered_local.c tests/repro/repro_ts_overload_return_chains.c tests/repro/repro_harness_cleanup.c tests/repro/repro_runner_filter.c
683+
TEST_PIPELINE_SRCS = tests/test_registry.c tests/test_pipeline.c tests/test_importance.c tests/test_cross_repo.c tests/test_fqn.c tests/test_route_canon.c tests/test_path_alias.c tests/test_configlink.c tests/test_doclinks.c tests/test_infrascan.c tests/test_worker_pool.c tests/test_parallel.c tests/test_index_resilience.c tests/test_index_format.c tests/test_call_reference_contract.c tests/repro/repro_call_scope_usages.c tests/repro/repro_call_argument_usages.c tests/repro/repro_reference_precision.c tests/repro/repro_lexical_binding_precision.c tests/repro/repro_call_argument_matrix_a.c tests/repro/repro_call_argument_matrix_b.c tests/repro/repro_call_node_behaviors.c tests/repro/repro_language_registry.c tests/repro/repro_call_node_manifest.c tests/repro/repro_lsp_ordered_signatures.c tests/repro/repro_lsp_ordered_local.c tests/repro/repro_ts_overload_return_chains.c tests/repro/repro_harness_cleanup.c tests/repro/repro_runner_filter.c
681684

682685
TEST_WATCHER_SRCS = tests/test_watcher.c
683686

@@ -839,8 +842,14 @@ PROD_SOURCE_OBJS = $(patsubst %.c,$(BUILD_DIR)/%.prod.o,$(PROD_SOURCE_SRCS))
839842
# ── Object file compilation (grammars need relaxed warnings) ─────
840843

841844
# Grammar + tree-sitter runtime: compiled without -Werror (upstream code has warnings)
845+
# $(TEST_SEAM_DEFINE): these prod objects (grammars, ts_runtime, lsp_all) are
846+
# linked into the same binary as the CFLAGS_PROD sources, so both halves must
847+
# agree on the seam switch. lsp_all holds seam code of its own (ts_lsp.c's
848+
# budget probes, any seam counter a resolver defines): without the define a
849+
# TEST_SEAMS=1 binary silently lacks those seams, and a seam symbol that cbm.c
850+
# references fails to link. GRAMMAR_CFLAGS_TEST always carries the define.
842851
GRAMMAR_CFLAGS = -std=c11 -D_DEFAULT_SOURCE -O2 -w -I$(CBM_DIR) -I$(TS_INCLUDE) -I$(TS_SRC) \
843-
$(MEMWASTE_VENDOR_CFLAGS)
852+
$(TEST_SEAM_DEFINE) $(MEMWASTE_VENDOR_CFLAGS)
844853
GRAMMAR_CFLAGS_TEST = -std=c11 -D_DEFAULT_SOURCE -g -O1 -w -I$(CBM_DIR) -I$(TS_INCLUDE) -I$(TS_SRC) \
845854
-DCBM_ENABLE_TEST_SEAMS=1 $(SANITIZED_DEFINE) $(SANITIZE)
846855
GRAMMAR_CFLAGS_TSAN = -std=c11 -D_DEFAULT_SOURCE -g -O1 -w -I$(CBM_DIR) -I$(TS_INCLUDE) -I$(TS_SRC) \
@@ -964,7 +973,9 @@ $(BUILD_DIR)/sqlite3.o: $(SQLITE3_SRC) | $(BUILD_DIR)
964973
$(BUILD_DIR)/tsan_sqlite3.o: $(SQLITE3_SRC) | $(BUILD_DIR)
965974
$(CC) $(SQLITE3_CFLAGS_TEST) $(TSAN_SANITIZE) -c -o $@ $<
966975

967-
$(BUILD_DIR)/prod_sqlite3.o: $(SQLITE3_SRC) Makefile.cbm | $(BUILD_DIR)
976+
# .build-config: SQLITE3_CFLAGS carries MEMWASTE_VENDOR_CFLAGS (see the stamp
977+
# under "Production binary").
978+
$(BUILD_DIR)/prod_sqlite3.o: $(SQLITE3_SRC) Makefile.cbm $(BUILD_DIR)/.build-config | $(BUILD_DIR)
968979
$(CC) $(SQLITE3_CFLAGS) -c -o $@ $<
969980

970981
# TRE regex (only compiled on Windows — POSIX uses system <regex.h>)
@@ -994,7 +1005,7 @@ $(BUILD_DIR)/tre.o: $(TRE_SRC) | $(BUILD_DIR)
9941005
$(BUILD_DIR)/tsan_tre.o: $(TRE_SRC) | $(BUILD_DIR)
9951006
$(CC) $(TRE_CFLAGS) $(TSAN_SANITIZE) -c -o $@ $<
9961007

997-
$(BUILD_DIR)/prod_tre.o: $(TRE_SRC) | $(BUILD_DIR)
1008+
$(BUILD_DIR)/prod_tre.o: $(TRE_SRC) $(BUILD_DIR)/.build-config | $(BUILD_DIR)
9981009
$(CC) $(TRE_CFLAGS) -O2 $(MEMWASTE_VENDOR_CFLAGS) -c -o $@ $<
9991010
endif
10001011

@@ -1138,28 +1149,35 @@ TS_RUNTIME_OBJ_PROD = $(BUILD_DIR)/prod_ts_runtime.o
11381149
LSP_OBJ_PROD = $(BUILD_DIR)/prod_lsp_all.o
11391150
PP_OBJ_PROD = $(BUILD_DIR)/prod_preprocessor.o
11401151

1141-
$(BUILD_DIR)/prod_%.o: $(CBM_DIR)/%.c $(GRAMMAR_UNITY_DEPS) | $(BUILD_DIR)
1152+
# Every prod object whose flags carry a BUILD_CONFIG_SIG variable
1153+
# (TEST_SEAM_DEFINE via GRAMMAR_CFLAGS, MEMWASTE_VENDOR_CFLAGS) depends on the
1154+
# $(BUILD_DIR)/.build-config stamp. These objects are compiled once and linked
1155+
# into every later binary, so without the stamp a TEST_SEAMS or MEMWASTE flip
1156+
# relinks the PREVIOUS flavour: a release binary keeps seam code compiled into
1157+
# lsp_all, or a seams binary misses it. prod_mimalloc.o and the unixcoder blob
1158+
# take no config variable and stay off the stamp.
1159+
$(BUILD_DIR)/prod_%.o: $(CBM_DIR)/%.c $(GRAMMAR_UNITY_DEPS) $(BUILD_DIR)/.build-config | $(BUILD_DIR)
11421160
$(CC) $(GRAMMAR_CFLAGS) -c -o $@ $<
11431161

1144-
$(BUILD_DIR)/prod_ts_runtime.o: $(TS_RUNTIME_DEPS) | $(BUILD_DIR)
1162+
$(BUILD_DIR)/prod_ts_runtime.o: $(TS_RUNTIME_DEPS) $(BUILD_DIR)/.build-config | $(BUILD_DIR)
11451163
$(CC) $(GRAMMAR_CFLAGS) -c -o $@ $<
11461164

1147-
$(BUILD_DIR)/prod_lsp_all.o: $(LSP_UNITY_DEPS) | $(BUILD_DIR)
1165+
$(BUILD_DIR)/prod_lsp_all.o: $(LSP_UNITY_DEPS) $(BUILD_DIR)/.build-config | $(BUILD_DIR)
11481166
$(CC) $(GRAMMAR_CFLAGS) -c -o $@ $<
11491167

1150-
$(BUILD_DIR)/prod_preprocessor.o: $(CBM_DIR)/preprocessor.cpp | $(BUILD_DIR)
1168+
$(BUILD_DIR)/prod_preprocessor.o: $(CBM_DIR)/preprocessor.cpp $(BUILD_DIR)/.build-config | $(BUILD_DIR)
11511169
$(CXX) $(CXXFLAGS_PROD) -w -I$(CBM_DIR)/vendored -c -o $@ $<
11521170

11531171
# Vendored LZ4 (compiled separately, not unity-built via lz4_store.c)
11541172
LZ4_OBJ_PROD = $(BUILD_DIR)/prod_lz4.o $(BUILD_DIR)/prod_lz4hc.o
1155-
$(BUILD_DIR)/prod_lz4.o: $(CBM_DIR)/vendored/lz4/lz4.c | $(BUILD_DIR)
1173+
$(BUILD_DIR)/prod_lz4.o: $(CBM_DIR)/vendored/lz4/lz4.c $(BUILD_DIR)/.build-config | $(BUILD_DIR)
11561174
$(CC) -std=c11 -D_DEFAULT_SOURCE -O2 -w $(MEMWASTE_VENDOR_CFLAGS) -I$(CBM_DIR) -c -o $@ $<
1157-
$(BUILD_DIR)/prod_lz4hc.o: $(CBM_DIR)/vendored/lz4/lz4hc.c | $(BUILD_DIR)
1175+
$(BUILD_DIR)/prod_lz4hc.o: $(CBM_DIR)/vendored/lz4/lz4hc.c $(BUILD_DIR)/.build-config | $(BUILD_DIR)
11581176
$(CC) -std=c11 -D_DEFAULT_SOURCE -O2 -w $(MEMWASTE_VENDOR_CFLAGS) -I$(CBM_DIR)/vendored/lz4 -c -o $@ $<
11591177

11601178
# Vendored zstd (compiled separately, not unity-built via zstd_store.c)
11611179
ZSTD_OBJ_PROD = $(BUILD_DIR)/prod_zstd.o
1162-
$(BUILD_DIR)/prod_zstd.o: $(CBM_DIR)/vendored/zstd/zstd.c | $(BUILD_DIR)
1180+
$(BUILD_DIR)/prod_zstd.o: $(CBM_DIR)/vendored/zstd/zstd.c $(BUILD_DIR)/.build-config | $(BUILD_DIR)
11631181
$(CC) -std=c11 -D_DEFAULT_SOURCE -O2 -w $(MEMWASTE_VENDOR_CFLAGS) -I$(CBM_DIR)/vendored/zstd -c -o $@ $<
11641182

11651183
OBJS_VENDORED_PROD = $(MIMALLOC_OBJ_PROD) $(SQLITE3_OBJ_PROD) $(TRE_OBJ_PROD) $(GRAMMAR_OBJS_PROD) $(TS_RUNTIME_OBJ_PROD) $(LSP_OBJ_PROD) $(PP_OBJ_PROD) $(LZ4_OBJ_PROD) $(ZSTD_OBJ_PROD) $(UNIXCODER_OBJ)
@@ -1175,7 +1193,8 @@ MAIN_SRC = src/main.c
11751193
# configuration and it is rewritten only when that content changes, so ordinary
11761194
# incremental builds are unaffected. (Single-quoted: the signature may contain
11771195
# the escaped quotes of -DCBM_VERSION, which are literal inside '', but must not
1178-
# contain a literal single quote.)
1196+
# contain a literal single quote.) The config-flavoured prod objects above
1197+
# depend on the stamp for the same reason as the binary.
11791198
# SANITIZE belongs here: it changes the instrumentation compiled into every
11801199
# object, so a build that only changes it MUST rebuild. Its absence made
11811200
# `scripts/test.sh SANITIZE= ...` -- the documented way to get a plain build

‎THIRD_PARTY.md‎

Lines changed: 12 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -67,15 +67,22 @@ License summary:
6767
smart-coin language, written for the upstream project because no usable public
6868
grammar exists; its source and corpus tests live in
6969
`tools/tree-sitter-chialisp/`.
70-
- Seven further grammars (`arkts`, `assembly`, `cfml`,
71-
`cfscript`, `dotenv`, `pine`, `qml`) are self-maintained forks that retain
72-
their original upstream authors' licenses — see the manifest for per-grammar
73-
provenance. `arkts` is an upstream DeusData first-party derivative of
70+
- Nine further grammars (`arkts`, `assembly`, `cfml`,
71+
`cfscript`, `dotenv`, `javascript`, `pine`, `qml`, `tsx`) are self-maintained
72+
forks that retain their original upstream authors' licenses — see the manifest
73+
for per-grammar provenance. `arkts` is an upstream DeusData first-party derivative of
7474
[tree-sitter/tree-sitter-typescript](https://github.com/tree-sitter/tree-sitter-typescript)
7575
(MIT, (c) 2017 Max Brunsfeld; on the
7676
[tree-sitter-javascript](https://github.com/tree-sitter/tree-sitter-javascript)
7777
base, MIT, (c) 2014 Max Brunsfeld) with (c) 2026 DeusData ArkTS additions;
78-
its grammar source lives in `tools/tree-sitter-arkts/`.
78+
its grammar source lives in `tools/tree-sitter-arkts/`. `javascript`
79+
([tree-sitter/tree-sitter-javascript](https://github.com/tree-sitter/tree-sitter-javascript),
80+
MIT, (c) 2014 Max Brunsfeld) and `tsx`
81+
([tree-sitter/tree-sitter-typescript](https://github.com/tree-sitter/tree-sitter-typescript)'s
82+
`tsx` dialect, MIT, (c) 2017 Max Brunsfeld, on the same javascript base) are
83+
the upstream grammars with one (c) 2026 DeusData patch that lets a lone `&`
84+
appear in JSX strings and text; their grammar sources live in
85+
`tools/tree-sitter-javascript/` and `tools/tree-sitter-tsx/`.
7986

8087
### tree-sitter-plsql
8188

‎docs/CONFIGURATION.md‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -182,6 +182,20 @@ same validation as the default, and a value that fails it is refused rather than
182182
silently ignored. Because the rendezvous is how sessions find each other, every
183183
process that should share one daemon must see the same value — set it consistently for any shells or agent processes that should share the same retained runtime coordination.
184184

185+
**WSL2 and Windows drives (`/mnt/c`, `/mnt/e`, ...).** `CBM_CACHE_DIR` goes through
186+
the same private-directory check. With WSL's default automount options, DrvFs
187+
reports every directory as `0777` and ignores `chmod`, so a cache under
188+
`/mnt/<drive>` is refused, and the refusal names this remedy. Either turn on
189+
permission metadata in `/etc/wsl.conf`, then run `wsl --shutdown` and reopen WSL:
190+
191+
```ini
192+
[automount]
193+
options = "metadata,umask=22,fmask=11"
194+
```
195+
196+
or keep the cache on the Linux filesystem (the default `~/.cache/codebase-memory-mcp`),
197+
which is also much faster than a 9p-mounted Windows drive.
198+
185199
Environment used by retained runtime-owned components—such as diagnostics, logging, and process-wide indexing resource limits—is captured when that runtime starts. Later participants cannot replace those values. `CBM_ALLOWED_ROOT` remains caller-specific, a conflicting `CBM_CACHE_DIR` is rejected, and one-shot CLI commands use their own current environment.
186200

187201

‎docs/cbmignore.md‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,12 @@ For directories:
7070

7171
1. **Built-in skip list** — `.git`, `node_modules`, `dist`, `target`,
7272
`vendor`, tool caches, etc. (60+ names; the fast/moderate index modes add
73-
more, e.g. `docs`, `examples`, `testdata`). A `.cbmignore` negation
73+
more, e.g. `docs`, `examples`, `testdata`). Generated output whose
74+
folder name alone is too common to skip everywhere is matched by path
75+
suffix instead: Laravel's compiled Blade view cache
76+
`storage/framework/views/` (at any depth) is skipped even without
77+
Laravel's stock `.gitignore` there, while other `views/` folders stay
78+
indexed. A `.cbmignore` negation
7479
(for example `!target/`) can un-skip these directories, except the
7580
non-negatable safety core: `.git`, `node_modules`, `.worktrees`, and
7681
`.claude-worktrees`.

‎internal/cbm/cbm.c‎

Lines changed: 42 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@
2121
#include "lsp/kotlin_lsp.h"
2222
#include "lsp/rust_lsp.h"
2323
#include "preprocessor.h"
24+
#include "sql_values.h" // #1735: literal INSERT rows kept out of the SQL parse
2425
#include "foundation/compat.h"
2526
#include "foundation/compat_fs.h" // cbm_fopen — crash-supervisor per-file marker write
2627
#include "foundation/hash_table.h" // CBMHashTable — crash-supervisor quarantine set
@@ -1775,8 +1776,9 @@ static void cbm_subtract_recovered_regions(cbm_error_regions_t *regs, const CBMD
17751776
}
17761777

17771778
#ifdef CBM_ENABLE_TEST_SEAMS
1778-
/* Count source bytes read by the #1071 line lookup, so tests can assert cost
1779-
* without depending on timing. */
1779+
/* Source bytes this thread has read to locate lines for the #1071 macro check
1780+
* (walks and line-table builds). Lets a test pin the check's cost as a count
1781+
* instead of a clock (#1735). Compiled only into seam-enabled test artifacts. */
17801782
static CBM_TLS uint64_t tl_macro_line_scan_bytes = 0;
17811783
uint64_t cbm_test_macro_line_scan_bytes(void) {
17821784
return tl_macro_line_scan_bytes;
@@ -1911,9 +1913,10 @@ static bool cbm_region_inside_callable(uint32_t rs, uint32_t re, const CBMDefArr
19111913
return false;
19121914
}
19131915

1914-
/* Use one source line table for all in-callable regions. Regions outside
1915-
* callables need no source scan, and a failed table allocation keeps the
1916-
* original per-region walk as a safe fallback. */
1916+
/* cbm_span_is_macro_invocation over a cbm_line_offsets table: offs[k] is where
1917+
* 1-based line k+1 starts and offs[nlines] is the end of the source, so a span
1918+
* costs two reads instead of a walk from byte 0. Same answer as the walk; falls
1919+
* back to it when there is no table. */
19171920
static bool cbm_lines_are_macro_invocation(const char *src, int src_len, const uint32_t *offs,
19181921
uint32_t nlines, uint32_t start_line, uint32_t end_line,
19191922
const CBMDefArray *defs) {
@@ -1929,6 +1932,13 @@ static bool cbm_lines_are_macro_invocation(const char *src, int src_len, const u
19291932
(int)offs[last], defs);
19301933
}
19311934

1935+
/* #1071 subtraction. Both questions are pure, so asking the cheap one first
1936+
* changes no answer: "is the region inside a function?" reads only the
1937+
* definition list, while "is it a macro call?" has to find the region's lines
1938+
* in the source. A region no function encloses — every region of a file with
1939+
* no functions, such as a SQL data dump — never touches the source, and the
1940+
* ones that do share one line table built on first use. The old order walked
1941+
* the source from byte 0 for every region: regions x file bytes (#1735). */
19321942
static void cbm_subtract_macro_invocation_regions(cbm_error_regions_t *regs,
19331943
const CBMDefArray *defs, const char *src,
19341944
int src_len) {
@@ -2117,6 +2127,21 @@ CBMFileResult *cbm_extract_file(const char *source, int source_len, CBMLanguage
21172127
enum { CBM_EXTRACT_SCRATCH_BLOCK = CBM_SZ_512 * CBM_SZ_1K };
21182128
enum { CBM_EXTRACT_SCRATCH_KEEP_BYTES = 4 * CBM_SZ_1K * CBM_SZ_1K };
21192129

2130+
/* The #1735 value-row exclusion is always on. Test builds can turn it off for
2131+
* one file (CBM_TEST_SQL_FULL_PARSE_ON=<rel_path substring>) so a test can
2132+
* compare the excluded parse against the full one on the same source. */
2133+
static bool cbm_sql_values_exclusion_on(const char *rel_path) {
2134+
#ifdef CBM_ENABLE_TEST_SEAMS
2135+
const char *full_on = getenv("CBM_TEST_SQL_FULL_PARSE_ON");
2136+
if (full_on && full_on[0] && rel_path && strstr(rel_path, full_on)) {
2137+
return false;
2138+
}
2139+
#else
2140+
(void)rel_path;
2141+
#endif
2142+
return true;
2143+
}
2144+
21202145
static CBMFileResult *extract_file_ex_body(const char *source, int source_len, CBMLanguage language,
21212146
const char *project, const char *rel_path,
21222147
int64_t timeout_micros, const char **extra_defines,
@@ -2225,7 +2250,19 @@ static CBMFileResult *extract_file_ex_body(const char *source, int source_len, C
22252250
#endif
22262251
}
22272252

2253+
/* #1735: a SQL data dump's literal-only INSERT rows carry no graph content
2254+
* but dominate its parse. Keep them out through included ranges; offsets
2255+
* and positions of everything kept are unchanged. The parser is
2256+
* thread-local and reused, so the ranges are cleared right after. */
2257+
CBMSqlKeptRanges sql_kept = {NULL, 0};
2258+
bool sql_ranged = language == CBM_LANG_SQL && cbm_sql_values_exclusion_on(rel_path) &&
2259+
cbm_sql_values_kept_ranges(source, (uint32_t)source_len, &sql_kept) &&
2260+
ts_parser_set_included_ranges(parser, sql_kept.items, sql_kept.count);
22282261
TSTree *tree = ts_parser_parse_with_options(parser, NULL, ts_input, opts);
2262+
if (sql_ranged) {
2263+
(void)ts_parser_set_included_ranges(parser, NULL, 0);
2264+
}
2265+
cbm_sql_kept_ranges_free(&sql_kept);
22292266
uint64_t t1 = now_ns();
22302267
#ifdef CBM_ENABLE_TEST_SEAMS
22312268
t1 += tl_parse_wall_seam_offset_ns; /* the stall seam inflates every wall reading */

0 commit comments

Comments
 (0)