Skip to content

Commit 3f6969a

Browse files
committed
Merge upstream main into fm/cbm-worktree-ignore-1
Signed-off-by: Nohj <vitalnohj@gmail.com>
2 parents 0d2f840 + 0f52d30 commit 3f6969a

140 files changed

Lines changed: 13565 additions & 1238 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/smoke.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# Smoke invariants across the WIDEST set of GitHub-hosted runners. Builds the
22
# production source path with explicit fault-injection test seams and runs
33
# scripts/smoke-invariants.sh
4-
# (version/help, MCP initialize handshake [#513], all 14 tools invocable, malformed-
4+
# (version/help, MCP initialize handshake [#513], all registered tools invocable, malformed-
55
# input resilience, clean EOF exit, shared-lib resolution, install dry-run).
66
#
77
# Maximizing platforms is the point: ubuntu-22.04 (older glibc → AlmaLinux/#182

‎Makefile.cbm‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -327,6 +327,7 @@ FOUNDATION_SRCS = \
327327
src/foundation/limits.c \
328328
src/foundation/index_policy.c \
329329
src/foundation/subprocess.c \
330+
src/foundation/git_env.c \
330331
src/foundation/sha256.c \
331332
src/foundation/secure_random.c \
332333
src/foundation/macos_acl.c \
@@ -348,6 +349,7 @@ EXTRACTION_SRCS = \
348349
$(CBM_DIR)/extract_channels.c \
349350
$(CBM_DIR)/extract_k8s.c \
350351
$(CBM_DIR)/extract_dbt.c \
352+
$(CBM_DIR)/sql_values.c \
351353
$(CBM_DIR)/helpers.c \
352354
$(CBM_DIR)/result_compact.c \
353355
$(CBM_DIR)/result_spill.c \
@@ -449,6 +451,7 @@ PIPELINE_SRCS = \
449451
src/pipeline/pass_gitdiff.c \
450452
src/pipeline/pass_configures.c \
451453
src/pipeline/pass_configlink.c \
454+
src/pipeline/pass_doclinks.c \
452455
src/pipeline/pass_route_nodes.c \
453456
src/pipeline/pass_enrichment.c \
454457
src/pipeline/pass_envscan.c \
@@ -675,7 +678,7 @@ TEST_DISCOVER_SRCS = \
675678

676679
TEST_GRAPH_BUFFER_SRCS = tests/test_graph_buffer.c
677680

678-
TEST_PIPELINE_SRCS = tests/test_registry.c tests/test_pipeline.c tests/test_importance.c tests/test_cross_repo.c tests/test_fqn.c tests/test_route_canon.c tests/test_path_alias.c tests/test_configlink.c tests/test_infrascan.c tests/test_worker_pool.c tests/test_parallel.c tests/test_index_resilience.c tests/test_index_format.c tests/test_call_reference_contract.c tests/repro/repro_call_scope_usages.c tests/repro/repro_call_argument_usages.c tests/repro/repro_reference_precision.c tests/repro/repro_lexical_binding_precision.c tests/repro/repro_call_argument_matrix_a.c tests/repro/repro_call_argument_matrix_b.c tests/repro/repro_call_node_behaviors.c tests/repro/repro_language_registry.c tests/repro/repro_call_node_manifest.c tests/repro/repro_lsp_ordered_signatures.c tests/repro/repro_lsp_ordered_local.c tests/repro/repro_ts_overload_return_chains.c tests/repro/repro_harness_cleanup.c tests/repro/repro_runner_filter.c
681+
TEST_PIPELINE_SRCS = tests/test_registry.c tests/test_pipeline.c tests/test_importance.c tests/test_cross_repo.c tests/test_fqn.c tests/test_route_canon.c tests/test_path_alias.c tests/test_configlink.c tests/test_doclinks.c tests/test_infrascan.c tests/test_worker_pool.c tests/test_parallel.c tests/test_index_resilience.c tests/test_index_format.c tests/test_call_reference_contract.c tests/repro/repro_call_scope_usages.c tests/repro/repro_call_argument_usages.c tests/repro/repro_reference_precision.c tests/repro/repro_lexical_binding_precision.c tests/repro/repro_call_argument_matrix_a.c tests/repro/repro_call_argument_matrix_b.c tests/repro/repro_call_node_behaviors.c tests/repro/repro_language_registry.c tests/repro/repro_call_node_manifest.c tests/repro/repro_lsp_ordered_signatures.c tests/repro/repro_lsp_ordered_local.c tests/repro/repro_ts_overload_return_chains.c tests/repro/repro_harness_cleanup.c tests/repro/repro_runner_filter.c
679682

680683
TEST_WATCHER_SRCS = tests/test_watcher.c
681684

‎README.md‎

Lines changed: 34 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -201,7 +201,7 @@ The install script placed beside the binary is **reported, not deleted** — uni
201201
- **Cypher-like queries**: `MATCH (f:Function)-[:CALLS]->(g) WHERE f.name = 'main' RETURN g.name`
202202

203203
### Search
204-
- **Semantic search** (`semantic_query`): vector search across the entire graph, powered by bundled Nomic `nomic-embed-code` embeddings (40K tokens, 768d int8) compiled into the binary — no API key, no Ollama, no Docker. 11-signal combined scoring (TF-IDF, RRI, API/Type/Decorator signatures, AST profiles, data flow, Halstead-lite, MinHash, module proximity, graph diffusion).
204+
- **Semantic search** (the `semantic_query` parameter of `search_graph`, not a separate tool): vector search across the entire graph, powered by bundled Nomic `nomic-embed-code` embeddings (40K tokens, 768d int8) compiled into the binary — no API key, no Ollama, no Docker. 11-signal combined scoring (TF-IDF, RRI, API/Type/Decorator signatures, AST profiles, data flow, Halstead-lite, MinHash, module proximity, graph diffusion).
205205
- **BM25 full-text search** via SQLite FTS5 with `cbm_camel_split` tokenizer (camelCase / snake_case aware)
206206
- **Structural search** (`search_graph`): regex name patterns, label filters, min/max degree, file scoping
207207
- **Code search** (`search_code`): graph-augmented grep over indexed files only
@@ -245,12 +245,12 @@ The install script placed beside the binary is **reported, not deleted** — uni
245245

246246
Commit a single compressed file to your repo and your teammates skip the reindex.
247247

248-
`.codebase-memory/graph.db.zst` is a zstd-compressed snapshot of the knowledge graph that lives next to your source. When you index, the artifact is written or refreshed; when a teammate clones the repo and runs `codebase-memory-mcp` for the first time, the artifact is decompressed and incremental indexing fills in their local diff.
248+
`.codebase-memory/graph.db.zst` is a zstd-compressed snapshot of the knowledge graph that lives next to your source. The artifact is opt-in: `index_repository` writes it only with `persistence: true` (the default is `false`), and once it exists later indexes refresh it; when a teammate clones the repo and runs `codebase-memory-mcp` for the first time, the artifact is decompressed and incremental indexing fills in their local diff.
249249

250250
- **Format**: SQLite database, indexes stripped, `VACUUM INTO` compacted, then zstd 1.5.7 compressed (8–13:1 ratio typical)
251251
- **Two tiers**:
252-
- **Best** (`zstd -9` + index strip + `VACUUM INTO`) — written on explicit `index_repository`
253-
- **Fast** (`zstd -3`) — written by the watcher for low-latency incremental updates
252+
- **Best** (`zstd -9` + index strip + `VACUUM INTO`) — written by `index_repository` with `persistence: true`
253+
- **Fast** (`zstd -3`) — refreshes an existing artifact on every other index (including the watcher's low-latency incremental updates)
254254
- **Bootstrap**: when no local DB exists but the artifact is present, `index_repository` imports the artifact first, then runs incremental indexing — avoiding the full reindex cost
255255
- **No merge pain**: a `.codebase-memory/.gitattributes` line with `merge=ours` is auto-created on first export, so concurrent edits don't produce conflicts on the binary artifact
256256
- **Commit it deliberately**: the artifact is rewritten on every index, including the watcher's Fast tier, and git stores each rewrite as a full new blob. Committing every refresh is what turns a 20 MB file into gigabytes of history — one team reached ~6 GB across ~350 commits of this single path. Pick a cadence (a release, a milestone, a nightly job) rather than committing every save.
@@ -671,12 +671,39 @@ JSON arguments can also be piped on stdin, for tools that take arguments. A tool
671671

672672
| Tool | Description |
673673
|------|-------------|
674-
| `index_repository` | Index a repository into the graph. Auto-sync keeps it fresh after that. |
674+
| `index_repository` | Index a repository into the graph. Auto-sync keeps it fresh after that. Waits for the whole index by default; pass `async: true` to start it in the daemon and return at once, then poll with `status: true` (see below). |
675675
| `list_projects` | List all indexed projects with node/edge counts. |
676676
| `delete_project` | Remove a project and all its graph data. |
677677
| `index_status` | Check indexing status of a project. |
678678
| `check_index_coverage` | Check whether exact paths or a scope are indexed and fresh. A clean result means no recorded gap, not proof of completeness. |
679679

680+
**Long indexes and client call deadlines.** A synchronous `index_repository` on a large
681+
repository can take longer than an MCP client allows one tool call (some IDE clients give up
682+
after a fixed deadline). When a client cancels or disconnects, the daemon cancels an index that
683+
nobody else is waiting for, so retrying the same blocking call never finishes. Use the async
684+
mode instead:
685+
686+
1. `index_repository(repo_path="/abs/path", async: true)` starts the index in the daemon (or
687+
joins the one already running for that project) and returns immediately with
688+
`state` (`queued`/`running`). The job keeps running even if the client cancels, times out or
689+
disconnects; only stopping the daemon ends it.
690+
2. `index_repository(repo_path="/abs/path", status: true)` reports `state`
691+
(`queued`, `running`, `cancelling`, `succeeded`, `failed`, `cancelled`), `started_at`,
692+
`finished_at` and an `error` summary. Poll it until the state is `succeeded`, `failed` or
693+
`cancelled`. Pass the same `repo_path` (and `name`, if the index call used one).
694+
695+
`async` and `status` are exclusive; `async` does not apply to `cross-repo-intelligence`. Both
696+
need the daemon-backed server (the default `codebase-memory-mcp` entry point). A temporary
697+
daemon (started on demand rather than by `codebase-memory-mcp daemon start`) stops, and
698+
cancels its jobs, when its last client disconnects. A connected MCP session keeps it alive, so
699+
async from your editor works. A one-shot `codebase-memory-mcp cli index_repository --async`
700+
that is the daemon's only client is refused with a clear error, because the job would die the
701+
moment the command exits: run `codebase-memory-mcp daemon start` first, keep an MCP session
702+
open, or call without `--async`. `status` works everywhere. When a synchronous call was cut
703+
short, the next
704+
`index_repository` or `status` call for that project carries a `notice` suggesting the async
705+
mode. `index_status` keeps describing the published graph and its freshness.
706+
680707
### Querying
681708

682709
| Tool | Description |
@@ -706,7 +733,7 @@ JSON arguments can also be piped on stdin, for tools that take arguments. A tool
706733

707734
### Edge Types
708735

709-
`CONTAINS_PACKAGE`, `CONTAINS_FOLDER`, `CONTAINS_FILE`, `DEFINES`, `DEFINES_METHOD`, `IMPORTS`, `CALLS`, `CALL_REFERENCE`, `HTTP_CALLS`, `ASYNC_CALLS`, `IMPLEMENTS`, `HANDLES`, `USAGE`, `CONFIGURES`, `WRITES`, `MEMBER_OF`, `TESTS`, `USES_TYPE`, `FILE_CHANGES_WITH`
736+
`CONTAINS_PACKAGE`, `CONTAINS_FOLDER`, `CONTAINS_FILE`, `DEFINES`, `DEFINES_METHOD`, `IMPORTS`, `CALLS`, `CALL_REFERENCE`, `HTTP_CALLS`, `ASYNC_CALLS`, `IMPLEMENTS`, `HANDLES`, `USAGE`, `CONFIGURES`, `REFERENCES_FILE`, `WRITES`, `MEMBER_OF`, `TESTS`, `USES_TYPE`, `FILE_CHANGES_WITH`
710737

711738
### Qualified Names
712739

@@ -802,6 +829,7 @@ SQLite databases stored at `~/.cache/codebase-memory-mcp/`. Persists across rest
802829
|---------|-----|
803830
| `/mcp` doesn't show the server | Check `.mcp.json` path is absolute. Restart agent. Test: `echo '{}' \| /path/to/binary` should output JSON. |
804831
| `index_repository` fails | Pass absolute path: `index_repository(repo_path="/absolute/path")` |
832+
| `index_repository` times out in the client | Start it with `async: true`, then poll with `status: true` (see [Indexing](#indexing)). |
805833
| `trace_path` returns 0 results | Use `search_graph(name_pattern=".*PartialName.*")` first to find the exact name. |
806834
| Queries return wrong project results | Add `project="name"` parameter. Use `list_projects` to see names. |
807835
| Binary not found after install | Add to PATH: `export PATH="$HOME/.local/bin:$PATH"` |

‎docs/CONFIGURATION.md‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -230,6 +230,20 @@ process that should share one daemon must see the same value — set it in the
230230
environment of your MCP client and your shell alike, or a CLI invocation without
231231
it will coordinate through the default location instead.
232232
233+
**WSL2 and Windows drives (`/mnt/c`, `/mnt/e`, ...).** `CBM_CACHE_DIR` goes through
234+
the same private-directory check. With WSL's default automount options, DrvFs
235+
reports every directory as `0777` and ignores `chmod`, so a cache under
236+
`/mnt/<drive>` is refused, and the refusal names this remedy. Either turn on
237+
permission metadata in `/etc/wsl.conf`, then run `wsl --shutdown` and reopen WSL:
238+
239+
```ini
240+
[automount]
241+
options = "metadata,umask=22,fmask=11"
242+
```
243+
244+
or keep the cache on the Linux filesystem (the default `~/.cache/codebase-memory-mcp`),
245+
which is also much faster than a 9p-mounted Windows drive.
246+
233247
Environment used by daemon-owned components—such as diagnostics, daemon logging, and process-wide indexing resource limits—is captured from the first daemon-backed session that starts the daemon. Later sessions join the existing process and cannot replace those values. To change them, close every daemon-backed session, update the relevant agent configurations consistently, and restart a session. `CBM_ALLOWED_ROOT` remains session-specific, a conflicting `CBM_CACHE_DIR` is rejected, and one-shot CLI commands use their own current environment without starting the daemon.
234248
235249

‎docs/cbmignore.md‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,12 @@ For directories:
7070

7171
1. **Built-in skip list** — `.git`, `node_modules`, `dist`, `target`,
7272
`vendor`, tool caches, etc. (60+ names; the fast/moderate index modes add
73-
more, e.g. `docs`, `examples`, `testdata`). A `.cbmignore` negation
73+
more, e.g. `docs`, `examples`, `testdata`). Generated output whose
74+
folder name alone is too common to skip everywhere is matched by path
75+
suffix instead: Laravel's compiled Blade view cache
76+
`storage/framework/views/` (at any depth) is skipped even without
77+
Laravel's stock `.gitignore` there, while other `views/` folders stay
78+
indexed. A `.cbmignore` negation
7479
(for example `!target/`) can un-skip these directories, except the
7580
non-negatable safety core: `.git`, `node_modules`, `.worktrees`, and
7681
`.claude-worktrees`.

0 commit comments

Comments
 (0)