Skip to content

Commit 0e91b3d

Browse files
authored
Merge branch 'main' into fix/index-worker-explicit-memory-budget
2 parents cc37792 + 638b116 commit 0e91b3d

71 files changed

Lines changed: 357704 additions & 111 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.gitignore‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -86,3 +86,7 @@ soak-results-query-leak/
8686
# Scorecard as a binary artifact in source). Source only, never the build.
8787
/pkg/go/codebase-memory-mcp
8888
/pkg/go/codebase-memory-mcp.exe
89+
90+
# Nix build symlinks (nix build → ./result, ./result-1, …)
91+
/result
92+
/result-*

‎Makefile.cbm‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -356,6 +356,7 @@ PIPELINE_SRCS = \
356356
src/pipeline/pass_complexity.c \
357357
src/pipeline/pass_cross_repo.c \
358358
src/pipeline/artifact.c \
359+
src/pipeline/pass_ensemble_routing.c \
359360
src/pipeline/pass_pkgmap.c
360361

361362
# SimHash / MinHash module
@@ -559,7 +560,7 @@ TEST_DISCOVER_SRCS = \
559560

560561
TEST_GRAPH_BUFFER_SRCS = tests/test_graph_buffer.c
561562

562-
TEST_PIPELINE_SRCS = tests/test_registry.c tests/test_pipeline.c tests/test_cross_repo.c tests/test_fqn.c tests/test_route_canon.c tests/test_path_alias.c tests/test_configlink.c tests/test_infrascan.c tests/test_worker_pool.c tests/test_parallel.c tests/test_index_resilience.c tests/test_call_reference_contract.c tests/repro/repro_call_scope_usages.c tests/repro/repro_call_argument_usages.c tests/repro/repro_reference_precision.c tests/repro/repro_lexical_binding_precision.c tests/repro/repro_call_argument_matrix_a.c tests/repro/repro_call_argument_matrix_b.c tests/repro/repro_call_node_behaviors.c tests/repro/repro_language_registry.c tests/repro/repro_call_node_manifest.c tests/repro/repro_lsp_ordered_signatures.c tests/repro/repro_lsp_ordered_local.c tests/repro/repro_ts_overload_return_chains.c tests/repro/repro_harness_cleanup.c tests/repro/repro_runner_filter.c
563+
TEST_PIPELINE_SRCS = tests/test_registry.c tests/test_pipeline.c tests/test_cross_repo.c tests/test_fqn.c tests/test_route_canon.c tests/test_path_alias.c tests/test_configlink.c tests/test_infrascan.c tests/test_worker_pool.c tests/test_parallel.c tests/test_index_resilience.c tests/test_index_format.c tests/test_call_reference_contract.c tests/repro/repro_call_scope_usages.c tests/repro/repro_call_argument_usages.c tests/repro/repro_reference_precision.c tests/repro/repro_lexical_binding_precision.c tests/repro/repro_call_argument_matrix_a.c tests/repro/repro_call_argument_matrix_b.c tests/repro/repro_call_node_behaviors.c tests/repro/repro_language_registry.c tests/repro/repro_call_node_manifest.c tests/repro/repro_lsp_ordered_signatures.c tests/repro/repro_lsp_ordered_local.c tests/repro/repro_ts_overload_return_chains.c tests/repro/repro_harness_cleanup.c tests/repro/repro_runner_filter.c
563564

564565
TEST_WATCHER_SRCS = tests/test_watcher.c
565566

‎README.md‎

Lines changed: 54 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -4,9 +4,9 @@
44
[![License](https://img.shields.io/badge/license-MIT-green)](LICENSE)
55
[![CI](https://img.shields.io/github/actions/workflow/status/DeusData/codebase-memory-mcp/dry-run.yml?label=CI)](https://github.com/DeusData/codebase-memory-mcp/actions/workflows/dry-run.yml)
66
[![Tests](https://img.shields.io/badge/tests-6768_passing-brightgreen)](https://github.com/DeusData/codebase-memory-mcp)
7-
[![Languages](https://img.shields.io/badge/languages-158-orange)](https://github.com/DeusData/codebase-memory-mcp)
7+
[![Languages](https://img.shields.io/badge/languages-161-orange)](https://github.com/DeusData/codebase-memory-mcp)
88
[![Hybrid LSP](https://img.shields.io/badge/Hybrid_LSP-10_languages-blue)](#hybrid-lsp)
9-
[![Agents](https://img.shields.io/badge/agent_surfaces-43-purple)](https://github.com/DeusData/codebase-memory-mcp)
9+
[![Agents](https://img.shields.io/badge/agent_surfaces-45-purple)](https://github.com/DeusData/codebase-memory-mcp)
1010
[![Pure C](https://img.shields.io/badge/pure_C-no_language_runtime-blue)](https://github.com/DeusData/codebase-memory-mcp)
1111
[![Platform](https://img.shields.io/badge/macOS_%7C_Linux_%7C_Windows-supported-lightgrey)](https://github.com/DeusData/codebase-memory-mcp/releases/latest)
1212
[![OpenSSF Scorecard](https://api.scorecard.dev/projects/github.com/DeusData/codebase-memory-mcp/badge)](https://scorecard.dev/viewer/?uri=github.com/DeusData/codebase-memory-mcp)
@@ -16,7 +16,7 @@
1616

1717
**The fastest and most efficient code intelligence engine for AI coding agents.** Full-indexes an average repository in milliseconds, the Linux kernel (28M LOC, 75K files) in 3 minutes. Answers structural queries in under 1ms. Ships as a native executable with a small verified runtime-asset set for macOS, Linux, and Windows — download, run `install`, done.
1818

19-
High-quality parsing through [tree-sitter](https://tree-sitter.github.io/tree-sitter/) AST analysis across all 158 languages, enhanced with [**Hybrid LSP** semantic type resolution](#hybrid-lsp) for Python, TypeScript / JavaScript / JSX / TSX, PHP, C#, Go, C, C++, Java, Kotlin, Rust, and Perl — producing a persistent knowledge graph of functions, classes, call chains, HTTP routes, and cross-service links. 15 MCP tools. No language runtime, hosted service, or API key. Plug and play across 43 supported automatic/conditional client surfaces.
19+
High-quality parsing through [tree-sitter](https://tree-sitter.github.io/tree-sitter/) AST analysis across all 161 languages, enhanced with [**Hybrid LSP** semantic type resolution](#hybrid-lsp) for Python, TypeScript / JavaScript / JSX / TSX, PHP, C#, Go, C, C++, Java, Kotlin, Rust, and Perl — producing a persistent knowledge graph of functions, classes, call chains, HTTP routes, and cross-service links. 15 MCP tools. No language runtime, hosted service, or API key. Plug and play across 45 supported automatic/conditional client surfaces.
2020

2121
> **Research** — The design and benchmarks behind this project are described in the preprint [*Codebase-Memory: Tree-Sitter-Based Knowledge Graphs for LLM Code Exploration via MCP*](https://arxiv.org/abs/2603.27277) (arXiv:2603.27277). Evaluated across 31 real-world repositories: 83% answer quality, 10× fewer tokens, 2.1× fewer tool calls vs. file-by-file exploration.
2222
@@ -32,9 +32,9 @@ High-quality parsing through [tree-sitter](https://tree-sitter.github.io/tree-si
3232

3333
- **Extreme indexing speed** — Linux kernel (28M LOC, 75K files) in 3 minutes. RAM-first pipeline: LZ4 compression, in-memory SQLite, fused Aho-Corasick pattern matching. Memory released after indexing.
3434
- **Plug and play** — native executable plus authenticated release-owned assets for macOS (arm64/amd64), Linux (arm64/amd64), and Windows (amd64). The native install needs no Docker, language runtime, or API keys. Download → `install` → restart agent → done.
35-
- **158 languages** — vendored tree-sitter grammars compiled into the binary. Nothing to install, nothing that breaks.
35+
- **161 languages** — vendored tree-sitter grammars compiled into the binary. Nothing to install, nothing that breaks.
3636
- **120x fewer tokens** — 5 structural queries: ~3,400 tokens vs ~412,000 via file-by-file search. One graph query replaces dozens of grep/read cycles.
37-
- **43 supported automatic/conditional client surfaces** — `install` configures detected clients and safely activates conditional clients only when their documented platform, marker, or explicit existing config path is present. See [Multi-Agent Support](#multi-agent-support) for the complete matrix and manual/UI-only boundaries.
37+
- **45 supported automatic/conditional client surfaces** — `install` configures detected clients and safely activates conditional clients only when their documented platform, marker, or explicit existing config path is present. See [Multi-Agent Support](#multi-agent-support) for the complete matrix and manual/UI-only boundaries.
3838
- **Built-in graph visualization** — 3D interactive UI at `localhost:9749`, served from the binary itself.
3939
- **Infrastructure-as-code indexing** — Dockerfiles, Kubernetes manifests, and Kustomize overlays indexed as graph nodes with cross-references. `Resource` nodes for K8s kinds, `Module` nodes for Kustomize overlays with `IMPORTS` edges to referenced resources.
4040
- **15 MCP tools** — search, trace, architecture, impact analysis, targeted index-coverage checks, Cypher queries, dead code detection, cross-service HTTP linking, ADR management, and more.
@@ -153,6 +153,8 @@ When enabled, new projects are indexed automatically on first connection. Previo
153153

154154
Watcher registration is controlled separately by `auto_watch` (default `true`). Set `config set auto_watch false` to keep a session from registering its project with the background watcher — useful when working across many projects and you want each session contained to explicit indexing.
155155

156+
To turn the watcher off entirely, set `config set watcher_enabled false` (default `true`): the background poll thread never starts and no project is registered, while `auto_index` and manual `index_repository` keep working. Unlike `auto_watch` — which is consulted per session — `watcher_enabled` is read once when the background daemon starts, so run `codebase-memory-mcp daemon stop` after changing it; reconnecting your MCP client alone will not restart the daemon. See [docs/CONFIGURATION.md](docs/CONFIGURATION.md#2-cli-managed-runtime-settings).
157+
156158
### Keeping Up to Date
157159

158160
**Updates run from the install script on every platform, not from inside the running binary.** `codebase-memory-mcp update` validates your flags and then prints the exact command to run:
@@ -353,6 +355,38 @@ paru -S codebase-memory-mcp-bin
353355

354356
The `codebase-memory-mcp-bin` package is available at: https://aur.archlinux.org/packages/codebase-memory-mcp-bin
355357

358+
### Nix (flake)
359+
360+
The flake exposes two server packages plus the standalone frontend:
361+
362+
| Package | Contents |
363+
|---------|----------|
364+
| `default` (`codebase-memory-mcp`) | Standard server, no UI |
365+
| `codebase-memory-mcp-ui` | Server with the graph UI embedded (`--ui=true` works) |
366+
| `graph-ui` | Just the built frontend assets (`dist/`) |
367+
368+
Run directly without installing:
369+
370+
```bash
371+
# Standard server
372+
nix run github:DeusData/codebase-memory-mcp
373+
374+
# Server with the embedded graph UI
375+
nix run github:DeusData/codebase-memory-mcp#codebase-memory-mcp-ui -- --ui=true --port=9749
376+
# then open http://127.0.0.1:9749
377+
```
378+
379+
Or build a binary into `./result/bin/codebase-memory-mcp`:
380+
381+
```bash
382+
nix build github:DeusData/codebase-memory-mcp # standard
383+
nix build github:DeusData/codebase-memory-mcp#codebase-memory-mcp-ui # with UI
384+
```
385+
386+
Working in a clone? Use `.` in place of the flake URL, e.g. `nix run .#codebase-memory-mcp-ui -- --ui=true`, or drop into a shell that puts the binary on `PATH` with `nix shell .#codebase-memory-mcp-ui`.
387+
388+
> **Note:** launched by hand (not from an MCP client) the server exits as soon as `stdin` closes — that's normal MCP behaviour. Keep `stdin` open while testing the UI, e.g. `sleep infinity | codebase-memory-mcp --ui=true --port=9749`. The `codebase-memory-mcp-ui` package embeds the UI at build time; `nix run`'ing the standard `default` package with `--ui=true` will refuse to start the HTTP server.
389+
356390
### Install via Claude Code
357391

358392
```
@@ -425,7 +459,7 @@ Restart your agent. Verify with `/mcp` — you should see `codebase-memory-mcp`
425459

426460
## Multi-Agent Support
427461

428-
`install` configures 43 client surfaces: 37 detected automatically and 6
462+
`install` configures 45 client surfaces: 39 detected automatically and 6
429463
conditional or explicit. “Conditional” means the installer writes only when the
430464
documented platform or an explicit, already-existing config path proves the
431465
target is active. It never flips experimental feature flags, enables plugins,
@@ -473,6 +507,7 @@ overwrite user-modified agents.
473507
| Crush | Detected | `.config/crush/crush.json` | Managed context path with explicit parent-to-child handoff |
474508
| Goose | Detected | `.config/goose/config.yaml` | `.goosehints` |
475509
| Mistral Vibe | Detected | `$VIBE_HOME/config.toml` | `AGENTS.md`, skill, and three matched agent/prompt pairs with explicit read-only graph-tool allowlists |
510+
| Grok Build | Detected | `$GROK_HOME/config.toml` | Owned `rules/codebase-memory.md`, skill, three graph agents with named-server `mcpInheritance` and exact `server__tool` dispatcher ids; context hooks withheld because its passive hook events discard stdout |
476511
| Qoder CLI | Detected | `~/.qoder/settings.json` | Skill, three directly MCP-attached agents with named-server scoping and exact per-tier graph-tool lists; `SessionStart`, `SubagentStart`, and post-`Read` coverage, including documented PowerShell execution on Windows |
477512
| Kimi Code CLI | Detected | `$KIMI_CODE_HOME/mcp.json` (default `~/.kimi-code`) | Same-root `AGENTS.md` + skill; fail-open `UserPromptSubmit` hook in `config.toml` |
478513
| GitLab Duo CLI | Detected | `$GLAB_CONFIG_DIR/duo/mcp.json` or platform fallback | Fail-open user `SessionStart` on macOS/Linux; hook withheld on Windows; no experimental global skill enablement |
@@ -490,6 +525,7 @@ overwrite user-modified agents.
490525
| Pochi | Detected | `~/.pochi/config.jsonc` (`mcp`) | `README.pochi.md`, skill, and three `readFile`-only parent-handoff agents |
491526
| Pi | Detected | — | `~/.pi/agent/AGENTS.md` + skill; MCP/subagents require an explicit reviewed extension |
492527
| IBM Bob IDE | Conditional | Existing `~/.bob/mcp.json` | Shared rule + IDE skill; no invented hook or agent |
528+
| Oh My Pi (omp) | Detected | Effective agent directory (`OMP_PROFILE` / `PI_CODING_AGENT_DIR`; default `~/.omp/agent/mcp.json`) | Skill and three direct-MCP graph-tool subagents (Scout/Verify/Auditor); preserves user `AGENTS.md` |
493529
| Sourcegraph Cody | Explicit opt-in | Existing `$CBM_CODY_CONFIG_PATH` | MCP only |
494530

495531
### Sessions, compaction, and subagents
@@ -521,7 +557,7 @@ carry the contract across fresh sessions and compaction: verify the graph projec
521557
and index freshness, query structural facts in the parent, then pass the project,
522558
qualified symbols, paths, and call-chain evidence in every delegated task.
523559
Claude, Codex, Gemini, Kiro, Qwen, Copilot, CodeBuddy, OpenCode, Kilo, Vibe,
524-
Qoder, Junie, and Factory receive Scout, Verify, and Auditor graph profiles.
560+
Qoder, Junie, Factory, and Grok Build receive Scout, Verify, and Auditor graph profiles.
525561
Kiro embeds this MCP server with `--tool-profile scout` for Scout and
526562
`--tool-profile analysis` for Verify/Auditor. Junie registers equivalent named
527563
server aliases because its subagent schema filters by server rather than by
@@ -547,7 +583,12 @@ project or plugin agents before user agents with the same name; reload the
547583
client after installation or profile changes.
548584
Cursor context
549585
hooks are withheld: session context injection has a known race, `subagentStart`
550-
is control-only, and read-only subagents cannot safely receive MCP access. Rovo
586+
is control-only, and read-only subagents cannot safely receive MCP access. Grok Build's
587+
passive hook events (`SessionStart`, `SubagentStart`, `PostToolUse`) discard
588+
stdout and `PreToolUse` honors only deny/rewrite decisions, so its context hooks
589+
are withheld; Grok also reads Claude and Cursor MCP, skill, and hook files
590+
through its compat layer, and the native `config.toml` entry shadows that copy
591+
by name. Rovo
551592
has no documented session context-output hook, and Bob
552593
documents neither a suitable hook nor a custom-agent surface. Those surfaces are
553594
not approximated with invented augmentation. Kimi plugins, Amp plugins, and
@@ -667,6 +708,7 @@ codebase-memory-mcp config list # show all settings
667708
codebase-memory-mcp config set auto_index true # auto-index on session start
668709
codebase-memory-mcp config set auto_index_limit 50000 # max files for auto-index
669710
codebase-memory-mcp config set auto_watch false # don't register background git watcher (default: true)
711+
codebase-memory-mcp config set watcher_enabled false # stop the watcher thread entirely (default: true)
670712
codebase-memory-mcp config reset auto_index # reset to default
671713
```
672714

@@ -754,14 +796,14 @@ codebase-memory-mcp ships a **lightweight C implementation of language type-reso
754796

755797
**Two-layer architecture:**
756798

757-
1. **Tree-sitter pass** — fast, syntactic, runs for every one of the 158 languages. Extracts definitions, calls, imports.
799+
1. **Tree-sitter pass** — fast, syntactic, runs for every one of the 161 languages. Extracts definitions, calls, imports.
758800
2. **Hybrid LSP pass** — type-aware, runs above the tree-sitter pass per-language. Refines call edges using the import graph plus a per-file or pre-built cross-file definition registry. Languages without a Hybrid LSP pass yet fall back to textual resolution, so you always get *some* answer.
759801

760802
The result is a knowledge graph accurate enough to drive `trace_path` across packages, inheritance hierarchies, and stdlib calls — without paying for a language server process per project.
761803

762804
## Language Support
763805

764-
158 languages, all parsed via vendored tree-sitter grammars compiled into the binary. Benchmarked against 64 real open-source repositories (78 to 49K nodes):
806+
161 languages, all parsed via vendored tree-sitter grammars compiled into the binary. Benchmarked against 64 real open-source repositories (78 to 49K nodes):
765807

766808
| Tier | Score | Languages |
767809
|------|-------|-----------|
@@ -778,7 +820,7 @@ src/
778820
main.c Entry point (MCP stdio server + CLI + install/update/config)
779821
daemon/ Per-account session coordination, IPC, lifecycle, shared jobs/watchers
780822
mcp/ MCP server (15 tools, JSON-RPC 2.0, session detection, auto-index)
781-
cli/ Install/uninstall/update/config (43 client surfaces, hooks, instructions)
823+
cli/ Install/uninstall/update/config (45 client surfaces, hooks, instructions)
782824
store/ SQLite graph storage (nodes, edges, traversal, search, Louvain)
783825
pipeline/ Multi-pass indexing (structure → definitions → calls → HTTP links → config → tests)
784826
cypher/ Cypher query lexer, parser, planner, executor
@@ -787,7 +829,7 @@ src/
787829
traces/ Runtime trace ingestion
788830
ui/ Local HTTP server + verified external 3D-UI asset pack
789831
foundation/ Platform abstractions (threads, filesystem, logging, memory)
790-
internal/cbm/ Vendored tree-sitter grammars (158 languages) + AST extraction engine
832+
internal/cbm/ Vendored tree-sitter grammars (161 languages) + AST extraction engine
791833
```
792834

793835
## Security

‎THIRD_PARTY.md‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,21 @@ License summary:
6464
base, MIT, (c) 2014 Max Brunsfeld) with (c) 2026 DeusData ArkTS additions;
6565
its grammar source lives in `tools/tree-sitter-arkts/`.
6666

67+
### tree-sitter-plsql
68+
69+
- **Project:** [AndreasMaierDe/tree-sitter-plsql](https://github.com/AndreasMaierDe/tree-sitter-plsql)
70+
- **License:** MIT
71+
- **Copyright:** (c) 2022 AndreasMaierDe
72+
- **Vendored at:** `internal/cbm/vendored/grammars/plsql/`
73+
- **Pinned commit:** `28aebef209be`
74+
- **Notes:** Community-maintained grammar for Oracle PL/SQL; not in the
75+
nvim-treesitter or Helix registries (`community-niche` in the manifest). No
76+
external scanner. One local patch: `parser.c`'s
77+
`#include <tree_sitter/parser.h>` is changed to the quoted form used by every
78+
other vendored grammar, as documented in
79+
`internal/cbm/vendored/grammars/MANIFEST.md`. PL/SQL support was originally
80+
contributed in PR #1033 by Oğuz (@ouzsrcm).
81+
6782
### tree-sitter-objectscript (UDL + routine)
6883

6984
- **Project:** [intersystems/tree-sitter-objectscript](https://github.com/intersystems/tree-sitter-objectscript)

0 commit comments

Comments
 (0)