Skip to content

Commit 9d53f32

Browse files
committed
Merge branch 'main' into fix/1355-external-import-shadow
# Conflicts: # src/pipeline/pass_calls.c # src/pipeline/pass_parallel.c
2 parents 4eb3af3 + 5958f54 commit 9d53f32

79 files changed

Lines changed: 6333 additions & 356 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/pr.yml‎

Lines changed: 22 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -64,9 +64,29 @@ jobs:
6464
run: |
6565
# The full .diff endpoint rejects large-but-valid PRs at 20k lines.
6666
# The paginated files endpoint remains filename-only for this gate.
67-
FILES=$(gh api --paginate "repos/$REPO/pulls/$PR/files?per_page=100" --jq '.[].filename')
67+
if FILES=$(gh api --paginate "repos/$REPO/pulls/$PR/files?per_page=100" --jq '.[].filename'); then
68+
echo "file list: pulls/files API"
69+
else
70+
# The files endpoint renders the diff server-side too, and answers
71+
# 422 "this diff is taking too long to generate" for a PR that
72+
# regenerates a vendored parser (#2246: a 42 MB sql/parser.c) --
73+
# so does the compare endpoint. The PR merge ref's first parent is
74+
# the base, so a name-only diff across it is the same file list.
75+
# depth 2 + blob:none fetches commits and trees only (well under
76+
# 1 MB, under a second); nothing from the PR is checked out or run.
77+
echo "::notice::pulls/files API failed; file list taken from refs/pull/$PR/merge"
78+
CHANGES_GIT="$RUNNER_TEMP/changes.git"
79+
git init -q --bare "$CHANGES_GIT"
80+
git --git-dir="$CHANGES_GIT" fetch -q --depth=2 --filter=blob:none \
81+
"https://github.com/$REPO" "refs/pull/$PR/merge"
82+
FILES=$(git --git-dir="$CHANGES_GIT" diff --name-only --no-renames FETCH_HEAD^1 FETCH_HEAD)
83+
fi
6884
printf '%s\n' "$FILES"
69-
if printf '%s\n' "$FILES" | grep -qE '^(src/|internal/|install\.(sh|ps1)|scripts/build\.sh|scripts/smoke-test\.sh|scripts/smoke-local\.sh|scripts/smoke-fixture-server\.py|scripts/gen-third-party-notices\.sh|scripts/env\.sh|test-infrastructure/vm/(vm-smoke\.sh|windows-user-path-guard\.ps1)|Makefile\.cbm)'; then
85+
# A here-string, not a pipe: under this shell's pipefail, grep -q
86+
# exits on its first match, printf dies of SIGPIPE once the list
87+
# outgrows the pipe buffer, and the MATCH is then reported as a
88+
# failure -- product=false, smoke skipped, on the largest PRs.
89+
if grep -qE '^(src/|internal/|install\.(sh|ps1)|scripts/build\.sh|scripts/smoke-test\.sh|scripts/smoke-local\.sh|scripts/smoke-fixture-server\.py|scripts/gen-third-party-notices\.sh|scripts/env\.sh|test-infrastructure/vm/(vm-smoke\.sh|windows-user-path-guard\.ps1)|Makefile\.cbm)' <<<"$FILES"; then
7090
echo "product=true" >> "$GITHUB_OUTPUT"
7191
else
7292
echo "product=false" >> "$GITHUB_OUTPUT"

‎CONTRIBUTING.md‎

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,7 @@ src/
5555
foundation/ Arena allocator, hash table, string utils, platform compat
5656
store/ SQLite graph storage (WAL mode, FTS5)
5757
cypher/ Cypher query → SQL translation
58-
mcp/ MCP server (JSON-RPC 2.0 over stdio, 14 tools)
58+
mcp/ MCP server (JSON-RPC 2.0 over stdio, 17 tools)
5959
pipeline/ Multi-pass indexing pipeline
6060
pass_*.c Individual pipeline passes (definitions, calls, usages, etc.)
6161
httplink.c HTTP route extraction (Go/Express/Laravel/Ktor/Python)
@@ -149,6 +149,25 @@ If in doubt, open an issue and ask.
149149
- Run `scripts/test.sh` and `scripts/lint.sh` before submitting
150150
- Keep PRs focused — avoid unrelated reformatting or refactoring
151151

152+
## AI-assisted and AI-authored contributions
153+
154+
Contributions written with the help of an AI tool, or written by an AI agent acting for a person, are welcome here on the same terms as any other. What we review is the change, not who or what typed it. Three things make that work:
155+
156+
- **Say so.** If an agent wrote the PR or is answering in the thread, disclose it — a line in the PR description or at the top of a comment is enough ("written with X", "this account is operated by an AI agent on behalf of Y"). Disclosure is never held against a contribution. Finding out later is.
157+
- **A person is accountable.** The [DCO sign-off](#license-and-sign-off-dco--required-on-every-commit) is a legal certification, and only a person can make it. The human behind the account certifies every commit, answers for it, and is who we talk to if something goes wrong. An agent may carry the conversation; it cannot carry the responsibility.
158+
- **The bar does not move.** Every claim in a PR must be something a reviewer can check: the failing test, the command that was run, the numbers and the machine they came from. "The model says it works" is not evidence, and a confident description does not replace a test that fails without the fix. We read AI-written PRs exactly as carefully as human-written ones — no more suspiciously, and no less.
159+
160+
Two practical notes:
161+
162+
- `Co-authored-by:` trailers naming a tool are fine. Links to private chat or agent sessions are not — they are dead to everyone else and do not belong in permanent history; we will ask you to remove them.
163+
- High-volume, low-effort submissions — many near-identical PRs, changes nobody ran, replies that do not engage with the review — will be closed, whoever or whatever produced them. One careful PR is worth more than twenty generated ones.
164+
165+
### Who speaks for the project
166+
167+
Only the maintainer account ([@DeusData](https://github.com/DeusData)) speaks for this project. A review, an approval, a "this will be merged" or a "this is a duplicate" from any other account is that person's opinion, however official it sounds — helpful reviews from the community are very welcome, and that is what they are.
168+
169+
**Impersonating a maintainer is not acceptable**, by a person or by an agent: presenting yourself as part of the maintainer team, posting approvals or change requests styled as maintainer decisions, or telling other contributors what the project has decided. We dismiss such reviews, hide the comments, say publicly in the affected threads that they were not ours, and report and block the account. If you are unsure whether a response you received is from the project, check the account name — and ask.
170+
152171
## Security
153172

154173
We take security seriously. All PRs go through:

‎Makefile.cbm‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -325,6 +325,7 @@ FOUNDATION_SRCS = \
325325
src/foundation/profile.c \
326326
src/foundation/dump_verify.c \
327327
src/foundation/limits.c \
328+
src/foundation/index_policy.c \
328329
src/foundation/subprocess.c \
329330
src/foundation/sha256.c \
330331
src/foundation/secure_random.c \
@@ -617,6 +618,7 @@ TEST_FOUNDATION_SRCS = \
617618
tests/test_str_intern.c \
618619
tests/test_log.c \
619620
tests/test_str_util.c \
621+
tests/test_index_policy.c \
620622
tests/test_workspace.c \
621623
tests/test_platform.c \
622624
tests/test_diagnostics.c \

‎README.md‎

Lines changed: 13 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@
1616

1717
**The fastest and most efficient code intelligence engine for AI coding agents.** Full-indexes an average repository in milliseconds, the Linux kernel (28M LOC, 75K files) in 3 minutes. Answers structural queries in under 1ms. Ships as a native executable with a small verified runtime-asset set for macOS, Linux, and Windows — download, run `install`, done.
1818

19-
High-quality parsing through [tree-sitter](https://tree-sitter.github.io/tree-sitter/) AST analysis across all 162 languages, enhanced with [**Hybrid LSP** semantic type resolution](#hybrid-lsp) for Python, TypeScript / JavaScript / JSX / TSX, PHP, C#, Go, C, C++, Java, Kotlin, Rust, and Perl — producing a persistent knowledge graph of functions, classes, call chains, HTTP routes, and cross-service links. 15 MCP tools. No language runtime, hosted service, or API key. Plug and play across 45 supported automatic/conditional client surfaces.
19+
High-quality parsing through [tree-sitter](https://tree-sitter.github.io/tree-sitter/) AST analysis across all 162 languages, enhanced with [**Hybrid LSP** semantic type resolution](#hybrid-lsp) for Python, TypeScript / JavaScript / JSX / TSX, PHP, C#, Go, C, C++, Java, Kotlin, Rust, and Perl — producing a persistent knowledge graph of functions, classes, call chains, HTTP routes, and cross-service links. 17 MCP tools. No language runtime, hosted service, or API key. Plug and play across 45 supported automatic/conditional client surfaces.
2020

2121
> **Research** — The design and benchmarks behind this project are described in the preprint [*Codebase-Memory: Tree-Sitter-Based Knowledge Graphs for LLM Code Exploration via MCP*](https://arxiv.org/abs/2603.27277) (arXiv:2603.27277). Evaluated across 31 real-world repositories: 83% answer quality, 10× fewer tokens, 2.1× fewer tool calls vs. file-by-file exploration.
2222
@@ -37,7 +37,7 @@ High-quality parsing through [tree-sitter](https://tree-sitter.github.io/tree-si
3737
- **45 supported automatic/conditional client surfaces** — `install` configures detected clients and safely activates conditional clients only when their documented platform, marker, or explicit existing config path is present. See [Multi-Agent Support](#multi-agent-support) for the complete matrix and manual/UI-only boundaries.
3838
- **Built-in graph visualization** — 3D interactive UI at `localhost:9749`, served from the binary itself.
3939
- **Infrastructure-as-code indexing** — Dockerfiles, Kubernetes manifests, and Kustomize overlays indexed as graph nodes with cross-references. `Resource` nodes for K8s kinds, `Module` nodes for Kustomize overlays with `IMPORTS` edges to referenced resources.
40-
- **15 MCP tools** — search, trace, architecture, impact analysis, targeted index-coverage checks, Cypher queries, dead code detection, cross-service HTTP linking, ADR management, and more.
40+
- **17 MCP tools** — search, trace, architecture, impact analysis, targeted index-coverage checks, Cypher queries, dead code detection, cross-service HTTP linking, ADR management, and more.
4141

4242
## Quick Start
4343

@@ -461,7 +461,7 @@ Add to `~/.claude.json` (user scope) or project `.mcp.json`:
461461
}
462462
```
463463

464-
Restart your agent. Verify with `/mcp` — you should see `codebase-memory-mcp` with 15 tools.
464+
Restart your agent. Verify with `/mcp` — you should see `codebase-memory-mcp` with 17 tools.
465465

466466
</details>
467467

@@ -673,6 +673,7 @@ JSON arguments can also be piped on stdin, for tools that take arguments. A tool
673673
| `list_projects` | List all indexed projects with node/edge counts. |
674674
| `delete_project` | Remove a project and all its graph data. |
675675
| `index_status` | Check indexing status of a project. |
676+
| `check_index_coverage` | Check whether exact paths or a scope are indexed and fresh. A clean result means no recorded gap, not proof of completeness. |
676677

677678
### Querying
678679

@@ -683,7 +684,9 @@ JSON arguments can also be piped on stdin, for tools that take arguments. A tool
683684
| `detect_changes` | Map git diff to affected symbols + blast radius with risk classification. |
684685
| `query_graph` | Execute Cypher-like graph queries (read-only). |
685686
| `get_graph_schema` | Node/edge counts, relationship patterns, property definitions per label. Run this first. |
687+
| `compare_graphs` | Compare two indexed snapshots: node/edge additions and removals between a base and a target. |
686688
| `get_code_snippet` | Read source code for a function by qualified name. |
689+
| `get_file_outline` | Declaration outline of one repository-relative file in source order, with optional label filter and paging. |
687690
| `get_architecture` | Codebase overview: languages, packages, routes, hotspots, clusters, ADR. |
688691
| `search_code` | Grep-like text search within indexed project files. |
689692
| `manage_adr` | CRUD for Architecture Decision Records (`get` reads, `update` replaces the whole document, `set_sections` rewrites only the named sections and leaves every other byte untouched, `sections` lists headings). Query modes do not wait behind a same-project reindex; writes remain serialized. |
@@ -733,9 +736,15 @@ codebase-memory-mcp config set auto_index true # auto-index on session
733736
codebase-memory-mcp config set auto_index_limit 50000 # max files for auto-index
734737
codebase-memory-mcp config set auto_watch false # don't register background git watcher (default: true)
735738
codebase-memory-mcp config set watcher_enabled false # stop the watcher thread entirely (default: true)
739+
codebase-memory-mcp config set index_max_files 250000 # optional per-index source-file limit
740+
codebase-memory-mcp config set index_max_source_mb 16384 # optional per-index source-size limit
736741
codebase-memory-mcp config reset auto_index # reset to default
737742
```
738743

744+
The two `index_max_*` settings default to `off`. Exceeding one fails the complete
745+
index attempt rather than publishing a partial graph; an existing serving index
746+
is preserved. See [Index resource limits](docs/INDEX_RESOURCE_LIMITS.md).
747+
739748
### Environment Variables
740749

741750
| Variable | Default | Description |
@@ -843,7 +852,7 @@ Also supported (not yet benchmarked): Ada, Agda, Apex, Assembly (NASM), Astro, A
843852
src/
844853
main.c Entry point (MCP stdio server + CLI + install/update/config)
845854
daemon/ Per-account session coordination, IPC, lifecycle, shared jobs/watchers
846-
mcp/ MCP server (15 tools, JSON-RPC 2.0, session detection, auto-index)
855+
mcp/ MCP server (17 tools, JSON-RPC 2.0, session detection, auto-index)
847856
cli/ Install/uninstall/update/config (45 client surfaces, hooks, instructions)
848857
store/ SQLite graph storage (nodes, edges, traversal, search, Louvain)
849858
pipeline/ Multi-pass indexing (structure → definitions → calls → HTTP links → config → tests)

‎docs/CONFIGURATION.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -89,6 +89,8 @@ Current keys:
8989
| `auto_index_limit` | `50000` | Maximum file count allowed for automatic indexing of a new project. |
9090
| `auto_watch` | `true` | Register the session's project with the background git watcher on connect. Set `false` to keep a session from registering its project (the watcher still runs for other projects). |
9191
| `watcher_enabled` | `true` | Master switch for the background watcher subsystem. Set `false` to stop the watcher from starting at all — no poll thread and no project registration. Reindex manually with `index_repository` when disabled. |
92+
| `index_max_files` | `off` | Optional maximum number of accepted source files in one discovery run. |
93+
| `index_max_source_mb` | `off` | Optional maximum accepted source size in MiB in one discovery run. |
9294

9395
> **`watcher_enabled` vs `auto_watch`.** `watcher_enabled` controls whether the
9496
> watcher *subsystem* starts at all (the background poll thread). `auto_watch` is
@@ -116,6 +118,14 @@ Current keys:
116118
> `auto_index` still runs, and `index_repository` stays available for manual
117119
> reindexing.
118120
121+
The two `index_max_*` settings are independent and disabled by default. They
122+
apply to explicit indexing, automatic indexing, and watcher re-indexing, but not
123+
to `cross-repo-intelligence`, which does not scan repository source files.
124+
Equality is allowed; exceeding either setting fails the complete index request
125+
and preserves any previously serving database. See
126+
[Index resource limits](INDEX_RESOURCE_LIMITS.md) for counting, validation, and
127+
error-response details.
128+
119129
## 3. UI Settings
120130
121131
The optional built-in graph UI stores its settings in:

‎docs/INDEX_RESOURCE_LIMITS.md‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
# Index resource limits
2+
3+
Index resource limits are optional operator controls for repositories whose
4+
discovery breadth is not known in advance. They are disabled by default so
5+
existing large-repository workloads retain their current behavior.
6+
7+
## Discovery settings
8+
9+
| Key | Default | Accepted value | Protects |
10+
|---|---:|---:|---|
11+
| `index_max_files` | `off` | `off` or `1..10000000` | Accepted source-file count |
12+
| `index_max_source_mb` | `off` | `off` or `1..1048576` | Accepted source-file bytes |
13+
14+
Set or reset them with the normal configuration command:
15+
16+
```bash
17+
codebase-memory-mcp config set index_max_files 250000
18+
codebase-memory-mcp config set index_max_source_mb 16384
19+
codebase-memory-mcp config reset index_max_files
20+
```
21+
22+
Values use base-10 integers. MiB means 1,048,576 bytes. Empty values, zero,
23+
negative values, suffixes, trailing characters, and values outside the stated
24+
ranges are rejected without changing the stored value.
25+
26+
## Counting and failure semantics
27+
28+
`index_max_files` counts a file only after it passes directory pruning, ignore
29+
rules, filename and suffix filters, language detection, and the existing
30+
per-file size rule. `index_max_source_mb` sums the filesystem sizes of that same
31+
accepted set.
32+
33+
Equality is allowed. The first file or byte that makes an observed value greater
34+
than its limit stops discovery. CBM discards the partial file list and does not
35+
publish a partial graph as a complete index.
36+
37+
For an explicit MCP request the error payload contains:
38+
39+
```json
40+
{
41+
"status": "error",
42+
"code": "resource_limit_exceeded",
43+
"stage": "discovery",
44+
"resource": "files",
45+
"observed": 250001,
46+
"limit": 250000,
47+
"unit": "files",
48+
"retryable": true,
49+
"serving_index_preserved": true,
50+
"message": "Index discovery exceeded index_max_files"
51+
}
52+
```
53+
54+
The previous database remains available because publication occurs only after a
55+
complete discovery and successful staged build. If no previous database exists,
56+
`serving_index_preserved` is false.
57+
58+
## Trust and compatibility
59+
60+
Limits are read from the CLI-managed `_config.db`; they are not MCP request
61+
arguments. A supervised parent replaces any caller-supplied internal policy
62+
before spawning its worker, and the worker rejects a missing or incomplete
63+
parent policy.
64+
65+
These settings do not replace or increase `auto_index_limit`, change the 512 MiB
66+
single-file cap, alter workspace-root authorization, or affect
67+
`cross-repo-intelligence`. With both settings `off`, discovery follows the
68+
existing unbounded path.

‎docs/index.html‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,7 @@
5656
"featureList": [
5757
"Indexes 158 programming languages via vendored tree-sitter grammars",
5858
"Hybrid LSP semantic type resolution for Python, TypeScript/JavaScript, PHP, C#, Go, C/C++, Java, Kotlin, and Rust",
59-
"15 MCP tools for structural search, call-path tracing, targeted coverage checks, and Cypher graph queries",
59+
"17 MCP tools for structural search, call-path tracing, targeted coverage checks, and Cypher graph queries",
6060
"Semantic vector code search via bundled nomic-embed-code embeddings (no API key, fully local)",
6161
"Semantic graph edges (SEMANTICALLY_RELATED) and near-clone detection (SIMILAR_TO, MinHash + LSH)",
6262
"Cross-service linking for HTTP, gRPC, GraphQL, tRPC, and pub/sub channels with confidence scoring",
@@ -739,8 +739,8 @@ <h3>3D graph visualization</h3>
739739
<p>The optional UI variant adds one verified, content-addressed frontend pack and serves an interactive 3D graph at <code>localhost:9749</code> to explore nodes, edges, and clusters visually.</p>
740740
</div>
741741
<div class="feature">
742-
<h3>15 MCP tools</h3>
743-
<p><code>search_graph</code>, <code>trace_path</code>, <code>detect_changes</code>, <code>query_graph</code> (Cypher), <code>get_architecture</code>, <code>get_code_snippet</code>, <code>check_index_coverage</code>, <code>manage_adr</code>, and 7 more.</p>
742+
<h3>17 MCP tools</h3>
743+
<p><code>search_graph</code>, <code>trace_path</code>, <code>detect_changes</code>, <code>query_graph</code> (Cypher), <code>get_architecture</code>, <code>get_code_snippet</code>, <code>check_index_coverage</code>, <code>manage_adr</code>, and 9 more.</p>
744744
</div>
745745
<div class="feature">
746746
<h3>Cypher graph queries</h3>

0 commit comments

Comments
 (0)