Skip to content

Commit 4f82a1e

Browse files
authored
Merge branch 'main' into fix/svelte-html-astro-embedded-structure
2 parents 24f54c0 + fe85a6b commit 4f82a1e

86 files changed

Lines changed: 19636 additions & 4491 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/_test.yml‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -311,8 +311,14 @@ jobs:
311311
steps:
312312
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
313313

314+
# Pinned: the unversioned `llvm` formula tracks Homebrew stable, so the
315+
# leg's compiler changed under us when stable moved 22 -> 23 (a new
316+
# -Wunused-but-set-global under -Werror broke the build on whichever
317+
# runner drew the 23 bottle, and passed on the ones that drew 22). A
318+
# test verdict must be a function of the code, not of the image snapshot;
319+
# moving to a newer major is a deliberate one-line bump here.
314320
- name: Install LSan-capable toolchain
315-
run: brew install llvm ccache
321+
run: brew install llvm@22 ccache
316322

317323
- name: Compiler cache (content-verified, restore)
318324
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
@@ -327,7 +333,7 @@ jobs:
327333
env:
328334
ASAN_OPTIONS: "detect_leaks=1:halt_on_error=1"
329335
run: |
330-
LLVM_PREFIX="$(brew --prefix llvm)"
336+
LLVM_PREFIX="$(brew --prefix llvm@22)"
331337
scripts/test.sh CC="$LLVM_PREFIX/bin/clang" CXX="$LLVM_PREFIX/bin/clang++"
332338
333339
- name: Compiler cache (save)

‎.github/workflows/pages.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,4 +41,4 @@ jobs:
4141
with:
4242
path: ./docs
4343
- id: deployment
44-
uses: actions/deploy-pages@cd2ce8fcbc39b97be8ca5fce6e763baed58fa128 # v5.0.0
44+
uses: actions/deploy-pages@368f82528645a54fb793d4d04e342629a3f51346 # v5.0.1

‎.github/workflows/release.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -259,7 +259,7 @@ jobs:
259259
git tag -f "$VERSION" "$GITHUB_SHA"
260260
git push origin "$VERSION" --force
261261
262-
- uses: softprops/action-gh-release@3d0d9888cb7fd7b750713d6e236d1fcb99157228 # v2
262+
- uses: softprops/action-gh-release@efb35369e0ad2afab669f228072c1b0d510eae64 # v3.0.3
263263
with:
264264
tag_name: ${{ inputs.version }}
265265
draft: true

‎Makefile.cbm‎

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -181,10 +181,35 @@ endif
181181
# ~259 MB of tree-sitter parse tables was mapped executable at runtime while the
182182
# amd64 build of the same source mapped them R only. That is a large ROP gadget
183183
# surface for data that is never executed, and it is invisible to section flags.
184+
#
185+
# -z relro + -z now finish the same job on the OTHER writable-but-shouldn't-be
186+
# region: the GOT. Both are free at runtime here (a static binary resolves
187+
# everything at link time, so there is no lazy binding left to pay for), and
188+
# they are added because the SHIPPED artifact measurably lacked the property --
189+
# not because the linker accepts the flags. Measured on this binary, built the
190+
# way .github/workflows/_build.yml builds the Linux release (STATIC=1, gcc 13.3
191+
# / GNU ld 2.42, Ubuntu 24.04 aarch64); PT_GNU_RELRO ends at 0x11DA0000 in both:
192+
#
193+
# without -z now: .got 0x11d9ec20+0x13c8, .got.plt 0x11d9ffe8+0x40
194+
# -> .got.plt ends at 0x11DA0028, i.e. 40 bytes PAST the
195+
# window: those GOT slots stayed writable for the whole
196+
# process lifetime, in every Linux binary we have shipped
197+
# with -z now: .got.plt folded into .got (0x11d9ec20+0x13c8, ends
198+
# 0x11D9FFE8) -> the whole GOT is inside the window and is
199+
# re-mapped read-only after startup
200+
#
201+
# -z relro is already this toolchain's default and changes nothing here; it is
202+
# named anyway so the property stops depending on one distro's spec file (the
203+
# musl/portable and glibc-floor images are different toolchains, and a default
204+
# is not a guarantee). A1c/A1d in scripts/ci/check-binary-composition.sh assert
205+
# the OUTCOME on the produced binary, because a flag the compiler accepts is
206+
# not evidence that the artifact gained anything -- the same reason A1 exists
207+
# next to the .note.GNU-stack annotation.
184208
ELF_HARDENING_FLAGS :=
185209
ifeq ($(IS_LINUX),yes)
186210
ifneq ($(IS_MINGW),yes)
187-
ELF_HARDENING_FLAGS := -Wl,-z,noexecstack -Wl,-z,separate-code
211+
ELF_HARDENING_FLAGS := -Wl,-z,noexecstack -Wl,-z,separate-code \
212+
-Wl,-z,relro -Wl,-z,now
188213
endif
189214
endif
190215

‎README.md‎

Lines changed: 20 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -250,7 +250,13 @@ Commit a single compressed file to your repo and your teammates skip the reindex
250250
- **Best** (`zstd -9` + index strip + `VACUUM INTO`) — written on explicit `index_repository`
251251
- **Fast** (`zstd -3`) — written by the watcher for low-latency incremental updates
252252
- **Bootstrap**: when no local DB exists but the artifact is present, `index_repository` imports the artifact first, then runs incremental indexing — avoiding the full reindex cost
253-
- **No merge pain**: a `.gitattributes` line with `merge=ours` is auto-created on first export, so concurrent edits don't produce conflicts on the binary artifact
253+
- **No merge pain**: a `.codebase-memory/.gitattributes` line with `merge=ours` is auto-created on first export, so concurrent edits don't produce conflicts on the binary artifact
254+
- **Commit it deliberately**: the artifact is rewritten on every index, including the watcher's Fast tier, and git stores each rewrite as a full new blob. Committing every refresh is what turns a 20 MB file into gigabytes of history — one team reached ~6 GB across ~350 commits of this single path. Pick a cadence (a release, a milestone, a nightly job) rather than committing every save.
255+
- **Git LFS, if it must move on every commit**: track it from the **repo-root** `.gitattributes` and leave the auto-created `.codebase-memory/.gitattributes` in place — the nearer file goes on supplying `merge=ours`, and only `filter` comes from the root:
256+
```gitattributes
257+
.codebase-memory/graph.db.zst filter=lfs diff=lfs merge=lfs -text
258+
```
259+
Track only the `.zst`; `artifact.json` is small and carries the schema version. The attribute applies to future commits only, so a repo that already has the blobs in history needs `git-filter-repo` to rewrite them first. Two costs to weigh before adopting it: GitHub meters LFS storage and bandwidth, and its objects cannot be pruned without contacting support; and every teammate needs `git lfs install` — without it their checkout leaves a pointer file where the artifact should be, the integrity-checked import refuses it, and they fall back to a full reindex.
254260
- **Optional**: never committed unless you want it. Add `.codebase-memory/` to `.gitignore` if you prefer everyone to reindex from scratch.
255261

256262
The result is similar in spirit to graphify's `graphify-out/` directory, but as a single compressed file with explicit two-tier export, integrity-checked import, and zero merge friction.
@@ -623,7 +629,13 @@ Every MCP tool can be invoked as a local, one-shot command. CLI tools neither st
623629

624630
Commands that mutate graph data use shared OS-backed, per-project locks. This serializes conflicting work from CLI and MCP sessions on the same project while allowing unrelated projects to proceed independently.
625631

626-
When stderr is an interactive terminal, the CLI automatically shows lifecycle and indexing progress. Pass `--progress` to force the same feedback when stderr is redirected or the command is run non-interactively. Progress is written only to stderr; stdout remains reserved for the command result, so pipes and scripts stay machine-safe. Pass `--json` when the full MCP result envelope is needed.
632+
When stderr is an interactive terminal, the CLI automatically shows lifecycle and indexing progress. Pass `--progress` to force the same feedback when stderr is redirected or the command is run non-interactively. Pass `--quiet` to disable automatic terminal progress and ordinary diagnostics while retaining errors; it cannot be combined with `--progress` or outer `cli --verbose`. Routine informational logs are quiet by default; pass outer `cli --verbose` to include them. Progress and logs use stderr while stdout remains reserved for the command result. Read tools return a compact tree by default; pass a tool's `--format json` for machine-readable payload JSON, or outer `--json` for the full MCP envelope.
633+
634+
Large compact-tree tables may start with a response-local `<section>_refs` directory and an explicit `<section>_ref_rule`. A cell such as `@0+handler.go` reconstructs to ref `0`'s prefix plus `handler.go`. References are local to that sibling `<section>` table and expansion is non-recursive: entries inside `<section>_refs` are always literal prefixes. This is limited to declared path and qualified-name columns and activates only when the exact rendered table is at least 15% and 64 bytes smaller and a conservative model-neutral token-shape proxy also improves by at least 1%. Search and trace likewise render direct and prefix-grouped tree shapes and keep the smaller complete representation, so singleton or scattered answers do not pay directory overhead. Keys are declared once per table but never cryptically abbreviated, and `--format json` keeps stable literal strings for machine consumers. Both gates are deterministic; exact token counts still depend on the caller's tokenizer.
635+
636+
Lean responses truncate semantically, never by cutting arbitrary bytes from code or identifiers. Ranked graph rows are retained ahead of raw grep rows and diagnostic summaries; omitted rows/sections report totals, `has_more`, and a strictly advancing continuation offset or cursor. If even the first whole row cannot fit, CBM asks for a higher budget and emits no self-looping cursor. `max_output_tokens` is model-neutral sizing guidance: CBM enforces a deterministic ceiling of four UTF-8 bytes per requested token, so it is not a tokenizer-exact count. Detail flags such as `diagnostics`, `source_mode`, and `detail` opt into heavier fields. `search_code` pages ranked rows with `result_limit`/`result_offset` (`limit` remains a compatibility alias), raw rows with `raw_limit`/`raw_offset`, and directory summaries with `directory_limit`/`directory_offset`. Raw lines default to a UTF-8-safe match-centered preview; each row reports `content_start_byte`, returned/total byte counts, match byte bounds when known, and a content continuation offset. Pass `raw_content_offset` to page the original line without moving the raw-row cursor. `match_limit` and `source_max_lines` bound per-result details, with exact omission metadata. `detect_changes` pages changed files, impacted symbols, and module summaries independently; prefer its snapshot-bound `*_cursor` continuations, which reject changed commits, worktree bytes, graph generation, or semantic arguments instead of silently skipping or duplicating rows.
637+
638+
Every response is standard UTF-8. Identifiers, paths, and raw search previews preserve POSIX byte-string identities: a preserved value containing malformed UTF-8 is emitted reversibly as `@bytes:<lowercase hex of every original byte>`. A valid preserved value that literally begins with the reserved `@bytes:` or `@utf8:` prefix is emitted as `@utf8:<original value>`, so decoding is unambiguous: strip one `@utf8:` prefix for literal UTF-8, or hex-decode one `@bytes:` prefix for original bytes. Ordinary valid UTF-8 is unchanged and pays no output-token overhead. To keep code readable, source bodies replace malformed UTF-8 with U+FFFD; use the pageable raw search preview when byte-exact source inspection is required.
627639

628640
Use `cli <tool> --help` to see the flags generated from that tool's input schema:
629641

@@ -638,7 +650,10 @@ codebase-memory-mcp cli query_graph --project my-project --query 'MATCH (f:Funct
638650

639651
# Force human-readable progress without contaminating stdout.
640652
codebase-memory-mcp cli --progress index_repository --repo-path /path/to/repo
641-
codebase-memory-mcp cli search_graph --project my-project --label Function | jq '.results[].name'
653+
# Suppress automatic terminal progress and non-error diagnostics.
654+
codebase-memory-mcp cli --quiet list_projects --format json
655+
codebase-memory-mcp cli search_graph --project my-project --label Function --format json
656+
codebase-memory-mcp cli list_projects --format json --detail stats | jq '.projects[].name'
642657
```
643658

644659
JSON arguments can also be piped on stdin, for tools that take arguments. A tool whose input schema declares none — `list_projects` — never reads stdin, so it stays responsive when it inherits a pipe the caller never closes (the default for `child_process.spawn` and similar wrappers). Inline JSON remains accepted for backward compatibility but is deprecated in favor of flags, `--args-file`, or stdin.
@@ -658,7 +673,7 @@ JSON arguments can also be piped on stdin, for tools that take arguments. A tool
658673

659674
| Tool | Description |
660675
|------|-------------|
661-
| `search_graph` | Structured search by label, name pattern, file pattern, degree filters. Pagination via limit/offset. |
676+
| `search_graph` | Structural, BM25, and semantic search. Page structural rows with `offset`/`limit` and ranked semantic rows independently with `semantic_offset`/`semantic_limit`. |
662677
| `trace_path` | BFS traversal — who calls a function and what it calls (alias: `trace_call_path`). Depth 1-5. |
663678
| `detect_changes` | Map git diff to affected symbols + blast radius with risk classification. |
664679
| `query_graph` | Execute Cypher-like graph queries (read-only). |
@@ -724,7 +739,7 @@ codebase-memory-mcp config reset auto_index # reset to default
724739
| `CBM_CACHE_DIR` | `~/.cache/codebase-memory-mcp` | Override the database storage directory. All project indexes and config are stored here. One account can use only one canonical cache root at a time; close active CBM sessions/commands before switching it. |
725740
| `CBM_DIAGNOSTICS` | `false` | Set to `1` or `true` to enable the shared daemon's periodic `snapshot.json` and retained `trajectory.ndjson` below a fresh owner-private directory in the system temp directory. Exact paths are logged by `diagnostics.start`. |
726741
| `CBM_DOWNLOAD_URL` | *(GitHub releases)* | Override the download URL for updates. Used for testing or self-hosted deployments. |
727-
| `CBM_LOG_LEVEL` | `info` | Set the minimum log level. Accepted values (case-insensitive): `debug`, `info`, `warn`, `error`, `none` — or their numeric equivalents `0`–`4` matching the internal enum. Thin-frontend messages go to that session's stderr; detached daemon events go to `${CBM_CACHE_DIR}/logs/cbm-daemon.log`. Stdout is reserved for MCP JSON-RPC. |
742+
| `CBM_LOG_LEVEL` | role-aware | Set the minimum log level. Thin MCP/CLI/hook frontends default to `warn`; the detached daemon and its supervised index workers default to `info` so lifecycle and liveness records remain available. Accepted values (case-insensitive): `debug`, `info`, `warn`, `error`, `none` — or their numeric equivalents `0`–`4`. A physical worker retains INFO liveness records even under a stricter override because its private log drives the supervisor's no-progress timeout. Frontend messages go to that session's stderr; detached daemon events go to `${CBM_CACHE_DIR}/logs/cbm-daemon.log`. Stdout is reserved for MCP JSON-RPC. |
728743
| `CBM_WORKERS` | *(detected)* | Override the parallel-indexing worker count returned by `cbm_default_worker_count`. Useful inside containers where `sysconf(_SC_NPROCESSORS_ONLN)` reports host CPUs rather than the cgroup's effective quota. Range 1–256; invalid values are ignored with a warning. |
729744
| `CBM_MEM_BUDGET_MB` | *(detected)* | Override the in-memory graph budget with an explicit cap in MiB, taking precedence over the `ram_fraction × total_RAM` default. Useful on bare-metal hosts without a cgroup limit, or to pin a budget *below* the cgroup limit so headroom is left for sibling processes. Must be a positive integer; it is clamped to detected total RAM (logged as `mem.budget.clamped`), and non-numeric or non-positive values are ignored with a warning (`mem.budget.env.invalid`). |
730745
| `CBM_DUMP_VERIFY_MIN_RATIO` | `0.5` | After indexing, compare persisted SQLite node count to the in-memory dump count. When persisted nodes fall below this fraction of committed nodes (and committed > 50), `index_repository` returns `status:"degraded"` instead of silent `indexed`. Range 0–1; set `0` to disable. Invalid values are ignored with a warning. |

‎docs/CONFIGURATION.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -149,7 +149,7 @@ These environment variables affect runtime behavior:
149149
| `CBM_CACHE_DIR` | `~/.cache/codebase-memory-mcp` | Override the cache directory used for indexes, `_config.db`, and UI `config.json`. |
150150
| `CBM_DIAGNOSTICS` | `false` | Enable periodic `snapshot.json` and retained `trajectory.ndjson` below a fresh owner-private directory in the system temp directory. The daemon records the randomized paths in the `diagnostics.start` discovery record (a single JSON line) in `${CBM_CACHE_DIR}/logs/cbm-daemon.log`; that one record is emitted even when `CBM_LOG_LEVEL` suppresses ordinary logging, so the paths always remain discoverable. |
151151
| `CBM_DOWNLOAD_URL` | GitHub releases | Override the update download URL. |
152-
| `CBM_LOG_LEVEL` | `info` | Set the log level to `debug`, `info`, `warn`, `error`, or `none` (or `0`-`4`). Thin-frontend messages use that session's stderr; detached daemon events use `${CBM_CACHE_DIR}/logs/cbm-daemon.log`. |
152+
| `CBM_LOG_LEVEL` | role-aware | Set the log level to `debug`, `info`, `warn`, `error`, or `none` (or `0`-`4`). Thin MCP/CLI/hook frontends default to `warn`; the detached daemon and supervised index workers default to `info`. Physical workers retain INFO liveness records because their private logs drive the supervisor's no-progress timeout. Frontend messages use that session's stderr; detached daemon events use `${CBM_CACHE_DIR}/logs/cbm-daemon.log`. |
153153
| `CBM_RUNTIME_DIR` | `%LOCALAPPDATA%` (Windows), `/private/tmp` (macOS), `/tmp` (other) | Parent directory for the daemon/CLI rendezvous directory, which CBM creates inside it as `cbm-daemon-<uid>` (`cbm-daemon-<key>` on Windows). Set it when the default ancestry cannot pass the private-directory check — see below. `CBM_CACHE_DIR` does **not** move the rendezvous. |
154154
| `CBM_WORKERS` | auto-detected | Override the indexing worker count. |
155155

0 commit comments

Comments
 (0)