Skip to content

Commit f4f9870

Browse files
committed
Merge upstream DeusData/codebase-memory-mcp through 13b7d06 (step 1 of upstream sync)
First of several real merges of upstream into the CLI fork. Earlier "milestone" integrations hand-ported upstream changes without recording upstream ancestry, so GitHub's behind count never moved. This merge brings 97 upstream commits (2026-09-02 .. 2026-09-04) into history. Conflict resolution (28 files): - extract_defs.c, language.c, pass_route_nodes.c, registry.c, cypher.c and their tests: the fork had hand-ported the same upstream fixes (Razor/Blazor @page routes, DeusData#1893 receiver-chain guard, RETURN * dedup, WITH width bound). Took upstream's wording where code was equivalent so later merges conflict less. Kept fork-only tests (wide-pattern cypher refusals, indented @page, await-generic TS calls). - src/mcp/{mcp.c,mcp.h,mcp_internal.h}, tests/test_mcp.c stay deleted. The upstream changes in them were moved to the CLI operation layer: * trace: an unreadable edge confidence is reported as not recorded instead of 0.0 (62b4451) * search_code: Windows suffix-glob prefilter applied while building the scoped file list rather than in the PowerShell pipeline (2f37f61) * read ops: new cbm_store_host_open_query helper opens stores query-only, never creates or rebuilds them, and reports a corrupt store as "project store is corrupt (left untouched); run 'codebase-memory-cli index .' to rebuild it" instead of "project not found" (93d2ee5). Wired into query, schema, architecture, file_outline, coverage, snippet, trace, status, search, changes and source search. adr keeps autorecovery. * daemon: CBM_INDEX_MAX_RESTARTS accepts 0 (no recovery rounds) and warns on unreadable or out-of-range values (6b801ee) MCP-protocol-only pieces (tool annotations, notification handling, MCP stdio smoke script) were dropped. - cli.c: kept CLI binary naming; added upstream's FreeBSD pkg upgrade hint as `pkg upgrade codebase-memory-cli`. - application.c: kept fork session-context API; removed a duplicate application_canonical_directory_exists introduced by the merge. - http_server.c: /api/ui-config returns lang, version and the fork's issues URL. - test_cli.c: kept fork install-plan assertions and added upstream's check that nothing is planned under Qoder/Pi plugins/. - Windows python tests: kept fork CLI drivers; adopted upstream timeout handling. Dropped section_stop_refuses_busy, which needs an attached MCP stdio session the CLI does not have (coverage gap noted). - Dockerfile.msan: upstream retry hardening, fork's LLVM 21 pin kept. - README: fork content kept; added a short shared-graph-artifact section. - vendored-checksums.txt recomputed; sha256sum -c passes.
2 parents f48aa9b + 13b7d06 commit f4f9870

74 files changed

Lines changed: 3133 additions & 762 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/_test.yml‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -324,8 +324,14 @@ jobs:
324324
steps:
325325
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
326326

327+
# Pinned: the unversioned `llvm` formula tracks Homebrew stable, so the
328+
# leg's compiler changed under us when stable moved 22 -> 23 (a new
329+
# -Wunused-but-set-global under -Werror broke the build on whichever
330+
# runner drew the 23 bottle, and passed on the ones that drew 22). A
331+
# test verdict must be a function of the code, not of the image snapshot;
332+
# moving to a newer major is a deliberate one-line bump here.
327333
- name: Install LSan-capable toolchain
328-
run: brew install llvm ccache
334+
run: brew install llvm@22 ccache
329335

330336
- name: Compiler cache (content-verified, restore)
331337
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
@@ -340,7 +346,7 @@ jobs:
340346
env:
341347
ASAN_OPTIONS: "detect_leaks=1:halt_on_error=1"
342348
run: |
343-
LLVM_PREFIX="$(brew --prefix llvm)"
349+
LLVM_PREFIX="$(brew --prefix llvm@22)"
344350
scripts/test.sh CC="$LLVM_PREFIX/bin/clang" CXX="$LLVM_PREFIX/bin/clang++"
345351
346352
- name: Compiler cache (save)

‎README.md‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -247,6 +247,18 @@ The upstream indexing engine is intentionally preserved through the CLI-first mi
247247

248248
The first vertical slice does **not** rewrite the graph schema, parser pipeline, store format, or existing indexes.
249249

250+
### Shared graph artifact
251+
252+
`.codebase-memory/graph.db.zst` is an optional zstd-compressed snapshot of the graph that sits next to the source. `codebase-memory-cli index` writes or refreshes it (the watcher refreshes it with a faster, lower-compression tier), and a first `index` on a clone with no local database imports it before running incremental indexing. A `.codebase-memory/.gitattributes` line with `merge=ours` is created on first export so concurrent edits do not conflict on the binary file.
253+
254+
- **Commit it deliberately.** The artifact is rewritten on every index, including the watcher's fast tier, and git stores each rewrite as a full new blob. Committing every refresh turns a 20 MB file into gigabytes of history. Pick a cadence (a release, a milestone, a nightly job) rather than committing every save.
255+
- **Git LFS, if it must move on every commit.** Track it from the repo-root `.gitattributes` and leave the auto-created `.codebase-memory/.gitattributes` in place; the nearer file keeps supplying `merge=ours`, and only `filter` comes from the root:
256+
```gitattributes
257+
.codebase-memory/graph.db.zst filter=lfs diff=lfs merge=lfs -text
258+
```
259+
Track only the `.zst`; `artifact.json` is small and carries the schema version. The attribute applies to future commits only, so a repo that already has the blobs in history needs `git-filter-repo` first. GitHub meters LFS storage and bandwidth, and every teammate needs `git lfs install`; without it the checkout leaves a pointer file, the integrity-checked import refuses it, and they fall back to a full reindex.
260+
- **Optional.** Add `.codebase-memory/` to `.gitignore` if everyone should reindex from scratch.
261+
250262
## Configuration
251263

252264
Common commands:

‎graph-ui/src/App.test.tsx‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
/* @vitest-environment jsdom */
2+
import "@testing-library/jest-dom/vitest";
3+
import { cleanup, render, screen, waitFor } from "@testing-library/react";
4+
import { afterEach, describe, expect, it, vi } from "vitest";
5+
import { App } from "./App";
6+
import { messages } from "./lib/i18n";
7+
8+
vi.mock("./components/GraphTab", () => ({ GraphTab: () => null }));
9+
vi.mock("./components/StatsTab", () => ({ StatsTab: () => null }));
10+
vi.mock("./components/ControlTab", () => ({ ControlTab: () => null }));
11+
vi.mock("./lib/i18n", async (importOriginal) => {
12+
const actual = await importOriginal<typeof import("./lib/i18n")>();
13+
return { ...actual, useUiMessages: () => messages.en };
14+
});
15+
16+
describe("App", () => {
17+
afterEach(() => {
18+
cleanup();
19+
vi.unstubAllGlobals();
20+
window.history.replaceState(null, "", "/");
21+
});
22+
23+
it("shows the serving binary version", async () => {
24+
vi.stubGlobal("fetch", vi.fn(async () =>
25+
new Response(JSON.stringify({ lang: "en", version: "0.10.8" }), {
26+
status: 200,
27+
headers: { "Content-Type": "application/json" },
28+
}),
29+
));
30+
31+
render(<App />);
32+
33+
expect(await screen.findByText("v0.10.8")).toBeVisible();
34+
});
35+
36+
it("hides the version when the config has no string version", async () => {
37+
const fetchMock = vi.fn(async () =>
38+
new Response(JSON.stringify({ lang: "en", version: 108 }), { status: 200 }),
39+
);
40+
vi.stubGlobal("fetch", fetchMock);
41+
42+
render(<App />);
43+
44+
await waitFor(() => expect(fetchMock).toHaveBeenCalledWith("/api/ui-config"));
45+
expect(screen.queryByTitle("Server version")).not.toBeInTheDocument();
46+
});
47+
48+
it("hides the version when the config request fails", async () => {
49+
const fetchMock = vi.fn(async () => {
50+
throw new Error("offline");
51+
});
52+
vi.stubGlobal("fetch", fetchMock);
53+
54+
render(<App />);
55+
56+
await waitFor(() => expect(fetchMock).toHaveBeenCalledWith("/api/ui-config"));
57+
expect(screen.queryByTitle("Server version")).not.toBeInTheDocument();
58+
});
59+
});

‎graph-ui/src/App.tsx‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,8 +33,24 @@ function routeUrl(tab: TabId, project: string | null): string {
3333
export function App() {
3434
const t = useUiMessages();
3535
const [route, setRoute] = useState<RouteState>(readRoute);
36+
const [version, setVersion] = useState<string | null>(null);
3637
const { tab: activeTab, project: selectedProject } = route;
3738

39+
useEffect(() => {
40+
let cancelled = false;
41+
void fetch("/api/ui-config")
42+
.then((response) => (response.ok ? response.json() : null))
43+
.then((config) => {
44+
if (!cancelled && typeof config?.version === "string" && config.version) {
45+
setVersion(config.version);
46+
}
47+
})
48+
.catch(() => {});
49+
return () => {
50+
cancelled = true;
51+
};
52+
}, []);
53+
3854
/* Normalize the URL on first load so it always carries the current route. */
3955
useEffect(() => {
4056
const initial = readRoute();
@@ -73,6 +89,14 @@ export function App() {
7389
<span className="text-[13px] font-semibold text-foreground/90 tracking-tight">
7490
Codebase Memory
7591
</span>
92+
{version && (
93+
<span
94+
className="translate-y-px text-[10px] font-mono text-foreground/30"
95+
title="Server version"
96+
>
97+
{version.startsWith("v") ? version : `v${version}`}
98+
</span>
99+
)}
76100
</div>
77101

78102
{/* Tabs inline in header */}

‎internal/cbm/cbm.c‎

Lines changed: 55 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,7 @@
1+
/* Full declaration set for the same CBMArena, and it must precede cbm.h:
2+
* internal/cbm/arena.h declares a subset and the two share the CBM_ARENA_H
3+
* guard, so whichever is included first is the one this file sees. */
4+
#include "foundation/arena.h" // cbm_arena_init_sized
15
#include "cbm.h"
26
#include "arena.h" // CBMArena, cbm_arena_init/alloc/strdup/destroy
37
#include "helpers.h"
@@ -871,9 +875,13 @@ static void cbm_error_regions_push(cbm_error_regions_t *acc, TSNode n) {
871875
* untouched.
872876
*
873877
* #1746: the Dockerfile grammar places that zero-width missing newline before
874-
* trailing horizontal whitespace rather than at raw EOF. Preserve the broad
875-
* exact-EOF rule above; only extend it past spaces/tabs when the missing token
876-
* is specifically a newline. */
878+
* trailing whitespace rather than at raw EOF. Preserve the broad exact-EOF
879+
* rule above; only extend it past blanks when the missing token is specifically
880+
* a newline. */
881+
static bool cbm_is_blank_not_newline(char c) {
882+
return c == ' ' || c == '\t' || c == '\v' || c == '\f' || c == '\r';
883+
}
884+
877885
static bool cbm_is_eof_terminator_miss(TSNode n, const char *source, int source_len) {
878886
if (!ts_node_is_missing(n) || source_len < 0) {
879887
return false;
@@ -890,7 +898,7 @@ static bool cbm_is_eof_terminator_miss(TSNode n, const char *source, int source_
890898
return false;
891899
}
892900
for (uint32_t i = end; i < (uint32_t)source_len; i++) {
893-
if (source[i] != ' ' && source[i] != '\t') {
901+
if (!cbm_is_blank_not_newline(source[i])) {
894902
return false;
895903
}
896904
}
@@ -1323,11 +1331,27 @@ CBMFileResult *cbm_extract_file(const char *source, int source_len, CBMLanguage
13231331
return r;
13241332
}
13251333

1326-
CBMFileResult *cbm_extract_file_ex(const char *source, int source_len, CBMLanguage language,
1327-
const char *project, const char *rel_path,
1328-
int64_t timeout_micros, const char **extra_defines,
1329-
const char **include_paths, const CBMMacroTable *macro_table,
1330-
const CBMReturnTypeTable *return_type_table) {
1334+
/* Initial block for the per-file traversal scratch arena, chosen by measuring
1335+
* arena_grow on a 14k-file TypeScript tree: it fires on one file in 12,000 at
1336+
* both this size and at 1 MB, and on most files at 256 KB, where the two
1337+
* channel walks alone are exactly 262144 bytes. 512 KB therefore buys the same
1338+
* growth behaviour as 1 MB for half the resident block per worker. It is also
1339+
* exactly MI_LARGE_MAX_OBJ_SIZE in the vendored mimalloc
1340+
* (vendored/mimalloc/include/mimalloc/types.h:426, MI_LARGE_PAGE_SIZE/8 with
1341+
* MI_ENABLE_LARGE_PAGES defaulting to 1 at :115 and not overridden here), so
1342+
* the block is still bin-allocated from a large page. Growth is not free at
1343+
* this size for the same reason: arena_grow doubles to 1 MiB, which is above
1344+
* that bound and so a singleton OS allocation. One file in twelve thousand
1345+
* pays it, which is why the cost is accepted. */
1346+
enum { CBM_EXTRACT_SCRATCH_BLOCK = CBM_SZ_512 * CBM_SZ_1K };
1347+
1348+
static CBMFileResult *extract_file_ex_body(const char *source, int source_len, CBMLanguage language,
1349+
const char *project, const char *rel_path,
1350+
int64_t timeout_micros, const char **extra_defines,
1351+
const char **include_paths,
1352+
const CBMMacroTable *macro_table,
1353+
const CBMReturnTypeTable *return_type_table,
1354+
CBMArena *scratch) {
13311355
// Allocate result on heap (arena inside for all string data)
13321356
enum { SINGLE = 1 };
13331357
CBMFileResult *result = (CBMFileResult *)calloc(SINGLE, sizeof(CBMFileResult));
@@ -1446,6 +1470,7 @@ CBMFileResult *cbm_extract_file_ex(const char *source, int source_len, CBMLangua
14461470
// Build extraction context
14471471
CBMExtractCtx ctx = {
14481472
.arena = a,
1473+
.scratch = scratch,
14491474
.result = result,
14501475
.source = source,
14511476
.source_len = source_len,
@@ -1574,6 +1599,7 @@ CBMFileResult *cbm_extract_file_ex(const char *source, int source_len, CBMLangua
15741599
// Build context for expanded source — extract only calls via unified extractor
15751600
CBMExtractCtx pp_ctx = {
15761601
.arena = a,
1602+
.scratch = scratch,
15771603
.result = result,
15781604
.source = expanded,
15791605
.source_len = expanded_len,
@@ -1810,6 +1836,26 @@ CBMFileResult *cbm_extract_file_ex(const char *source, int source_len, CBMLangua
18101836
return result;
18111837
}
18121838

1839+
/* Public entry. Owns the traversal scratch arena for the whole of one file's
1840+
* extraction: created here, handed to the body as ctx->scratch, destroyed on
1841+
* the way out. The body has seven early returns, so bracketing it in a wrapper
1842+
* is what keeps that to one create and one destroy. If the arena cannot be
1843+
* created, the body is handed NULL and the traversal stacks fall back to the
1844+
* result arena, which is what shipped before #1997. */
1845+
CBMFileResult *cbm_extract_file_ex(const char *source, int source_len, CBMLanguage language,
1846+
const char *project, const char *rel_path,
1847+
int64_t timeout_micros, const char **extra_defines,
1848+
const char **include_paths, const CBMMacroTable *macro_table,
1849+
const CBMReturnTypeTable *return_type_table) {
1850+
CBMArena scratch;
1851+
cbm_arena_init_sized(&scratch, CBM_EXTRACT_SCRATCH_BLOCK);
1852+
CBMFileResult *result = extract_file_ex_body(
1853+
source, source_len, language, project, rel_path, timeout_micros, extra_defines,
1854+
include_paths, macro_table, return_type_table, scratch.nblocks > 0 ? &scratch : NULL);
1855+
cbm_arena_destroy(&scratch);
1856+
return result;
1857+
}
1858+
18131859
void cbm_free_result(CBMFileResult *result) {
18141860
if (!result) {
18151861
return;

‎internal/cbm/cbm.h‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -269,6 +269,10 @@ typedef struct {
269269
// pass_lsp_cross.c. Default false.
270270
bool requires_lsp_resolution; // synthetic semantic candidate (for example an implicit
271271
// C++ operator). Never fall back to textual resolution.
272+
bool callee_is_locally_bound; // bare call foo() whose callee identifier is bound as a
273+
// parameter of an enclosing function, so it cannot be the
274+
// module-level foo. Python only today. Read by the
275+
// weak-local-binding guard. Default false.
272276
} CBMCall;
273277

274278
typedef struct {
@@ -589,6 +593,13 @@ typedef struct {
589593

590594
typedef struct {
591595
CBMArena *arena;
596+
/* Scratch for AST traversal, owned by the cbm_extract_file_ex call that
597+
* built this context and destroyed when it returns. Nothing a
598+
* CBMFileResult points at may be allocated here: `arena` is the result's
599+
* own, and it outlives extraction by the whole pipeline (#1997). NULL in a
600+
* context built without one, in which case the stacks fall back to
601+
* `arena`. */
602+
CBMArena *scratch;
592603
CBMFileResult *result;
593604
const char *source;
594605
int source_len;

0 commit comments

Comments
 (0)