[futile]

What does this PR do?

Fix a Linux startup segfault exposed by glibc 2.44 by updating vendored mimalloc from 3.3.2 to 3.4.4 (upstream commit 1f06f694972279bbc7ec72902e8570f5784d0fc9).

During libstdc++ initialization, glibc's newlocale() can call the globally interposed free(NULL) before mimalloc's constructor runs. The old release allocator dereferences its uninitialized page map, leading to this SIGSEGV immediately and reproducibly when the cli binary starts:

$ codebase-memory-mcp cli --progress index_status --project home-felix-gits-pale-darks-online
fish: Job 1, 'codebase-memory-mcp cli --progr…' terminated by signal SIGSEGV (Address boundary error)

mimalloc 3.4.4, the version this PR updates to, includes the upstream fix:

• mimalloc 3.x segfaults on free(NULL) called before mimalloc is initialized microsoft/mimalloc#1341: mimalloc 3.x segfaults on free(NULL) called before mimalloc is initialized microsoft/mimalloc#1341
• Upstream fix: microsoft/mimalloc@b2226c1
• Matching CBM report that should be resolved by this PR: fix(cs-lsp): bind same-name C# types by declared namespace (#2120) #2347 (comment)

Changes

• Refresh vendored include/, src/, and LICENSE from the upstream release. The existing reproducible-banner patch is preserved; one macOS comment is reworded to avoid a false positive in the vendored security scanner. Both local differences and the immutable source pin are documented in vendored/mimalloc/PATCHES.md.
• Update the SBOM version, third-party documentation, and vendored integrity manifest.
• Add a Linux ELF-preinit regression test linked against prod_mimalloc.o with production allocator interposition. It calls free(NULL) before allocator initialization and then checks normal allocation/free in main. Wire it into the production checks in scripts/test.sh.

The normal sanitizer runner does not enable global allocator interposition, so it cannot cover this startup regression. The new fixture fails with SIGSEGV (exit 139) when linked to the original mimalloc 3.3.2 production object, and passes with 3.4.4.

Validation

Tested locally on x86_64 Linux with GCC 16.2.0 and glibc 2.44:

• PASS: scripts/build.sh; production binary --version.
• PASS: bash tests/test_mimalloc_startup.sh build/c, with the failing old-allocator comparison described above.
• FAIL (exit 1): all listed C suites through the supported iteration entry point: scripts/test.sh --suites "$(build/tests/test-runner --list-suites)" BUILD_DIR=build/tests (ASan + UBSan build). The log contains 832 assertion-failure lines, many from probe fixtures reporting no graph DB, recovering UBSan diagnostics, and a LeakSanitizer report of 13,399 bytes in 11 allocations. These additional failures have not been compared against clean upstream main; they are not claimed to be baseline failures or unrelated to this change.
• PASS: scripts/smoke-local.sh build/c/codebase-memory-mcp, including indexing, MCP queries, install/uninstall, and daemon cleanup. The UI-only leg was skipped because the frontend was not built.
• PASS: the reported cli --progress index_status --project ... form, against a freshly indexed private fixture rather than an existing user index.
• PASS: scripts/security-audit.sh and scripts/security-vendored.sh. The broader make -f Makefile.cbm security target also passed source, binary-string, source-only UI, sandbox-install, and strace network checks, but stopped on a 10-second MCP fuzz timeout (shell injection semicolon). That exact input passed three isolated retries; the subsequent full scripts/security-fuzz.sh retry passed 32/32. The combined security target itself is not claimed as passing, and built-frontend integrity was not verified.

Existing baseline failures

The following checks fail on this branch and were reproduced on a clean checkout of upstream main bf93f0b7c3b82683b3ac3b00bb0342465878cdf8, with the same local tool versions:

• bash tests/test_version_metadata_contract.sh: the Scoop manifest pin is 0.11.0, equal to the latest stable tag; the contract requires changing that surface's classification from pin:0.11.0 to release and dropping its PIN_REASONS entry. This stops the default scripts/test.sh before the C suites, hence the separate all-suite invocation above.
• Cppcheck 2.22: src/mcp/mcp.c:962 reports a possible null dereference of allowed. Reproduced independently with make -f Makefile.cbm lint-cppcheck LINT_SRCS=src/mcp/mcp.c. The branch's full lint gate also fails; formatting, NOLINT, memory-core checks, and applicable diff-scoped clang-tidy passed.

The full scripts/lint.sh audit additionally encounters GCC-only warning flags passed to Clang and pre-existing findings in unchanged files. This is not a claim that the full unpatched test/lint suite or GitHub CI was run. Baseline failures were not changed or suppressed in this focused allocator fix. The local pre-commit gate was bypassed because these seemed potentially unrelated to me (the human), and I wanted to see what PR CI says/what discussion says.

AI disclosure

This contribution and description were prepared with an OpenAI coding agent in Pi on behalf of Felix Rath (futile). Felix is the accountable human contributor and explicitly authorized the DCO sign-off on every commit. No private agent-session links are included.

Checklist

• Every commit is signed off (git commit -s); scripts/check-dco.sh upstream/main..HEAD passed for both commits.
• Full default tests pass locally — blocked by the unchanged baseline version contract; focused regression and smoke checks pass, but the separate all-suite run also fails as detailed above.
• Lint passes — unchanged baseline failure described above.
• New behavior is covered by a reproduce-first regression test.

[github-actions]

Thanks for opening this — it has been seen, and it is queued.

This note is automated, but it is not a brush-off: it exists so you know where your PR stands instead of having to guess from silence.

Current review status: working through a backlog. 0.9.1-rc.1 is out, so the release freeze that held reviews is over — but it left a large queue of open pull requests behind it, and we are reading through them oldest-first. The background is in discussion #1144.

What that means for this PR, concretely:

• It will not be closed for inactivity. No stale bot touches pull requests here.
• It may still sit a while before a human reads it. That is on us, not on you.
• Older PRs are read first, so a recent one is not being skipped — it is behind a queue.

Things that will genuinely speed it up whenever review does happen:

• Keep it rebased on main — the tree is moving quickly right now, and a conflicting branch cannot be reviewed as the diff you intended.
• Get CI green, or say which failures you believe are pre-existing.
• Keep the change to one claim. Bundled features and refactors get split before they get merged, which costs you a round trip.
• Every commit needs a sign-off (git commit -s) — CI enforces DCO.

If this fixes a bug, a reproduction we can run is worth more than a description of the symptom.

Thanks for contributing, and sorry in advance for the wait.

[DeusData]

Thanks for the allocator startup reproduction and for separating the pre-init regression from the broader failing checks. I have read the update and its validation limits. The vendored mimalloc update remains pending review; the un-compared full-suite failures are not being treated as baseline-green evidence.

[DeusData]

Thank you, @futile -- this is an excellent report and an even better fix. Your analysis is spot on: with a recent glibc, newlocale() can call free(NULL) before mimalloc's constructor runs, and our current release allocator then reads a page map that does not exist yet. We reproduced it with our production allocator flags and a preinit free(NULL), exactly as your test does, and confirmed that the upstream fix (microsoft/mimalloc@b2226c1) makes it go away. We also checked your vendored tree against the upstream v3.4.4 tag -- it matches exactly apart from the two documented local edits, which made the review much easier.

One thing shapes how we land it: we vendor third-party code ourselves, and main now carries a second local mimalloc patch (a macOS memory-footprint change in src/prim/unix/prim.c, which is why this branch conflicts). We are deciding between a small backport of the upstream fix and the full 3.4.4 refresh, and will come back here shortly with the route. Either way your diagnosis and your preinit regression test are the basis, with credit to you. Thank you for the careful write-up and the reproduce-first test.

[DeusData]

Thank you again, @futile. As promised, here is the route we took.

We went with a full refresh rather than a backport, and since we vendor third-party code ourselves, it lands as a maintainer PR: #2731. It vendors mimalloc v3.5.3, upstream's latest published release (newer than the 3.4.4 in this branch, and it contains microsoft/mimalloc@b2226c1, the fix you identified), and keeps the two local changes main still needs: the macOS decommit change in src/prim/unix/prim.c and your comment rewording in the zone file, which our vendored-code gate still requires. The reproducible-banner edit is no longer needed, because upstream dropped the build clock in v3.4.5. Each remaining local change is now documented in vendored/mimalloc/PATCHES.md and pinned by a check that fails without it.

Your preinit test is the heart of the new contract test: tests/test_prod_allocator_contract.sh links a free(NULL)-from-.preinit_array fixture against the production allocator object, exactly as your test does, and runs on every Linux and macOS leg. On Linux it fails on current main (segfault, every run, with gcc and clang) and passes on the new branch. You are credited as co-author on both commits.

We will close this PR once #2731 is merged. Thank you for the diagnosis, the upstream pointers and the reproduce-first test -- they made this a well-understood fix.

[futile]

Thank you very much for the detailed updates, the credit on the commits, and for the nice, improved solution! Feel free to close as you like :)