Skip to content

Commit f9f44b5

Browse files
DeusDatafutile
andcommitted
fix(mem): vendor mimalloc 3.5.3 so free(NULL) before init is safe
With mimalloc 3.3.2's release build, free(NULL) before the allocator's constructor dereferenced a page map that does not exist yet. glibc 2.44's newlocale() does exactly that during libstdc++ start-up, so every Linux binary died with SIGSEGV before main (reported with a reproduction in #2562). Upstream fixed it in b2226c19 (mimalloc #1341), first released in v3.4.4; v3.5.3, upstream's latest published release, contains it. Vendored from microsoft/mimalloc tag v3.5.3 (tag object deac3dcf, commit d4881d338125e1cb7c47ba4cfb398d6f7c0c8d45): LICENSE, include/ and src/, written blob by blob from a bare clone. Before the local patches were reapplied, the staged LICENSE blob and include/ and src/ trees had the tag's exact object ids (50 files; src/arena-meta.c is gone upstream). Afterwards the only differences are two documented hunks: - src/prim/unix/prim.c: macOS always recommits on decommit, carried from 63fb7b3 (the phys_footprint fix). - src/prim/osx/alloc-override-zone.c: one comment reworded, because scripts/security-vendored.sh rejects the upstream "fork()" spelling even inside a comment. No code change. The reproducible-banner patch (no __DATE__/__TIME__) is dropped: upstream removed the build clock in v3.4.5 (mimalloc #1355). The two Makefile comments that pointed at that patch now say so. The production object is built with -DMI_PROFILE=0: mimalloc 3.5.2+ compiles its allocation-sampling profiler in by default, with a sample countdown on the allocation path, and the product never uses it. vendored/mimalloc/PATCHES.md records the pin, a re-verification recipe, the check that pins each patch and the refresh procedure; THIRD_PARTY.md points at it, the SBOM version is 3.5.3 and scripts/vendored-checksums.txt is regenerated (mimalloc lines only; security-vendored.sh passes). Measured on Apple M5 Pro (macOS 26.6), full indexes, 2 interleaved reps, main fceb25d (3.3.2) -> this change: TypeScript peak phys_footprint 1.07/1.01 -> 1.04/1.04 GB, peak RSS 1.87/1.84 -> 1.85/1.86 GB, wall 30.6/29.1 -> 29.3/30.0 s; django footprint 0.41/0.41 -> 0.38/0.39 GB, RSS 0.59/0.59 -> 0.58/0.60 GB, wall 6.5/6.2 -> 6.5/6.5 s. The graphs are identical: full node and edge hashes, node and edge census, no dangling edges (django 68022 nodes / 322464 edges, TypeScript 306239 / 835984). The same tree without the recommit patch peaks at 2.09/2.04 GB footprint on TypeScript, so 63fb7b3 still does its job. Linux arm64 (glibc 2.39, gcc 13.3): both trees build without warnings; django peak RSS 0.67/0.68 GB on main, 0.69/0.68 GB with this change. Co-authored-by: Felix Rath <felixm.rath@gmail.com> Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
1 parent d5bb016 commit f9f44b5

49 files changed

Lines changed: 6707 additions & 4319 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎Makefile.cbm‎

Lines changed: 11 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -45,8 +45,9 @@ endif
4545

4646
# -Wdate-time: with -Werror, any use of __DATE__/__TIME__/__TIMESTAMP__ fails the
4747
# build. Build-time entropy is how one identical source tree produced a new hash
48-
# on every compile (see the local patch in vendored/mimalloc/src/options.c) —
49-
# once removed, it must not creep back through a new call site.
48+
# on every compile (mimalloc's version banner, patched locally until upstream
49+
# dropped the clock in v3.4.5; see vendored/mimalloc/PATCHES.md) — once
50+
# removed, it must not creep back through a new call site.
5051
CFLAGS_COMMON = -std=c11 -D_DEFAULT_SOURCE -D_GNU_SOURCE -Wall -Wextra -Werror \
5152
-Wno-unused-parameter -Wno-sign-compare -Wdate-time \
5253
$(GCC_ONLY_FLAGS) \
@@ -182,8 +183,9 @@ endif
182183
ifeq ($(IS_MINGW),yes)
183184
# --no-insert-timestamp: the PE header otherwise carries the link wall clock, so
184185
# two Windows builds of identical source are never byte-identical and every
185-
# release is a brand-new file to a reputation system. Same motivation as the
186-
# mimalloc __DATE__ patch; ELF and Mach-O have no equivalent field to clear.
186+
# release is a brand-new file to a reputation system. Same motivation as
187+
# dropping mimalloc's __DATE__ banner; ELF and Mach-O have no equivalent field
188+
# to clear.
187189
WIN32_LIBS := -lws2_32 -lpsapi -lshell32 -ladvapi32 -lbcrypt -Wl,--allow-multiple-definition -Wl,--stack,8388608 -Wl,--no-insert-timestamp -static
188190
endif
189191

@@ -653,10 +655,14 @@ endif
653655
# work inside the allocator (2026-09-17). Committed-bytes accounting, which the
654656
# memory budget reads, is kept in every build (subproc stats, not MI_STAT).
655657
# The TEST build below keeps mimalloc's debug checks.
658+
# MI_PROFILE=0: mimalloc 3.5.2+ compiles its allocation-sampling profiler in by
659+
# default, which puts a sample countdown on the allocation path. The product
660+
# never uses it, so the production allocator leaves it out (the contract in
661+
# tests/test_prod_allocator_contract.sh checks the banner has no profiling line).
656662
MIMALLOC_CFLAGS = -std=c11 -O2 -w \
657663
-Ivendored/mimalloc/include \
658664
-Ivendored/mimalloc/src \
659-
-DMI_OVERRIDE=1 -DMI_DEBUG=0 \
665+
-DMI_OVERRIDE=1 -DMI_DEBUG=0 -DMI_PROFILE=0 \
660666
$(MIMALLOC_OVERRIDE_DEFINE)
661667
MIMALLOC_CFLAGS_TEST = -std=c11 -g -O1 -w \
662668
-Ivendored/mimalloc/include \

‎THIRD_PARTY.md‎

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -121,10 +121,12 @@ License summary:
121121
| wyhash | `internal/cbm/vendored/wyhash/` | Unlicense (public domain) | [wangyi-fudan/wyhash](https://github.com/wangyi-fudan/wyhash) |
122122

123123
Local modifications to these libraries are documented next to the
124-
vendored sources (currently only SQLite: `vendored/sqlite3/PATCHES.md`,
125-
raising the Unix VFS `MAX_PATHNAME` ceiling from 512 to 4096 to match
126-
CBM's 4 KiB path support). Patches must be reapplied on every upstream
127-
refresh and are covered by `scripts/vendored-checksums.txt`.
124+
vendored sources: `vendored/sqlite3/PATCHES.md` (raising the Unix VFS
125+
`MAX_PATHNAME` ceiling from 512 to 4096 to match CBM's 4 KiB path support)
126+
and `vendored/mimalloc/PATCHES.md` (the upstream source pin, mimalloc v3.5.3
127+
at commit `d4881d33`, plus the macOS decommit-recommit change and one comment
128+
rewording). Patches must be reapplied on every upstream refresh and are
129+
covered by `scripts/vendored-checksums.txt`.
128130

129131
The graph-UI HTTP server is a first-party implementation
130132
(`src/ui/httpd.c` + `src/ui/http_server.c`) — no third-party HTTP library

‎scripts/ci/generate-sbom.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@
4040
"packages": [
4141
{"SPDXID": "SPDXRef-Package-sqlite3", "name": "sqlite3", "versionInfo": "3.51.3", "licenseDeclared": "blessing", "downloadLocation": "https://sqlite.org", "filesAnalyzed": False},
4242
{"SPDXID": "SPDXRef-Package-yyjson", "name": "yyjson", "versionInfo": "0.12.0", "licenseDeclared": "MIT", "downloadLocation": "https://github.com/ibireme/yyjson", "filesAnalyzed": False},
43-
{"SPDXID": "SPDXRef-Package-mimalloc", "name": "mimalloc", "versionInfo": "3.3.2", "licenseDeclared": "MIT", "downloadLocation": "https://github.com/microsoft/mimalloc", "filesAnalyzed": False},
43+
{"SPDXID": "SPDXRef-Package-mimalloc", "name": "mimalloc", "versionInfo": "3.5.3", "licenseDeclared": "MIT", "downloadLocation": "https://github.com/microsoft/mimalloc", "filesAnalyzed": False},
4444
{"SPDXID": "SPDXRef-Package-xxhash", "name": "xxhash", "versionInfo": "0.8.3", "licenseDeclared": "BSD-2-Clause", "downloadLocation": "https://github.com/Cyan4973/xxHash", "filesAnalyzed": False},
4545
{"SPDXID": "SPDXRef-Package-tre", "name": "tre", "versionInfo": "0.8.0", "licenseDeclared": "BSD-2-Clause", "downloadLocation": "https://github.com/laurikari/tre", "filesAnalyzed": False},
4646
{"SPDXID": "SPDXRef-Package-tree-sitter", "name": "tree-sitter", "versionInfo": "0.24.4", "licenseDeclared": "MIT", "downloadLocation": "https://github.com/tree-sitter/tree-sitter", "filesAnalyzed": False},

‎scripts/vendored-checksums.txt‎

Lines changed: 44 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -992,50 +992,56 @@ a86b7735823b0408a4f3b922be86d4d25b6f4ab0dab32e03fb77246ca439dcf2 internal/cbm/v
992992
9b4bc8245565c98ccfc61c07749928b57e7c0f6fddb0530c4f6aa1971893d88b internal/cbm/vendored/zstd/zstd.h
993993
66a8c3f71d12ea6e797e4f622f31f3f8f81c41b36f48cad4f5de7d8bfb6aac0a internal/cbm/vendored/zstd/zstd_errors.h
994994
82ade5d7d9b029044b5fbbc326207fc47c2d44ee4dd71e8a559c36d728217de9 vendored/mimalloc/LICENSE
995-
dd4f25cae53209d45d73f8e6a2b9c219e8fc7434d97f20eba9af1a8b850030fd vendored/mimalloc/include/mimalloc-new-delete.h
995+
edb95ac4f912bbcde6c1b658e147e757896c881eae249dbb5bbbae66ebdf6deb vendored/mimalloc/PATCHES.md
996+
0a9c7e6daea2814876eab49697922f98b630c27ab9b089fbe4bd74f6e56edd68 vendored/mimalloc/include/mimalloc-new-delete.h
996997
21fcf61c4443341ac6bf6ea528af31dc7267e8e3456fc64bfd07704503032175 vendored/mimalloc/include/mimalloc-override.h
997-
5260df227ea1612b8678a06b063cc8f48e739b551554541cdeb4d430a7e257b3 vendored/mimalloc/include/mimalloc-stats.h
998-
869161433ab2d807a75752d0762accf5b68b6b24ce67e417af46bf181b9b6c72 vendored/mimalloc/include/mimalloc.h
999-
e9512b521a82cc0cbc262f6060dd70bb4243ac4a7d69d6ab8b1ef8e56f181a21 vendored/mimalloc/include/mimalloc/atomic.h
1000-
57a7297e18ae464a7bfd3fc25a46229008602385697305c69ed289b89ece7631 vendored/mimalloc/include/mimalloc/bits.h
1001-
5aeeff21d667daf4aadcc66f634aa957ecd688d15b277828f367c8c2274251a8 vendored/mimalloc/include/mimalloc/internal.h
1002-
555302d2fcd05e974441d0c9c9208c26e860db4b3f17b9025a08f42e0f4807ba vendored/mimalloc/include/mimalloc/prim.h
1003-
1f378b4b6063c078d2aeb5d916c1d30b5db787a44ab9118e7daf673b086882c6 vendored/mimalloc/include/mimalloc/track.h
1004-
b54e760da2a8d9d8a9cb708e280268c2d194c99bc518b21f72db70100578ad6a vendored/mimalloc/include/mimalloc/types.h
1005-
70420b4430da4bb6b5197fd27c6c0df485249c53a986bc9536c9fb643fc7613b vendored/mimalloc/src/alloc-aligned.c
1006-
6c8e9b0d042481c55791edb7330daec64bedaf26f0e70c9ca40de3cc17a690fe vendored/mimalloc/src/alloc-override.c
1007-
98d0eb877d7b9cdf7e9f857aaf3c739b69f746b91221e0e427e82bdd7cec90b3 vendored/mimalloc/src/alloc-posix.c
1008-
a92d58e47220e5b6d138b83d92498771be4195ee59136a134e4461f6f726b9b8 vendored/mimalloc/src/alloc.c
1009-
86fde644ede2e8d582e53499568a1713516a20326e0350eaa06a28029cece91b vendored/mimalloc/src/arena-meta.c
1010-
a1d6900d2b9a17461aa2a72a7791dfb22f6167b86f7ac6f15f981d9735aa2adb vendored/mimalloc/src/arena.c
1011-
de16033e029488e9ade4152ea1274e1a5ecbdc2c8a8d8ee50b745fd037ee6f6c vendored/mimalloc/src/bitmap.c
1012-
2e7a54ff44592a1fe8fe929e691322622f8aa1e0b1e80954781f332a88dd6cd1 vendored/mimalloc/src/bitmap.h
1013-
c9a845dd7b9020f26d73761d235fb75e7bc4a3682576de750050ac4098211b43 vendored/mimalloc/src/free.c
1014-
ae8e88737c9861e8e071fe5f1e7acc52093bcffd136a359a6b473101e968c14f vendored/mimalloc/src/heap.c
1015-
eca1fe77d72ab2aaccc447c8119056e9c4e8851bda9a95472b573539f76107ad vendored/mimalloc/src/init.c
1016-
ebd95b475f15d5387e6ef3c88a9cb47b88e64d7e55998ae1dc656614676a443b vendored/mimalloc/src/libc.c
1017-
192ce06ac9076445d223205fb340d8a1fc818dbf2b4c4a801dfd2c10bc6c1162 vendored/mimalloc/src/options.c
1018-
011e965f28c0e2c885a6834d51bd756a458dd37ebfed6fd4d9ad6607ce0390ed vendored/mimalloc/src/os.c
1019-
1be3a208c678fc34187c2a8ffb75f3fe1eecc674bbf8b2af4d1f329506b1fad5 vendored/mimalloc/src/page-map.c
1020-
7ec275805c9f00eb66710ffa99817b3ed1dba9826435d08f4866f7935152f6ae vendored/mimalloc/src/page-queue.c
1021-
d3614a59f2407a32e75b051deb1aff31b233f587d325dcc5420ba508551b687e vendored/mimalloc/src/page.c
1022-
7488e9e562ca85d2d757ec48e4d334935263a2333aa05b6061d8520f427c5890 vendored/mimalloc/src/prim/emscripten/prim.c
1023-
ce14eb6dfd55f241095b55f70983931a2d92f7a6a622a9b360cdd70ab3b53208 vendored/mimalloc/src/prim/osx/alloc-override-zone.c
998+
c0aabbf2fbe54ce55e7dee23cb1891c2d65e0e1f5b18d9fe1a4e733c0c91d996 vendored/mimalloc/include/mimalloc-profile.h
999+
57aaf0b0db23521a32b0c6d7e0969e37ce69466fcfb0cb9b2d0114c2b56be182 vendored/mimalloc/include/mimalloc-stats.h
1000+
8ec1bff9902bab6682e3ecbf7613ccc915e082ca743fdd0c396d3954323fe8bf vendored/mimalloc/include/mimalloc.h
1001+
7be611200328a99b052eda81abb8ad78b7e03446a0f6d302c1eb31cc9f096398 vendored/mimalloc/include/mimalloc/atomic.h
1002+
1ecfb1bd1a7b6f73c335d21f3411c9ce0de3cbf6710e2d209e6cb6df2baf1ed3 vendored/mimalloc/include/mimalloc/bits.h
1003+
26c10f7b990c51629618c74c64e3263a1fe00da787c2e72c56a79f87393f602c vendored/mimalloc/include/mimalloc/internal.h
1004+
a701ed0c182b4d0039bf683b4ac197011b8b794971a616c76ca64f036bc30d8b vendored/mimalloc/include/mimalloc/prim-tls.h
1005+
1987e8e2eedc07bb181bf2a11a27bec80a5309c32cfa66a56900fb4cbb64b172 vendored/mimalloc/include/mimalloc/prim.h
1006+
31d94e981640bc4c340f21887610a9cd7242af4c5adc578f722ad03cb852db2e vendored/mimalloc/include/mimalloc/track.h
1007+
01abbaf1cc5572675b97f4115f205335e319b93249ee5163fb3bf2ad0fbbceae vendored/mimalloc/include/mimalloc/types.h
1008+
8390ff8d0e6c2954db1d95bfc1a4388b302605a1c5102c0495e5163838186e7f vendored/mimalloc/src/alloc-aligned.c
1009+
6576949f30978f3b7544d9696af12dac9615074a5c2fc05e3681475575bc6ca3 vendored/mimalloc/src/alloc-override.c
1010+
a6dcdef4694964c972e6cf37f197f93c95f321566a3b5ccc1761129257255e93 vendored/mimalloc/src/alloc-posix.c
1011+
e074a8a622f6317d42eb43e48db8cde3345696c96d6749523ef4fd58aeb55bb7 vendored/mimalloc/src/alloc.c
1012+
27fce7003a5108fb9d0c61e92e0cd7870f687f7e3cf12ad19c7fb3954b2a0bc2 vendored/mimalloc/src/arena.c
1013+
b33f7b220358667fb0eddcd6569ec7b79f7ef61daf08448c8448c8d1a8741b09 vendored/mimalloc/src/bitmap.c
1014+
f3a9375ec0b56d515bea4b23732bf9a04964efecc516eb8aaac8b58ebf4512d9 vendored/mimalloc/src/bitmap.h
1015+
56a1c57bd08c49359e19e8cdcd2a11b50c30bb12a53063375374fb3f5c8d6189 vendored/mimalloc/src/free.c
1016+
a33d339568e5a7d16d6bae5650a79b1aef649e9b4ad0261be982580c2971eb64 vendored/mimalloc/src/heap.c
1017+
1452ca684eca6d959eec9eae91750162f7738055f995bd4863f79b87aab88a8c vendored/mimalloc/src/init.c
1018+
b774d5750409006b690ab6262f037d78362ca978729314493636b358249e7fbd vendored/mimalloc/src/libc.c
1019+
5bfe80a0c473371d299840e8ed20831601d4998e4164e2e0bf71303697395824 vendored/mimalloc/src/options.c
1020+
06b2d6d8a4cf3bfcc1822bb09aeaa4e7b741e7a5241f70b4f693b8038f455b00 vendored/mimalloc/src/os.c
1021+
ff3509ae3d4185e9cb2a95e1f35daba3329e998bac917ac7938f1bf06dba0f78 vendored/mimalloc/src/page-map.c
1022+
4128ca8b8d4df1116f73bf9a397903ca9941701c635345b1241295c3a31ff47d vendored/mimalloc/src/page-queue.c
1023+
ad0df1217e96669ef63539c345ea7dc4d637c3269be2d4de36ac3a5a42fb0c12 vendored/mimalloc/src/page.c
1024+
896fc1016a26e10c6cf96259d0524c57e123ac9dcef1ba04f23d0add445286dd vendored/mimalloc/src/prim/emscripten/prim.c
1025+
502efbc8616f18be83713759438a0e5b8017ed215947ddfd48e74b85fd899114 vendored/mimalloc/src/prim/osx/alloc-override-zone.c
10241026
247a9952465eb105be03a9962e922b085ce5d52034775551a61cd8594275be73 vendored/mimalloc/src/prim/osx/prim.c
1025-
cc771788e5ba591efb681829a8724f8662a1f50f3db39cf5ebe7738b8ed57e44 vendored/mimalloc/src/prim/prim.c
1026-
792531421c247fca0766ff7630057ec491173578ca4d1919ae34f05eb289ccf4 vendored/mimalloc/src/prim/readme.md
1027-
20594d38ef4fc0c641f20a8664d17ee03ce195ca21b44e036f43cdb7bcd0bd0d vendored/mimalloc/src/prim/unix/prim.c
1028-
bee9c655a17e1e93bf4908e0a902fd3b566628447b1446fcbfa92bca2e76ca15 vendored/mimalloc/src/prim/wasi/prim.c
1027+
884443f66808143d645dc6254c38e27f91b540a667ea23fe0ba1ef1caaf2b114 vendored/mimalloc/src/prim/prim-tls.c
1028+
44419a450e742812d8f3c881d957bdd6cf18f40b90dfe1153084d2b64666f12d vendored/mimalloc/src/prim/prim.c
1029+
28510fce6710d7d08224135e2e986e85995d7907232d3d6e278a82213ca78935 vendored/mimalloc/src/prim/readme.md
1030+
a5ce291e50e965fbeedc632f54bc9f10b78c0548cf7739cd477e21c6e2751b3a vendored/mimalloc/src/prim/unix/prim.c
1031+
2bc862cddb522314d58b3c6ef041e101c25cf04dfdda35cf78f91dbb6140cf46 vendored/mimalloc/src/prim/wasi/prim.c
10291032
f70e61a1f2fe63da2f51c581895cf4a0918a9bffb0cf8600171f297141446962 vendored/mimalloc/src/prim/windows/etw-mimalloc.wprp
10301033
4f3110ef2054c95cb275be96cb279224d3d46a728c5117bd1435425f382de778 vendored/mimalloc/src/prim/windows/etw.h
10311034
445e6a70f9728fcc76a5ba52079336064d5e2257c2969023f1c64bf943776f54 vendored/mimalloc/src/prim/windows/etw.man
1032-
5cde60fcef94cd79ab816030be00f8c1bdf18fab343563508c681ec6e762d1f1 vendored/mimalloc/src/prim/windows/prim.c
1035+
41d96c081cbda6bcad0e8270eab7c3d69f94dfe3d3dcd8a9567569c595984cb0 vendored/mimalloc/src/prim/windows/prim.c
10331036
1bcb43eef7fcddf6d8f18c6d5ee9bd350090ed183a1009d4f1c9b4d81a205bb9 vendored/mimalloc/src/prim/windows/readme.md
1034-
a96a1acc75fd6595811b28420cf0495b87fac98c4dbc73aa0d742b6824f607c8 vendored/mimalloc/src/random.c
1035-
4059be2a6676693be3433f5aedc711f56301772e65f3573654c236d6d1cfd074 vendored/mimalloc/src/static.c
1036-
16825bbc252de422982b5728df7eda181d952824f5e3ce958c7fc85feae64b8e vendored/mimalloc/src/stats.c
1037-
7f76bcace4b0578f2063477752138adabd99f7b10a276998e296a312c47a7cfe vendored/mimalloc/src/theap.c
1038-
49be69a4b674e09690a63870830f3ffb05698ed4210949ef5add11957a08325b vendored/mimalloc/src/threadlocal.c
1037+
2bd12844f9de0392fdb00ea70f5ad15649ffd8e49f9a4d2388e15d6469a7b703 vendored/mimalloc/src/random.c
1038+
4edc7700c2dd911cd03919d1098e973d887900ab08a5f41fd9648a79517fca57 vendored/mimalloc/src/sample-guarded.c
1039+
3df74fee931244a302293f7d215dc87e3b5c120ab7df207edd92d0ff19eb3729 vendored/mimalloc/src/sample-profile.c
1040+
fadfb75216098793f9f81e57d6ff3b63914084976526ed7867c9ca212825d784 vendored/mimalloc/src/static.c
1041+
d78e0467008946b40d5356315a9ba76e04354ead65b1aafabbb7bd091c3826a6 vendored/mimalloc/src/stats.c
1042+
41c147e8eb6e960687b79a201dd4498fb80f58923a0a4d73e04e80ef315b8eb6 vendored/mimalloc/src/subproc.c
1043+
91271bc76502c743d109f32201c02271f63a9f6ec51ef207f8b533f6de39df27 vendored/mimalloc/src/theap.c
1044+
01dbaacd5ac6685dc59c40d82a3f8932e5290d4d05f9a441424e4ea53e1ba973 vendored/mimalloc/src/threadlocal.c
10391045
cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30 vendored/nomic/LICENSE
10401046
893a74d9e3f0a74358344edc999e165739df343b36297967f3fd9cd462022f39 vendored/nomic/NOTICE
10411047
dab3009c0d76b0c5e05ad8abc7c9f8f6effca547fea3c0394be96418a85c1081 vendored/nomic/code_tokens.h

‎vendored/mimalloc/PATCHES.md‎

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
# Vendored mimalloc: source pin and local patches
2+
3+
## Source pin
4+
5+
- Upstream: <https://github.com/microsoft/mimalloc>
6+
- Tag: `v3.5.3` (annotated tag object `deac3dcf5f0614884013d411a5c07e16bec96c9f`)
7+
- Commit: `d4881d338125e1cb7c47ba4cfb398d6f7c0c8d45`
8+
(tree `fbc4e8f72287d127c079ada5ede870b7d63e55f5`)
9+
- Vendored: `LICENSE`, every file under `include/` and every file under `src/`.
10+
Nothing else (no build files, tests or docs).
11+
12+
v3.5.3 is upstream's latest published release at the time of the pin
13+
(GitHub release of 2026-09-17, on the `main3` branch).
14+
15+
v3.5.3 contains upstream commit `b2226c195fe884223f6d6f68ea52df476e57e5c2`
16+
(mimalloc #1341, first released in v3.4.4): `free(NULL)` must work before the
17+
allocator is initialized. With 3.3.2's release build, glibc 2.44's
18+
`newlocale()` freed NULL during libstdc++ start-up, before mimalloc's
19+
constructor, and every Linux binary died with SIGSEGV (reported with a
20+
reproduction by Felix Rath in #2562). The preinit fixture in
21+
`tests/test_prod_allocator_contract.sh` pins it.
22+
23+
## Verifying the vendored bytes
24+
25+
Every vendored file is byte-identical to the tag except the two hunks below.
26+
Re-check from a fresh bare clone (nothing in it is built or run):
27+
28+
```sh
29+
git clone --bare https://github.com/microsoft/mimalloc.git /tmp/mimalloc.git
30+
git --git-dir=/tmp/mimalloc.git rev-parse 'v3.5.3^{commit}' # d4881d33...
31+
for f in $(git --git-dir=/tmp/mimalloc.git ls-tree -r --name-only v3.5.3 -- LICENSE include src); do
32+
[ "$(git --git-dir=/tmp/mimalloc.git rev-parse "v3.5.3:$f")" = \
33+
"$(git hash-object "vendored/mimalloc/$f")" ] || echo "differs: $f"
34+
done
35+
```
36+
37+
The expected output is exactly the two files named below, and
38+
`vendored/mimalloc/include` plus `vendored/mimalloc/src` must hold no file the
39+
tag lacks.
40+
41+
## Local patches (reapply on every refresh)
42+
43+
Each patch is pinned by a check that fails when the patch is dropped.
44+
45+
### 1. macOS decommit always recommits: `src/prim/unix/prim.c`
46+
47+
In `_mi_prim_decommit`, macOS takes the recommit path that upstream reserves for
48+
`MI_DEBUG` / `MI_SECURE>2` builds, so a purged range is not re-accounted as a
49+
whole by `MADV_FREE_REUSE` when the arena reuses it. This cut the indexer's peak
50+
`phys_footprint` substantially (measurements in the inline comment and in commit
51+
63fb7b3d; re-measured on v3.5.3 when this pin was taken). Linux and Windows keep
52+
upstream's behaviour.
53+
54+
Pinned by: the decommit fixture in `tests/test_prod_allocator_contract.sh`
55+
(`needs_recommit` must be true on macOS and upstream's false elsewhere).
56+
57+
### 2. Comment wording: `src/prim/osx/alloc-override-zone.c`
58+
59+
Comment only, no code change: "crash in fork()." reads "crash while forking.".
60+
`scripts/security-vendored.sh` rejects call spellings such as `fork(` anywhere
61+
in vendored C, comments included, so the upstream sentence fails that gate.
62+
(Wording first proposed in #2562.)
63+
64+
Pinned by: `scripts/security-vendored.sh` itself, which reports the pristine
65+
upstream line as a blocked subprocess call.
66+
67+
## Build configuration
68+
69+
`Makefile.cbm` compiles the production object with `-DMI_PROFILE=0`, which
70+
leaves out mimalloc's allocation-sampling profiler (on by default since 3.5.2)
71+
and its sample countdown on the allocation path. Pinned by: the banner fixture
72+
in `tests/test_prod_allocator_contract.sh` (a profiling build prints a
73+
`profiling` line). The test build keeps upstream's default.
74+
75+
## Dropped patches
76+
77+
- Reproducible version banner (`src/options.c`, carried from 1f674c80 up to
78+
3.4.4): upstream removed `__DATE__`/`__TIME__` from the banner in v3.4.5
79+
(mimalloc #1355). The banner fixture in
80+
`tests/test_prod_allocator_contract.sh` still asserts that the first line is
81+
exactly `vX.Y.Z` with no build clock, so a future refresh cannot bring it
82+
back unnoticed.
83+
84+
## Refresh procedure
85+
86+
1. Replace `LICENSE`, `include/` and `src/` with the new tag's files, deleting
87+
files the tag no longer has, and verify the staged tree against the tag
88+
before patching.
89+
2. Reapply the patches above, or drop the ones upstream has absorbed (and move
90+
them to "Dropped patches" with the upstream reference).
91+
3. Update the pin in this file and `versionInfo` in
92+
`scripts/ci/generate-sbom.py`, then run `scripts/security-vendored.sh --update`.
93+
4. After `make -f Makefile.cbm cbm`, run
94+
`bash tests/test_prod_allocator_contract.sh build/c` on Linux and macOS
95+
(`scripts/test.sh` runs it on every CI leg), and `scripts/security-vendored.sh`.

0 commit comments

Comments
 (0)