Repository navigation
Description
What happens
collect_grep_matches() reads the scan output with fgets into a 2048-byte buffer and parses each line as file:line:content. A grep record longer than 2047 bytes is split across two fgets calls, and the continuation chunk is then parsed as if it were a fresh record: the text before its first : becomes the file, and the text after it becomes the line number.
Any file with a line longer than ~2 KB containing colons reproduces it — a minified bundle, generated code, a long data row.
Reproduction
mkdir -p /tmp/fx && cd /tmp/fx && git init -q
python3 -c "
pairs = ','.join('k%d:\"v%d\"' % (i, i) for i in range(260))
open('big.js','w').write('var CFG=({' + pairs + ',NEEDLEmarker:1});\n')
open('normal.js','w').write('const NEEDLEmarker = 42;\n')
"
git add -A && git -c user.email=t@t -c user.name=t commit -qm init
codebase-memory-mcp cli index_repository --repo-path /tmp/fx
codebase-memory-mcp cli --json search_code --project <project> --pattern NEEDLEmarkerbig.js is a single 2927-byte line. The response comes back isError: false, and contains:
raw: 1 (cols: file line content)
"v181",k182 0 "v183",k184:"v184",k185:"v185", … ,NEEDLEmarker:1});
total_grep_matches: 3
total_results: 2
The file is a fragment of JavaScript — "v181",k182 — and the line number is 0. No such file exists. total_grep_matches is 3 for a repository containing 2 matches, because the split produced a third record.
An agent that reads this calls get_code_snippet on a path that was never in the repository, or reports to the user that a symbol lives at a file and line that do not exist.
Mechanism
src/mcp/mcp.c:10261 collect_grep_matches():
char line[CBM_SZ_2K]; /* 2048 — src/foundation/constants.h:36 */
while (fgets(line, sizeof(line), fp) && gm_count < grep_limit) {
...
char *sep1 = strchr(line, (unsigned char)sep);
if (!sep1) { continue; }
char *sep2 = strchr(sep1 + SKIP_ONE, (unsigned char)sep);
if (!sep2) { continue; }
*sep1 = '\0';
*sep2 = '\0';
const char *path = line; /* mcp.c:10300 */
...
snprintf(gm[gm_count].file, sizeof(gm[0].file), "%s", file);
gm[gm_count].line = (int)strtol(sep1 + SKIP_ONE, NULL, CBM_DECIMAL_BASE);Nothing distinguishes a continuation chunk from a record start. When the tail happens to contain two colons the chunk becomes a fabricated match; when it does not, the continue silently drops a genuine one instead — so the same split can either invent a match or lose one.
Suggested fix
Detect the truncation fgets already tells you about: a chunk that does not end in \n is a partial record. Either accumulate until the newline arrives, or consume and discard the remainder of the over-long record and count it as skipped, rather than feeding the tail back into the file:line:content parser. Validating the parsed line number (reject a non-numeric or <= 0 field) would catch the fabricated rows as a second line of defence, but on its own it would still drop the real match silently.
Environment
- Built from
main@ 5fbab7b (plus the unrelated cypher patch in fix(cypher): reject a non-numeric SKIP/LIMIT operand instead of dropping it #2002;collect_grep_matchesis untouched by it) - macOS 15 (Darwin 25.6.0), arm64, clang
Activity
DeusData commentedon Sep 3, 2026
Thank you for the minimal long-line reproducer and the exact fabricated row.
Current
mainconfirms the bug:collect_grep_matches()reads into a fixed 2 KiBfgetsbuffer, has no partial-record guard, and parses the next chunk as a new delimiter-separated record. It also accepts the line field throughstrtolwithout validating that it is a positive integer. The reported invented path and extra match are therefore source-grounded.I have routed this as a high-priority correctness bug. No more evidence is needed from you at this stage; thank you for making the failure deterministic.
DeusData commentedon Sep 5, 2026
Fixed on main in 3c7427e (#1597): search_code now reads each grep record with cbm_getline (unbounded, doubling buffer) instead of a fixed 2 KiB fgets, so an over-long line can no longer split into two records and invent a match. The regression test from #2066 is being landed on top so the shape stays guarded. Thanks for the precise report.
test(mcp): over-long grep records never invent matches (DeusData#2011)
Merge pull request #2066 from kavish-19/fix-grep-record-alignment
test(mcp): over-long grep records never invent matches (DeusData#2011)