You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit b4e514e
Browse filesBrowse the repository at this point in the historyBrowse files
fix(query): a node of the query walk lets go of its folder once its own path is built - so a path term down a deep chain no longer keeps every folder above, and the whole copy of its path the term read, alive; and a test holds the deep hit's walk to a few hundred bytes per folder, so a walk that holds the depth squared fails on every machine, not only on CI's 2 GB heap (CI4)
The second way the query walk held the square of a deep chain's depth.
QueryNode kept its folder after its path was built, though the folder is
only there to build that path. A path term reads every node's path, which
copies it whole; each node then held its folder, which held its folder,
up to the root - every one with a whole copy of its path. With the store
fixed (the commit before), `path:leaf` down 10,000 folders still held
291 MB; with the folder let go it holds 0.2 MB at 30,000.
The guard (tests/queryWalkWork.test.ts, "CI4: a deep hit costs memory in
proportion to its depth, not its square"): down an 8,000-deep chain,
`name:leaf` and `path:leaf` each hold at most 2 KiB per folder at the
moment the deep path is built (the heap after a full collection,
tests/fixtures/heapProbe.ts; measured 243 and 16 bytes per folder - a
whole copy of every folder's path is 23 KB). The deep path is still built
once, from its folder's, without a stack frame per folder, and is exactly
right. The F1.2 chain builder is shared with it, unchanged.
Mutants, each RED for the right reason, restored and checked by SHA-256:
the folder kept (path:leaf: 184.7 MB held, 23091 bytes per folder); the
store's endsWith rule back (name:leaf: 184.8 MB, 23099 bytes per folder);
the path built by store.path() instead of from its folder's (the deep path
is built once, from its folder's: 0 builds - the probe never measured).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
assert.equal(out.total,1,`${q}: the one file matches`);
292
-
assert.equal(out.hits[0].path,`${deepest}/leaf.bin`,`${q}: its path, all ${DEPTH} folders of it`);
302
+
assert.equal(out.hits[0].path,leafPath,`${q}: its path, all ${DEPTH} folders of it`);
303
+
}
304
+
});
305
+
306
+
/**
307
+
* What a deep hit's path may hold while it is built: a few hundred bytes per
308
+
* folder on its way down (the node the walk holds, and the folder's share of
309
+
* a path joined from its folder's) — never a whole copy of every folder's
310
+
* path, which costs half the deep path's length per folder: 26 KB at 8,000
311
+
* folders deep.
312
+
*/
313
+
constHELD_PER_FOLDER=2048;
314
+
315
+
test('CI4: a deep hit costs memory in proportion to its depth, not its square — with a path term or without, no folder on its way down keeps a whole copy of its path',async()=>{
316
+
// CI run 39095045009: the 30k-deep chain above ran macOS's 2 GB heap out
317
+
// (it passed on this Mac's 4 GB). Each folder's path, built from its
318
+
// folder's, was read whole as the next was joined (the store's childPath
319
+
// asked whether it ended with the separator), so every folder kept a flat
320
+
// copy of its own path: the depth squared, about 2.7 GB at 30k. A path
321
+
// term reads every node's path whole anyway, and a folder kept reachable
322
+
// from below after its path was built then held that copy alive.
323
+
// Measured where the deep path is built: the most the walk holds at once.
assert.equal(out.total,1,`${q}: the one file matches`);
346
+
assert.equal(out.hits[0].path,leafPath,`${q}: its path, all ${DEPTH} folders of it`);
347
+
assert.equal(probes,1,`${q}: the deep path is built once, from its folder's`);
348
+
assert.ok(
349
+
held<HELD_PER_FOLDER*DEPTH,
350
+
`${q}: ${(held/1e6).toFixed(1)} MB held as the deep path is built, ${Math.round(held/DEPTH)} bytes per folder of ${DEPTH} (at most ${HELD_PER_FOLDER}; a whole copy of every folder's path is about ${Math.round(leafPath.length/2)})`,
0 commit comments