Make a failed priority-queue growth fatal instead of corrupting the heap - #334
Merged
Merged
Conversation
…he heap #333 changed pqueue to grow geometrically so Size and Belady could stop reserving 64MB apiece. It merged without this guard -- the review that found the problem landed while the merge was in flight -- so develop now has the growth policy and not the check. pqueue_insert returns 1 when the realloc fails, and the header documents "0 on success", but no caller in the tree checks it: Size.c:181, Belady.c:201, PG.c:344 and PG.c:399 all discard the return. They could not act on it if they did. Size_insert calls cache_insert_base first, so the object is already in the cache, then allocates the node with my_malloc -- plain malloc, not zeroed -- and assigns cached_obj->Size.pq_node = node whether or not the insert took. bubble_up never ran, so node->pos holds whatever was in that heap cell. Size_remove_obj sees a non-NULL pq_node and calls pqueue_remove: size_t posn = q->getpos(d); q->d[posn] = q->d[--q->size]; an unbounded write at that uninitialised index. Not a wrong miss ratio -- a heap out-of-bounds write. The path predates #333, but 8 million entries of slack stood in front of it. Growth is how the queue reaches its working size now, so it becomes reachable from about 65K objects onward, and specifically under the virtual-memory limits the smaller reservation was meant to fit inside. Terminate instead, which is how create_chained_hashtable_v2 already handles a failed table allocation, and which covers PG's two call sites by the same guard. Checking at the callers was the alternative and does not work: by the time the insert is attempted the object is in the cache, so there is no atomic recovery short of unwinding cache_insert_base. Verified by injecting a failed realloc on the third growth: the run aborts with "cannot grow priority queue to 136 entries (1088 bytes)" and rc=134 rather than continuing into the corrupting path. Suite passes 10/10 with 185 CLI checks, and neither memory nor miss ratios move. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YUq1vM4g82TkaX2jvLmuQc
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
13 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Follow-up to #333, which merged without this guard — the review that found the problem landed while the merge was in flight, so
developnow has the geometric growth policy and not the check that makes it safe.pqueue_insertreturns 1 when itsreallocfails, and the header documents "0 on success", but no caller in the tree checks it —Size.c:181,Belady.c:201,PG.c:344andPG.c:399all discard the return. They could not act on it if they did:Size_insertcallscache_insert_basefirst, so the object is already in the cache.my_malloc— plainmalloc, not zeroed.cached_obj->Size.pq_node = nodewhether or not the insert took.bubble_upnever ran, sonode->posholds whatever was in that heap cell.Size_remove_objsees a non-NULLpq_nodeand callspqueue_remove:An unbounded write at that uninitialised index. Not a wrong miss ratio — a heap out-of-bounds write.
The path predates #333, but eight million entries of slack stood in front of it. Growth is how the queue reaches its working size now, so it becomes reachable from roughly 65K objects onward, and specifically under the virtual-memory limits the smaller reservation was meant to fit inside.
This terminates instead, which is how
create_chained_hashtable_v2already handles a failed table allocation, and which covers PG's two call sites by the same guard. Checking at the callers was the alternative and does not work: by the time the insert is attempted the object is in the cache, so there is no atomic recovery short of unwindingcache_insert_base.Found by Codex review on #333.
Type of change
How was it tested?
The failure path cannot be reached with the sample traces —
cloudPhysicsIO's working set is ~49K objects, which fits inside the 65536-entry reservation, so the queue never grows. I verified it by fault injection instead, forcing the third growth'sreallocto return NULL:instead of continuing into the corrupting path. The injection was reverted before committing.
ctest --test-dir _build --output-on-failurepasses-Wall -Wextra -Werror)Results
No change to memory or miss ratios — this only replaces an unhandled error return with a diagnostic. The
sizeandbeladyfigures from #333 are unaffected.Checklist
clang-format(or the pre-commit hook fromscripts/setup_hooks.sh)test_cli.shcannot induce portably; fault injection is described above🤖 Generated with Claude Code
https://claude.ai/code/session_01YUq1vM4g82TkaX2jvLmuQc
Generated by Claude Code