Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 256a188ee6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| cache->obj_md_size = | ||
| MAX(params->LRU->obj_md_size, params->main_cache->obj_md_size); |
There was a problem hiding this comment.
Use the main cache overhead when checking promotions
When metadata accounting is enabled with a lower-overhead main policy such as FIFO or Clock, assigning the parent the maximum overhead makes WTinyLFU_evict() use the window's 16-byte overhead in its main-cache capacity check (params->req_local->obj_size + cache->obj_md_size). This incorrectly treats the main cache as full, causing unnecessary evictions and altered simulation results; if the main cache is empty and the promoted object fits exactly under its own zero-byte overhead, the code enters the full-cache branch and dereferences the null result of main_cache->to_evict(). The promotion check needs to use main_cache->obj_md_size rather than this parent-level value.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Confirmed, and this one was mine — fixed.
cache->obj_md_size is the MAX of the two sub-caches so that a caller asking the composite what it reserves is not told less than it really does. Using that same value for the main cache's capacity check was wrong, exactly as you describe: against a FIFO or Clock main cache, which reserve nothing, it billed the window's 16 bytes and called the main cache full early. The check now uses main_cache->obj_md_size:
if (main_cache->get_occupied_byte(main_cache) +
params->req_local->obj_size + main_cache->obj_md_size <=
main_cache->cache_size) {Worth being precise about what changed and when. Before this PR the same line read cache->obj_md_size too, but that field was copied from a malloc'd, un-memset struct before main_cache was assigned, so the whole configuration segfaulted and no value was ever correct. Fixing the uninitialized read is what made this line reachable, and my MAX is what made it wrong in a new way. So: not a pre-existing bug I inherited, a regression I introduced while fixing the crash.
Results move again as a result — main-cache=FIFO with metadata on is now 0.7836, where my previous number in this PR's description was 0.8070:
| main-cache | miss ratio |
|---|---|
| FIFO | 0.7836 |
| clock | 0.7679 |
| LRU | 0.7766 |
| SLRU | 0.8040 |
The null-dereference half is real but latent rather than observed: DEBUG_ASSERT compiles out in Release, so an empty main cache reaching the else branch would deref main_cache->to_evict()'s NULL. I did not manage to trigger it on the sample trace, so I am reporting it as reasoning rather than a reproduction — the arithmetic fix removes the path either way.
ctest 10/10.
Generated by Claude Code
There was a problem hiding this comment.
Pull request overview
This PR hardens libCacheSim’s CLI-reachable code paths by fixing crashes and teardown-visible leaks across multiple cache/prefetch implementations and the traceAnalyzer / mrcProfiler command-line parsers. It primarily targets early-exit parameter reporting (-e print), option parsing edge cases, and missing frees in cache destructors.
Changes:
- Fix
-e print/ parameter parsing early-exit leaks by tracking and freeing the originalstrdup()buffer across many algorithms. - Prevent crashes from NULL option arguments (
argpoptional args) and make output-path options require an argument intraceAnalyzer/mrcProfiler. - Address correctness/leak issues in specific algorithms (e.g., WTinyLFU init ordering/metadata sizing, SLRU param reporting before segment allocation, missing frees in several cache free functions, hashpower clamping).
Reviewed changes
Copilot reviewed 35 out of 38 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| libCacheSim/cache/prefetch/PG.c | Free duplicated param string on print/default and normal completion. |
| libCacheSim/cache/prefetch/Mithril.c | Free duplicated param string on print/default and normal completion. |
| libCacheSim/cache/eviction/WTinyLFU.c | Zero-init params, defer metadata sizing until subcaches exist, add print support, free params on teardown. |
| libCacheSim/cache/eviction/TwoQ.c | Free param buffer on print early exit. |
| libCacheSim/cache/eviction/SLRUv0.c | Zero-init params, clamp hashpower, free params on teardown, avoid leaking request on early return in cool path. |
| libCacheSim/cache/eviction/SLRU.c | Make current_params safe before segment allocation; validate n-seg / seg-size; free param buffer on print exit. |
| libCacheSim/cache/eviction/Size.c | Free params struct in Size_free(). |
| libCacheSim/cache/eviction/S3FIFOv0.c | Avoid deref of NULL main cache during -e print; free param buffer on print exit. |
| libCacheSim/cache/eviction/S3FIFOd.c | Clamp hashpower; free all subcaches (including eviction trackers); safe param reporting before build; free param buffer on print exit. |
| libCacheSim/cache/eviction/S3FIFO.c | Free param buffer on print exit. |
| libCacheSim/cache/eviction/RandomLRU.c | Free params struct on teardown; free param buffer on print exit. |
| libCacheSim/cache/eviction/QDLP.c | Avoid deref of NULL main cache during -e print; free param buffer on print exit. |
| libCacheSim/cache/eviction/plugin_cache.c | Allow bare print flag; free param buffer on print exit. |
| libCacheSim/cache/eviction/other/S3LRU.c | Free param buffer on print exit. |
| libCacheSim/cache/eviction/other/flashProb.c | Avoid deref of NULL RAM cache during -e print; free param buffer on print exit. |
| libCacheSim/cache/eviction/LRUProb.c | Free param buffer on print exit. |
| libCacheSim/cache/eviction/LeCaR.c | Free param buffer on print exit. |
| libCacheSim/cache/eviction/Hyperbolic.c | Free param buffer on print exit. |
| libCacheSim/cache/eviction/GLCache/GLCache.c | Free duplicated init-param string on print/default and normal completion. |
| libCacheSim/cache/eviction/fifo/SFIFOv0.c | Guard against divide-by-zero in hashpower divisor; free param buffer on print exit. |
| libCacheSim/cache/eviction/fifo/SFIFO.c | Free param buffer on print exit. |
| libCacheSim/cache/eviction/fifo/LP_TwoQ.c | Free param buffer on print exit. |
| libCacheSim/cache/eviction/fifo/LP_SFIFO.c | Clamp hashpower; free param buffer on print exit. |
| libCacheSim/cache/eviction/fifo/LP_ARC.c | Free param buffer on print exit. |
| libCacheSim/cache/eviction/FIFO_Reinsertion.c | Free param buffer on print exit. |
| libCacheSim/cache/eviction/FIFO_Merge.c | Free param buffer on print exit. |
| libCacheSim/cache/eviction/cpp/LRU_K.cpp | Free duplicated param buffer on print exit. |
| libCacheSim/cache/eviction/ClockPro.c | Free param buffer on print exit. |
| libCacheSim/cache/eviction/Clock2QPlus.c | Allow bare print flag; free param buffer on print exit. |
| libCacheSim/cache/eviction/Clock.c | Free param buffer on print exit. |
| libCacheSim/cache/eviction/CAR.c | Free param buffer on print exit. |
| libCacheSim/cache/eviction/Cacheus.c | Clamp hashpower reduction. |
| libCacheSim/cache/eviction/BeladySize.c | Free param buffer on print exit. |
| libCacheSim/cache/eviction/ARCv0.c | Free param buffer on print exit. |
| libCacheSim/cache/eviction/ARC.c | Free param buffer on print exit. |
| libCacheSim/bin/traceAnalyzer/cli_parser.cpp | Make --output/-o require an argument; update example path. |
| libCacheSim/bin/mrcProfiler/cli_parser.cpp | Make multiple options require arguments (incl. --output); avoid NULL arg derefs. |
| libCacheSim/bin/cli_reader_utils.c | Treat presence of optional argp flags (NULL arg) as true in is_true(). |
Suppressed comments (2)
libCacheSim/cache/eviction/SLRU.c:494
- In the parse loop, the unknown-parameter branch exits without freeing the strdup'd parameter buffer (old_params_str). This still leaks when the process exits normally (e.g., in builds where ERROR() doesn't abort), even though the print path was fixed.
free(old_params_str);
exit(0);
} else {
ERROR("%s does not have parameter %s\n", cache->cache_name, key);
exit(1);
libCacheSim/cache/eviction/TwoQ.c:359
- TwoQ_parse_params() frees old_params_str for the "print" early-exit, but the unknown-parameter branch still exits without freeing the strdup'd buffer. That leaves a leak on exit(1) in builds where ERROR() doesn't abort, and is inconsistent with the other parsers in this PR.
free(old_params_str);
exit(0);
} else {
ERROR("%s does not have parameter %s\n", cache->cache_name, key);
exit(1);
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| if (params->window_size < 0 || params->window_size >= 1) { | ||
| ERROR("window_size must be in [0, 1)\n"); | ||
| exit(1); | ||
| } | ||
| } else if (strcasecmp(key, "print") == 0) { |
89eb478 to
d3dda43
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d3dda43c07
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| cache->obj_md_size = | ||
| MAX(params->LRU->obj_md_size, params->main_cache->obj_md_size); |
There was a problem hiding this comment.
Charge window insertions their actual metadata
With --consider-obj-metadata=true and a main policy whose overhead exceeds the window LRU's 16 bytes—ARC, LeCaR, and Cacheus each charge 40—this maximum is also used by the generic cache_get_base() capacity loop. Every incoming miss is inserted into the window, so reserving 40 bytes there instead of 16 can invoke WTinyLFU_evict() even when the object's actual window footprint fits; that causes an unnecessary admission comparison or eviction and changes simulation results. Preserve the composite's advertised maximum separately or override the parent capacity calculation to charge the window's overhead.
Useful? React with 👍 / 👎.
d3dda43 to
f6189ee
Compare
|
Consolidated response to the review comments on this PR. Window metadata, second finding (P1) — fixed. Measured at 100 MB with metadata on: LeCaR moves 0.7894 → 0.7895 and ARC, LRU, SLRU and FIFO do not move, because objects are large relative to a 24-byte difference. Correct rather than consequential, but it removes the last place the composite billed an overhead belonging to a cache the object was not entering.
$ cachesim ... slru 100mb -e bogus=1 # LSan build
[ERROR] SLRU.c:493 SLRU does not have parameter bogus
rc=134 # SIGABRT, no leak report27 files share that shape. The frees I did add are on the Everything else raised here was already covered: the Generated by Claude Code |
f6189ee to
a65dc8a
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a65dc8af41
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ccache_params_local.hashpower = | ||
| MAX(4, MIN(16, ccache_params_local.hashpower - 4)); |
There was a problem hiding this comment.
Preserve the zero hashpower sentinel
When an API caller uses a designated common_cache_params_t initializer without setting hashpower (as several repository tests do), its value is zero, which cache_struct_init() deliberately interprets as HASH_POWER_DEFAULT (23). This clamp converts that sentinel to 4, so every SLRUv0 segment starts with only 16 buckets and must repeatedly reallocate and rehash while processing a large trace, creating a substantial initialization/runtime regression. Keep nonpositive values unchanged and apply the reduction only to an explicitly supplied positive hashpower.
Useful? React with 👍 / 👎.
a65dc8a to
7d460ce
Compare
7d460ce to
a2c9750
Compare
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
The `s3fifo_hashpower_for_size` helper and its three call sites came in with staircased indentation, so `clang-format --dry-run --Werror` fails on S3FIFO.c. The Code Quality workflow runs that check over the files a pull request touches, so any change to this file fails CI until the indentation is repaired. The reformat is whitespace-only -- `diff -w` between the old and new file is empty -- and covers only the lines clang-format objects to; no behavior changes. Full test suite passes (10/10). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YUq1vM4g82TkaX2jvLmuQc
a2c9750 to
6ca8d7f
Compare
|
Code Quality failed on the previous head ( The failing file was
Generated by Claude Code |
S3FIFO_init sizes each sub-cache's hash table from that sub-cache's byte size and writes the result over whatever the caller passed, so a requested hashpower reached only the parent table. cachesim's --hashpower documents itself as a way to cut memory, and for S3FIFO it mostly did not: measured on cloudPhysicsIO at a 1GB cache, --hashpower=12 left the run at 207MiB peak RSS where lru at the same setting used 14MiB, the difference being three sub-cache tables sized as though nothing had been asked for. Cap each computed sub-cache hashpower by the requested one. 12 now lands at 15.7MiB and 18 at 23.3MiB, against lru's 14.0MiB and 15.9MiB. A hashpower of 0 is the "use HASH_POWER_DEFAULT" sentinel rather than a request, so it caps nothing, and the default path is byte-for-byte what it was: 335.3MiB before and after, so the proportional sizing that #316 added to avoid OOM is intact. No behavior change. S3FIFO's miss ratio is identical at hashpower 12, 18 and 24 (0.4922 at 1GB) -- the sub-caches are FIFOs and do not draw eviction candidates from the table, so a smaller one only costs rehashing. Full suite passes 10/10. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YUq1vM4g82TkaX2jvLmuQc
ht_test tracks objects in their test period and is private to ClockPro, so cache_struct_init never sizes it; ClockPro_init created it at HASH_POWER_DEFAULT no matter what the caller asked for. cachesim's --hashpower therefore reached only the main table: measured on cloudPhysicsIO at a 1GB cache, --hashpower=12 left the run at 77.7MiB peak RSS where lru at the same setting used 14.0MiB, the difference being this table's 2^23 pointer slots. Cap it by whatever the main table settled on. 12 now lands at 13.6MiB and 18 at 17.4MiB, against lru's 14.0MiB and 15.9MiB. Cap rather than mirror, which is not the same thing here: cachesim's default hashpower is 24 while HASH_POWER_DEFAULT is 23, so mirroring the main table would have raised the no-flag default from 205.4MiB to 269.4MiB. Measured both ways -- a request to use less memory must not become more for everyone who makes no request. The default path is unchanged at 205.4MiB. No behavior change: ClockPro's miss ratio is identical at hashpower 12, 18 and 24 (0.5053 at 1GB), since ht_test is looked up by object id and never sampled. Full suite passes 10/10. This was the last hard-coded table in the tree -- cache.c and this line were the only two create_hashtable() call sites, and cache.c already honors the requested hashpower. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YUq1vM4g82TkaX2jvLmuQc
…allocator guard
Two low-level paths that cannot do what they were written to do.
create_chained_hashtable_v2 computes its size from hashtable->hashpower, but
the memset two lines above has just zeroed that field, and the caller's value
is the hashpower parameter, which is not stored until the end of the
function. So size was always 8 bytes. Instrumented at hashpower 20:
PROBE hashpower=0 requested=20 size=8 expected=8388608
That feeds two statements. The memset clears one slot instead of the table,
which is invisible today only because my_malloc_n resolves to calloc under
the default allocator -- under HEAP_ALLOCATOR_ALIGNED_MALLOC, which does not
zero, the table would be full of garbage pointers. And the madvise() call is
compiled in by default (USE_HUGEPAGE defaults to ON and MADV_HUGEPAGE exists
on Linux), so the transparent-hugepage hint the project deliberately enables
has been advising 8 bytes rather than the whole table. Throughput on this
container is too noisy to claim a win from restoring it -- nine runs each
gave medians 5.02 against 4.58 MQPS with overlapping spreads -- so the claim
here is coverage, not speed.
mem.h guards its first branch on HEAP_ALLOCTOR, missing the second A, so
HEAP_ALLOCATOR_G_NEW can never be selected: the identifier expands to 0, no
branch matches, and none of my_malloc/my_malloc_n/my_free get defined. One of
the four documented allocator choices simply fails to compile. Verified with
a probe translation unit before and after.
No behavior change: miss ratios are identical across lru, fifo, arc, s3fifo,
sieve, twoq and clockpro, and the suite passes 10/10.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YUq1vM4g82TkaX2jvLmuQc
ARC_init, ARCv0_init and LP_ARC_init never called the *_parse_params they each define, so the parser was dead code and the -e flag was dropped. Two consequences, both reachable from a documented command. `-e print` replayed the whole trace instead of reporting, which is what let it hide behind the sweep in test_cli.sh. Worse, a mistyped parameter was accepted in silence: `cachesim ... arc 1gb -e nonsense=42` ran a full simulation and exited 0, where s3fifo rejects the same input with "does not have parameter nonsense". A typo produced a plausible-looking result rather than an error. Call the parser the way every other algorithm with one does. arc and arcv0 now print and exit 0 for `-e print`, and reject an unknown key. Their parameter reports are empty because neither has a tunable, which is honest -- `print` is the only key either parser accepts. Miss ratios are unchanged when no -e is given (arc and arcv0 both 0.5688 at 1GB on cloudPhysicsIO.oracleGeneral). Suite passes 10/10. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YUq1vM4g82TkaX2jvLmuQc
These were the last two files in the tree that fail `clang-format --dry-run --Werror`. No open pull request touches them, so the Code Quality job -- which only checks the files a pull request changes -- would have failed for whoever edited them next, over formatting they did not introduce. Formatting only: with comments and whitespace normalized away, both files are byte-identical to before. The one non-whitespace character in the diff is the `*` clang-format adds when it rewraps a block comment onto a second line. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YUq1vM4g82TkaX2jvLmuQc
|
Closing as redundant: #324 was merged into The remaining Generated by Claude Code |
Part 3 of 5, split out of #324. Base of the code stack — #329 builds on this, and #330 on that. Independent of #326 and #327.
Every bug here is reachable from a command in the quickstart guides. The regression tests that pin them are in #330, split out so this PR stays a readable set of fixes.
Crashes
cachesim … slru/qdlp/s3fifod/s3fifov0/flashProb … -e printparse_paramsbuilds after ittraceAnalyzer -o my-output …-odeclaredOPTION_ARG_OPTIONAL, so argp passesarg == NULLfor the space-separated form andstrncpydereferences itmrcProfiler -o out …--algo,--size,--profiler,--profiler-paramstraceAnalyzer --verbose …is_true(NULL)→strcasecmp(NULL, …)cachesim … slru … -e n-seg=0SLRU_initcachesim … wtinyLFU … --consider-obj-metadata=trueWTinyLFU_initreadparams->main_cache->obj_md_sizefrom amalloc'd, un-memsetstruct beforemain_cachewas assignedSLRUalso wrote pastseg_size_array[SLRU_MAX_N_SEG]when given more than 16 colon-separated sizes, before the validity check ran.n-segandseg-sizeare now validated at parse time, so invalid input says what is wrong instead of aborting later with a figure like-9223372036854775808 bytes.Memory leaks
The ubuntu job builds with LeakSanitizer, but nothing ran the binaries to completion, which is where these live:
-e printearly exit leaked thestrdup'd parameter string in 29 files —exit(0)comes before the free at the end of the parser. GLCache, Mithril and PG never kept the original pointer at all, so they leaked on every call, not just the print path.Size_free,RandomLRU_freeandSLRUv0_freenever freed their params struct;S3FIFOd_freefreed three of its five sub-caches, leaving 73 KB;SLRUv0_coolallocated a request and early-returned oni == 0without freeing it, once per eviction from the bottom segment;WTinyLFU_freenever freed its params.Consistency
-e printis documented for every algorithm, but Clock2QPlus and pluginCache rejected the bareprintkey before reaching their own print branch, and WTinyLFU had no print branch at all despite taking parameters. All three now behave like the rest.One deliberate behavior change
With the uninitialized read fixed,
WTinyLFU_can_insert()charges the window its own per-object overhead rather than the main cache's. The two genuinely differ — LRU and SLRU reserve 16 bytes, FIFO and Clock none — somain-cache=FIFOwas charging the window nothing for an overhead it does reserve.This moves results:
main-cache=FIFOwith metadata on goes 0.8619 → 0.8070, andclock0.7732 → 0.7733. It is only reachable at all because the configuration used to segfault, so there is no working behavior on either side being changed. With metadata off nothing changes, and LRU, SLRU, sieve and ARC are unchanged either way.Testing
-Wall -Wextra -Werrorctest --output-on-failure— 9/9 (the tenth target,testCLI, arrives in Add a CLI regression test target #330)develop;lecarv0,cacheus,fifo-mergeandRandomLRUvary run to run on their own (FIFO_Mergehas an explicitnext_rand()tiebreaker) and were compared against themselvesGenerated by Claude Code