Skip to content

Commit ea0e6cc

Browse files
committed
fix(mrcProfiler): keep MINISIM's hash tables small, and require a thread count
MINISIM builds one cache per profile point and holds them all at once, so every point pays for its own hash table. Sizing those tables the way cachesim does -- hashpower 24, 128MiB apiece -- costs 12.8GiB at the default --size=0.01,1,100 and was OOM-killed on one run here; hashpower 20 peaks at 0.8GiB and finishes 27x faster. The larger table bought exactness for a single policy. Measured at 100MB on cloudPhysicsIO.oracleGeneral, five runs each, only randomTwo moves: 0.8215 against cachesim's 0.8220. lru, fifo, arc, s3fifo, sieve, twoq, lfu, clock, gdsf, hyperbolic and randomLRU already agree to four decimals, and random and lecar seed from the clock and vary between runs on both sides, so no table size makes those two match. One policy's 0.0005 is not worth 15x the memory; the comment records what closing that gap properly would take. Separately, MINISIM's thread_num is only assigned when a third parameter is present, so --profiler-params=FIX_RATE,1 left it at 0, skipped the validation that guards it, and aborted inside the thread pool with "cannot push data into thread_pool in get_miss_ratio" -- saying nothing about the parameter that was actually missing. Reject that form where the other parameter errors are raised. SHARDS is unaffected: its optional third field is a salt, and 0 is a valid salt. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YUq1vM4g82TkaX2jvLmuQc
1 parent eae9bed commit ea0e6cc

2 files changed

Lines changed: 40 additions & 16 deletions

File tree

libCacheSim/mrcProfiler/mrcProfiler.cpp

Lines changed: 29 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -25,14 +25,6 @@ static constexpr double kHashSpaceSize = 18446744073709551616.0;
2525
* caches are small, so a smaller table than cachesim's is appropriate. */
2626
static constexpr int kMiniSimHashPower = 20;
2727

28-
/* what cachesim uses, mirroring DEFAULT_HASHPOWER in bin/cachesim/cache_init.h.
29-
* Above sample rate 0.5 MINISIM replays the whole trace, and that run is meant
30-
* to be exact rather than approximate, so it has to size the table the way
31-
* cachesim would: Random, RandomTwo, RandomLRU and Hyperbolic draw eviction
32-
* candidates through the hash mask, so a different table gives a different
33-
* curve. */
34-
static constexpr int kCacheSimHashPower = 24;
35-
3628
/* whether a reader fills in req->next_access_vtime, which the Belady policies
3729
* need; every other reader leaves it at -2.
3830
*
@@ -380,14 +372,35 @@ void mrcProfiler::MRCProfilerMINISIM::run() {
380372

381373
/* BeladySize picks its victim by drawing samples from the hash table, so an
382374
* oversized table costs memory and leaves the sampler probing empty buckets.
383-
* cachesim shrinks it by 8 before constructing the cache; do the same here,
384-
* since the miniature caches are built straight from the registry and would
385-
* otherwise get a 1M-slot table each. Hyperbolic gets it for a different
386-
* reason: Hyperbolic_init shrinks its own table as well, so cachesim ends up
387-
* two reductions down, and matching that is what makes an unsampled run
388-
* reproduce cachesim rather than land 0.0001 away. */
389-
int minisim_hashpower =
390-
(sampler == nullptr) ? kCacheSimHashPower : kMiniSimHashPower;
375+
* cachesim shrinks it by 8 before constructing the cache, landing on 16;
376+
* the miniature caches are built straight from the registry and would
377+
* otherwise keep a 1M-slot table each, so the floor below puts them on 16
378+
* too. Hyperbolic gets the same treatment for a different reason:
379+
* Hyperbolic_init shrinks its own table as well, so cachesim ends up two
380+
* reductions down, and starting from the same 16 is what makes an unsampled
381+
* hyperbolic run reproduce cachesim to four decimals rather than land beside
382+
* it. */
383+
/* The table stays small even when sampling is off and MINISIM replays the
384+
* whole trace, rather than growing to the hashpower cachesim would use.
385+
* MINISIM builds one cache per profile point and holds them all at once, so
386+
* the table is paid for once per point: at the default --size=0.01,1,100
387+
* that is 100 of them, and cachesim's 24 would be 128MiB each. Measured on
388+
* cloudPhysicsIO, that peaks at 12.8GiB and was OOM-killed on one run;
389+
* hashpower 20 peaks at 0.8GiB and finishes 27x faster. --size accepts up to
390+
* MAX_MRC_PROFILE_POINTS points, so the ceiling is higher still.
391+
*
392+
* What the smaller table costs: policies that draw eviction candidates
393+
* through the hash mask can land slightly off a cachesim run at the same
394+
* size. Measured at 100MB on cloudPhysicsIO.oracleGeneral, five runs each,
395+
* the only reproducible gap is randomTwo -- 0.8215 here against cachesim's
396+
* 0.8220. lru, fifo, arc, s3fifo, sieve, twoq, lfu, clock, gdsf, hyperbolic
397+
* and randomLRU agree to four decimals. random and lecar disagree by a
398+
* comparable amount, but they seed from the clock and move between runs on
399+
* both sides, so no table size makes those two match. Closing randomTwo's
400+
* gap would need simulate_with_multi_caches to build each cache when a
401+
* thread picks it up instead of all of them up front -- a change to the
402+
* simulator, not to this profiler. */
403+
int minisim_hashpower = kMiniSimHashPower;
391404
if (strcasecmp(params_.cache_algorithm_str, "hyperbolic") == 0) {
392405
minisim_hashpower = MAX(minisim_hashpower - 8, 16);
393406
}

libCacheSim/mrcProfiler/mrcProfiler.h

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -178,6 +178,17 @@ typedef struct profiler_params {
178178
start = end + 1;
179179
}
180180
}
181+
182+
/* thread_num is only assigned when a third field is present, so
183+
* --profiler-params=FIX_RATE,1 leaves it at 0 and skips the check just
184+
* above. simulate_with_multi_caches then fails to queue any work and
185+
* aborts with "cannot push data into thread_pool in get_miss_ratio",
186+
* which says nothing about the parameter that was actually missing. */
187+
if (current_param_idx < 3) {
188+
ERROR("minisim params need FIX_RATE,<sample_rate>,<thread_num>: %s\n",
189+
str);
190+
exit(1);
191+
}
181192
}
182193
} minisim_params;
183194

0 commit comments

Comments
 (0)