Commit 3331335
perf(bench): add tail-latency percentiles and optional jemalloc/tcmalloc baselines (#116)
## Summary
Implements **ROADMAP item 9.4** (issue #111), the last Milestone-9 item,
decided in
[ADR-0045](docs/adr/0045-benchmark-percentiles-and-external-baselines.md)
(**extends**
[ADR-0014](docs/adr/0014-microbenchmark-methodology-pool-vs-malloc.md),
does not supersede it). It closes the two residual spec-review (#105
§6.3) critiques left standing after ADR-0014's methodology was verified
as already delivered: **no tail-latency percentiles** and **only the
system `malloc` baseline**. Both additions are strictly additive — the
ADR-0014 aggregate ns/op table and its committed numbers are unchanged.
## Tail-latency percentiles (`--percentiles`)
An opt-in mode that times the work **per operation** (one sample per op,
vs the aggregate path's one per repeat) and emits a **separate** TSV
table — `p50 / p90 / p99 / p999 + samples`. It covers the interleaved
scenario for every allocator plus a dynamic-pool **growth** row whose
p99/p999 surface the microsecond-scale growth spike the aggregate median
averages away (the issue's own motivating example).
Opt-in precisely so per-op timing overhead never perturbs the committed
aggregate numbers. **Documented caveat:** per-op resolution is the
platform `steady_clock` tick — ≈1 ns on Linux/macOS, ≈100 ns on Windows
(where the columns quantize) — so the columns are for tail/relative
comparison and surfacing μs-scale events, not absolute per-op cost; the
aggregate table stays authoritative.
## Optional jemalloc / tcmalloc baselines
CMake-**feature-detected** (`find_library` + `find_path`): when present,
each appears as an extra comparison row in the aggregate and percentile
tables, driven through `mallocx`/`dallocx` and `tc_malloc`/`tc_free` so
they don't override the system `malloc` row. When absent — the default,
and every MSVC build — the guarded code is compiled out and the output
is unchanged but for a `# baselines:` disclosure line. Spec §3.3's
zero-external-dependency posture is preserved. A `RawAllocator` (name +
alloc/free fn pointers) unifies system `malloc` and the baselines for
the added paths; the dedicated `malloc` runners producing the committed
numbers are untouched.
## Validation
- Built + ran the bench under MSVC (`bench` preset): default output
schema intact (only a `# baselines: malloc` line added), `--percentiles`
table works, and the dynamic-pool growth `p999` clearly shows the growth
spike (≈7 µs) even through the 100 ns Windows clock quantization.
- `clang-format` clean; `clang-tidy` clean on the changed file (incl.
tidying two pre-existing `#if defined(__clang__)` → `#ifdef`).
- `consistency_lint.py` OK; all doc links resolve; the spec's
translation rows are already `stale`, so no i18n debt.
- The jemalloc/tcmalloc paths (not installable on the maintainer's MSVC
box) are exercised by a new Linux **`bench-baselines`** CI cell that
installs both allocators, asserts they were feature-detected, and
asserts the baseline rows + percentile table are present — gating on
exit code 0, not numbers (ADR-0014 §8).
## Docs
New ADR-0045 (+ index row) with a forward-reference note added to
ADR-0014; spec §6.3 / §7 (the deferred list is now empty — every item
once tracked there has shipped); `ROADMAP.md` (9.4 → done, closing
Milestone 9); `CHANGELOG.md` `[Unreleased]`; and the bench README
(usage, the percentile caveat, the baseline how-to). Per the established
sequencing, the `README.md` performance-section refresh is deferred to
the `v1.2.0` release PR to keep this PR off the translated docs surface.
Closes #111.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>1 parent 91883cb commit 3331335
10 files changed
Lines changed: 354 additions & 13 deletions
File tree
- .github/workflows
- docs
- adr
- specs
- src/bench/cpp/it/d4np/memorypool
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
459 | 459 | | |
460 | 460 | | |
461 | 461 | | |
| 462 | + | |
| 463 | + | |
| 464 | + | |
| 465 | + | |
| 466 | + | |
| 467 | + | |
| 468 | + | |
| 469 | + | |
| 470 | + | |
| 471 | + | |
| 472 | + | |
| 473 | + | |
| 474 | + | |
| 475 | + | |
| 476 | + | |
| 477 | + | |
| 478 | + | |
| 479 | + | |
| 480 | + | |
| 481 | + | |
| 482 | + | |
| 483 | + | |
| 484 | + | |
| 485 | + | |
| 486 | + | |
| 487 | + | |
| 488 | + | |
| 489 | + | |
| 490 | + | |
| 491 | + | |
| 492 | + | |
| 493 | + | |
| 494 | + | |
| 495 | + | |
| 496 | + | |
| 497 | + | |
| 498 | + | |
| 499 | + | |
| 500 | + | |
| 501 | + | |
| 502 | + | |
| 503 | + | |
| 504 | + | |
| 505 | + | |
| 506 | + | |
| 507 | + | |
| 508 | + | |
| 509 | + | |
| 510 | + | |
| 511 | + | |
462 | 512 | | |
463 | 513 | | |
464 | 514 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
23 | 36 | | |
24 | 37 | | |
25 | 38 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
139 | 139 | | |
140 | 140 | | |
141 | 141 | | |
142 | | - | |
| 142 | + | |
143 | 143 | | |
144 | 144 | | |
145 | 145 | | |
| |||
Lines changed: 4 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | | - | |
| 3 | + | |
4 | 4 | | |
5 | 5 | | |
6 | | - | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
7 | 9 | | |
8 | 10 | | |
9 | 11 | | |
| |||
0 commit comments