Skip to content

Commit 275edc9

Browse files
committed
perf(benchmark): record startup separately [skip jenkins] [skip ci]
1 parent 6e3e691 commit 275edc9

4 files changed

Lines changed: 10 additions & 7 deletions

File tree

‎docs/LINUX_BENCHMARK_RUNBOOK.md‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -65,8 +65,9 @@ The manifest currently selects from:
6565
- repeated `status`;
6666
- optional `cross_repo` indexing for the companion repositories.
6767

68-
Each repeated read operation receives one unrecorded warm-up, followed by the configured number of
69-
measured runs. Daemon startup is kept outside steady-state read timings.
68+
Daemon startup is recorded as its own `startup` case. Indexing is recorded separately, and each
69+
repeated read operation receives one unrecorded warm-up followed by the configured number of
70+
measured runs. This separates cold startup, indexing, and warm action timings.
7071

7172
A required warm-up or measured invocation failure fails that repository. Failed invocations remain
7273
in `timings.tsv` and are also written to `FAILURES.txt`; they are not silently removed from the

‎docs/PERFORMANCE_BASELINE.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -164,7 +164,7 @@ CBM_BENCH_REPEATS=7 \
164164

165165
Each result directory contains:
166166

167-
- `timings.tsv` — every recorded operation/run, elapsed milliseconds, and exit code;
167+
- `timings.tsv` — startup, indexing, warm action operation/run timings, and exit codes;
168168
- `summary.tsv` — successful-run min/median/max by operation;
169169
- `environment.txt` — binary/repository/harness/host metadata;
170170
- raw stdout/stderr for every recorded case;

‎scripts/benchmark-agent-workflows.ps1‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -46,8 +46,9 @@ function Invoke-Repeat([string]$Label,[string[]]$Args) {
4646
function Wants([string]$Name) { return $Operations -contains $Name }
4747

4848
& $Binary daemon stop *> $null
49-
& $Binary daemon start *> $null
50-
if ($LASTEXITCODE -ne 0) { throw 'daemon start failed before benchmark workload' }
49+
[void](Invoke-CbmCase 'startup' 1 @('daemon','start'))
50+
$startup = @(Import-Csv $timings -Delimiter "`t" | Where-Object { $_.case -eq 'startup' -and $_.exit_code -eq '0' }).Count
51+
if ($startup -ne 1) { throw 'daemon start failed before benchmark workload' }
5152
try {
5253
if (-not (Wants 'index')) { throw 'benchmark operations must include index' }
5354
$indexRc=Invoke-CbmCase 'index' 1 @('index',$Repo,'--mode','full','--json')

‎scripts/benchmark-agent-workflows.sh‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -92,9 +92,10 @@ repeat_case() {
9292
cleanup() { "$BINARY" daemon stop >/dev/null 2>&1 || true; }
9393
trap cleanup EXIT
9494

95-
# Keep daemon startup out of steady-state read timings.
95+
# Record daemon startup separately; steady-state action timings begin afterward.
9696
"$BINARY" daemon stop >/dev/null 2>&1 || true
97-
if ! "$BINARY" daemon start >/dev/null; then
97+
run_once startup 1 "$BINARY" daemon start || true
98+
if ! awk -F '\t' 'NR>1 && $1=="startup" && $4==0 {ok=1} END{exit ok?0:1}' "$RESULTS_DIR/timings.tsv"; then
9899
echo "error: daemon start failed before benchmark workload" >&2
99100
exit 3
100101
fi

0 commit comments

Comments
 (0)