Skip to content

Commit 96a25b2

Browse files
committed
fix(test-harness): run extraction in the quiet tail, not the 18-job wave
The wide-flat SCALING-RATIO guard grows the input 20x and asserts the time grows ~20x (linear) rather than ~128x (quadratic), with the bound at 40x between them. Contention does not cancel out of that ratio: the 400k-node measurement loses far more to memory pressure and scheduling than the 20k one, so oversubscription inflates the ratio itself. Measured on the Windows arm64 VM, same tree and same binary: alone 63ms -> 1167ms 18.5x passes in the 18-job wave 168ms -> 9045ms 53.8x fails 163ms -> 9019ms 55.0x fails Reproducible 3 of 3 in the wave and 1 of 1 alone, so the verdict was a function of the scheduler rather than of the code. The suite is ~22s; running it alone is cheap next to a ~40min ladder. The bound is deliberately NOT widened. The calibration note in tests/test_extraction.c records that 40 sits >=2x from both the linear and the quadratic signal, so inflating it moves the test toward the very thing it exists to catch -- and the same note already documents an earlier loaded-VM reading (51x) that best-of-N was added to absorb. Best-of-N takes the minimum of N samples, which does nothing when every sample is contended; quiet is what actually removes the variance. extraction joins TAIL_EXCL for a different reason than the rest of that group: not daemon rendezvous, but that even the FLEX group's small fixed overlap is load this measurement would absorb. Both comments say so. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
1 parent db91b88 commit 96a25b2

1 file changed

Lines changed: 18 additions & 1 deletion

File tree

‎scripts/run-tests-parallel.sh‎

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -168,7 +168,19 @@ shard_filter() {
168168
# deadlines into deterministic failures while an idle machine passes 6/6.
169169
# They also all rendezvous through the shared per-account runtime namespace,
170170
# which the quiet tail keeps free of cross-suite admission traffic.
171+
# extraction carries the wide-flat SCALING-RATIO guard, which grows the input
172+
# 20x and asserts the time grows ~20x (linear) rather than ~128x (quadratic),
173+
# with a bound of 40x between them. Contention does not cancel out of that
174+
# ratio: the 400k-node measurement loses far more to memory pressure and
175+
# scheduling than the 20k one, so oversubscription inflates the ratio itself.
176+
# Measured on the Windows arm64 VM: 18.5x alone (passes) vs 53.8x and 55x in
177+
# the 18-job wave (fails) — reproducible, 3 of 3. The bound is deliberately NOT
178+
# widened; see the calibration note in tests/test_extraction.c, which records
179+
# that 40 sits >=2x from both the linear and quadratic signals, so inflating it
180+
# would move the test toward the very thing it exists to catch. Quiet is the
181+
# fix, and at ~22s the suite is cheap to run alone.
171182
SERIAL_SUITES="cli subprocess watcher incremental httpd ui index_resilience mcp \
183+
extraction \
172184
stack_overflow_a stack_overflow_b stack_overflow_c \
173185
index_supervisor daemon_application daemon_runtime daemon_frontend \
174186
daemon_bootstrap daemon_ipc"
@@ -245,8 +257,13 @@ run_wave "$PAR_FILE" "$JOBS"
245257
# time on an idle machine. The EXCL group (daemon-family plus the suites
246258
# that drive daemon one-shots or supervisor rendezvous) then runs strictly
247259
# sequentially on a machine exactly as quiet as the old tail gave it.
260+
# extraction is in this group for a DIFFERENT reason than the rest: it does not
261+
# rendezvous through the daemon namespace, it measures a scaling ratio, and even
262+
# the FLEX group's small fixed overlap is load the measurement would absorb.
263+
# Strictly sequential is what makes its verdict a function of the code instead
264+
# of the scheduler.
248265
TAIL_EXCL="cli mcp index_supervisor daemon_application daemon_runtime \
249-
daemon_frontend daemon_bootstrap daemon_ipc"
266+
daemon_frontend daemon_bootstrap daemon_ipc extraction"
250267
is_tail_excl() {
251268
case " $TAIL_EXCL " in *" $1 "*) return 0 ;; *) return 1 ;; esac
252269
}

0 commit comments

Comments
 (0)