Skip to content

Commit 5af40c4

Browse files
copeusclaude
andcommitted
spec: a metric overclaimed and a stale present tense, both caught by the ADR job
'100% of real jobs ran without a mechanical scope gate / 73 of 73 recorded jobs' is not reproducible as phrased. No state.json in this repo records a backend at all and only 6 of 12 run dirs have one, so the figure counts jobs AS DECLARED IN MANIFESTS. That supports 'no non-claude backend was ever scheduled' -- the claim the design actually needs -- but not '73 jobs each escaped the gate'. Reworded, with what the number is stated beside it. B5 still said none of tests/ is executed by CI 'on this branch'. Task-0 landed that sweep an hour ago: true about the past, false as present tense. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent 1488ef2 commit 5af40c4

1 file changed

Lines changed: 2 additions & 2 deletions

File tree

docs/superpowers/specs/2026-09-01-v3.0-triage-tests-orchestration-design.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ Three defects, confirmed against this repository, none of them the one we assume
1212

1313
**Small work is not over-processed — it escapes entirely.** All 10 recorded runs served releases of ≥788 changed lines. All 10 releases under 200 lines ran with no manifest and no run directory. The measured failure is an unrecorded bypass, not excess ceremony.
1414

15-
**Nothing scopes tests, and 100% of real jobs ran without a mechanical scope gate.** The manifest has no test field; `agents/spec-reviewer.md:138` demands the whole suite unconditionally. 73 of 73 recorded jobs ran `backend: claude` — the one backend where the scope gate is prose the dispatcher must remember to type.
15+
**Nothing scopes tests, and every job ever scheduled ran on the backend whose scope gate is prose.** The manifest has no test field; `agents/spec-reviewer.md:138` demands the whole suite unconditionally. All 73 jobs declared across the historical run manifests carry `backend: claude` — the one backend where the scope gate is prose the dispatcher must remember to type. **What that number is:** a count of jobs *as declared in manifests* — no `state.json` in this repo records a backend at all, and only 6 of 12 run dirs have one. It supports "no non-`claude` backend was ever scheduled", not "73 jobs each escaped the gate".
1616

1717
**Evidence limit, stated plainly.** The maintainer's "~20,000 tests for a one-word change" was not reproduced by any lane: no downstream repository, CI config or recorded measurement in this repo exhibits it. **Feature B's defaults are therefore chosen on principle, not on measurement.** They must be re-derived against a real instance before anyone claims a measured improvement.
1818

@@ -187,7 +187,7 @@ Both are in scope:
187187

188188
"The full suite still runs in CI before merge" is fragile as stated: GitHub reports a conditionally-skipped job as `Success`, so a path filter can turn a required check green without running it. The backstop must be a job that always runs and is required, dispatching internally.
189189

190-
Separately, and independently of this feature: **none of the three files under `tests/` is executed by CI on this branch** — the only sweep is `for s in scripts/*.py`. Any test written for Feature C would land dead on arrival. That is the v2.14.1 false-green shape recurring and is fixed here.
190+
Separately, and independently of this feature: until task-0 landed, **no file under `tests/` was executed by CI on this branch** — the only sweep was `for s in scripts/*.py`. Any test written for Feature C would land dead on arrival. That is the v2.14.1 false-green shape recurring and is fixed here.
191191

192192
### B6. The ADR
193193

0 commit comments

Comments
 (0)