Skip to content

Commit 3db669b

Browse files
committed
Run a whole review against the fixture and score it against the answer key
Work item W7: the demo that makes this app judgeable before it has a screen. It builds two git repositories with known defects planted in them, reviews the branch pair for real, and prints which of those defects the review actually found. The manifest is the answer key, so the output is a score rather than an impression: every planted defect is FOUND or MISSED, and any finding in the files that are deliberately fine is a false positive named as such. `--fake` routes every model call through the test double, so the whole demo is free and deterministic. That mode proves the plumbing carries a correct review end to end; it proves nothing about review quality, which is what a real run is for. `npm run demo:fixture -- --fake` currently prints: Built the fixture: 8 planted defect(s), 2 file(s) that are deliberately fine. ... 8/8 planted defect(s) found. No findings in the files that are deliberately fine. outcome completed discarded quotes 0 open questions 0 tokens 500 in, 100 out, 10,000 cached read cost equivalent $0.0210 wall time 0.9s Step zero of the plan was to prove tsx resolves the @/ alias rather than assume it. It does, so the script runs directly and the tsconfig.scripts.json fallback the plan held in reserve is not needed. Two smaller decisions the output forced. Numbers are grouped by hand rather than by toLocaleString, because this output is captured as gate evidence and a locale-dependent separator would make two runs on two machines look different when nothing about the review changed. And there is no colour: escape sequences are control characters, which the house style gate refuses, and the output is written to a file where escapes are noise rather than emphasis. A real model run writes its event log and score to review/<date>-fg2/, which is gitignored, and is the maintainer's to judge.
1 parent 5199e6c commit 3db669b

4 files changed

Lines changed: 390 additions & 2 deletions

File tree

docs/DECISIONS.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -604,3 +604,13 @@ verified evidence, in writing, here.
604604
prompt hash. Editing it after a review has started means the stages are asked
605605
again rather than replayed, which is correct: a review judged against a
606606
different account of the change is a different review.
607+
- 2026-07-31 PROVEN (W7 step zero): tsx resolves the `@/` alias from
608+
tsconfig, so the demo script runs directly with no build step and the
609+
`tsconfig.scripts.json` fallback the plan held in reserve is not needed.
610+
- 2026-07-31 DECIDED (W7): the demo prints numbers with its own grouping rather
611+
than toLocaleString. The output is captured as gate evidence, and a
612+
locale-dependent separator would make two runs on two machines look
613+
different when nothing about the review changed.
614+
- 2026-07-31 DECIDED (W7): no colour in the demo output. Escape sequences are
615+
control characters, which the house style gate refuses, and the output is
616+
written to a file as evidence where escapes are noise rather than emphasis.

docs/plans/M2-FINISH-PLAN.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -401,7 +401,7 @@ maintainer at the top of the demo output directory:
401401
| W4 | review service, artifact lifecycle, its tests | W1, W2, W3 | DONE |
402402
| W5 | pause/resume/cancel determinism tests | W4 | DONE |
403403
| W6 | bus, manager, instrumentation, SSE route, tests | W4 | DONE |
404-
| W7 | demo script, tsx alias proof, npm script | W2, W6 | |
404+
| W7 | demo script, tsx alias proof, npm script | W2, W6 | DONE |
405405
| W8 | docs, gate evidence, FG-2 checklist | W7 | |
406406

407407
## 3. Decisions fixed by this plan (append to DECISIONS.md as work lands)

package.json

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,8 @@
2828
"test": "vitest run",
2929
"test:watch": "vitest",
3030
"e2e": "playwright test",
31-
"db:generate": "drizzle-kit generate"
31+
"db:generate": "drizzle-kit generate",
32+
"demo:fixture": "tsx scripts/demo-fixture.ts"
3233
},
3334
"overrides": {
3435
"postcss": "^8.5.25",

0 commit comments

Comments
 (0)