Skip to content

Commit 7f5043f

Browse files
JuliaEdomclaude
andcommitted
fix(eval): sync machine-readable artifacts with the headline; add consistency gate
The audit found the machine and the README disagreeing: - v31 report's top-level `overall` said 185/0.925 while its own records recomputed to 188/0.94 — refreshed to 188/0.94 (records are the source of truth). - GET /eval/latest hardcoded a stale 60.5% hybrid baseline. It now serves a new committed reproducible baseline (eval/baselines/reproducible_n200.json: config E, codestral, no voting, no hints = 57.5% EA / 115 of 200), i.e. the number this API's own pipeline actually achieves. - FastAPI description "57% hybrid" → honest reproducible/hint-assisted split. - runtime.txt python-3.12 → 3.13 (CI/ruff/mypy already target 3.13). New regression test pins the invariant that every canonical report's `overall` equals the recomputation from its records[], so summaries can't silently drift again. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 742647e commit 7f5043f

5 files changed

Lines changed: 6943 additions & 6 deletions

File tree

0 commit comments

Comments
 (0)