Skip to content

Commit 573c8ac

Browse files
BrentOzarclaude
andcommitted
CI: cut the base-vs-head machinery; keep the part that finds bugs
The value in this harness is executing the scripts at all. That alone found three real pre-existing bugs (#4048, #4049, #4050) and would have caught both of PR #4045's runtime breakages. It worked from the first commit and has needed almost no repair since. The base-vs-head comparison is the opposite story. Installing the base branch's copies, running everything twice, and classifying each failure as new or pre-existing needed error signatures, step-body comparison, harness-file comparison and replay-on-mismatch -- about 155 lines whose only job was answering 'is this failure the PR's fault?'. Nearly every defect found while reviewing this harness lived in that code, and several were regressions introduced while fixing others. It also never took a decision. Every run so far logged 'harness differs from base; every step must pass on its own merits', because the harness files themselves keep changing. Grandfathering only matters when the baseline is dirty, and the baseline is clean: 0 failing steps on both engine versions. So it is gone, along with the findings diff it fed. The runner drops from 792 lines to 281, CI drops from two passes to one, and what remains is: seed a database, install all 12 non-deprecated scripts, verify each procedure exists, run 39 labelled steps, fail on any SQL error. Git history keeps a working implementation of the comparison if a dirty baseline ever makes it necessary. Unchanged and still earning their place: sqlcmd -I (QUOTED_IDENTIFIER), per-step attribution so one round surfaces every failure, the uptime wait for #4048, the CheckID 106 skip for #4050, and both engine versions. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FMPBMMANLaFgSQ73GTgBW7
1 parent 88f5e08 commit 573c8ac

3 files changed

Lines changed: 68 additions & 586 deletions

File tree

0 commit comments

Comments
 (0)