Skip to content

Commit 7809756

Browse files
authored
fix: give three tests distinct screenshot names, so a failure keeps its evidence (#290)
`index-without-img-cropped` was asserted by THREE tests, not the two #284 reported: the `color_distance_limit` case (:117), `crop accepts css selector` (:129), and the `skip_area: "input"` case (:200). A differing screenshot is written to its baseline path by design, so when one of them fails the next re-captures the same name, matches, and leaves the file clean. The failure's own evidence is gone before artifacts upload. Measured on the real failing run 32740082766, artifact `screenshots-selenium_headless-chunky_png-diffs`: the job reported a **26.06% difference** and its uploaded `.base.png`, `.png` and the committed baseline were all `5fccb56f4a47d41b3a8c8cdf38531da6`. Anyone opening that artifact to debug the flake finds two identical PNGs and concludes it was spurious. That is why #285 has never been diagnosed. All three visit `/index-without-img.html` and crop `"form"`, so they share one rendering -- which is exactly why the collision went unnoticed for so long. Each renamed test therefore gets a copy of that same baseline in all six os/driver directories; nothing is re-recorded and no comparison changes. The `color_distance_limit` test keeps the original name so its baseline history stays continuous. NOT verified locally, and deliberately not claimed to be: this file reports `0 verified -- NOTHING WAS VERIFIED` on this machine, and does so on unmodified master too, so a local green here is not evidence. `Test Drivers` is where these baselines actually compare; this PR carries `full-ci` for that reason.
1 parent 9987b43 commit 7809756

13 files changed

Lines changed: 2 additions & 2 deletions
928 Bytes
Loading
928 Bytes
Loading
1.73 KB
Loading
1.73 KB
Loading
1.88 KB
Loading
1.88 KB
Loading
3.6 KB
Loading
3.6 KB
Loading
3.16 KB
Loading
3.16 KB
Loading

0 commit comments

Comments
 (0)