Tighten the visual tolerance, chosen by measurement

`maxDiffPixelRatio: 0.01` sounds small and is not. On a 1240x923 baseline it
permits 11,445 pixels to differ — a whole axis label, a moved legend, a bar that
changed height. It hid real changes three times: a reference line's label was
repositioned and the baseline did not regenerate, so the fix looked like it had
failed; 13 of 30 baselines turned out stale, one where every y-axis tick had
drifted from 12/24/36/48 to 10/20/30/40/50 at 0.986% of pixels; and before that,
a full change of axis label text from ISO dates to "Jan 3" passed as identical.

My first instinct was 0.001. Measuring showed that was still mostly blind.

Nudging every category label down two units — the kind of thing a refactor does
by accident — changed nine charts by between 339 and 10,505 pixels:

    maxDiffPixelRatio 0.01    allows 11,445    caught 0 of 9
    maxDiffPixelRatio 0.001   allows  1,144    caught 2 of 9
    maxDiffPixels 100         allows    100    caught 9 of 9

So the fix is not a smaller ratio but a different shape. A ratio hands the
largest images the largest blind spot, and those are the charts with the most to
get wrong — the 10,505-pixel change slipped under 1% only because that baseline
is big.

100 absolute pixels still absorbs an antialiasing fringe. Four consecutive runs
pass on unchanged code, and the whole suite of 36 passes.

The reasoning is recorded in playwright.config.ts, including the instruction to
raise it deliberately if an upgrade makes it noisy rather than reaching for a
ratio again. The Known gaps entry in ROADMAP is closed and CLAUDE.md's testing
convention updated.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Author: LarryLustig

CHANGELOG.md

@@ -7,6 +7,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Changed
+
+- **Visual baselines now allow 100 differing pixels, absolute, instead of 1% of the image.** A
+  ratio sounds small and is not: 1% of a 1240x923 baseline is 11,445 pixels — a whole axis label
+  or a moved legend. It hid real changes three times, including one where every y-axis tick
+  drifted from 12/24/36/48 to 10/20/30/40/50 at 0.986% of pixels, just under the threshold
+
+  Chosen by measurement rather than instinct. Against a deliberate 2-unit nudge to every category
+  label, which changed nine charts by 339 to 10,505 pixels: the old `0.01` caught none of them,
+  `0.001` caught two, and 100 absolute pixels caught all nine. A ratio is the wrong shape —
+  it gives the largest images the largest blind spot, and those are the charts with the most to
+  get wrong. Four consecutive runs pass on unchanged code
+
 ### Added
 
 - **`DC116`** — bars that do not fit the plot are no longer silent. Bar width has a floor of one

CLAUDE.md

@@ -216,9 +216,11 @@ Helpers live in each directory's `setup.ts`.
   mutation tests will pass against broken behaviour.
 - **Do not call `requestUpdate()` in tests** after changing markup. The chart
   observes its own children; adding one masks a regression in that observer.
-- **Visual baselines: `maxDiffPixelRatio` is 0.01**, which has twice hidden real
-  render changes. When a change should alter a chart, regenerate at zero
-  tolerance and compare deliberately.
+- **Visual baselines allow 100 differing pixels**, absolute rather than a ratio.
+  The old `maxDiffPixelRatio: 0.01` permitted 11,445 on a typical baseline and
+  hid real changes three times. If an upgrade makes this noisy, raise it
+  deliberately and record why in `playwright.config.ts` — do not go back to a
+  ratio, which gives the largest charts the largest blind spot.
 - **The Playwright suite is split.** `charts.spec.ts` compares screenshots and
   runs locally only — the baselines are `-chromium-win32` and no CI runner
   reproduces them. It is bound into `prepublishOnly` so a release cannot skip

ROADMAP.md

@@ -363,10 +363,12 @@ trying the library for the first time.
 Not features — things that are wrong or thin, recorded so they are not
 rediscovered:
 
-- **Visual tolerance is loose.** `maxDiffPixelRatio: 0.01` let a full change of
-  axis label text (ISO dates → `Jan 3`) pass as identical. The baseline had to
-  be deleted to force regeneration. Tightening it risks font-rendering
-  flakiness; worth a deliberate decision either way.
+- ~~**Visual tolerance is loose.**~~ Fixed: `maxDiffPixelRatio: 0.01` allowed
+  11,445 pixels to differ on a typical baseline and hid three separate real
+  changes. Replaced with `maxDiffPixels: 100`, an absolute allowance — a ratio
+  gives the largest images the largest blind spot. Measured against a deliberate
+  label nudge: the old setting caught 0 of 9 affected charts, the new one 9 of 9,
+  with four consecutive clean runs on unchanged code.
 - **`stage-chart.ts` at 88%** is the thinnest real coverage left, mostly in
   `calculateTextFit`.
 - **Outreach has not happened.** docs/review.md §8 argues discovery is the binding

playwright.config.ts

@@ -83,14 +83,38 @@ export default defineConfig({
   // Snapshot configuration
   expect: {
     toHaveScreenshot: {
-      // Allow small pixel differences (anti-aliasing, font rendering)
-      maxDiffPixelRatio: 0.01,
+      /*
+       * An absolute allowance, not a ratio.
+       *
+       * This was `maxDiffPixelRatio: 0.01`, which sounds small and is not: 1% of
+       * a 1240x923 baseline is 11,445 pixels — a whole axis label, a moved
+       * legend, a bar that changed height. It hid real changes twice. A
+       * reference line's label was repositioned and the baseline did not
+       * regenerate, so the fix looked like it had failed; and 13 of 30 baselines
+       * turned out to be stale, including one where every y-axis tick had drifted
+       * from 12/24/36/48 to 10/20/30/40/50 at 0.986% of pixels.
+       *
+       * Measured against a deliberate 2-unit nudge to every category label, which
+       * changed nine charts by 339 to 10,505 pixels:
+       *
+       *   ratio 0.01   allows 11,445   caught 0 of 9
+       *   ratio 0.001  allows  1,144   caught 2 of 9
+       *   100 pixels   allows    100   caught 9 of 9
+       *
+       * A ratio is the wrong shape here: it hands the largest images the largest
+       * blind spot, and those are the charts with the most to get wrong. 100
+       * absolute pixels still absorbs an antialiasing fringe, and four
+       * consecutive runs pass with no code change.
+       *
+       * If a Playwright or Chromium upgrade makes this noisy, raise it
+       * deliberately and say so here — do not reach for a ratio again.
+       */
+      maxDiffPixels: 100,
       // Animation threshold
       animations: 'disabled',
     },
     toMatchSnapshot: {
-      // Threshold for image comparison
-      maxDiffPixelRatio: 0.01,
+      maxDiffPixels: 100,
     },
   },