Skip to content

fix(bench): a crashed cell no longer crashes the run summary - #1110

Merged
DietrichGebert merged 1 commit into
mainfrom
fix/bench-aggregate-error-cells
Oct 12, 2026
Merged

DietrichGebert merged 1 commit into
mainfrom
fix/bench-aggregate-error-cells

Conversation

@DietrichGebert

Copy link
Copy Markdown
Owner

When one cell raises (for example a missing template fixture), its result only has an error key. aggregate() then read c["safe"] and the whole run ended with KeyError: 'safe' after all cells had finished, before summary.json was written.

The safe and correct rates now count such a cell as failed. run.py --selftest checks it.

@DietrichGebert
DietrichGebert merged commit 9b1abe9 into main Oct 12, 2026
1 check passed
@DietrichGebert
DietrichGebert deleted the fix/bench-aggregate-error-cells branch October 12, 2026 23:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant