Repository navigation
refactor(e2e): parse the fuzz summary table with the mdast GFM table stack - #566
Merged
Merged
Conversation
The fuzzer's summaryTableProblems found the step summary's table by hand (the first run of lines opening with a pipe), counted each row's unescaped pipes with a backslash-run state machine, checked the delimiter row against a regex, and compared the pipe counts. The mdast GFM table stack already in the repository (mdast-util-from-markdown with micromark-extension-gfm-table and mdast-util-gfm-table, as generated-regions.ts uses it) reads the table as GitHub renders it: it recognizes the table only when the header and delimiter row agree, treats an escaped pipe as cell text and a pipe after an escaped backslash as a delimiter, and keeps each row's real cell count. What remains compares each row's cell count to the header's and reports a missing table, which a malformed delimiter row now also becomes. Differential check, the old function against the new over engine-rendered summaries: 700 generated through writeSummary, writeMultiSummary, writeSnapshotDirSummary, and writeRenderSummary with cells drawn from the hostile names plus raw pipes, lone and trailing backslashes, code spans, and blank cells; 200 real summaries from generated fuzz scenarios run through the runner with the oracle's predicted section count; 62 real summaries from a sample of the curated corpus. All 962 agree on the verdict and on the problem kinds. Of 75 hand-made malformed inputs and 10501 mutations (a delimiter row dropped or emptied, an escape broken, a row dropped, a cell appended, the table removed or indented, a leading pipe stripped, a raw pipe inserted, a blank line inserted, the text truncated), the verdicts agree on 8580 mutations, and every split is a shape the engine cannot write: an indented table, a row without its leading or trailing pipe, text glued to the table, or a multi-target summary with its first table destroyed. The fuzz passes this function single-run summaries alone. The diagnostics change wording: a malformed delimiter row reads "summary carries no markdown table" rather than naming the separator row, the row-count message counts data rows, and the cell-count message numbers the data row. No engine-produced summary changes verdict.
Contributor
File size check0 over a hard cap (fails), 6 warning(s).
Split the file, wrap the line, shorten or exempt the comment, or list the path in 5 managed file(s) skipped; repo-platform owns them. |
There was a problem hiding this comment.
🟢 Approval recommended
The focused test-only refactor preserves the intended validation checks, with no unresolved defects identified.
No open issues 🎉
What changed in this PR
Replaces the end-to-end fuzzer's custom Markdown table parser with the existing mdast GFM table stack, aligning summary validation with GFM parsing.
Changes:
- Parses the first table and checks its data-row count and cell widths against expectations.
- Updates diagnostics for missing tables, insufficient rows, and mismatched cell counts.
| File | Description |
|---|---|
test/e2e/fuzz.ts |
Uses mdast GFM parsing for summary-table validation and updates diagnostics. |
Review effort: Balanced
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
How
.github/scripts/lib/generated-regions.ts.Proof
Technical details
fuzz.tsrunsmain()at import, so its functions cannot be imported; a follow-up could guard it withimport.meta.mainand pin the excess-cell fact.test/e2e/fuzz.ts.