build(scripts): guard the generated markdown tables with a round trip instead of a regex

The sections-table and policy-defaults-table regions admitted authored text their renderers never write, so a hand edit inside the markers was erased on the next regeneration instead of refused. The sections-table shape used `[^\n]*` cells and admitted a pipe, a carriage return, a sixth cell, a blank cell, a repeated section with other prose, and an Undeclared default outside the three displays. The policy-defaults-table shape admitted an override naming the same policy as the default, a keep row above a delete row, and a repeated section.

Both shapes are now one shared round trip, `tableRoundTrip` in lib/markdown-table.ts: the body splits into rows and cells by the inverse of `tableRow()`, the rows pass the table rule (the header's column count, the cell rule `tableCell()` writes by, a key span opening every row, no key repeated), a per-table parser turns the cells into the record the table renders from, and the records re-rendered must reproduce the body byte for byte. The refusal names the first differing line through `renderedMismatch`, now shared with the action.yml round trip. The Defaults table re-renders through its real renderer, so the delete-first order comes from the renderer, and its parser checks the opposite-policy binding through the same `OPPOSITE` table the renderer writes from. The Sections table re-renders through the table helper with the prose cells opaque, since its PAT cell paraphrases prose no parser inverts.

The helper is the grammar on both sides. `tableFault()` is the one statement of the table rule: `renderTable()` throws its text and `tableRoundTrip()` refuses by the same rule, so a renderer cannot write a body its own guard would refuse, which would otherwise turn the next regeneration of the committed page into a throw. The Defaults statement `policyTableFault()` adds each row's prose by `proseFault`, the trimmed-blank rule `tableCell()` uses, which the parser consults too, since blank prose renders `delete ()` or a bare policy span that the cell rule admits. A compile-time proof of non-blank prose was tried and dropped: a structural type cannot keep a hidden property out of a parameter, so only the run-time check closes the boundary.

Two property-style tests hold both sides to the statement over generated inputs (keys inside and outside the key span, both policies, prose with blanks, padding, pipes, carriage returns, and a Unicode separator, duplicate and reordered sections): the renderer throws exactly the statement's fault, the guard admits what the renderer wrote, and the guard refuses the same cells joined without the renderer whenever the statement faults, so a check added to one side only turns a generated input red. A rule test in each registry's suite refuses a RegExp body on any region whose rendering is a table. Refusal tables for both regions pin each input above: nine were admitted by the regexes on main, the rest were refused with the generic message and now name the cell or line.

Author: Vivswan

.github/scripts/gen-action-docs.ts

@@ -30,9 +30,14 @@ import {
 } from "../../src/sections/contract/permissions.js";
 import { SECTIONS } from "../../src/sections/registry.js";
 import { agree } from "../../src/text.js";
+import type { UndeclaredPolicy } from "../../src/types.js";
 import { countWord } from "./lib/count-word.js";
-import { type GeneratedRegion, regenerateRegions } from "./lib/generated-regions.js";
-import { tableCell } from "./lib/markdown-table.js";
+import {
+  type GeneratedRegion,
+  regenerateRegions,
+  renderedMismatch,
+} from "./lib/generated-regions.js";
+import { renderTable, tableFault, tableRoundTrip } from "./lib/markdown-table.js";
 
 const ROOT = join(import.meta.dir, "..", "..");
 
@@ -174,30 +179,102 @@ export interface PolicyRowProse {
   readonly override: string;
 }
 
-const DEFAULTS_TABLE_HEADER = "| Section | Default | The override buys you |\n|---|---|---|";
+/**
+ * Why a row's prose cannot be rendered: blank prose would render `delete ()` or a bare policy span, cells
+ * tableCell() admits since the policy word fills them. The renderer refuses it at its own boundary and the parser
+ * through this same check, by the trimmed-blank rule tableCell() uses, so what the renderer writes is exactly what
+ * the guard reads back. A structural type cannot vouch for a property a narrowing hid, so the value is inspected
+ * as unknown.
+ */
+function proseFault(row: PolicyRowProse): string | undefined {
+  const blank = (value: unknown): boolean => typeof value !== "string" || value.trim() === "";
+  return blank(row.override) || (row.caveat !== undefined && blank(row.caveat))
+    ? "has a blank caveat or override"
+    : undefined;
+}
+
+const OPPOSITE: Readonly<Record<UndeclaredPolicy, UndeclaredPolicy>> = {
+  delete: "keep",
+  keep: "delete",
+};
+
+export const DEFAULTS_TABLE_HEADER = "| Section | Default | The override buys you |\n|---|---|---|";
+
+/** One Defaults row: the section and the prose its cells carry. */
+export interface PolicyRow {
+  readonly section: KnobbedSection;
+  readonly prose: PolicyRowProse;
+}
+
+/** A row's three cells, as the renderer writes them and the guard reads them back. */
+export function policyCells(row: PolicyRow): readonly string[] {
+  const caveat = row.prose.caveat === undefined ? "" : ` (${row.prose.caveat})`;
+  return [
+    `\`${row.section.key}\``,
+    `${row.section.undeclaredDefault}${caveat}`,
+    `\`${OPPOSITE[row.section.undeclaredDefault]}\`: ${row.prose.override}`,
+  ];
+}
+
+/**
+ * The one statement of what a Defaults table may hold: each row's prose by proseFault(), then the table rule over
+ * the cells. The renderer consults it whole before writing; the guard consults the same two rules as it parses,
+ * so a test can hold both sides to it.
+ */
+export function policyTableFault(rows: readonly PolicyRow[]): string | undefined {
+  for (const row of rows) {
+    const fault = proseFault(row.prose);
+    if (fault !== undefined) {
+      return `the "${row.section.key}" Defaults row ${fault}`;
+    }
+  }
+  return tableFault(DEFAULTS_TABLE_HEADER, rows.map(policyCells));
+}
 
 export function renderPolicyDefaultsTable(
   sections: readonly KnobbedSection[],
   prose: Readonly<Record<string, PolicyRowProse>>,
 ): string {
-  const rows = deleteFirst(sections).map((section) => {
+  const rows = deleteFirst(sections).map((section): PolicyRow => {
     const text = prose[section.key];
     if (text === undefined) {
       throw new Error(`no Defaults-per-section prose for the "${section.key}" section`);
     }
-    const caveat = text.caveat === undefined ? "" : ` (${text.caveat})`;
-    const opposite = section.undeclaredDefault === "delete" ? "keep" : "delete";
-    const cells = [
-      `\`${section.key}\``,
-      tableCell(`${section.undeclaredDefault}${caveat}`, `the ${section.key} Default cell`),
-      tableCell(
-        `\`${opposite}\`: ${text.override}`,
-        `the ${section.key} "The override buys you" cell`,
-      ),
-    ];
-    return `| ${cells.join(" | ")} |`;
+    // Read once, so the fault check and the cells see the same values.
+    return { section, prose: { caveat: text.caveat, override: text.override } };
   });
-  return [DEFAULTS_TABLE_HEADER, ...rows].join("\n");
+  const fault = policyTableFault(rows);
+  if (fault !== undefined) {
+    throw new Error(fault);
+  }
+  return renderTable(DEFAULTS_TABLE_HEADER, rows.map(policyCells));
+}
+
+/** A Defaults row read back to the section and prose it renders from; the delete-first order is then the
+ * renderer's to check, through the byte compare. The dotAll flag keeps the two Unicode line separators
+ * tableCell() admits inside the prose. */
+function policyRow(
+  cells: readonly string[],
+  key: string,
+): { readonly section: KnobbedSection; readonly prose: PolicyRowProse } | string {
+  const policy = /^(delete|keep)(?: \((.*)\))?$/s.exec(cells[1] ?? "");
+  const buys = /^`(delete|keep)`: (.*)$/s.exec(cells[2] ?? "");
+  if (policy === null) {
+    return "states no delete or keep default";
+  }
+  if (buys === null) {
+    return "names no policy span before its override prose";
+  }
+  const undeclaredDefault = policy[1] === "delete" ? "delete" : "keep";
+  if (buys[1] !== OPPOSITE[undeclaredDefault]) {
+    return `names \`${buys[1]}\` where the override is the opposite policy, \`${OPPOSITE[undeclaredDefault]}\``;
+  }
+  const row = { caveat: policy[2], override: buys[2] ?? "" };
+  const fault = proseFault(row);
+  if (fault !== undefined) {
+    return `${fault}, which the renderer refuses`;
+  }
+  return { section: { key, undeclaredDefault }, prose: row };
 }
 
 /** The row every secret family shares: the value is write-only, so a wrong delete is a loss, not a drift. */
@@ -347,10 +424,6 @@ function blockShape(lines: string): RegExp {
   return new RegExp(String.raw`^\n(?:${lines}|\n)?$`);
 }
 
-function tableShape(header: string, cells: string): RegExp {
-  return blockShape(String.raw`${RegExp.escape(header)}\n(?:\| ${cells} \|\n)*`);
-}
-
 /** The emitter is its own grammar: a body is admitted when re-rendering the declarations it parses to reproduces it
  * byte for byte, so no spelling the emitter does not write gets through. A freshly placed region holds "\n" and
  * renders next. */
@@ -386,15 +459,7 @@ function roundTrip<T extends { readonly description: string }>(
     if (typeof declared === "string") {
       return `it is not a set of ${key} declarations (${declared})`;
     }
-    const rendered = `\n${render(declared)}\n`;
-    if (rendered === body) {
-      return undefined;
-    }
-    const authored = body.split("\n");
-    const expected = rendered.split("\n");
-    const differing = authored.findIndex((line, i) => line !== expected[i]);
-    const at = differing === -1 ? authored.length : differing;
-    return `line ${at} reads ${JSON.stringify(authored[at] ?? "")} where the generator writes ${JSON.stringify(expected[at] ?? "")}`;
+    return renderedMismatch(body, `\n${render(declared)}\n`);
   };
 }
 
@@ -434,10 +499,11 @@ export const GENERATED_REGIONS: Readonly<Record<string, readonly GeneratedRegion
     {
       name: "policy-defaults-table",
       placement: { kind: "under-heading", heading: "## Defaults per section" },
-      // A cell holds no pipe and no line break: tableCell() refuses both, so a row carrying one is authored, not stale output.
-      body: tableShape(
-        DEFAULTS_TABLE_HEADER,
-        String.raw`\x60[a-z_]+\x60 \| (?:delete|keep)(?: \([^\r\n|]+\))? \| \x60(?:delete|keep)\x60: [^\r\n|]+`,
+      body: tableRoundTrip(DEFAULTS_TABLE_HEADER, policyRow, (rows) =>
+        renderPolicyDefaultsTable(
+          rows.map((row) => row.section),
+          Object.fromEntries(rows.map((row) => [row.section.key, row.prose])),
+        ),
       ),
       render: block(() => renderPolicyDefaultsTable(knobbedSections(), POLICY_ROW_PROSE)),
     },

.github/scripts/gen-docs.ts

@@ -16,7 +16,7 @@ import { readArchitecture, renderArchitectureMermaid } from "./arch-lint.js";
 import { COVERAGE_DATA, type CoverageData } from "./coverage-data.js";
 import { ENDPOINT_ANCHORS, type EndpointAnchors } from "./endpoint-docs.js";
 import { type GeneratedRegion, regenerateRegions, regionBounds } from "./lib/generated-regions.js";
-import { tableCell } from "./lib/markdown-table.js";
+import { renderTable, tableCell, tableRoundTrip, tableRow } from "./lib/markdown-table.js";
 
 const ROOT = join(import.meta.dir, "..", "..");
 export const COVERAGE_PATH = "docs/reference/coverage.md";
@@ -86,27 +86,37 @@ export function renderPatCell(grant: string): string {
   return `${cell}; ${rendered}`;
 }
 
-const TABLE_HEADER =
+export const SECTIONS_TABLE_HEADER =
   "| Section | Endpoints | PAT permission | Undeclared default | Notes |\n|---|---|---|---|---|";
 
-export function renderSectionsTable(
+/**
+ * The cells of each section's row, as the renderer writes them and the guard reads them back; the table rule
+ * judges them.
+ */
+export function sectionsCells(
   sections: readonly SectionsTableRow[],
   docs: Readonly<Record<string, Pick<SectionDocs, "sections_table">>>,
-): string {
-  const rows = sections.map((section) => {
+): ReadonlyArray<readonly string[]> {
+  return sections.map((section) => {
     const doc = docs[section.key];
     if (doc === undefined) {
       throw new Error(`gen-docs: section "${section.key}" has no docs entry`);
     }
     return [
       `\`${section.key}\``,
-      tableCell(doc.sections_table.endpoints, `the ${section.key} Endpoints cell`),
-      tableCell(renderPatCell(sectionGrant(section)), `the ${section.key} PAT permission cell`),
+      doc.sections_table.endpoints,
+      renderPatCell(sectionGrant(section)),
       UNDECLARED_DEFAULT_DISPLAY[section.undeclaredDefault],
-      tableCell(doc.sections_table.notes, `the ${section.key} Notes cell`),
+      doc.sections_table.notes,
     ];
   });
-  return [TABLE_HEADER, ...rows.map((cells) => `| ${cells.join(" | ")} |`)].join("\n");
+}
+
+export function renderSectionsTable(
+  sections: readonly SectionsTableRow[],
+  docs: Readonly<Record<string, Pick<SectionDocs, "sections_table">>>,
+): string {
+  return renderTable(SECTIONS_TABLE_HEADER, sectionsCells(sections, docs));
 }
 
 /** Text for inside a markdown code span: a cell, and a backtick would close the span early. */
@@ -244,7 +254,7 @@ function renderSection(
       `[\`${section.key}\`](${SECTIONS_PAGE})${keys}`,
       tableCell(renderCalls(section, row, claimed, carrier, anchors), `${where}'s Endpoints cell`),
     ];
-    return `| ${cells.join(" | ")} |`;
+    return tableRow(cells);
   });
   const declared = [...Object.keys(section.endpoints), ...Object.keys(section.graphql ?? {})];
   const unclaimed = declared.filter((role) => !claimed.has(role));
@@ -310,7 +320,7 @@ export function renderCoverage(
               tableCell(row.endpoints.join(", "), `${where}'s Endpoints cell`),
               tableCell(row.why, `${where}'s Why cell`),
             ];
-            return `| ${cells.join(" | ")} |`;
+            return tableRow(cells);
           }),
         ];
   return [
@@ -407,14 +417,21 @@ function patFormUrl(): string {
   );
 }
 
-// The body shape is this generator's own output (or an empty body), built from the renderer constants, so a marker
-// moved over authored prose or another table fails instead of erasing it.
+// The three prose cells are opaque; a row is this table's when its Undeclared default cell is one of the displays,
+// the one cell the table rule does not judge.
+function sectionsRow(cells: readonly string[]): readonly string[] | string {
+  if (!Object.values(UNDECLARED_DEFAULT_DISPLAY).includes(cells[3] ?? "")) {
+    return "shows an Undeclared default the renderer has no display for";
+  }
+  return cells;
+}
+
 function sectionsTableRegion(name: string, heading: string): GeneratedRegion {
   return {
     name,
     placement: { kind: "under-heading", heading },
-    body: new RegExp(
-      String.raw`^\n(?:${RegExp.escape(TABLE_HEADER)}\n(?:\| \x60[a-z_]+\x60 \| [^\n]* \|\n)*)?$`,
+    body: tableRoundTrip(SECTIONS_TABLE_HEADER, sectionsRow, (rows) =>
+      renderTable(SECTIONS_TABLE_HEADER, rows),
     ),
     render: () => `\n${renderSectionsTable(SECTIONS, DOCS)}\n`,
   };

.github/scripts/lib/generated-regions.ts

@@ -159,6 +159,18 @@ export function bodyRefusal(shape: BodyShape, body: string): string | undefined
   return shape(body);
 }
 
+/** A round-trip guard's verdict: undefined when `body` is `rendered` byte for byte, else the first differing line. */
+export function renderedMismatch(body: string, rendered: string): string | undefined {
+  if (rendered === body) {
+    return undefined;
+  }
+  const authored = body.split("\n");
+  const expected = rendered.split("\n");
+  const differing = authored.findIndex((line, i) => line !== expected[i]);
+  const at = differing === -1 ? authored.length : differing;
+  return `line ${at} reads ${JSON.stringify(authored[at] ?? "")} where the generator writes ${JSON.stringify(expected[at] ?? "")}`;
+}
+
 interface MarkdownScan {
   /** Fenced code blocks, an unclosed one running to the end of the text. */
   readonly fenced: readonly MarkerSpan[];