fix(release): never publish an empty changelog (commit-based fallback) (#1766)

* fix(release): fall back to a commit-based changelog when generate-notes finds no PRs

releases/generate-notes aggregates merged pull requests over the compared tag range. Work landing as direct commits on dev (or through PRs based on dev rather than the release branch) leaves that range with nothing to aggregate, so the body collapsed to the npm line plus a compare link: v2.17.0..v2.18.2 had 0 of 36 commits PR-associated and v2.18.2..v2.19.0 had 2 of 21, and both releases shipped a 169-char stub.

The workflow now detects an empty PR delta and renders the commit log instead, categorized by conventional-commit prefix, excluding merge and release-bump commits. The renderer keeps only PR-numbered entries, so the fallback travels on its own --commit-fallback channel and is emitted only when the PR pipeline produced no sections at all.

* fix(release): harden the commit fallback against untrusted commit metadata

Audit findings on the first commit: (1) a preview whose notes came from the fallback would carry into a stable release as meaningful, but the PR renderer discards non-PR bullets, collapsing the stable body back to the stub; (2) %an is a free-form git display name, so a contributor named e.g. Abhishek Sharma rendered as a live @abhishek mention, and subjects could inject markdown or a forged unit separator; (3) real merges using the merge: conventional prefix were not treated as plumbing.

extractCommitBulletSections carries PR-free bullets through the renderer, sanitizeCommitText neutralizes mentions/markdown/separators, authors render as plain text in a (sha, Name) trailer, non-hex shas are dropped, parseCommitLog splits the author from the right, and merge: joins the plumbing filter. Nine regression tests cover each.

* fix(release): NUL-frame the commit log and decouple the fallback from carried notes

Second audit round: (1) U+001F is legal in git author names too, so right-splitting still let author metadata shift the subject; the log is now NUL-delimited (git log -z), which git forbids inside commit content, and records parse as exactly three fields. (2) The workflow gated fallback generation on carried notes as well as the delta, so a stable release carrying commit-based preview notes silently dropped every post-preview direct commit; the guard now depends on this range delta only and the renderer decides whether to emit. (3) The sanitizer escaped by deletion, mangling technical text like Map<K, V> | CLI; metacharacters are now backslash-escaped instead.

* fix(release): merge fallback sections by category and de-vacuum two tests

Third audit round: concatenating carried and current fallback bodies emitted a duplicate heading when both touched the same category; mergeCommitBulletSections now merges by category with bullet de-duplication. Two converted tests still carried literal \\u001f fixtures, so parseCommitLog returned nothing and they asserted on an empty string; both now use real NUL fields and assert the entry renders.

* docs(release): put the commit-log JSDoc back on parseCommitLog

Author: lidge-jun

.github/workflows/release.yml

@@ -374,8 +374,10 @@ jobs:
           notes_file="$(mktemp)"
           carried_file="$(mktemp)"
           delta_file="$(mktemp)"
+          commit_fallback_file="$(mktemp)"
           : > "$carried_file"
           : > "$delta_file"
+          : > "$commit_fallback_file"
 
           # Stable releases after matching previews: aggregate every matching preview
           # changelog (oldest→newest; each preview body is incremental), then only
@@ -449,6 +451,31 @@ jobs:
             pr_notes="$(gh api "${generate_notes_api[@]}" --jq '.body')"
             # Drop generate-notes' trailing compare link; we re-append it after the commit list.
             printf '%s\n' "$pr_notes" | sed '/^\*\*Full Changelog\*\*:/d' > "$delta_file"
+
+            # generate-notes counts MERGED PULL REQUESTS in the tag range. Work that lands
+            # as direct commits on the integration branch (or via PRs based on `dev` rather
+            # than this release branch) leaves that range with nothing to aggregate, and the
+            # body collapses to the npm line plus a compare link — v2.17.0..v2.18.2 shipped
+            # exactly that, 0 of 36 commits PR-associated. Fall back to the commit log so a
+            # release can never publish an empty changelog.
+            # The renderer only keeps entries carrying a PR number, so the fallback
+            # travels in its own channel (--commit-fallback). The decision depends on
+            # THIS range's PR delta only: carried preview notes cover the pre-preview
+            # span, so gating on them too would silently drop every post-preview
+            # direct commit. The renderer decides whether to emit the channel.
+            if ! bun scripts/release-notes.ts has-meaningful "$delta_file"; then
+              commit_log_file="$(mktemp)"
+              # NUL-delimited: Git forbids NUL in commit content, so neither a crafted
+              # subject nor an author name can forge a field boundary.
+              git log -z --format='%H%x00%s%x00%an' "${notes_range_start}..${GITHUB_SHA}" > "$commit_log_file"
+              bun scripts/release-notes.ts commit-fallback "$commit_log_file" > "$commit_fallback_file"
+              if bun scripts/release-notes.ts has-meaningful "$commit_fallback_file"; then
+                echo "::notice::generate-notes returned no PR categories for ${notes_range_start}..${release_tag}; using the commit-based changelog fallback"
+              else
+                : > "$commit_fallback_file"
+                echo "::notice::No PR categories and no eligible commits in ${notes_range_start}..${release_tag}; release notes stay minimal"
+              fi
+            fi
           else
             # First release on this channel: never call generate-notes without previous_tag_name.
             # GitHub would baseline the newest repo tag, which may belong to the other channel.
@@ -476,6 +503,7 @@ jobs:
             --npm-metadata "$npm_metadata"
             --carried "$carried_file"
             --delta "$delta_file"
+            --commit-fallback "$commit_fallback_file"
             --out "$notes_file"
             --compare-to "$release_tag"
             --repository "$GITHUB_REPOSITORY"

scripts/release-notes.ts

@@ -10,6 +10,7 @@
  *   bun scripts/release-notes.ts matching-preview-tags <version>
  *   bun scripts/release-notes.ts previous-release-tag <version>
  *   bun scripts/release-notes.ts has-meaningful [body-file]
+ *   bun scripts/release-notes.ts commit-fallback [commit-log-file]
  *   bun scripts/release-notes.ts credit-takeovers --repo <owner/name> --in <file> --out <file>
  *   bun scripts/release-notes.ts render --npm-metadata ... --out ... [--carried ...] [--delta ...] [--compare-from ...] [--compare-to ...] [--repository ...]
  *   bun scripts/release-notes.ts polish --in <file> --out <file> [--model ...] [--base-url ...]
@@ -177,6 +178,223 @@ export function hasMeaningfulCarriedNotes(stripped: string): boolean {
   return !isEmptyGeneratedNotes(stripped);
 }
 
+/**
+ * A single commit considered for the commit-based changelog fallback.
+ * `sha` is the full or short hash; `subject` is the commit subject line.
+ */
+export type ReleaseNoteCommit = {
+  sha: string;
+  subject: string;
+  author: string;
+};
+
+/** Category order shared by the PR renderer and the commit fallback. */
+const RENDER_CATEGORY_ORDER = ["New Features", "Bug Fixes", "Documentation", "Chores", "Other Changes"];
+
+/** Conventional-commit type -> release.yml category title. */
+const COMMIT_TYPE_CATEGORY: Record<string, string> = {
+  feat: "New Features",
+  fix: "Bug Fixes",
+  perf: "Bug Fixes",
+  docs: "Documentation",
+  chore: "Chores",
+  build: "Chores",
+  ci: "Chores",
+  refactor: "Chores",
+  style: "Chores",
+  test: "Chores",
+};
+
+/**
+ * Commits that are release plumbing rather than shipped work. A merge commit's
+ * content is already represented by the commits it brings in, and a `release:`
+ * bump is the release itself.
+ */
+export function isReleasePlumbingCommit(subject: string): boolean {
+  const text = subject.trim();
+  if (/^Merge\s/i.test(text)) return true;
+  // Real two-parent merges in this repo also use a `merge:` conventional prefix.
+  if (/^merge(?:\([^)]*\))?!?:\s/i.test(text)) return true;
+  if (/^release(?:\([^)]*\))?!?:\s/i.test(text)) return true;
+  return false;
+}
+
+/**
+ * Neutralize Markdown and mention syntax from untrusted commit text before it
+ * lands in a release body. Commit subjects and author names are attacker- or
+ * accident-controlled: a bare `@name` renders as a real GitHub mention (and
+ * notifies that account), and backticks/brackets can restructure the notes.
+ */
+export function sanitizeCommitText(text: string): string {
+  return text
+    .replace(/\r?\n/g, " ")
+    // Strip the ASCII unit separator so a subject can never forge a log field.
+    .replace(/[\u0000\u001f]/g, " ")
+    // Escape rather than delete: `Map<K, V> | CLI` must stay readable.
+    .replace(/([`<>|[\]\\])/g, "\\$1")
+    // `@name` -> `@\u200bname`: reads identically, never notifies.
+    .replace(/@(?=[A-Za-z0-9_-])/g, "@\u200b")
+    .replace(/\s+/g, " ")
+    .trim();
+}
+
+/**
+ * Render commits as a generate-notes-shaped body so the existing category
+ * parser/renderer can consume them unchanged.
+ *
+ * Why this exists: `releases/generate-notes` aggregates MERGED PULL REQUESTS
+ * against the compared tag range. When work lands as direct commits on the
+ * integration branch (or through PRs whose base is `dev` rather than the
+ * release branch), that range contains no PRs the API will count and the body
+ * collapses to the npm line plus a compare link — v2.17.0..v2.18.2 had 0 of 36
+ * commits associated with a main-merged PR, and both releases shipped an empty
+ * changelog. The fallback keeps the release body honest regardless of how the
+ * work reached the branch.
+ *
+ * Commits carry no PR number, so the synthetic entries use `#0` — a sentinel
+ * the renderer never prints as a link because these are emitted as plain
+ * bullets under their category heading.
+ */
+export function renderCommitFallbackNotes(commits: ReleaseNoteCommit[]): string {
+  const buckets = new Map<string, string[]>();
+  for (const commit of commits) {
+    const subject = commit.subject.trim();
+    if (!subject) continue;
+    if (isReleasePlumbingCommit(subject)) continue;
+    const match = /^([a-zA-Z]+)(?:\(([^)]*)\))?!?:\s*(.+)$/.exec(subject);
+    const type = match?.[1]?.toLowerCase();
+    const scope = sanitizeCommitText(match?.[2] ?? "");
+    const summary = sanitizeCommitText(match?.[3] ?? subject);
+    if (!summary) continue;
+    const category = (type && COMMIT_TYPE_CATEGORY[type]) ?? "Other Changes";
+    // Hex-only short hash: a crafted `sha` field can never inject markup.
+    const shortSha = /^[0-9a-f]{7,40}$/i.test(commit.sha.trim())
+      ? commit.sha.trim().slice(0, 9)
+      : "";
+    const scopePrefix = scope ? `${scope}: ` : "";
+    // `%an` is a free-form Git display name, not a GitHub login, so it is
+    // rendered as plain text rather than an @mention that would notify a
+    // same-named (or non-existent) account.
+    const author = sanitizeCommitText(commit.author).replace(/^@\u200b/, "");
+    const trailer = [shortSha, author].filter(Boolean).join(", ");
+    const line = trailer ? `- ${scopePrefix}${summary} (${trailer})` : `- ${scopePrefix}${summary}`;
+    const existing = buckets.get(category);
+    if (existing) existing.push(line);
+    else buckets.set(category, [line]);
+  }
+  if (buckets.size === 0) return "";
+  const parts: string[] = [];
+  for (const title of RENDER_CATEGORY_ORDER) {
+    const lines = buckets.get(title);
+    if (!lines || lines.length === 0) continue;
+    parts.push([`## ${title}`, "", ...lines].join("\n"));
+  }
+  return parts.join("\n\n").replace(/\n+$/, "") + "\n";
+}
+
+/**
+ * Extract commit-style category sections (bullets with no `(#N)` reference)
+ * from an already-rendered body.
+ *
+ * A preview release whose notes came from the commit fallback carries bullets
+ * like `- gui: fix a thing (abc1234, Name)`. Those are meaningful prose, so the
+ * workflow keeps them as carried notes and skips regenerating a fallback — but
+ * the PR renderer only retains entries carrying a PR number, so without this
+ * the stable release would silently collapse back to the npm-line stub.
+ */
+export function extractCommitBulletSections(body: string): string {
+  const out: string[] = [];
+  let current: { title: string; lines: string[] } | null = null;
+  const flush = (): void => {
+    if (current && current.lines.length > 0) {
+      out.push([`## ${current.title}`, "", ...current.lines].join("\n"));
+    }
+    current = null;
+  };
+  for (const rawLine of body.replace(/\r\n/g, "\n").split("\n")) {
+    const line = rawLine.trim();
+    if (!line || line.startsWith("<!--")) continue;
+    if (line.startsWith("## ") || line.startsWith("### ")) {
+      flush();
+      const title = line.replace(/^#{2,3}\s+/, "").trim();
+      if (!SCAFFOLD_HEADINGS.has(title)) current = { title, lines: [] };
+      continue;
+    }
+    if (!current) continue;
+    if (!line.startsWith("- ")) continue;
+    // Anything carrying a PR reference belongs to the PR pipeline, not here.
+    if (/\(#\d+(?:\s*,\s*#\d+)*\)\s*$/.test(line)) continue;
+    if (/^-\s+#\d+\s/.test(line)) continue;
+    current.lines.push(line);
+  }
+  flush();
+  return out.join("\n\n").replace(/\n+$/, "") + (out.length > 0 ? "\n" : "");
+}
+
+/**
+ * Merge several already-rendered commit-bullet bodies into one set of category
+ * sections, preserving order within a category and de-duplicating identical
+ * bullets. Concatenating the bodies directly would repeat a shared heading.
+ */
+export function mergeCommitBulletSections(bodies: string[]): string {
+  const buckets = new Map<string, string[]>();
+  const seen = new Set<string>();
+  for (const body of bodies) {
+    let current: string | null = null;
+    for (const rawLine of (body ?? "").replace(/\r\n/g, "\n").split("\n")) {
+      const line = rawLine.trim();
+      if (!line) continue;
+      if (line.startsWith("## ") || line.startsWith("### ")) {
+        current = line.replace(/^#{2,3}\s+/, "").trim();
+        if (!buckets.has(current)) buckets.set(current, []);
+        continue;
+      }
+      if (!current || !line.startsWith("- ")) continue;
+      const key = `${current}\u0000${line}`;
+      if (seen.has(key)) continue;
+      seen.add(key);
+      buckets.get(current)!.push(line);
+    }
+  }
+  const titles = [...buckets.keys()].sort((x, y) => {
+    const ix = RENDER_CATEGORY_ORDER.indexOf(x);
+    const iy = RENDER_CATEGORY_ORDER.indexOf(y);
+    const rx = ix === -1 ? RENDER_CATEGORY_ORDER.length : ix;
+    const ry = iy === -1 ? RENDER_CATEGORY_ORDER.length : iy;
+    return rx - ry;
+  });
+  const merged: string[] = [];
+  for (const title of titles) {
+    const lines = buckets.get(title)!;
+    if (lines.length === 0) continue;
+    merged.push([`## ${title}`, "", ...lines].join("\n"));
+  }
+  return merged.join("\n\n").trim();
+}
+
+/**
+ * Parse `git log -z --format=%H%x00%s%x00%an` output into commits.
+ *
+ * Records and fields are NUL-separated. Git forbids NUL in commit content, so
+ * — unlike the unit separator, which Git accepts in both subjects and author
+ * names — no field value can forge a boundary. Every record is read as exactly
+ * three fields.
+ */
+export function parseCommitLog(raw: string): ReleaseNoteCommit[] {
+  const commits: ReleaseNoteCommit[] = [];
+  const fields = raw.split("\u0000");
+  // Trailing separator from `git log -z` leaves an empty final element.
+  if (fields.length > 0 && fields[fields.length - 1]!.trim() === "") fields.pop();
+  for (let i = 0; i + 2 < fields.length + 1; i += 3) {
+    const sha = (fields[i] ?? "").replace(/^\n+/, "").trim();
+    const subject = fields[i + 1] ?? "";
+    const author = fields[i + 2] ?? "";
+    if (!sha || !subject.trim()) continue;
+    commits.push({ sha, subject, author });
+  }
+  return commits;
+}
+
 export function hasNonWhitespace(text: string): boolean {
   return text.replace(/\s+/g, "").length > 0;
 }
@@ -425,8 +643,6 @@ export function groupPrsByScope(prs: ReleaseNotePr[]): Array<{ scope: string | n
   return groups;
 }
 
-const RENDER_CATEGORY_ORDER = ["New Features", "Bug Fixes", "Documentation", "Chores", "Other Changes"];
-
 /**
  * Render OpenAI-Codex-style release notes from the generate-notes pieces:
  * H2 category sections with scope-grouped, prefix-free summary bullets, then a
@@ -439,6 +655,12 @@ export function renderReleaseNotes(input: {
   npmMetadata: string;
   carriedPreviewNotes?: string;
   deltaPrNotes?: string;
+  /**
+   * Pre-rendered category sections for commit-based entries (no PR numbers).
+   * Used only when the PR pipeline yields nothing, so a release body can never
+   * collapse to the npm line plus a compare link.
+   */
+  commitFallbackNotes?: string;
   compareFrom?: string | null;
   compareTo?: string;
   repository?: string;
@@ -494,6 +716,20 @@ export function renderReleaseNotes(input: {
     parts.push(lines.join("\n"));
   }
 
+  // Commit fallback: only when the PR pipeline produced no category content at
+  // all. Its sections are already rendered, so they are appended verbatim.
+  const renderedAnyPrSection = parts.length > (npmMetadata ? 1 : 0);
+  if (!renderedAnyPrSection) {
+    // Carried commit bullets first (older preview work), then this range's own.
+    // They are merged BY CATEGORY: concatenating two rendered bodies would emit
+    // `## Bug Fixes` twice when both halves touched the same category.
+    const merged = mergeCommitBulletSections([
+      extractCommitBulletSections(input.carriedPreviewNotes ?? ""),
+      input.commitFallbackNotes ?? "",
+    ]);
+    if (merged) parts.push(merged);
+  }
+
   const allPrs = [...categories.values()].flat().sort((a, b) => a.number - b.number);
   const from = input.compareFrom?.trim();
   const to = input.compareTo?.trim();
@@ -734,6 +970,13 @@ async function main(argv: string[]): Promise<void> {
     process.exit(hasMeaningfulCarriedNotes(stripped) ? 0 : 1);
   }
 
+  if (cmd === "commit-fallback") {
+    // stdin: `git log --format=%H%x1f%s%x1f%an <range>` output.
+    const rendered = renderCommitFallbackNotes(parseCommitLog(await readStdinOrFile(rest[0])));
+    process.stdout.write(rendered);
+    return;
+  }
+
   if (cmd === "join-carried") {
     let out: string | undefined;
     const files: string[] = [];
@@ -888,6 +1131,7 @@ async function main(argv: string[]): Promise<void> {
       "out",
       "carried",
       "delta",
+      "commit-fallback",
       "compare-from",
       "compare-to",
       "repository",
@@ -909,6 +1153,7 @@ async function main(argv: string[]): Promise<void> {
       npmMetadata,
       carriedPreviewNotes: await readOptional("carried"),
       deltaPrNotes: await readOptional("delta"),
+      commitFallbackNotes: await readOptional("commit-fallback"),
       compareFrom: args.get("compare-from") ?? null,
       compareTo: args.get("compare-to"),
       repository: args.get("repository"),
@@ -972,6 +1217,7 @@ async function main(argv: string[]): Promise<void> {
 Usage:
   bun scripts/release-notes.ts strip-carried [body-file]
   bun scripts/release-notes.ts has-meaningful [body-file]
+  bun scripts/release-notes.ts commit-fallback [commit-log-file]
   bun scripts/release-notes.ts join-carried --out <file> <part-file>...
   bun scripts/release-notes.ts matching-preview-tag <version>   # tags on stdin
   bun scripts/release-notes.ts matching-preview-tags <version>  # tags on stdin, oldest→newest