Skip to content

Commit f3e0bdf

Browse files
committed
Make stepSummary's write outcome honest and carry teardown failure stacks through
stepSummary now reports whether its write actually succeeded instead of assuming success from the size check; a teardown-only failure carries its own stack into the report instead of a generic one; resolves a remaining design-doc contradiction about which trial run is pending.
1 parent f6c8094 commit f3e0bdf

5 files changed

Lines changed: 41 additions & 10 deletions

File tree

‎dev-docs/ci-test-log-grouping-design.md‎

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -462,14 +462,17 @@ are ASCII by construction while file/test names keep going through
462462
unit-test fixtures, never against a real CI log, and a cascade bug in its
463463
own logic was found and fixed as recently as commit 396c6e2e7. Gating this
464464
PR's own CI on an unproven checker risks either false failures (hard gate)
465-
or noise (report-only). Revisit once the pre-ready trial run (item 3
466-
below) produces real CI logs to validate the checker against.
465+
or noise (report-only). The item-3 trial below validated Phases 1–2
466+
against real CI logs, but predates the reporting-hardening commits made
467+
since; revisit once a fresh CI run — the still-pending run against the
468+
PR once it leaves draft — produces logs to validate the checker against.
467469
2. **Unit tests** for the new pure logic: annotation cap counter, the
468470
orchestrated-mode gate (no annotations/groups when
469471
`QUARTO_TESTS_GHA_ORCHESTRATED` is set), ANSI stripping, the stat-based
470472
summary size budget (including the degrade-to-name-only path), escaping.
471-
3. **Trial CI runs** on a fork via `workflow_dispatch`, with one deliberately
472-
failing test committed temporarily, covering all four cells:
473+
3. **Trial CI runs** (done — see status above) on a fork via
474+
`workflow_dispatch`, with one deliberately failing test committed
475+
temporarily, covering all four cells:
473476
{bucketed, default} × {Linux, Windows} — plus one bucket run with **more
474477
than 10 failing files**, to observe GitHub's silent annotation dropping
475478
and confirm the step summary still lists every failure. Check in the real

‎src/tools/github.ts‎

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -294,19 +294,23 @@ export const kStepSummaryBudgetBytes = 512 * 1024;
294294
// $GITHUB_STEP_SUMMARY; `null` (or "") → guaranteed no-op. Tests must pass
295295
// `null` or a temp path, never an explicit `undefined` — that triggers the
296296
// default parameter and, on CI, writes to the REAL step summary (found by
297-
// trial run cderv/quarto-cli#29767179626).
297+
// trial run cderv/quarto-cli#29767179626). Returns whether the write actually
298+
// happened, so a caller deciding what got recorded doesn't have to guess.
298299
export function stepSummary(
299300
markdown: string,
300301
path?: string | null,
301-
): void {
302+
): boolean {
302303
const p = path === undefined ? Deno.env.get("GITHUB_STEP_SUMMARY") : path;
303-
if (!p) return;
304+
if (!p) return false;
304305
try {
305306
Deno.writeTextFileSync(p, markdown, { append: true });
307+
return true;
306308
} catch {
307309
// Best-effort: callers append while a test failure is already in flight
308310
// (and from the unload handler, after the run), so a summary write error
309-
// must not replace the failure being reported. A lost append costs a row.
311+
// must not replace the failure being reported. A lost append costs a row,
312+
// which the returned `false` lets the caller account for.
313+
return false;
310314
}
311315
}
312316

@@ -421,8 +425,7 @@ function appendIfFits(markdown: string, path: string): boolean {
421425
if (!fitsInStepSummary(markdown, kStepSummaryContentBudgetBytes, path)) {
422426
return false;
423427
}
424-
stepSummary(markdown, path);
425-
return true;
428+
return stepSummary(markdown, path);
426429
}
427430

428431
// Append the truncation notice in place of content that no longer fits.

‎tests/test.ts‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -481,6 +481,12 @@ function reportFailure(
481481
}
482482
if (primary?.stack) {
483483
rawExcerpt.push(primary.stack);
484+
} else if (finallyFailure) {
485+
// A teardown-only failure has no primary, so its own stack is the
486+
// only pointer to where it threw - without it the banner above names
487+
// the failure but not its location.
488+
const stack = describeThrow(finallyFailure.value).stack;
489+
if (stack) rawExcerpt.push(stack);
484490
}
485491
const logMessages = primary?.logMessages;
486492
if (logMessages && logMessages.length > 0) {

‎tests/unit/github-actions-reporting.test.ts‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -426,6 +426,21 @@ unitTest("gha-reporting - appendStepSummaryBounded no-ops when the path is unset
426426
assertEquals(appendStepSummaryBounded("ignored", ""), false);
427427
});
428428

429+
// deno-lint-ignore require-await
430+
unitTest("gha-reporting - appendStepSummaryBounded reports failure when the write itself fails, not just when the budget check refuses", async () => {
431+
// A small candidate can pass the size-budget check (stepSummarySize reads
432+
// 0 on an unreadable path) and then fail the actual write. The return
433+
// value must reflect the write, or a caller mistakes a lost row for a
434+
// recorded one.
435+
const dir = Deno.makeTempDirSync({ prefix: "quarto-summary" });
436+
try {
437+
const unwritable = join(dir, "no-such-dir", "summary.md");
438+
assertEquals(appendStepSummaryBounded("| row |\n", unwritable), false);
439+
} finally {
440+
Deno.removeSync(dir, { recursive: true });
441+
}
442+
});
443+
429444
// appendStepSummaryFirstFit exists because trying a full-size candidate and
430445
// emitting the truncation notice on refusal (as appendStepSummaryBounded
431446
// does) consumes the notice's reserved headroom before a smaller fallback

‎tests/unit/harness-failure-reporting.test.ts‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -153,6 +153,10 @@ unitTest("harness-reporting - a throwing teardown on a passing test is reported"
153153
run.summary.includes("FIXTURE_TEARDOWN_BOOM"),
154154
"the teardown error text reaches the summary",
155155
);
156+
assert(
157+
run.summary.includes("harness-reporting-fixture.ts:34"),
158+
`a teardown-only failure (no primary) must still carry its stack, so the reader can find where the teardown itself threw; summary was:\n${run.summary}`,
159+
);
156160
});
157161

158162
unitTest("harness-reporting - a lifecycle failure is reported", async () => {

0 commit comments

Comments
 (0)