Skip to content

Commit 4a66491

Browse files
committed
Adjust workding
1 parent 304c616 commit 4a66491

1 file changed

Lines changed: 25 additions & 10 deletions

File tree

‎llm-docs/github-actions-test-reporting.md‎

Lines changed: 25 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -16,18 +16,24 @@ key_files:
1616

1717
Quarto's test harness groups full-run logs by test file and reports failures
1818
through annotations and the GitHub Actions step summary. Bucketed workflows
19-
already group and annotate each file in their shell loops, so the two paths must
20-
not emit competing workflow commands.
19+
already group each bucket and emit bucket-level annotations in their shell
20+
loops, so the harness must not emit competing workflow commands.
2121

2222
## Ownership
2323

2424
There are two execution paths:
2525

26-
- **Harness-owned:** A full run invokes one `deno test` process. The harness
27-
emits groups, annotations, and step-summary entries.
28-
- **Orchestrated:** A workflow loop invokes `run-tests` once per file and sets
29-
`QUARTO_TESTS_GHA_ORCHESTRATED=1`. The loop owns groups and annotations; the
30-
harness emits only step-summary entries.
26+
- **Harness-owned:** `QUARTO_TESTS_GHA_ORCHESTRATED` is unset. The harness
27+
emits groups, annotations, and step-summary entries. This does not imply a
28+
single `deno test` process: `run-tests.sh`'s timing mode (`QUARTO_TEST_TIMING`,
29+
used by `test-smokes.yml`) invokes separate `deno test` processes for test
30+
files, and invokes `smoke-all.test.ts` separately for each smoke-all document.
31+
It leaves the flag unset, so each invocation is independently harness-owned.
32+
- **Orchestrated:** A workflow loop invokes `run-tests` once per file (or
33+
bucket of files) and sets `QUARTO_TESTS_GHA_ORCHESTRATED=1`. The loop owns
34+
groups and bucket-level annotations; the harness emits only step-summary
35+
entries. A direct test runner can still emit its own file-specific
36+
annotations.
3137

3238
`harnessOwnsStep()` in `src/tools/github.ts` implements this distinction.
3339
Workflow commands are otherwise gated by `GITHUB_ACTIONS`.
@@ -79,7 +85,9 @@ Each failure contributes:
7985
Rows are written immediately. Detail blocks are buffered until the file's
8086
`unload` event because GitHub-flavored Markdown ends a table at the first
8187
non-row line. Failures with the same first three non-empty excerpt lines share
82-
one detail block.
88+
one detail block. `pendingClusters` is a module-level map, and Deno gives each
89+
test-file execution its own module graph, so this clustering only merges
90+
failures within one module instance, not across the step.
8391

8492
Summary writes are best-effort and limited to 512 KiB, leaving margin below
8593
GitHub's 1 MiB per-step limit. Full rows degrade to name-only rows when
@@ -141,5 +149,12 @@ GITHUB_ACTIONS=true ./run-tests.sh <subset> | tee log.txt
141149
deno run --allow-read tests/tools/check-gha-log.ts log.txt
142150
```
143151

144-
Direct `Deno.test` files that bypass the Quarto harness do not receive harness
145-
groups, annotations, or summary entries. Their output remains ungrouped.
152+
Direct `Deno.test` files that bypass the Quarto harness (e.g.
153+
`playwright-tests.test.ts`) receive none of the harness's own grouping,
154+
annotation, or summary logic. They can still use file-specific reporting or be
155+
wrapped by workflow-level reporting. On GitHub Actions,
156+
`tests/integration/playwright-tests.test.ts` emits its own annotations; when a
157+
bucket loop owns the step, it calls `gha.error` directly instead of using the
158+
step-wide annotation budget. Bucket loops in `test-smokes.yml` also wrap each
159+
bucket in their own `::group::`/`::endgroup::` pair and emit a bucket-level
160+
failure annotation regardless of which files the bucket runs.

0 commit comments

Comments
 (0)