docs: streamline CI test reporting guidance

Author: cderv

.claude/rules/testing/built-version-ci.md

@@ -24,4 +24,6 @@ Invariants when editing:
 - `test-ff-matrix.yml` owns the ff-matrix bucket glob.
 - Scheduler jobs in `test-smokes-built.yml` set per-leg OS scope through their `runners:` inputs.
 - Keep the per-call suffix in `test-ff-matrix.yml`'s concurrency group so sibling calls cannot cancel one another.
-- Every `test-smokes.yml` / `test-ff-matrix.yml` call site passes a `label-tag`: it is what keeps CI failure labels distinct between same-OS jobs in a run's concatenated step summary. Tags need only differ among call sites that can land on the same OS — the per-OS nightly legs share one tag deliberately, since the label already carries an OS prefix.
+- Every `test-smokes.yml` and `test-ff-matrix.yml` call site passes a
+  `label-tag` that distinguishes same-OS jobs in the combined run summary.
+  Per-OS jobs may share a tag because labels also include the OS.

.claude/rules/testing/github-actions-reporting.md

@@ -0,0 +1,41 @@
+---
+paths:
+  - .github/workflows/test-smokes.yml
+  - .github/workflows/test-smokes-built.yml
+  - .github/workflows/test-smokes-parallel.yml
+  - .github/workflows/test-ff-matrix.yml
+  - .github/workflows/update-test-timing.yml
+  - src/tools/github.ts
+  - tests/test.ts
+  - tests/gha-grouping.ts
+  - tests/integration/playwright-tests.test.ts
+  - tests/tools/check-gha-log.ts
+  - tests/unit/check-gha-log.test.ts
+  - tests/unit/gha-grouping.test.ts
+  - tests/unit/github-actions-reporting.test.ts
+  - tests/unit/harness-failure-reporting.test.ts
+  - tests/unit/harness-reporting-fixture.ts
+---
+
+# GitHub Actions test reporting
+
+`llm-docs/github-actions-test-reporting.md` is the deep dive. Read only the
+sections relevant to the edit:
+
+- Workflow ownership or environment wiring: "Ownership" and "Workflow contract".
+- Group opening, closing, or log validation: "Log grouping" and "Verification".
+- Annotations, labels, excerpts, or step summaries: "Failure reporting" and
+  "Annotation budget and labels".
+
+For a trivial edit, the invariants below are sufficient.
+
+Keep these invariants:
+
+- A step has one group and annotation owner. Bucket loops set
+  `QUARTO_TESTS_GHA_ORCHESTRATED=1`; otherwise the harness owns them.
+- Step-summary entries are emitted in both paths.
+- Failure lines and Deno's final failure sections remain outside groups.
+- The annotation count is step-wide; Deno module state is only per test file.
+- Every reusable-workflow call passes a `label-tag` that distinguishes same-OS
+  jobs in the run summary.
+- Do not enable parallel test-file execution without redesigning grouping.

.github/workflows/test-ff-matrix.yml

@@ -39,7 +39,7 @@ on:
         type: string
         default: '["ubuntu-latest", "windows-latest"]'
       label-tag:
-        description: "Short alphanumeric tag identifying this call in CI failure labels (forwarded to test-smokes.yml; empty means the standalone dev default)"
+        description: "Short alphanumeric tag for distinguishing same-OS jobs in failure summaries"
         required: false
         type: string
         default: ""

.github/workflows/test-smokes-parallel.yml

@@ -104,6 +104,5 @@ jobs:
     with:
       buckets: ${{ matrix.buckets.files }}
       extra-r-packages: ${{ inputs.extra-r-packages }}
-      # `num` is the zero-padded bucket index built in jobs-matrix, the only
-      # thing that differs between these ~20 same-OS legs.
+      # The bucket index distinguishes same-OS matrix jobs.
       label-tag: b${{ matrix.buckets.num }}

.github/workflows/test-smokes.yml

@@ -50,7 +50,7 @@ on:
         type: string
         default: '["ubuntu-latest", "windows-latest"]'
       label-tag:
-        description: "Short alphanumeric tag identifying this call in CI failure labels (src/tools/github.ts failureLabel). Several jobs of one run share an OS and the label is the Ctrl+F navigation target in the concatenated run summary, so every caller must pass a value. Empty means labels are not unique across same-OS jobs."
+        description: "Short alphanumeric tag for distinguishing same-OS jobs in failure summaries"
         required: false
         type: string
         default: ""
@@ -97,8 +97,7 @@ jobs:
     runs-on: ${{ matrix.os }}
     env:
       PLAYWRIGHT_BROWSERS_PATH: ${{ github.workspace }}/ms-playwright-browsers
-      # Read by the test harness to keep this call's failure labels distinct
-      # from sibling jobs on the same OS in the concatenated run summary.
+      # Distinguishes this call from other jobs on the same OS.
       QUARTO_TESTS_GHA_LABEL_TAG: ${{ inputs.label-tag }}
     steps:
       - name: Checkout Repo
@@ -499,8 +498,7 @@ jobs:
           # Useful as TinyTeX latest release is checked in run-test.sh
           GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
           BUCKETS_JSON: ${{ inputs.buckets }}
-          # This loop owns the per-file ::group:: and ::error:: markers, so the
-          # test harness must not also emit annotations for this step.
+          # Prevent the harness from duplicating this loop's workflow commands.
           QUARTO_TESTS_GHA_ORCHESTRATED: "1"
         run: |
           haserror=0
@@ -551,8 +549,7 @@ jobs:
           # Useful as TinyTeX latest release is checked in run-test.sh
           GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
           BUCKETS_JSON: ${{ inputs.buckets }}
-          # This loop owns the per-file ::group:: and ::error:: markers, so the
-          # test harness must not also emit annotations for this step.
+          # Prevent the harness from duplicating this loop's workflow commands.
           QUARTO_TESTS_GHA_ORCHESTRATED: "1"
         run: |
           $haserror=$false