Skip to content

Commit 970022e

Browse files
coverage: emit a markdown job summary from the report generator (#440)
* coverage: emit a markdown job summary from the report generator Every consumer that wants a human-readable coverage summary on its workflow run page has been re-parsing the LCOV itself; the report generator now emits it directly. - New py_binary coverage_summary (stdlib only): parses the pipeline's LCOV (which includes the exact-0% baseline records), aggregates line/branch/file totals, and renders markdown — overall table with text progress bars, raw-vs-effective when a justification report exists, a per-directory rollup (first one/two path segments, worst first), and collapsible least-covered / exact-0% file lists, closing with a pointer to the HTML artifact. - generate_coverage_html: new optional --summary-md <path>; when the flag is absent and GITHUB_STEP_SUMMARY is set (GitHub Actions), the summary is appended there automatically; with neither, behavior is unchanged. Emitted BEFORE the threshold gate decides the exit code, so a failing gate still leaves the summary on the run page. - Edge cases covered by unit tests: missing/empty LCOV (note instead of crash), records with LF but no BRF (branch cells render as an em dash), BRDA fallback counting, zero denominators, non-UTF8 bytes in paths, markdown cell escaping. - Integration workspace: asserts summary content markers, append semantics for GITHUB_STEP_SUMMARY (existing content preserved), and that the summary survives a failing gate. The test script unsets GITHUB_STEP_SUMMARY first so CI runs don't spam the real run page. - Docs: adoption guide (README) and COVERAGE_GUIDE updated. * coverage: clarify summary test resets, import-lib pattern and bar docstring Review follow-up for #440: comment why each summary test section deletes summary.md before its own run (stale-file false passes; the failing-gate section must prove re-creation), note that imports=[".."] is confined to the *_lib unit-test import helpers, and state in the docstring that the progress bar visualizes the coverage percentage.
1 parent b7d7599 commit 970022e

9 files changed

Lines changed: 645 additions & 0 deletions

File tree

‎.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,3 +20,5 @@ coverage-html/
2020
coverage/integration_tests/coverage_artifacts.zip
2121
coverage/integration_tests/lcov.dat
2222
coverage/integration_tests/coverage_linux/
23+
coverage/integration_tests/summary.md
24+
coverage/integration_tests/step_summary.md

‎coverage/BUILD‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,13 @@ py_binary(
5656
srcs = ["effective_coverage.py"],
5757
)
5858

59+
# Markdown job summary (GITHUB_STEP_SUMMARY / --summary-md), invoked by
60+
# generate_coverage_html.sh. Stdlib only.
61+
py_binary(
62+
name = "coverage_summary",
63+
srcs = ["coverage_summary.py"],
64+
)
65+
5966
sh_binary(
6067
name = "generate_coverage_html",
6168
srcs = ["generate_coverage_html.sh"],
@@ -75,6 +82,15 @@ py_library(
7582
deps = ["@rules_python//python/runfiles"],
7683
)
7784

85+
# Import-only test helper like merger_lib/reporter_lib above: imports=[".."]
86+
# makes `from coverage.coverage_summary import ...` resolvable from the unit
87+
# tests. Only the *_lib targets carry it; the binaries are unaffected.
88+
py_library(
89+
name = "coverage_summary_lib",
90+
srcs = ["coverage_summary.py"],
91+
imports = [".."],
92+
)
93+
7894
# These compile time options are required to cover abnormal termination cases
7995
# (death tests). LLVM provides them in combination with a specific profile
8096
# setting which is enabled in Bazel via LLVM_PROFILE_CONTINUOUS_MODE.

‎coverage/COVERAGE_GUIDE.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -181,6 +181,14 @@ bazel run @score_tooling//coverage:generate_coverage_html -- \
181181
(exit 1)** when below. The model: every uncovered line must eventually be
182182
either tested or justified; the threshold is ratcheted up as gaps close.
183183

184+
- Optionally a **markdown job summary** is emitted (`--summary-md <path>`, or
185+
appended to `GITHUB_STEP_SUMMARY` automatically inside GitHub Actions when
186+
the flag is absent): overall/per-directory tables computed from the LCOV
187+
data (which includes the exact-0% baseline records), raw-vs-effective
188+
numbers when justifications ran, and collapsible least-covered/0% file
189+
lists. It is written before the gate decides the exit code, so a failing
190+
gate still leaves the summary on the run page.
191+
184192
### 2.3 Day-to-day commands
185193

186194
```bash

‎coverage/README.md‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,7 @@ is copied from it.
4343
| `@score_tooling//coverage:generate_coverage_html` | Orchestration: unpacks the report, runs justifications, enforces the threshold, optionally archives. |
4444
| `@score_tooling//coverage:justify` | Parses the justification YAML + in-code markers into a manifest. |
4545
| `@score_tooling//coverage:effective_coverage` | Post-processes the HTML: restyles justified lines, computes effective coverage, detects stale justifications. |
46+
| `@score_tooling//coverage:coverage_summary` | Renders the markdown job summary from the LCOV data (invoked by `generate_coverage_html` for `--summary-md` / `GITHUB_STEP_SUMMARY`). |
4647
| `@score_tooling//coverage:enable_llvm_coverage_for_death_tests` | `cc_feature` adding `-mllvm -runtime-counter-relocation` (continuous-mode profiling for death tests). |
4748

4849
## Prerequisites
@@ -207,6 +208,16 @@ COVERAGE_THRESHOLD=95 bazel run @score_tooling//coverage:generate_coverage_html
207208
a YAML; add one (`version: 1` + `justifications: []`) when you introduce your
208209
first `COV_JUSTIFIED` marker.
209210

211+
**GitHub job summary:** inside GitHub Actions no extra flags are needed — when
212+
`GITHUB_STEP_SUMMARY` is set (and `--summary-md` is not given), a markdown
213+
summary is appended to the workflow run page automatically: overall
214+
line/branch/file tables with progress bars, raw-vs-effective when a
215+
justification YAML is in play, a per-directory rollup (worst first), and
216+
collapsible lists of the least-covered and exact-0% files. Outside Actions,
217+
pass `--summary-md <path>` to write the same summary to a file. The summary is
218+
emitted before the threshold gate decides the exit code, so a failing gate
219+
still leaves it on the run page. No consumer-side LCOV parsing needed.
220+
210221
`--build_tests_only` matters: without it, coverage builds (not runs) every
211222
target matched by the pattern, including e.g. `manual`-tagged or
212223
platform-incompatible test binaries.
@@ -219,6 +230,7 @@ platform-incompatible test binaries.
219230
| Output directory | positional `output-dir` argument (default `coverage_<platform>`) |
220231
| Platform-specific justifications | `--platform linux\|qnx` (default linux) |
221232
| JUnit XMLs subtree in the archive | `--testlogs-subdir <dir>` (default: whole `bazel-testlogs`) |
233+
| Markdown job summary | `--summary-md <path>`; auto-append to `GITHUB_STEP_SUMMARY` when the flag is absent and the variable is set |
222234
| Different LLVM version | your own `llvm.toolchain(...)`; pass its labels in step 3 |
223235
| Rust branch coverage | `-Zcoverage-options=branch` (needs a nightly-based/rolling Ferrocene; drop the flag on stable) |
224236

0 commit comments

Comments
 (0)