Skip to content

Commit 23b53f6

Browse files
shenxianpengclaude
andcommitted
feat: render a skipped check as skipped, not as a green tick
PR #258 in this repository reported "All 5 checks passed" over five green ticks while validating nothing: its author is dependabot[bot], which the org config lists in ignore_authors, so every rule was bypassed. The report had no way to say so -- ScopeResult knew only pass and fail, and a skip arrived looking exactly like a pass. The bare labels in that summary were the only hint anything was different, and only because a skipped check reports no value. That was incidental, not a signal. commit-check 2.13.4 reports "status": "skip", so this consumes it: * ScopeResult.status gains "skip", set when every check in the scope skipped. One real verdict outranks the skips. * Skipped scopes render "⊘ <label> (skipped)" -- deliberately not a ✔, and with no value, because nothing was examined. * The headline distinguishes the three cases. All skipped reads "⊘ All N checks skipped — nothing was validated"; a partial skip reads "✅ 3 of 5 checks passed, 2 skipped" rather than claiming all passed. * The step log line follows the same rule. Back-compatible by construction: against an engine that never emits "skip" no branch here is reachable, and a test pins that the old rendering is unchanged. Adds a golden test for the fully skipped report so the layout stays exact, alongside tests for the partial-skip headline, failure precedence, and the one-real-verdict rule. The output specification comment and the README gain the skipped case, the README's rendered from the real renderer. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U9zFxq8V4qxG4aMzJhGBFn
1 parent 124de73 commit 23b53f6

3 files changed

Lines changed: 183 additions & 7 deletions

File tree

‎README.md‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -303,6 +303,40 @@ A scope is one thing that was checked — a commit message, the branch, the auth
303303
— not one rule evaluation, so the total matches the ✔/✖ lines you can count and
304304
does not grow with the number of rules in your config.
305305

306+
### Skipped Job Summary
307+
308+
Some runs validate nothing at all — most commonly when the commit author is
309+
listed in `ignore_authors`, which is how Dependabot and other bots are usually
310+
exempted. Those runs report `⊘`, never `✔`:
311+
312+
> <img src="https://raw.githubusercontent.com/commit-check/commit-check-action/main/assets/logo.png" width="20" align="top" alt=""> **Commit Check**
313+
>
314+
> ⊘ **All 3 checks skipped** — nothing was validated
315+
>
316+
> <details>
317+
> <summary>Show all 3 checks</summary>
318+
>
319+
> ```text
320+
> Commit message
321+
> ⊘ PR title (skipped)
322+
> ⊘ Commit 1/1 (skipped)
323+
> Branch
324+
> ⊘ Branch (skipped)
325+
> ```
326+
>
327+
> </details>
328+
>
329+
> _commit-check &lt;version&gt; · [Rules reference](https://commit-check.com/rules/)_
330+
331+
A skipped scope carries no checked value, because nothing was examined. When
332+
only some scopes skip, the verdict counts them separately —
333+
`✅ **3 of 5 checks passed**, 2 skipped` — so the headline never claims a pass
334+
that did not happen. Failures still take precedence over skips.
335+
336+
This needs commit-check 2.13.4 or newer, which reports `"status": "skip"` in
337+
its JSON. Against an older engine every check is `pass` or `fail` as before,
338+
and the report is unchanged.
339+
306340
## GitHub Pull Request Comments
307341

308342
With `pr-comments: true` the same report is posted as a pull request comment.

‎main.py‎

Lines changed: 71 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -98,10 +98,24 @@ class ScopeResult:
9898

9999
@property
100100
def status(self) -> str:
101-
"""Overall status: ``pass`` when every check passed."""
101+
"""Overall status: ``pass``, ``fail``, or ``skip``.
102+
103+
``skip`` means every rule in this scope declined to run — the author
104+
is on an ``ignore_authors`` list, or there was nothing to check. It
105+
is reported separately from ``pass`` because a skipped scope
106+
validated nothing, and rendering the two identically let a bypassed
107+
policy read as an enforced one.
108+
109+
A single real verdict outranks the skips: a scope is ``skip`` only
110+
when *all* of its checks skipped.
111+
"""
102112
if self.raw_text and not self.checks:
103113
return "fail"
104-
return "fail" if any(c["status"] == "fail" for c in self.checks) else "pass"
114+
if any(c["status"] == "fail" for c in self.checks):
115+
return "fail"
116+
if self.checks and all(c["status"] == "skip" for c in self.checks):
117+
return "skip"
118+
return "pass"
105119

106120
@property
107121
def failures(self) -> list[dict[str, str]]:
@@ -381,6 +395,11 @@ def _render_scopes(scopes: list[ScopeResult], include_docs: bool) -> list[str]:
381395
"""
382396
lines: list[str] = []
383397
for scope in scopes:
398+
if scope.status == "skip":
399+
# Deliberately not a ✔. Nothing was validated here, and a tick
400+
# claiming otherwise is what made a bypassed policy look enforced.
401+
lines.append(f" ⊘ {scope.label} (skipped)")
402+
continue
384403
if scope.status == "pass":
385404
value = _scope_value(scope)
386405
lines.append(f" ✔ {scope.label}{f' ({value})' if value else ''}")
@@ -464,7 +483,13 @@ def render_step_log(results: list[ScopeResult]) -> None:
464483
)
465484

466485
if not annotations:
467-
print("\u2714 commit-check: all checks passed")
486+
skipped, total = _skip_count(results), len(results)
487+
if total and skipped == total:
488+
print("\u2298 commit-check: all checks skipped, nothing was validated")
489+
elif skipped:
490+
print(f"\u2714 commit-check: all checks passed ({skipped} skipped)")
491+
else:
492+
print("\u2714 commit-check: all checks passed")
468493

469494

470495
def _check_counts(results: list[ScopeResult]) -> tuple[int, int]:
@@ -492,6 +517,15 @@ def _failure_count(results: list[ScopeResult]) -> int:
492517
return _check_counts(results)[0]
493518

494519

520+
def _skip_count(results: list[ScopeResult]) -> int:
521+
"""Number of scopes that never ran.
522+
523+
Reported separately from the pass count so the headline cannot claim
524+
that checks passed when they were skipped.
525+
"""
526+
return sum(1 for scope in results if scope.status == "skip")
527+
528+
495529
def _markdown_table(results: list[ScopeResult]) -> str:
496530
"""Render the failure table shared by summary and PR comment.
497531
@@ -589,6 +623,25 @@ def _scope_value(scope: ScopeResult, max_len: int = 60) -> str:
589623
#
590624
# _commit-check 2.13.1 · [Rules reference](https://commit-check.com/rules/)_
591625
#
626+
# Skipped (every rule declined to run — e.g. the author is in ignore_authors):
627+
#
628+
# ⊘ **All 5 checks skipped** — nothing was validated
629+
#
630+
# ```text
631+
# Commit message
632+
# ⊘ PR title (skipped)
633+
# ⊘ Commit 1/1 (skipped)
634+
# Branch
635+
# ⊘ Branch (skipped)
636+
# Author
637+
# ⊘ Author name (skipped)
638+
# ⊘ Author email (skipped)
639+
# ```
640+
#
641+
# A skipped scope deliberately carries no ✔ and no checked value: nothing was
642+
# examined, so there is no value to report and no pass to claim. When only some
643+
# scopes skip, the verdict reads "✅ **3 of 5 checks passed**, 2 skipped".
644+
#
592645
# Failure:
593646
#
594647
# <!-- commit-check-action -->
@@ -663,15 +716,26 @@ def render_report(results: list[ScopeResult]) -> str:
663716
failure table (failures only) and the collapsible per-scope details.
664717
"""
665718
failed, total = _check_counts(results)
719+
skipped = _skip_count(results)
666720
unit = "check" if total == 1 else "checks"
667721

668722
lines = [COMMENT_MARKER, REPORT_TITLE, ""]
669-
if failed == 0:
670-
lines.append(f"✅ **All {total} {unit} passed**")
671-
lines.append("")
672-
else:
723+
if failed:
673724
lines.append(f"❌ **{failed} of {total} {unit} failed**")
674725
lines.extend(["", _markdown_table(results), ""])
726+
elif skipped == total:
727+
# Nothing ran, so there is no success to announce. Saying "all
728+
# checks passed" here is the defect this branch exists to prevent.
729+
lines.append(f"⊘ **All {total} {unit} skipped** — nothing was validated")
730+
lines.append("")
731+
elif skipped:
732+
lines.append(
733+
f"✅ **{total - skipped} of {total} {unit} passed**, {skipped} skipped"
734+
)
735+
lines.append("")
736+
else:
737+
lines.append(f"✅ **All {total} {unit} passed**")
738+
lines.append("")
675739
lines.extend([_markdown_details(results), "", _report_footer()])
676740
return "\n".join(lines)
677741

‎main_test.py‎

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1355,3 +1355,81 @@ def test_no_comments_yields_nothing_to_update(self):
13551355
target, stale = main._find_own_comments([])
13561356
self.assertIsNone(target)
13571357
self.assertEqual(stale, [])
1358+
1359+
1360+
def skip_scope(label: str = "PR title") -> main.ScopeResult:
1361+
"""A scope whose every rule declined to run (e.g. author in ignore_authors)."""
1362+
return main.ScopeResult(
1363+
label=label,
1364+
checks=[make_check("message", status="skip", rule_id="CC001", value="")],
1365+
)
1366+
1367+
1368+
class TestSkippedScopes(unittest.TestCase):
1369+
"""A skipped scope must never render as a passing one.
1370+
1371+
commit-check-action#258 is the case this guards: every rule was skipped
1372+
because the author is dependabot[bot], and the report announced
1373+
"All 5 checks passed" over five green ticks.
1374+
"""
1375+
1376+
def test_scope_status_is_skip_not_pass(self):
1377+
self.assertEqual(skip_scope().status, "skip")
1378+
1379+
def test_one_real_verdict_outranks_the_skips(self):
1380+
"""A scope only counts as skipped when nothing in it ran."""
1381+
mixed = main.ScopeResult(
1382+
label="PR title",
1383+
checks=[
1384+
make_check("message", status="skip", value=""),
1385+
make_check("subject_max_length", status="pass", value="feat: x"),
1386+
],
1387+
)
1388+
self.assertEqual(mixed.status, "pass")
1389+
1390+
@pin_version
1391+
def test_all_skipped_golden_output(self):
1392+
"""Pin the fully skipped report: no ✔, no pass claim."""
1393+
results = [skip_scope("PR title"), skip_scope("Commit 1/1")]
1394+
body = main.render_report(results)
1395+
self.assertEqual(
1396+
body,
1397+
f"{main.COMMENT_MARKER}\n"
1398+
f"{main.REPORT_TITLE}\n"
1399+
"\n"
1400+
"⊘ **All 2 checks skipped** — nothing was validated\n"
1401+
"\n"
1402+
"<details>\n"
1403+
"<summary>Show all 2 checks</summary>\n"
1404+
"\n"
1405+
"```text\n"
1406+
"Commit message\n"
1407+
" ⊘ PR title (skipped)\n"
1408+
" ⊘ Commit 1/1 (skipped)\n"
1409+
"```\n"
1410+
"\n"
1411+
"</details>\n"
1412+
"\n"
1413+
f"{FOOTER}",
1414+
)
1415+
1416+
def test_partial_skip_does_not_claim_all_passed(self):
1417+
results = [pass_scope("Branch", value="main"), skip_scope("PR title")]
1418+
body = main.render_report(results)
1419+
self.assertIn("✅ **1 of 2 checks passed**, 1 skipped", body)
1420+
self.assertNotIn("All 2 checks passed", body)
1421+
1422+
def test_failure_still_wins_over_skips(self):
1423+
results = [fail_scope("Commit 1/1"), skip_scope("PR title")]
1424+
body = main.render_report(results)
1425+
self.assertIn("❌ **1 of 2 checks failed**", body)
1426+
1427+
def test_skipped_scope_reports_no_checked_value(self):
1428+
"""The ⊘ line carries no value, because nothing was examined."""
1429+
body = main.render_report([skip_scope("PR title")])
1430+
self.assertIn(" ⊘ PR title (skipped)", body)
1431+
1432+
def test_older_engine_without_skip_is_unaffected(self):
1433+
"""Back-compat: engines that only emit pass/fail render as before."""
1434+
results = [pass_scope("Branch", value="main")]
1435+
self.assertIn("✅ **All 1 check passed**", main.render_report(results))

0 commit comments

Comments
 (0)