Skip to content

Commit 3f3da36

Browse files
shenxianpengclaude
andcommitted
fix: count things checked, not rule evaluations
The header on this pull request read "1 of 100 checks failed". The number was arithmetically right — sixteen commit messages against six enabled rules, plus two branch rules and one each for author name and email — but it was the wrong number to show. It measured the size of the pull request rather than the strictness of the policy: the same configuration reports 100 on a fifteen-commit branch and 10 on a one-commit branch. Ninety-six of the hundred were the same six rules run again per commit, so the total also overstated the work done. Worst of all, the inflated denominator made a real failure look negligible: one bad commit out of fifteen is not "1 of 100". A check is now one thing that was checked — a commit message, the branch, the author — so the same report reads "1 of 19 checks failed", and the total is what the reader can count in the details block. Rule-level detail is unchanged; it lives in the table and the details, which is where someone goes to find out what to fix. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U9zFxq8V4qxG4aMzJhGBFn
1 parent 56e6f50 commit 3f3da36

2 files changed

Lines changed: 52 additions & 33 deletions

File tree

‎main.py‎

Lines changed: 25 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -403,29 +403,27 @@ def render_step_log(results: list[ScopeResult]) -> None:
403403

404404

405405
def _check_counts(results: list[ScopeResult]) -> tuple[int, int]:
406-
"""Return ``(failed, total)`` counted in checks, not scopes.
407-
408-
One unit throughout: the header used to read ``2 failures \u00b7 9 passed
409-
(11 scopes)``, where the first number counted checks and the other two
410-
counted scopes, so the three never added up.
411-
412-
A scope whose output could not be parsed contributes one failed check \u2014
413-
it has no check list to count, but it is a problem the reader has to act
414-
on and must not vanish from the totals.
406+
"""Return ``(failed, total)`` where one check is one thing that was checked.
407+
408+
A "check" here is a scope \u2014 one commit message, the branch, the author name
409+
\u2014 not one rule evaluation. Counting rule evaluations produced a number that
410+
grew with the size of the pull request rather than with the strictness of
411+
the policy: sixteen commit messages against six enabled rules reported
412+
"1 of 100 checks failed", where 96 of the 100 were the same six rules run
413+
again per commit. The large denominator also made a real failure look
414+
negligible \u2014 one bad commit out of fifteen reads very differently from
415+
1 of 100.
416+
417+
This number matches what the reader can count: the rows in the table plus
418+
the \u2714/\u2716 lines in the details block. Which rules failed is not lost, it is
419+
just reported where it belongs \u2014 in the table and the details.
415420
"""
416-
failed = total = 0
417-
for scope in results:
418-
if scope.raw_text and not scope.checks:
419-
failed += 1
420-
total += 1
421-
continue
422-
failed += len(scope.failures)
423-
total += len(scope.checks)
424-
return failed, total
421+
failed = sum(1 for scope in results if scope.status == "fail")
422+
return failed, len(results)
425423

426424

427425
def _failure_count(results: list[ScopeResult]) -> int:
428-
"""Number of failed checks across every scope."""
426+
"""Number of scopes that failed."""
429427
return _check_counts(results)[0]
430428

431429

@@ -528,10 +526,10 @@ def _scope_value(scope: ScopeResult, max_len: int = 60) -> str:
528526
# <!-- commit-check-action -->
529527
# ## <img src="..." width="20" align="top" alt=""> Commit Check
530528
#
531-
# ✅ **All 28 checks passed**
529+
# ✅ **All 5 checks passed**
532530
#
533531
# <details>
534-
# <summary>Show all 28 checks</summary>
532+
# <summary>Show all 5 checks</summary>
535533
#
536534
# ```text
537535
# Commit message
@@ -553,14 +551,14 @@ def _scope_value(scope: ScopeResult, max_len: int = 60) -> str:
553551
# <!-- commit-check-action -->
554552
# ## <img src="..." width="20" align="top" alt=""> Commit Check
555553
#
556-
# ❌ **2 of 28 checks failed**
554+
# ❌ **1 of 5 checks failed**
557555
#
558556
# | Scope | Checked value | Failed checks |
559557
# |---|---|---|
560558
# | Commit 2/11 | `bad msg` | [CC001 message](https://commit-check.com/rules/#cc001) |
561559
#
562560
# <details>
563-
# <summary>Show all 28 checks</summary>
561+
# <summary>Show all 5 checks</summary>
564562
#
565563
# ```text
566564
# Commit message
@@ -577,8 +575,10 @@ def _scope_value(scope: ScopeResult, max_len: int = 60) -> str:
577575
# _commit-check 2.13.1 · [Rules reference](https://commit-check.com/rules/)_
578576
#
579577
# Notes:
580-
# - Counts are in checks, never scopes, so the numbers in the header always
581-
# reconcile against the number in the details summary.
578+
# - One check is one thing that was checked — a commit message, the branch, the
579+
# author — not one rule evaluation. The total therefore matches the number of
580+
# ✔/✖ lines the reader can count in the details block, and does not grow with
581+
# the number of commits in the pull request or rules in the config.
582582
# - The table lists only failed scopes; there is no per-row result column
583583
# because it would read ❌ on every row. Passing scopes live in the details.
584584
# - Values are capped at 60 characters with a literal "..." suffix and shown

‎main_test.py‎

Lines changed: 27 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -711,12 +711,14 @@ def test_failure_golden_output(self):
711711
f"{FOOTER}",
712712
)
713713

714-
def test_counts_are_checks_not_scopes(self):
715-
"""The header numbers must reconcile: failed + passed == total.
716-
717-
They previously did not — failures were counted in checks while the
718-
rest of the line counted scopes, so a scope failing two rules rendered
719-
"2 failures · 2 passed (3 scopes)".
714+
def test_a_check_is_a_thing_checked_not_a_rule_evaluation(self):
715+
"""The total counts scopes, so it tracks the policy, not the PR size.
716+
717+
Counting rule evaluations made the denominator grow with the number of
718+
commits: sixteen messages against six enabled rules reported "1 of 100
719+
checks failed", which both overstated the work done and made one bad
720+
commit out of fifteen look negligible. Two rules failing on one commit
721+
is still one thing to go and fix.
720722
"""
721723
two_failures = main.ScopeResult(
722724
label="Commit 1/2",
@@ -726,8 +728,25 @@ def test_counts_are_checks_not_scopes(self):
726728
],
727729
)
728730
body = main.render_report([pass_scope("Branch"), two_failures])
729-
self.assertIn("❌ **2 of 3 checks failed**", body)
730-
self.assertIn("<summary>Show all 3 checks</summary>", body)
731+
self.assertIn("❌ **1 of 2 checks failed**", body)
732+
self.assertIn("<summary>Show all 2 checks</summary>", body)
733+
# Both failing rules are still named, in the table and the details.
734+
self.assertIn("CC001 message", body)
735+
self.assertIn("CC005 subject-min-length", body)
736+
737+
def test_total_does_not_grow_with_the_number_of_rules(self):
738+
"""Adding rules to one scope must not change the headline total."""
739+
one_rule = [pass_scope("Branch")]
740+
many_rules = [
741+
main.ScopeResult(
742+
label="Branch",
743+
checks=[
744+
make_check(f"rule_{i}", rule_id=f"CC{i:03d}") for i in range(9)
745+
],
746+
)
747+
]
748+
self.assertIn("✅ **All 1 check passed**", main.render_report(one_rule))
749+
self.assertIn("✅ **All 1 check passed**", main.render_report(many_rules))
731750

732751
def test_table_has_no_constant_result_column(self):
733752
"""Only failures reach the table, so a result column would never vary."""

0 commit comments

Comments
 (0)