Skip to content

Code review, fixes, and unit tests for the reporting/logging plugins - #1264

Merged
mkarlesky merged 11 commits into
next_versionfrom
reporting-plugins-review
Sep 5, 2026
Merged

Code review, fixes, and unit tests for the reporting/logging plugins#1264
mkarlesky merged 11 commits into
next_versionfrom
reporting-plugins-review

Conversation

@mkarlesky

Copy link
Copy Markdown
Member

Summary

Code review pass across the test-reporting/logging plugins
(report_build_warnings_log, report_tests_raw_output_log, the four
report_tests_*_stdout plugins, their shared base classes, and
report_tests_log_factory), which collectively had zero unit test
coverage
apart from one trivial one-assertion spec. Found and fixed real
bugs — data corruption, malformed XML, a broken CI-integration escape
function, thread-safety races — none of which existing tests could have
caught, then built the coverage that should have caught them, with zero
filesystem/stdout interaction in any test.

Bugs fixed

  • Thread-safety race in report_build_warnings_log: the write path
    iterated the collected-warnings hash with no lock, while collection into
    that same hash was always mutex-guarded.
  • XML/HTML injection: report_tests_log_factory's JUnit reporter never
    escaped failure messages; CppUnit and HTML escaped nothing at all in test
    names, messages, or filepaths.
  • Cross-reporter data corruption: JUnit's escaping mutated the shared
    results structure's own strings in place (gsub!); since the same
    results object is handed to every configured reporter in turn, one
    reporter's escaping could silently leak into another's output depending
    on :reports: order in project configuration.
  • Mislabeled ignored tests: the GTest-like report tagged ignored tests
    as passes, rendering them [ OK ] with their message dropped, and its
    footer counts didn't sum to its header total when ignores existed.
  • Broken TeamCity escaping: escape used single-quoted '\r'/'\n'
    (literal backslash-r/backslash-n, not real control characters) — verified
    empirically that a real embedded newline passed straight through,
    corrupting/splitting ##teamcity[...] service messages.
  • Missing IDE report features: no multi-line message reindentation
    (unlike its "identical" pretty sibling) and no summary banner color
    decoration.

Refactoring

  • New ReportLogWriterPlugin base class shared by report_build_warnings_log
    and report_tests_raw_output_log — one mutex discipline, one FileWrapper-based
    write path, instead of two independently-hand-rolled (and inconsistently
    correct) copies.
  • New shared xml_escape/html_escape on TestsReporter, used
    non-destructively by all three affected reporters.
  • New shared test_report_preamble/reflow_message/decorate_summary_banner
    on PluginReportinator, now used by both the pretty and IDE stdout
    templates — the exact duplication that let IDE quietly drift out of sync
    with pretty in the first place.
  • TestsReporter#write and the TeamCity plugin's teamcity_service_message
    now write through injectable seams (FileWrapper, a @stream reference)
    instead of raw File.open/bare puts — the reason none of this had any
    test coverage before.
  • eval-based dynamic reporter instantiation replaced with Object.const_get.

Testing

  • Full unit suite: 3204 examples, 0 failures (up from 3112 baseline before
    this and the prior PR in this branch's lineage).
  • First-ever unit test coverage added for 13 previously-untested
    files/classes across all the plugins above.
  • Live end-to-end smoke tests: a fresh example project's full 86-test suite
    through the default (pretty) template, and all four report_tests_log_factory
    report formats generated and validated as well-formed JSON/XML against a
    real build.
  • Existing report_tests_raw_output_log system test re-verified green
    after the shared-base refactor.

No documentation or Changelog updates — deferred to a follow-on plan per
plan.

🤖 Generated with Claude Code

mkarlesky and others added 11 commits September 5, 2026 00:52
New base for build-artifact log-writing plugins: owns the mutex
discipline around collecting concurrent build-thread output and the
actual file write, so a subclass's collection logic and its own flush
logic can't drift out of sync the way they previously could when each
plugin hand-rolled both independently.

report_build_warnings_log and report_tests_raw_output_log will be
refactored onto this base next.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
report_build_warnings_log and report_tests_raw_output_log now build on
the shared ReportLogWriterPlugin base: both plugins' writes go through
the same mutex discipline their own collection already used (write_logs
previously iterated the collected-warnings hash with no lock at all,
while collection into that same hash was always mutex-guarded), and
both write via the already-injected FileWrapper instead of a raw
File.open, making them unit-testable without touching disk.

report_build_warnings_log also now keeps only the line(s) that actually
look like a warning, rather than the entire shell-output blob for any
build step where the word "warning" appears anywhere in it. @warnings
is a plain Hash rather than one with an auto-vivifying default block,
so reading an absent context can no longer silently create an entry.

Adds first unit test coverage for both plugins (previously zero).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…_factory

JUnit escaped a failing test's own name but never its failure message,
producing malformed XML on any message containing ", <, &, or a
newline. CppUnit and HTML escaped nothing at all in test names,
messages, or filepaths -- a real HTML-injection risk in the latter.

Escaping now goes through two small shared helpers (xml_escape,
html_escape) on the TestsReporter base class, both returning new
strings rather than mutating in place. JUnit previously used gsub! on
the shared results structure's own strings -- since the very same
results object is handed unmodified to every configured reporter in
turn, one reporter's escaping could silently leak into another's
output depending on :reports: order in project configuration.

JSON and HTML also recomputed a "passed" count via subtraction from
:total instead of using the already-aggregated counts[:passed] --
duplicated logic that could drift from the real count; both now use it
directly.

TestsReporter#write renders into an in-memory buffer and hands the
complete content to an injected FileWrapper instead of opening a raw
File itself, and load_reporters instantiates reporter subclasses via
Object.const_get instead of eval. First unit test coverage for the
main plugin, the TestsReporter base class, and all four built-in
reporters (previously zero).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
test_report_preamble, reflow_message, and decorate_summary_banner give
the pretty and IDE stdout report templates one shared place for logic
they'd otherwise each reimplement independently -- exactly the
duplication that already let them quietly drift apart (the IDE
template is missing both multi-line message reindentation and summary
banner color decoration that pretty's template already has).

Also fills in previously-nonexistent coverage for
run_test_results_report/run_report and assemble_test_results -- only
test_results_floor_verbosity had any test before this.

Templates are wired to call these next.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…pers

Preamble computation, multi-line message reindentation, and summary
banner color decoration now go through test_report_preamble,
reflow_message, and decorate_summary_banner instead of being
reimplemented inline in this template -- proven behavior-preserving by
a new spec asserting the real, rendered output byte-for-byte before
and after. Also drops a dead banner_width local left over from before
banner width became self-sizing.

Adds first unit test coverage for this template (previously zero).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
DEFAULT_TESTS_RESULTS_REPORT_TEMPLATE (this plugin's own template
asset, left in place in lib/ceedling/defaults.rb) never re-indented a
multi-line failure/ignore message, unlike its "identical" pretty
sibling -- a continuation line after an embedded newline had no
file:line:test prefix at all, undermining the plugin's whole
documented purpose (an IDE jumping straight to a failing test from
that line). The summary banner was also never color-decorated,
another undocumented divergence from "identical to pretty except one
difference."

Both now go through the same shared PluginReportinator helpers pretty
already uses. Adds first unit test coverage for this template
(previously zero).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Ignored tests were folded into the same per-file collection as real
passes, tagged :pass = true, and rendered as [ OK ] with their ignore
message dropped entirely -- and the header's total (which includes
ignores) never matched the footer's PASSED+FAILED sum.

GTest's own format has no "ignored" concept, so ignored tests are now
left out of this report entirely: never entered into the per-file
results hash, never counted in the header/footer total (now
passed+failed, not counts[:total]), never given a per-test line. A
file whose only tests are ignored now produces no per-file section at
all, and a build where every test was ignored renders "No tests
executed." -- the deliberate, fully-consistent consequence of treating
this format as only ever knowing about pass/fail, called out with an
in-template comment so it isn't mistaken for a regression later.

Adds first unit test coverage for this template (previously zero).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
teamcity_service_message called bare `puts`, making this plugin
untestable without stubbing Kernel#puts globally. @stream (defaulting
to $stdout, set once in setup) replaces it -- identical runtime
behavior, but a test can now substitute a StringIO.

Deliberately not routed through Loginator: that would risk both
suppression under a quiet --verbosity and batching/timing changes to
what TeamCity expects as immediate, real-time service messages.

Standalone, behavior-preserving -- no test coverage added yet.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
escape used single-quoted '\r'/'\n' -- the literal two-character
sequences backslash-r/backslash-n, not real carriage-return/newline
control characters. Verified empirically that a real embedded newline
passed straight through unescaped, letting a failure message split a
##teamcity[...] service message across physical lines and corrupt it
from TeamCity's line-based parser's point of view.

Adds first unit test coverage for this plugin (previously zero),
covering flow-id bookkeeping, message shapes for success/failure/
ignored test cases, and this fix.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
post_test_fixture_execute recompiled a Regexp from
Regexp.escape(PROJECT_TEST_RESULTS_PATH) on every test-fixture
completion just to check containment -- a String#include? does the
same job directly, without the regex-compilation overhead or the need
to reason about escaping at all. Behavior-preserving, including the
#104 bracket-path case a regex-based check once got wrong.

Adds first unit test coverage for this shared base class (previously
zero), covering post_test_fixture_execute's dedup/matching, post_build,
and summary -- all three stdout report subclasses inherit this
coverage rather than needing to re-test the same hook plumbing each.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@mkarlesky
mkarlesky merged commit 656ab0e into next_version Sep 5, 2026
5 of 12 checks passed
@mkarlesky
mkarlesky deleted the reporting-plugins-review branch September 5, 2026 13:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant