Code review, fixes, and unit tests for the reporting/logging plugins - #1264
Merged
Conversation
New base for build-artifact log-writing plugins: owns the mutex discipline around collecting concurrent build-thread output and the actual file write, so a subclass's collection logic and its own flush logic can't drift out of sync the way they previously could when each plugin hand-rolled both independently. report_build_warnings_log and report_tests_raw_output_log will be refactored onto this base next. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
report_build_warnings_log and report_tests_raw_output_log now build on the shared ReportLogWriterPlugin base: both plugins' writes go through the same mutex discipline their own collection already used (write_logs previously iterated the collected-warnings hash with no lock at all, while collection into that same hash was always mutex-guarded), and both write via the already-injected FileWrapper instead of a raw File.open, making them unit-testable without touching disk. report_build_warnings_log also now keeps only the line(s) that actually look like a warning, rather than the entire shell-output blob for any build step where the word "warning" appears anywhere in it. @warnings is a plain Hash rather than one with an auto-vivifying default block, so reading an absent context can no longer silently create an entry. Adds first unit test coverage for both plugins (previously zero). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…_factory JUnit escaped a failing test's own name but never its failure message, producing malformed XML on any message containing ", <, &, or a newline. CppUnit and HTML escaped nothing at all in test names, messages, or filepaths -- a real HTML-injection risk in the latter. Escaping now goes through two small shared helpers (xml_escape, html_escape) on the TestsReporter base class, both returning new strings rather than mutating in place. JUnit previously used gsub! on the shared results structure's own strings -- since the very same results object is handed unmodified to every configured reporter in turn, one reporter's escaping could silently leak into another's output depending on :reports: order in project configuration. JSON and HTML also recomputed a "passed" count via subtraction from :total instead of using the already-aggregated counts[:passed] -- duplicated logic that could drift from the real count; both now use it directly. TestsReporter#write renders into an in-memory buffer and hands the complete content to an injected FileWrapper instead of opening a raw File itself, and load_reporters instantiates reporter subclasses via Object.const_get instead of eval. First unit test coverage for the main plugin, the TestsReporter base class, and all four built-in reporters (previously zero). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
test_report_preamble, reflow_message, and decorate_summary_banner give the pretty and IDE stdout report templates one shared place for logic they'd otherwise each reimplement independently -- exactly the duplication that already let them quietly drift apart (the IDE template is missing both multi-line message reindentation and summary banner color decoration that pretty's template already has). Also fills in previously-nonexistent coverage for run_test_results_report/run_report and assemble_test_results -- only test_results_floor_verbosity had any test before this. Templates are wired to call these next. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…pers Preamble computation, multi-line message reindentation, and summary banner color decoration now go through test_report_preamble, reflow_message, and decorate_summary_banner instead of being reimplemented inline in this template -- proven behavior-preserving by a new spec asserting the real, rendered output byte-for-byte before and after. Also drops a dead banner_width local left over from before banner width became self-sizing. Adds first unit test coverage for this template (previously zero). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
DEFAULT_TESTS_RESULTS_REPORT_TEMPLATE (this plugin's own template asset, left in place in lib/ceedling/defaults.rb) never re-indented a multi-line failure/ignore message, unlike its "identical" pretty sibling -- a continuation line after an embedded newline had no file:line:test prefix at all, undermining the plugin's whole documented purpose (an IDE jumping straight to a failing test from that line). The summary banner was also never color-decorated, another undocumented divergence from "identical to pretty except one difference." Both now go through the same shared PluginReportinator helpers pretty already uses. Adds first unit test coverage for this template (previously zero). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Ignored tests were folded into the same per-file collection as real passes, tagged :pass = true, and rendered as [ OK ] with their ignore message dropped entirely -- and the header's total (which includes ignores) never matched the footer's PASSED+FAILED sum. GTest's own format has no "ignored" concept, so ignored tests are now left out of this report entirely: never entered into the per-file results hash, never counted in the header/footer total (now passed+failed, not counts[:total]), never given a per-test line. A file whose only tests are ignored now produces no per-file section at all, and a build where every test was ignored renders "No tests executed." -- the deliberate, fully-consistent consequence of treating this format as only ever knowing about pass/fail, called out with an in-template comment so it isn't mistaken for a regression later. Adds first unit test coverage for this template (previously zero). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
teamcity_service_message called bare `puts`, making this plugin untestable without stubbing Kernel#puts globally. @stream (defaulting to $stdout, set once in setup) replaces it -- identical runtime behavior, but a test can now substitute a StringIO. Deliberately not routed through Loginator: that would risk both suppression under a quiet --verbosity and batching/timing changes to what TeamCity expects as immediate, real-time service messages. Standalone, behavior-preserving -- no test coverage added yet. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
escape used single-quoted '\r'/'\n' -- the literal two-character sequences backslash-r/backslash-n, not real carriage-return/newline control characters. Verified empirically that a real embedded newline passed straight through unescaped, letting a failure message split a ##teamcity[...] service message across physical lines and corrupt it from TeamCity's line-based parser's point of view. Adds first unit test coverage for this plugin (previously zero), covering flow-id bookkeeping, message shapes for success/failure/ ignored test cases, and this fix. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
post_test_fixture_execute recompiled a Regexp from Regexp.escape(PROJECT_TEST_RESULTS_PATH) on every test-fixture completion just to check containment -- a String#include? does the same job directly, without the regex-compilation overhead or the need to reason about escaping at all. Behavior-preserving, including the #104 bracket-path case a regex-based check once got wrong. Adds first unit test coverage for this shared base class (previously zero), covering post_test_fixture_execute's dedup/matching, post_build, and summary -- all three stdout report subclasses inherit this coverage rather than needing to re-test the same hook plumbing each. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Code review pass across the test-reporting/logging plugins
(
report_build_warnings_log,report_tests_raw_output_log, the fourreport_tests_*_stdoutplugins, their shared base classes, andreport_tests_log_factory), which collectively had zero unit testcoverage apart from one trivial one-assertion spec. Found and fixed real
bugs — data corruption, malformed XML, a broken CI-integration escape
function, thread-safety races — none of which existing tests could have
caught, then built the coverage that should have caught them, with zero
filesystem/stdout interaction in any test.
Bugs fixed
report_build_warnings_log: the write pathiterated the collected-warnings hash with no lock, while collection into
that same hash was always mutex-guarded.
report_tests_log_factory's JUnit reporter neverescaped failure messages; CppUnit and HTML escaped nothing at all in test
names, messages, or filepaths.
results structure's own strings in place (
gsub!); since the sameresults object is handed to every configured reporter in turn, one
reporter's escaping could silently leak into another's output depending
on
:reports:order in project configuration.as passes, rendering them
[ OK ]with their message dropped, and itsfooter counts didn't sum to its header total when ignores existed.
escapeused single-quoted'\r'/'\n'(literal backslash-r/backslash-n, not real control characters) — verified
empirically that a real embedded newline passed straight through,
corrupting/splitting
##teamcity[...]service messages.(unlike its "identical" pretty sibling) and no summary banner color
decoration.
Refactoring
ReportLogWriterPluginbase class shared byreport_build_warnings_logand
report_tests_raw_output_log— one mutex discipline, one FileWrapper-basedwrite path, instead of two independently-hand-rolled (and inconsistently
correct) copies.
xml_escape/html_escapeonTestsReporter, usednon-destructively by all three affected reporters.
test_report_preamble/reflow_message/decorate_summary_banneron
PluginReportinator, now used by both the pretty and IDE stdouttemplates — the exact duplication that let IDE quietly drift out of sync
with pretty in the first place.
TestsReporter#writeand the TeamCity plugin'steamcity_service_messagenow write through injectable seams (
FileWrapper, a@streamreference)instead of raw
File.open/bareputs— the reason none of this had anytest coverage before.
eval-based dynamic reporter instantiation replaced withObject.const_get.Testing
this and the prior PR in this branch's lineage).
files/classes across all the plugins above.
through the default (pretty) template, and all four
report_tests_log_factoryreport formats generated and validated as well-formed JSON/XML against a
real build.
report_tests_raw_output_logsystem test re-verified greenafter the shared-base refactor.
No documentation or Changelog updates — deferred to a follow-on plan per
plan.
🤖 Generated with Claude Code