Summary
Codecov's coverage upload (.github/workflows/tests.yml's codecov/codecov-action step, from #51/#54) only reports line coverage. Codecov's separate Test Analytics product — test run times, failure rates, flaky-test detection, and a failed-tests report on PRs — needs a JUnit XML file of the actual test results (pass/fail/duration per test case), uploaded via a second action, codecov/test-results-action@v1.
Codecov's own setup instructions only list JUnit-generation snippets for pytest/Vitest/Jest/PHPUnit — no MATLAB option, since matlab.unittest isn't one of their first-party integrations. It's straightforward regardless: matlab.unittest.plugins.XMLPlugin.producingJUnitFormat(file) is MATLAB's own built-in JUnit-format test-report plugin, and Codecov's ingestion is format-based, not language-based — a JUnit XML file is a JUnit XML file to their action.
Proposed change
test_integration.m: add a second plugin to the existing TestRunner, alongside the current CodeCoveragePlugin:
import matlab.unittest.plugins.XMLPlugin
...
junitReportFile = fullfile(repoRoot, 'junit.xml');
runner.addPlugin(XMLPlugin.producingJUnitFormat(junitReportFile));
(Same repoRoot-relative pattern already used for coverage.xml — see the matlab-ci-testing skill's path-resolution guidance if this file is touched again.)
.github/workflows/tests.yml: add the upload step Codecov's wizard specifies, after the existing test-run step:
- name: Upload test results to Codecov
if: ${{ !cancelled() }}
uses: codecov/test-results-action@v1
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: junit.xml
Uses the same CODECOV_TOKEN secret as the existing coverage-upload step. Note the condition is !cancelled() (Codecov's own recommendation for this specific action), not always() like the existing coverage-upload/artifact steps — worth a moment's thought on whether to align them or whether the distinction is deliberate on Codecov's part.
- Consider also uploading
junit.xml as a build artifact (actions/upload-artifact), matching the existing coverage-report artifact, for local inspection without needing the Codecov dashboard — not required by Codecov itself, just consistent with the existing pattern.
Motivation
40 test cases across 4 matlab.unittest.TestCase classes is enough surface area that flaky-test detection and failure-rate tracking are genuinely useful, not just a formality. This is pure test-infrastructure tooling — no pipeline algorithm or .m production-code behaviour changes anywhere.
Scope note
Sized for v0.9.2, not milestone-assigned yet (no v0.9.2 milestone object exists in this repo yet — see #30/#50/#52/#53 for the same caveat). Not algorithmic, but a new deliverable rather than a fix to existing behaviour.
Acceptance criteria
Summary
Codecov's coverage upload (
.github/workflows/tests.yml'scodecov/codecov-actionstep, from #51/#54) only reports line coverage. Codecov's separate Test Analytics product — test run times, failure rates, flaky-test detection, and a failed-tests report on PRs — needs a JUnit XML file of the actual test results (pass/fail/duration per test case), uploaded via a second action,codecov/test-results-action@v1.Codecov's own setup instructions only list JUnit-generation snippets for pytest/Vitest/Jest/PHPUnit — no MATLAB option, since
matlab.unittestisn't one of their first-party integrations. It's straightforward regardless:matlab.unittest.plugins.XMLPlugin.producingJUnitFormat(file)is MATLAB's own built-in JUnit-format test-report plugin, and Codecov's ingestion is format-based, not language-based — a JUnit XML file is a JUnit XML file to their action.Proposed change
test_integration.m: add a second plugin to the existingTestRunner, alongside the currentCodeCoveragePlugin:repoRoot-relative pattern already used forcoverage.xml— see thematlab-ci-testingskill's path-resolution guidance if this file is touched again.).github/workflows/tests.yml: add the upload step Codecov's wizard specifies, after the existing test-run step:CODECOV_TOKENsecret as the existing coverage-upload step. Note the condition is!cancelled()(Codecov's own recommendation for this specific action), notalways()like the existing coverage-upload/artifact steps — worth a moment's thought on whether to align them or whether the distinction is deliberate on Codecov's part.junit.xmlas a build artifact (actions/upload-artifact), matching the existingcoverage-reportartifact, for local inspection without needing the Codecov dashboard — not required by Codecov itself, just consistent with the existing pattern.Motivation
40 test cases across 4
matlab.unittest.TestCaseclasses is enough surface area that flaky-test detection and failure-rate tracking are genuinely useful, not just a formality. This is pure test-infrastructure tooling — no pipeline algorithm or.mproduction-code behaviour changes anywhere.Scope note
Sized for
v0.9.2, not milestone-assigned yet (nov0.9.2milestone object exists in this repo yet — see #30/#50/#52/#53 for the same caveat). Not algorithmic, but a new deliverable rather than a fix to existing behaviour.Acceptance criteria
test_integration.mproduces ajunit.xmlalongside the existingcoverage.xmlcodecov/test-results-action@v1CODECOV_TOKENsecret already exists from the coverage-upload work — confirm the test-results action can reuse it (should be able to; same repo, same token scope)