Skip to content

fix(runner): propagate pytest exit status - #196

Open
FU-max-boop wants to merge 1 commit into
XS-MLVP:mainfrom
FU-max-boop:fix/propagate-pytest-exit-code
Open

fix(runner): propagate pytest exit status#196
FU-max-boop wants to merge 1 commit into
XS-MLVP:mainfrom
FU-max-boop:fix/propagate-pytest-exit-code

Conversation

@FU-max-boop

Copy link
Copy Markdown

Summary

  • propagate the return code from pytest.main() through main() to the shell
  • keep process_doc_result() after pytest so reports are still finalized for ordinary pytest failures
  • add hardware-free regression coverage for every pytest exit code (0 through 5) and the full main -> SystemExit failure path
  • document the isolated repository-test command with the Toffee pytest plugin disabled

Problem

run.py called pytest.main(...) but discarded its return value, then invoked main() without SystemExit. A failed, interrupted, invalid, or empty test run could therefore complete with shell status 0 and make Make/CI report a false success.

Baseline reproduced in the project container:

python3 -m pytest -q tests             -> 5
python3 run.py ... -- -q tests         -> 0

Result

python3 run.py ... tests/test_run.py                         -> 0
python3 run.py ... -k __unitychip_no_such_test__ tests/...   -> 5

The second path still invokes report post-processing before returning 5.

Validation

  • python3 -m pytest -q -p no:toffee-test tests
    • Python 3.12.3 / pytest 9.1.1: 7 passed
    • workflow image Python 3.10.12 / pytest 8.3.4: 7 passed
  • full CLI glue regression: pytest failure -> report processing -> SystemExit(1)
  • python3 -m py_compile run.py tests/test_run.py
  • git diff --check

The unit tests replace only the comm dependency tree, so they require no DUT build, RTL download, or current project Python dependencies.

Compatibility boundary

This change only propagates pytest return codes. Existing early-return behavior for configuration errors, --download-rtl, and --build, plus existing build_dut failure handling, is unchanged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant