Repository navigation
Description
Description
Not urgent, just documenting a chore. I tried to clean this up in a PR but I think this is more of a policy decision than coding work these days.
The Test Util Scripts workflow (.github/workflows/github-actions-cron-util-test.yml) runs flow/test/*.py. Some unit tests elsewhere in the tree are not run by CI, so they are run on demand and can break unnoticed:
| Tests | Cover | Runner |
|---|---|---|
flow/scripts/memories/{detect,gen_memories,idiomatic,schema}_test.py |
AUTO_MEMORIES |
none |
flow/util/genMetrics_test.py |
genMetrics.py |
none |
tools/FakeRAM2.0/test/*_test.py |
FakeRAM2.0 | tools/FakeRAM2.0/test/test_all.sh |
All of them pass on master (9ca2a9e7).
#4807 tried to fix this by documenting these tests as run on demand, and was closed. Moving them is little coding now that an AI can do the edit; what is open is policy:
- Does every unit test in the tree run in CI?
- Do unit tests live in
flow/test/, where CI and contributors find them, or next to the code they cover? - Does a vendored tool such as FakeRAM2.0 keep its own suite and runner, called from the workflow?
Suggested Solution
If the answers are yes, in flow/test/, and yes:
- Move the memories tests and
genMetrics_test.pyintoflow/test/astest_<name>.py. They use only the Python standard library (plusgitforgenMetrics_test.py), so the workflow needs nothing installed. Each reaches the code under test the wayflow/test/test_convertDrc.pyalready does, asys.pathentry for../scripts/memoriesor../util. - Add
flow/scripts/memories/*.pyto the workflow'spathstriggers. Today a pull request that changes only that directory does not run the workflow. - Run
tools/FakeRAM2.0/test/test_all.shas a workflow step instead of moving its tests: its tests import FakeRAM2.0's modules relative totools/FakeRAM2.0, and the suite has its own runner. Addtools/FakeRAM2.0/**to the triggers.
Additional Context
The workflow's loop, for file in *.py; do python3 "$file"; done, runs under the GitHub Actions default bash -e, so it stops at the first failing test.
Activity