Skip to content

Commit 89bf8e9

Browse files
test: give each test its own PredBat rather than sharing one (#5079)
Every registered test ran against a single PredBat built once at startup, so a test that mutated instance state and did not restore it broke whichever unrelated test happened to follow - depending only on the order they ran in. That has produced a long series of order-dependent failures (#5079), each previously fixed by hand-writing a snapshot/restore block naming the fields that particular test touched. Building the fixture per test removes the whole class instead. Measured on the 53 known-bad (culprit, victim) orderings collected so far: 53 of 53 fail before this change, 0 of 53 after (with #5100, which fixes the one test that also fails on its own). The reason this was not done sooner appears to be an assumption that rebuilding would be too slow. It is not - create_predbat() is about 22ms, and the full suite measures 174s shared against 173s per-test, i.e. no measurable difference, since the suite's cost is dominated by the prediction work inside the tests rather than by fixture setup. Also confirms the tests themselves are sound: all 316 pass when run individually in their own process, so every ordering failure was shared-fixture contamination rather than a test depending on an earlier one to set it up. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent 45e3562 commit 89bf8e9

1 file changed

Lines changed: 9 additions & 0 deletions

File tree

‎apps/predbat/unit_test.py‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -819,6 +819,8 @@ def main():
819819
sys.exit(0)
820820

821821
print("**** Starting Predbat tests ****")
822+
# Used by the debug-file, random-scenario and profiling paths below. The registry test
823+
# loop does not share this one - it builds a fresh instance per test (see GH#5079).
822824
my_predbat = create_predbat()
823825
print("**** Testing Predbat ****")
824826
failed = False
@@ -897,6 +899,13 @@ def main():
897899
# still running (or one that hung) is identified by its unmatched start stamp.
898900
print(f"**** Running: {name} - {desc} (start {time.strftime('%H:%M:%S')}) ****")
899901

902+
# Each test gets a PredBat of its own, so state an earlier test left behind cannot
903+
# reach it. Sharing one instance across the whole run made a test that mutated it
904+
# without restoring break whichever unrelated test happened to follow, depending on
905+
# the order they ran in (GH#5079). create_predbat() costs about 22ms, which is lost
906+
# in the noise of the suite as a whole.
907+
my_predbat = create_predbat()
908+
900909
start_time = time.time()
901910
test_failed = func(my_predbat)
902911
elapsed = time.time() - start_time

0 commit comments

Comments
 (0)