Repository navigation
Commit 2f33999
committed
test(vm): let the log decide the Windows leg, not the exit status
On 2026-09-18 a Windows leg printed "7925 passed, 0 failed" and
"=== All tests passed ===" and still exited 1, while the identical work run
through win.sh's other entry exited 0. The status travels
ssh -> cmd.exe -> msys2_shell.cmd and that chain loses it.
A channel that can turn 0 into 1 can turn 1 into 0, and that direction is a
false green: a red Windows leg reported as passing, which is the one thing the
local ladder exists to prevent. So the two channels are now split by what each
can actually know. The log decides the test outcome — it is written by the
runner on the VM and cannot be mangled in transit — and rc only decides what
the log cannot see. Green needs zero reported failures AND the completion
marker scripts/test.sh prints last, so a leg that stopped early has a summary
but no marker and is red. A lost exit status is reported, not obeyed.
The logic is a function, vm_verdict(), reachable as `--verdict <log> <rc>`, so
tests/test_vm_verdict_contract.sh drives the REAL code with synthetic logs
instead of a copy that could drift from it: lost exit status, failures under
rc=0, a marker that must not launder failures, failures in a later suite,
partial runs, and no summary at all (which keeps its own exit code 90, so
"never ran" stays distinguishable from "ran and failed"). Eight cases, wired
into scripts/test.sh as step 0y.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>1 parent 11cb476 commit 2f33999
2 files changed
Lines changed: 67 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
278 | 278 | | |
279 | 279 | | |
280 | 280 | | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
281 | 289 | | |
282 | 290 | | |
283 | 291 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
45 | 45 | | |
46 | 46 | | |
47 | 47 | | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
48 | 92 | | |
49 | 93 | | |
50 | 94 | | |
| |||
172 | 216 | | |
173 | 217 | | |
174 | 218 | | |
175 | | - | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
0 commit comments