The test run leaves child processes alive well past Test Run Successful. #321 works around this so CI is green, but the workaround treats the symptom.
How it surfaced
Pinning Fallout to 10.4.0 made CI fail with exit 143 roughly 30 seconds after every test had already passed (41 passed, 5 skipped, Test Run Successful), and with no Fallout target-summary table — so the build never got past the test target.
Fallout waits for the child's output streams to reach EOF. MSBuild's reusable worker nodes and build server outlive dotnet test while still holding the inherited stdout handle, so EOF never arrives, Fallout waits out its grace period, kills the process and surfaces 143.
Evidence
| Observation |
|
dotnet test invoked directly, Fallout out of the picture |
exits on its own — the tests are not what hangs |
Same command via ./build.sh TestNoUi on Fallout 10.4.0 |
hangs until killed |
| After a killed local run |
~45 orphaned dotnet processes plus 2 testhost |
One of those testhost processes |
reached 20.8 GB working set, and held a lock on .nuke/temp/build.log for ~13 minutes before exiting on its own |
With MSBUILDDISABLENODEREUSE=1 + DOTNET_CLI_USE_MSBUILD_SERVER=0 |
CI green, 2m55s |
Why the workaround isn't the fix
Disabling the MSBuild daemons stops them holding the pipe, so the build completes. It does not stop processes being left behind — it just stops them mattering to the build. A future tool that also waits on process exit hits the same wall, and the 20 GB figure suggests something worth understanding on its own.
Fallout 11.x tolerated this, which is why it went unnoticed: the repo had been silently resolving unlisted 11.0.8 for months (see #321). The behaviour was always there; only 10.4 surfaced it.
Worth finding out
Not urgent
CI is green and the workaround is stable. This is worth doing because a build that can't tell "finished" from "still holding a pipe" will bite again somewhere else — not because anything is broken today.
The test run leaves child processes alive well past
Test Run Successful. #321 works around this so CI is green, but the workaround treats the symptom.How it surfaced
Pinning Fallout to 10.4.0 made CI fail with exit 143 roughly 30 seconds after every test had already passed (41 passed, 5 skipped,
Test Run Successful), and with no Fallout target-summary table — so the build never got past the test target.Fallout waits for the child's output streams to reach EOF. MSBuild's reusable worker nodes and build server outlive
dotnet testwhile still holding the inherited stdout handle, so EOF never arrives, Fallout waits out its grace period, kills the process and surfaces 143.Evidence
dotnet testinvoked directly, Fallout out of the picture./build.sh TestNoUion Fallout 10.4.0dotnetprocesses plus 2testhosttesthostprocesses.nuke/temp/build.logfor ~13 minutes before exiting on its ownMSBUILDDISABLENODEREUSE=1+DOTNET_CLI_USE_MSBUILD_SERVER=0Why the workaround isn't the fix
Disabling the MSBuild daemons stops them holding the pipe, so the build completes. It does not stop processes being left behind — it just stops them mattering to the build. A future tool that also waits on process exit hits the same wall, and the 20 GB figure suggests something worth understanding on its own.
Fallout 11.x tolerated this, which is why it went unnoticed: the repo had been silently resolving unlisted 11.0.8 for months (see #321). The behaviour was always there; only 10.4 surfaced it.
Worth finding out
dotnet testseparately and watching for survivors should isolate it.Satisfactory.Presentation.Api.Testsis the prime suspect — it's last to finish and its shutdown log shows Wolverine and TickerQ still working (Application stopping signal received,TimeTickersUPDATE statements) right up to the end.testhosta genuine leak, or an artefact of a dev machine? Caveat worth checking first: that measurement is from a machine with both Satisfactory and Captain of Industry installed, so the real-fixture tests actually execute there and CI skips them.Satisfactory.Infrastructure.Teststook 3m39s locally for 53 tests. The number may not reproduce on CI.Not urgent
CI is green and the workaround is stable. This is worth doing because a build that can't tell "finished" from "still holding a pipe" will bite again somewhere else — not because anything is broken today.