Coverage job reports a number that varies by ~2000 lines between runs of the same code

[DennisOSRM]

Issue

The coverage job reports a number that moves by about 2000 lines between runs of the same code. #7705 tried to fix this by running the cucumber suite serially where coverage is measured. That did not work, and the variance is still there.

Data

Lines covered, as reported by lcov at the end of clang-20-debug-cov:

Commit	Cucumber	Lines covered	Of
adf0e04c	parallel	77716	96461
adf0e04c re-run, identical tree	parallel	75623	96461
0adc8be	parallel	75608	96461
0adc8be re-run, identical tree	parallel	75622	96461
6214e910	serial	75600	96461
6214e910 re-run, identical tree	serial	75597	96461
932a3aa (#7704)	serial	77934	96748
985f45c0 (#7704)	serial	75934	96795

Most runs land near 75600. Occasionally one lands near 77800, about 2100 lines higher. That happens under both settings, so parallelism was not the cause.

The last two rows are not quite the same tree, they differ by 58 lines of unit tests, but that cannot account for 2000 lines and the run with more tests is the lower one.

What has been ruled out

• Parallel writes to shared .gcda files. This was the theory behind ci: make the coverage number reproducible, and stop gating on it #7705. Serial runs still produce both the low and the high value.
• A different amount of work between runs. Both feat(nearest): support multiple coordinates for GET and POST #7704 runs completed all six matrix combinations with identical scenario counts, 1482 each, and ran all 14 unit test binaries.
• Processes dying before flushing coverage. features/lib/osrm_loader.js:63 sends SIGTERM and waits for close, and src/tools/routed.cpp:365 blocks and handles SIGTERM, so osrm-routed exits cleanly.

Why it matters

codecov/project compares against whichever sample the base commit happened to draw. When the base drew a high one, every later pull request looks like a 2.9% regression. This happened on #7702, where the only change in the commit that "regressed" was a 25 line cucumber feature file containing no C++ at all.

The status is informational since #7705, so it no longer blocks. It should stay that way until the number is reproducible.

Suggested next step

Take a high run and a low run and diff their per-file coverage, rather than the totals. Roughly 2100 lines is a large and specific amount, so it is likely one or two files or one subsystem appearing and disappearing rather than noise spread thinly. That would localise it.

Also worth considering: the serialisation added in #7705 costs wall clock on the coverage job and has not been shown to buy anything. Unless it turns out to help for another reason, it can be reverted.

Requirements / Relations

Follows #7705. Noticed on #7702 and again on #7704.