Repository navigation
Commit 5270162
committed
fix(extract): budget tree-sitter parses by thread CPU time, not wall clock
The per-file tree-sitter parse was bounded by a 5s WALL-CLOCK budget. Under CI
contention a worker thread can be descheduled long enough for 5s of wall time
to elapse against near-zero CPU, so a small, perfectly parseable file was
abandoned with a false "parse timeout" and yielded ZERO defs. When that file
defined a symbol other files resolve against (e.g. celery.app.Celery.send_task
backing an ASYNC_CALLS edge, or a userpb gRPC client backing GRPC_CALLS), the
cross-file service edge then failed to classify -- surfacing as the flaky
convergence_probe / edge_types_probe service-edge assertions (count 0), a
verdict decided by wall-clock scheduling rather than the input (violates O9,
"no timeout decides a test").
Gate the parse on per-thread CPU time instead, with a generous wall ceiling as
a backstop:
- New portable helper cbm_thread_cpu_time_ns() (compat.c/.h): POSIX/macOS
clock_gettime(CLOCK_THREAD_CPUTIME_ID), Windows GetThreadTimes (kernel+user);
returns 0 on clock failure, degrading safely to the wall ceiling. It is NOT
cbm_clock_gettime, which is wall on every platform (QueryPerformanceCounter
on Windows).
- The progress callback's payload becomes {cpu_deadline_ns, wall_ceiling_ns};
it trips when this thread's CPU time passes the budget OR wall time passes
t0 + budget * 12 (~60s for the 5s budget). Descheduling burns wall, not CPU,
so a starved-but-parseable file survives; a CPU-spinning parse still trips
the CPU deadline; a genuinely stuck parse still trips the wall ceiling
(bounded, no hang).
RED-repro (deterministic, CBM_ENABLE_TEST_SEAMS only): a thread-local wall-clock
offset armed by CBM_TEST_WALL_STALL_ON emulates a descheduled worker (wall
reads budget + 1s ahead while CPU is untouched -- above the old 1x wall budget,
below the ceiling). async_calls_celery_wall_stall_seam arms it for the Celery
DEFINITION file and asserts ASYNC_CALLS >= 1. With the fix the parse completes
and the edge resolves; with the budget reverted to wall-only + a 1x ceiling the
def file is dropped and the edge count is 0.
Verified (macOS, sanitized): edge_types_probe + convergence_probe 107 passed /
0 failed with the fix; 58 passed / 1 failed on revert (only the seam test, at
the ASYNC_CALLS assertion); 107 restored. Regression smoke (extraction
parse_coverage pipeline lang_contract grammar_regression) 698 passed / 0
failed. lint-ci clean (cppcheck + clang-format Homebrew LLVM + NOLINT); the
Windows GetThreadTimes path is compile-checked by CI (not runnable on macOS).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>1 parent 055fbb7 commit 5270162
4 files changed
Lines changed: 120 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
213 | 213 | | |
214 | 214 | | |
215 | 215 | | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
216 | 245 | | |
217 | | - | |
218 | | - | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
219 | 252 | | |
220 | 253 | | |
221 | 254 | | |
| |||
1620 | 1653 | | |
1621 | 1654 | | |
1622 | 1655 | | |
1623 | | - | |
| 1656 | + | |
1624 | 1657 | | |
1625 | | - | |
1626 | | - | |
| 1658 | + | |
| 1659 | + | |
| 1660 | + | |
| 1661 | + | |
| 1662 | + | |
| 1663 | + | |
| 1664 | + | |
1627 | 1665 | | |
| 1666 | + | |
| 1667 | + | |
| 1668 | + | |
| 1669 | + | |
| 1670 | + | |
| 1671 | + | |
| 1672 | + | |
| 1673 | + | |
| 1674 | + | |
| 1675 | + | |
1628 | 1676 | | |
1629 | 1677 | | |
1630 | 1678 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
316 | 316 | | |
317 | 317 | | |
318 | 318 | | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
319 | 343 | | |
320 | 344 | | |
321 | 345 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| 14 | + | |
14 | 15 | | |
15 | 16 | | |
16 | 17 | | |
| |||
99 | 100 | | |
100 | 101 | | |
101 | 102 | | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
102 | 113 | | |
103 | 114 | | |
104 | 115 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
842 | 842 | | |
843 | 843 | | |
844 | 844 | | |
| 845 | + | |
| 846 | + | |
| 847 | + | |
| 848 | + | |
| 849 | + | |
| 850 | + | |
| 851 | + | |
| 852 | + | |
| 853 | + | |
| 854 | + | |
| 855 | + | |
| 856 | + | |
| 857 | + | |
| 858 | + | |
| 859 | + | |
| 860 | + | |
| 861 | + | |
| 862 | + | |
| 863 | + | |
| 864 | + | |
| 865 | + | |
| 866 | + | |
| 867 | + | |
| 868 | + | |
| 869 | + | |
| 870 | + | |
| 871 | + | |
| 872 | + | |
| 873 | + | |
| 874 | + | |
| 875 | + | |
845 | 876 | | |
846 | 877 | | |
847 | 878 | | |
| |||
1625 | 1656 | | |
1626 | 1657 | | |
1627 | 1658 | | |
| 1659 | + | |
1628 | 1660 | | |
1629 | 1661 | | |
1630 | 1662 | | |
| |||
0 commit comments