Skip to content

perf(extraction): compute a call's URL context once, and only when read - #2621

Merged
DeusData merged 1 commit into
mainfrom
perf/url-context-once-per-call
Oct 13, 2026
Merged

DeusData merged 1 commit into
mainfrom
perf/url-context-once-per-call

Conversation

@DeusData

Copy link
Copy Markdown
Owner

Tonight's bench of the final release tree (10 LSP corpora, three interleaved runs each, plus the kernel) found csharp 10 % slower than this morning's main (135.5 s against 123.4 s, all three runs consistent).

A per-pass trace put the whole difference in parallel_extract: 24.0 s → 37.6 s. Bisecting over the merge batches pinned it to #2513's first commit (77bc492). A macOS sample diff showed the cost in handle_calls → cbm_service_pattern_match → match_qn → strstr.

Cause

extract_url_or_topic_arg computes url_context (is the callee an HTTP client or a global fetch?) once per positional argument of every call, in every language, through a linear strstr scan over all service patterns. Only the URL-folding functions read the answer, and they return NULL outside Python and JS/TS. The one other reader is the template-string branch.

Fix

A per-call url_context_t computes the answer at most once, and only when an argument reads it. The folding paths ask only in folding languages; the template-string branch asks for itself.

Proof (macOS arm64)

  • C# extraction: parallel_extract 37.4 s → 24.2 s, wall 127–132 s → 119.7 s. Before Preserve topic identities while folding local URL constants #2513 it was 24.7 s / 121.9 s.
  • Byte-identical graphs: the full node and edge sets, properties included, hash the same before and after on django (Python), typescript (TS) and csharp, HTTP_CALLS included (457 / 283 / 39). They also equal the current main bench binary's hashes.
  • Tests: extraction, extraction_imports, pipeline, parallel, infrascan and edge_types_probe: 1009 passed.
  • Lint: cppcheck 2.20 and clang-format are clean.

#2513 (77bc492) computes `url_context` -- is the callee an HTTP client or a
global fetch? -- for every positional argument of every call, in every
language, through cbm_service_pattern_match: a linear strstr over every
service pattern. Only the URL-folding functions read it, and they return
NULL outside Python and JS/TS; the one other reader is the template-string
branch. On the C# corpus (dotnet/runtime, 451k functions) extraction went
from 24.7 s to 37.4 s (+51 %); a macOS sample put the difference in
handle_calls -> cbm_service_pattern_match -> match_qn -> strstr.

A per-call url_context_t now computes the answer at most once and only when
an argument needs it: the folding paths ask only in folding languages, the
template-string branch asks for itself.

Proof (macOS arm64, Apple M5 Pro):
- C# parallel_extract 37.4 s -> 24.2 s, wall 127-132 s -> 119.7 s (the
  pre-#2513 level, 24.7 s / 121.9 s).
- Byte-identical graphs: the full node and edge sets (properties included)
  hash the same before and after on django (Python), typescript (TS) and
  csharp, HTTP_CALLS included (457 / 283 / 39).
- extraction, extraction_imports, pipeline, parallel, infrascan and
  edge_types_probe: 1009 passed.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
@DeusData
DeusData merged commit bb636d1 into main Oct 13, 2026
16 of 18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant