Repository navigation
Preserve topic identities while folding local URL constants - #2513
Merged
Merged
Conversation
Owner
Author
|
Checkpoint and handover (2026-10-03 UTC) Published head: URL/topic folding is published, including the formatting follow-up. #2519 is the dependent client-navigation change. Resume by checking this prerequisite before its dependent PR; no fresh local runtime validation was run. Hosted snapshot at 2026-10-03 21:54:11 UTC: 5 queued, 5 success. Confirm the required checks on this exact head before treating it as ready. No additional local build, test, lint, sanitizer, benchmark or CI runs were performed at this checkpoint, as requested. Earlier executed evidence remains historical; prepared tests and the newer source-reviewed changes must still be validated by the hosted gate. The campaign is paused at the maintainer’s request. Local monitoring has stopped; hosted jobs remain running. No merge was performed. Thanks for reviewing this change. |
Refs #1186 Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Use tracked allocations for binding arrays and namespace keys, share IMPORTS cleanup, and consolidate namespace construction failure cleanup. Preserve libc ownership for existing FQN values and JSON writer buffers. Remove the redundant condition in binding validation. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
collect_use_declaration dropped the group prefix of `use A\B\{C as D}`
and read the function/const keyword only from the declaration, so a
grouped alias mapped to the bare `C` with the declaration's kind. The
cross-file LSP then could not recognise a grouped vendor class import as
external, and a grouped `function`/`const` alias was registered as a
class. Prefix each grouped clause with the group namespace and take the
keyword from the clause when the declaration has none, mirroring
emit_php_use_decl in extract_imports.c.
Refs #1186
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
When a cross-file lsp_cross_crate or php_external_receiver row supersedes the per-file records of its exact parser occurrence, every matching record was rewritten in place. Two earlier records at one occurrence therefore became two identical rows, which the dedup table only indexes and never collapses, so a single call site carried a duplicated resolution. Rewrite the first matching record and drop the others at that occurrence; records at other occurrences are untouched. Refs #1186 Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Two gaps let a call on a vendor class escape the #1186 evidence: - A vendor class the PHP LSP knows from its seeded library data (for example GuzzleHttp.Client) resolved as php_static_resolved or php_method_typed. Its target has no graph node either, but without the php_external_receiver strategy the pipeline fell through to a name-only registry binding. Methods found on a class reached through a vendor `use` now carry php_external_receiver as well. - A vendor call skips the registry, and the extractor names a scoped call by its method alone (only a literal `Route::` scope is kept), so `GuzzleClient::get('/x')` reached the route-verb fallback and became a route registration instead of an HTTP call. When the callee text names no service, the service patterns now also see the vendor method the LSP proved (`GuzzleHttp.Client.get`, `Kafka.Producer.publish`), in both the sequential and the parallel pass. Refs #1186 Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The vendor-types test never got past its extraction precondition, so two of its assertions had never run against production: - The precondition looked for the import alias in the callee text of `GuzzleClient::get` and `KafkaProducer::publish`. The extractor names a scoped call by its method alone and keeps only a literal `Route::` scope (#952). The aliased scope is now proven on the occurrence text at the call's site span, which identifies the same call just as strictly. - The sequential/parallel parity fingerprint hashed the HTTP_CALLS and ASYNC_CALLS properties whole. The parallel emitter appends the call's "args" member last; the sequential pass leaves route/service properties without it by design (calls_emit_edge). Those two edge types are now compared with only that trailing member removed; their type, endpoints and every other property must still agree. Refs #1186 Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The parallel pass appends the call's ,"args":[...] member to every HTTP_CALLS and ASYNC_CALLS edge (finalize_and_emit). The sequential pass appended args to CALLS only, so the same service call produced different edge properties depending on which pass indexed the repo, and create_data_flows had no caller_args for small (sequential) repos. Split the args-and-closing-brace step out of finalize_and_emit as cbm_pipeline_close_call_props and let the sequential emit_http_async_edge build its props in the same CBM_SZ_2K buffer and finish them with that function. Field order, expression sanitizing, escaping and truncation are therefore one implementation; CALLS edges and their "line" member are unchanged in both passes. Refs #1186 Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Both passes now emit byte-identical HTTP_CALLS and ASYNC_CALLS properties, so the vendor-types parity fingerprint hashes them whole again instead of trimming the "args" member. The precondition change from the previous fixture correction stays. Refs #1186 Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…s probe CBMCallArg gained url_value after the probe's positional initializer was written, so -Wmissing-field-initializers (-Werror) stopped the test build once this branch met main. Spell the new field out as NULL. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…aries
A call no project symbol resolves, whose callee an import binds to an
external library, is now spelled through that import and classified with
the existing service patterns: `send` under
`from paho.mqtt import publish as send` is paho.mqtt.publish (ASYNC_CALLS
to its topic), `navigate` under `import { get as navigate } from 'axios'`
is axios.get (a typed GET HTTP_CALLS edge). Before, such calls had no
service edge at all.
- CBMImport gains `member`: the exported name an ES named import binds
(`get` in `{ get as navigate }`), NULL for default, namespace and other
imports and for every other language. Filled by process_named_imports,
carried through result compaction (and so through spill), counted in the
extraction census with the other import strings.
- One helper, cbm_pipeline_external_import_service, serves both call
resolvers: a per-file index (local name -> binding import), built on the
first unresolved call that asks, then one hash lookup per call.
- Precision: only unresolved calls, after the callee-name, route-suffix and
global-fetch fallbacks; never a relative import, an import an IMPORTS
edge materialized (a project module), a name bound by two different
imports, or a Python callee rebound as a parameter. An HTTP edge needs a
request verb in the spelled name, so urllib.parse.urljoin is not a
request. Python, JavaScript, TypeScript and TSX.
Tests: extraction_imports covers the member and its survival through
compaction; a pipeline test runs both resolvers over a typed GET, a typed
POST and a publish through aliases, and over controls that must stay
silent (urljoin, lodash and json aliases, a local definition and a
parameter shadowing the import, a materialized project package named
httpx). Each guard was reverted once and its control went red.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
DeusData
force-pushed
the
fix/issues-706-1147-url-fold-v2
branch
from
October 12, 2026 22:55
ae23fdd to
3248591
Compare
yermakoffivan
pushed a commit
to yermakoffivan/codebase-memory-mcp
that referenced
this pull request
Oct 13, 2026
DeusData#2513 (77bc492) computes `url_context` -- is the callee an HTTP client or a global fetch? -- for every positional argument of every call, in every language, through cbm_service_pattern_match: a linear strstr over every service pattern. Only the URL-folding functions read it, and they return NULL outside Python and JS/TS; the one other reader is the template-string branch. On the C# corpus (dotnet/runtime, 451k functions) extraction went from 24.7 s to 37.4 s (+51 %); a macOS sample put the difference in handle_calls -> cbm_service_pattern_match -> match_qn -> strstr. A per-call url_context_t now computes the answer at most once and only when an argument needs it: the folding paths ask only in folding languages, the template-string branch asks for itself. Proof (macOS arm64, Apple M5 Pro): - C# parallel_extract 37.4 s -> 24.2 s, wall 127-132 s -> 119.7 s (the pre-DeusData#2513 level, 24.7 s / 121.9 s). - Byte-identical graphs: the full node and edge sets (properties included) hash the same before and after on django (Python), typescript (TS) and csharp, HTTP_CALLS included (457 / 283 / 39). - extraction, extraction_imports, pipeline, parallel, infrascan and edge_types_probe: 1009 passed. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2 of 4 tasks
DeusData
added a commit
to lorenzozanee/codebase-memory-mcp
that referenced
this pull request
Oct 20, 2026
…e URL fold Since 77bc492 (merged with DeusData#2513), the extractor folds a template URL over a module string constant before the template flattener runs, so fetch(`${BASE_URL}/v1/payments`) with a literal BASE_URL already yields an HTTP_CALLS edge to the /v1/payments route (DeusData#2540). Resolving the constant a second time inside cbm_template_string_text is no longer needed, so the production part of this change is dropped and the helper keeps its original signature. The pipeline regression test from this PR stays: it fails on main before DeusData#2513 (no HTTP_CALLS edge) and passes on current main. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Python and JavaScript/TypeScript calls composed from module constants could lose their HTTP route paths. This adds leading-template-base recovery and local expression/object-member folding while preserving raw topic identities:
tenant/eventsandtenant//eventskeep their exact separators and prefixes.Raw argument values and URL projections are stored separately. HTTP consumers can recover a literal path after an unknown base or root a composed relative endpoint. ASYNC calls, including resolved aliases, and explicit topic keywords do not consume that projection. Request-config URL handling is retained; compaction, spill relocation and memory accounting include the candidate metadata.
Regression coverage includes direct, constant, object-member and keyword topics; unresolved prefixes; doubled separators; HTTP recovery controls; relative wrapper/config URLs; sequential and parallel ASYNC dispatch and aliases; and compact/spill preservation.
Validation: source formatting and
git diff --checkpassed. Compilation, tests, revert checks, lint and local CI were intentionally not run under the requested waiver; the actual proposed SHA still needs GitHub validation. Historical green results are not claimed for this tree. Existing indexes need reindexing.Imported/cross-file endpoint maps, constructor baseURL propagation, cross-repo suffix matching and a new sequential generic-argument heuristic remain outside this change.
Refs #2291, #706, #1147.