Skip to content

Commit 95689b5

Browse files
fix(python): suppress bare calls shadowed by an enclosing parameter
A Python `foo()` whose callee identifier is bound as a parameter of an enclosing scope cannot be the module-level `foo` -- the parameter shadows it for the whole body -- so resolving the call to a project Function/Method by a weak short-name strategy fabricates the edge by construction: def _run_with_heavy_slot(run): return run() # bound an unrelated SatoriLive.run The receiver-aware weak-member guard (DeusData#1276) cannot see this class at all: a bare call has no receiver, so is_method is false and the guard never fires. This is the bare-call counterpart of python_receiver_is_exempt. Keyed on the SCOPE FACT, not on the callee's spelling. A list of generic-looking names (get / run / execute) asserts that certain spellings are usually noise, which is a claim about corpus fashion rather than about what the resolver knew, and it ages invisibly: nothing fails when the distribution shifts, the graph just quietly loses different edges. A parameter binding is decidable from this file's AST outright. Parameters only, deliberately. A parameter is in scope for the entire body regardless of position and Python forbids `global` on one, so no flow analysis is needed. Local assignments are flow- and binding-form-sensitive (`for`, `with as`, `except as`, `:=`, unpacking, plus global/nonlocal overrides); a partial body scan would suppress the wrong edges invisibly -- the same failure mode that rules out the name list. Enclosing scopes are walked to the file root so a closure over an outer parameter counts. Wired at both pass_calls.c and pass_parallel.c with an identical language gate: a guard on one resolver only diverges the sequential and parallel paths. The weak-strategy drop-list is now a single shared predicate used by both the member guard and this one, so they cannot disagree about what "weak" means; a unit test pins that agreement. Tests pin both directions. The pipeline positive control is a cross-file bare call with NO import, so it resolves by a weak strategy this guard could have killed -- asserting a same_module edge would prove nothing, since no guard touches same_module for any input. Verified by breaking the guard in both directions: under-suppressing fails the negatives and the extraction flag; over-suppressing fails the positives, so they are not vacuous. Disabling only the pass_parallel.c gate fails only the >=50-file test and nothing else. 805 passed / 0 failed across pipeline registry parallel extraction complexity lsp_resolution_probe. The defect class was identified by Joseph-MingEn in DeusData#1386, which proposed a name-keyed shape; the diagnosis is theirs. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> Co-authored-by: Joseph-MingEn <125283161+Joseph-MingEn@users.noreply.github.com>
1 parent 5d96377 commit 95689b5

9 files changed

Lines changed: 436 additions & 12 deletions

File tree

‎internal/cbm/cbm.h‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -268,6 +268,10 @@ typedef struct {
268268
// pass_lsp_cross.c. Default false.
269269
bool requires_lsp_resolution; // synthetic semantic candidate (for example an implicit
270270
// C++ operator). Never fall back to textual resolution.
271+
bool callee_is_locally_bound; // bare call foo() whose callee identifier is bound as a
272+
// parameter of an enclosing function, so it cannot be the
273+
// module-level foo. Python only today. Read by the
274+
// weak-local-binding guard. Default false.
271275
} CBMCall;
272276

273277
typedef struct {

‎internal/cbm/extract_calls.c‎

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2800,6 +2800,86 @@ static bool python_receiver_is_exempt(CBMExtractCtx *ctx, TSNode receiver) {
28002800
return false;
28012801
}
28022802

2803+
/* Name bound by one Python parameter node, or NULL when the shape binds none.
2804+
* Covers every binding form a `parameters` / `lambda_parameters` list produces:
2805+
* a bare `identifier`, the `name` field of default/typed parameters, and the
2806+
* identifier under a `*args` / `**kwargs` splat. A shape with no identifier (the
2807+
* bare `*` keyword separator) yields NULL and simply matches nothing. */
2808+
static const char *python_parameter_name(CBMExtractCtx *ctx, TSNode param) {
2809+
if (ts_node_is_null(param)) {
2810+
return NULL;
2811+
}
2812+
if (strcmp(ts_node_type(param), "identifier") == 0) {
2813+
return cbm_node_text(ctx->arena, param, ctx->source);
2814+
}
2815+
TSNode name = ts_node_child_by_field_name(param, TS_FIELD("name"));
2816+
if (!ts_node_is_null(name) && strcmp(ts_node_type(name), "identifier") == 0) {
2817+
return cbm_node_text(ctx->arena, name, ctx->source);
2818+
}
2819+
/* `*args` / `**kwargs`, and any typed shape without a `name` field: the
2820+
* bound identifier is the first named child. */
2821+
TSNode first = ts_node_named_child(param, 0);
2822+
if (!ts_node_is_null(first) && strcmp(ts_node_type(first), "identifier") == 0) {
2823+
return cbm_node_text(ctx->arena, first, ctx->source);
2824+
}
2825+
return NULL;
2826+
}
2827+
2828+
/* True when the callee of a BARE Python call `foo()` is bound as a parameter of
2829+
* an enclosing function or lambda — the bare-call counterpart of
2830+
* python_receiver_is_exempt above.
2831+
*
2832+
* A parameter binding shadows any module-level `foo` for the whole body, so
2833+
* resolving such a call to a project Function/Method by short name alone
2834+
* fabricates the edge BY CONSTRUCTION: `def _run_with_heavy_slot(run): run()`
2835+
* must not bind an unrelated `SatoriLive.run`. Unlike a receiver type this is
2836+
* decidable from the AST outright, with no flow analysis and no list of
2837+
* "generic-looking" callee names — Python forbids `global` on a parameter, and a
2838+
* parameter is in scope for the entire body regardless of position, so there is
2839+
* no ordering subtlety to get wrong.
2840+
*
2841+
* Enclosing scopes are walked to the file root so a closure over an outer
2842+
* parameter counts (`def outer(run): def inner(): return run()`).
2843+
*
2844+
* LOCAL ASSIGNMENTS are deliberately NOT covered. They are flow- and
2845+
* binding-form-sensitive (`for`, `with ... as`, `except ... as`, `:=`,
2846+
* unpacking, plus `global`/`nonlocal` overrides), so a partial body scan would
2847+
* suppress the wrong edges invisibly — the same failure mode that rules out a
2848+
* hardcoded name list. Parameters alone already cover the Callable-parameter
2849+
* shape that motivated this guard. Cost is O(enclosing depth x params) per bare
2850+
* call, never the corpus.
2851+
*
2852+
* Known and accepted: `def inner(): global run; return run()` nested in a
2853+
* function whose parameter is `run` is still flagged. Detecting it needs exactly
2854+
* the body scan this helper avoids, and it costs one edge in a shape that
2855+
* essentially does not occur. */
2856+
static bool python_callee_is_bound_parameter(CBMExtractCtx *ctx, TSNode call_node,
2857+
TSNode callee_ident) {
2858+
const char *callee_name = cbm_node_text(ctx->arena, callee_ident, ctx->source);
2859+
if (!callee_name || !callee_name[0]) {
2860+
return false;
2861+
}
2862+
for (TSNode scope = ts_node_parent(call_node); !ts_node_is_null(scope);
2863+
scope = ts_node_parent(scope)) {
2864+
const char *kind = ts_node_type(scope);
2865+
if (strcmp(kind, "function_definition") != 0 && strcmp(kind, "lambda") != 0) {
2866+
continue;
2867+
}
2868+
TSNode params = ts_node_child_by_field_name(scope, TS_FIELD("parameters"));
2869+
if (ts_node_is_null(params)) {
2870+
continue;
2871+
}
2872+
uint32_t count = ts_node_named_child_count(params);
2873+
for (uint32_t i = 0; i < count; i++) {
2874+
const char *pname = python_parameter_name(ctx, ts_node_named_child(params, i));
2875+
if (pname && strcmp(pname, callee_name) == 0) {
2876+
return true;
2877+
}
2878+
}
2879+
}
2880+
return false;
2881+
}
2882+
28032883
static bool is_objectscript_language(CBMLanguage language) {
28042884
return language == CBM_LANG_OBJECTSCRIPT_UDL || language == CBM_LANG_OBJECTSCRIPT_ROUTINE;
28052885
}
@@ -3424,11 +3504,18 @@ CBMInvocationDescriptor handle_calls(CBMExtractCtx *ctx, TSNode node, const CBML
34243504
// (`accelerator.print()` must not bind MockAccelerator.print).
34253505
// Imported receivers stay unflagged: module.function() is Python's
34263506
// canonical cross-file call and the import map resolves it.
3507+
// A BARE Python call foo() whose callee is bound as a parameter of an
3508+
// enclosing scope cannot be the module-level foo, so short-name
3509+
// resolution would fabricate the edge (`def f(run): run()` must not
3510+
// bind SatoriLive.run). Distinct from is_method: there is no receiver
3511+
// here, so the weak-member guard cannot see this class at all.
34273512
if (ctx->language == CBM_LANG_PYTHON && strcmp(ts_node_type(node), "call") == 0) {
34283513
TSNode fn = ts_node_child_by_field_name(node, TS_FIELD("function"));
34293514
if (!ts_node_is_null(fn) && strcmp(ts_node_type(fn), "attribute") == 0) {
34303515
TSNode obj = ts_node_child_by_field_name(fn, TS_FIELD("object"));
34313516
call.is_method = !python_receiver_is_exempt(ctx, obj);
3517+
} else if (!ts_node_is_null(fn) && strcmp(ts_node_type(fn), "identifier") == 0) {
3518+
call.callee_is_locally_bound = python_callee_is_bound_parameter(ctx, node, fn);
34323519
}
34333520
}
34343521
// TS/JS/TSX receiver-aware guard (#592/#606 direction; same intent

‎src/pipeline/pass_calls.c‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -627,8 +627,16 @@ static int resolve_single_call(cbm_pipeline_ctx_t *ctx, CBMCall *call,
627627
bool suppress_weak_member = lang == CBM_LANG_PYTHON || lang == CBM_LANG_JAVASCRIPT ||
628628
lang == CBM_LANG_TYPESCRIPT || lang == CBM_LANG_TSX ||
629629
lang == CBM_LANG_ARKTS;
630+
/* Bare-call local-binding suppression. A member call has a receiver the
631+
* guard above can reason about; a bare `run()` has none, so that guard
632+
* cannot see this class at all. Python-only today because the extraction
633+
* flag is set only for Python — this gate MUST match pass_parallel.c's
634+
* exactly, for the same divergence reason noted above. */
635+
bool suppress_weak_local_binding = lang == CBM_LANG_PYTHON;
630636
bool drop_plain_call =
631-
cbm_suppress_weak_member_match(suppress_weak_member, call->is_method, res.strategy);
637+
cbm_suppress_weak_member_match(suppress_weak_member, call->is_method, res.strategy) ||
638+
cbm_suppress_weak_local_binding_call(suppress_weak_local_binding,
639+
call->callee_is_locally_bound, res.strategy);
632640

633641
/* Service-pattern HTTP/ASYNC calls to an EXTERNAL client library (e.g.
634642
* `requests.get("/api/orders/{id}")`) resolve to a QN containing the library

‎src/pipeline/pass_parallel.c‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2482,8 +2482,13 @@ static void resolve_file_calls(resolve_ctx_t *rc, resolve_worker_state_t *ws, CB
24822482
bool suppress_weak_member = lang == CBM_LANG_PYTHON || lang == CBM_LANG_JAVASCRIPT ||
24832483
lang == CBM_LANG_TYPESCRIPT || lang == CBM_LANG_TSX ||
24842484
lang == CBM_LANG_ARKTS;
2485+
/* Bare-call local-binding suppression — see the note in pass_calls.c.
2486+
* This gate MUST stay identical to the one there. */
2487+
bool suppress_weak_local_binding = lang == CBM_LANG_PYTHON;
24852488
bool drop_plain_call =
2486-
cbm_suppress_weak_member_match(suppress_weak_member, call->is_method, res.strategy);
2489+
cbm_suppress_weak_member_match(suppress_weak_member, call->is_method, res.strategy) ||
2490+
cbm_suppress_weak_local_binding_call(suppress_weak_local_binding,
2491+
call->callee_is_locally_bound, res.strategy);
24872492

24882493
/* Service-pattern HTTP/ASYNC client call (`requests.get(url)`): the
24892494
* service signal lives in the callee_name. The registry can mis-resolve

‎src/pipeline/pipeline.h‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -280,6 +280,18 @@ bool cbm_perl_suppress_generic_match(bool is_perl, bool is_method, const char *c
280280
* Pure; unit-tested in test_registry.c. */
281281
bool cbm_suppress_weak_member_match(bool enabled, bool is_method, const char *strategy);
282282

283+
/* Bare-call counterpart of the guard above. True when a resolved BARE call edge
284+
* binds a callee that is shadowed by an enclosing parameter, and the match came
285+
* from a weak short-name strategy — so the edge is fabricated by construction
286+
* (`def f(run): run()` must not bind an unrelated `SatoriLive.run`). Shares the
287+
* member guard's drop-list, so lsp_* / import / same-module matches are kept.
288+
* Deliberately keyed on the SCOPE FACT, not on the callee's spelling. The
289+
* language set lives at the call sites (pass_calls.c / pass_parallel.c) and must
290+
* be identical in both, or the sequential and parallel resolvers diverge.
291+
* Pure; unit-tested in test_registry.c. */
292+
bool cbm_suppress_weak_local_binding_call(bool enabled, bool callee_is_locally_bound,
293+
const char *strategy);
294+
283295
/* #725: drop a suffix_match CALLS edge when the caller language and the
284296
* target file's language disagree. unique_name (candidates == 1) is #1572
285297
* and is left alone; same_module / import_map / lsp_* are kept. JS/TS/TSX

‎src/pipeline/registry.c‎

Lines changed: 47 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -448,22 +448,59 @@ bool cbm_perl_suppress_generic_match(bool is_perl, bool is_method, const char *c
448448
* per-language decision made at the call sites in pass_calls.c and
449449
* pass_parallel.c, which MUST stay in lockstep — a gate added to only one of
450450
* them diverges the sequential and parallel resolvers. */
451-
bool cbm_suppress_weak_member_match(bool enabled, bool is_method, const char *strategy) {
452-
if (!enabled || !is_method || !strategy || !strategy[0]) {
451+
/* The weak short-name strategies that actually reach the call-resolution
452+
* guards: the registry's suffix_match / unique_name and the parallel
453+
* field_type_hint. "fuzzy" is listed as defensive insurance only —
454+
* cbm_registry_fuzzy_resolve is not wired into the sequential/parallel resolvers
455+
* today, so it never reaches these helpers, but naming it keeps a future wiring
456+
* from silently reintroducing the noise. Everything else — same_module /
457+
* import_map / import_map_suffix / qualified_suffix / callee_suffix /
458+
* service_pattern / lsp_* — is a receiver- or import-aware match and is KEPT.
459+
*
460+
* Shared by BOTH weak-call guards below so the drop-list exists exactly once: a
461+
* list that drifted between the member guard and the local-binding guard would
462+
* make the two disagree about what "weak" means. */
463+
static bool weak_short_name_strategy(const char *strategy) {
464+
if (!strategy || !strategy[0]) {
453465
return false;
454466
}
455-
/* Weak short-name strategies that actually reach the call-resolution guards:
456-
* the registry's suffix_match / unique_name and the parallel field_type_hint.
457-
* "fuzzy" is listed as defensive insurance only — cbm_registry_fuzzy_resolve
458-
* is not wired into the sequential/parallel resolvers today, so it never
459-
* reaches this helper, but naming it keeps a future wiring from silently
460-
* reintroducing the noise. Everything else — same_module / import_map /
461-
* import_map_suffix / qualified_suffix / callee_suffix / service_pattern /
462-
* lsp_* — is a receiver- or import-aware match and is KEPT. */
463467
return strcmp(strategy, "suffix_match") == 0 || strcmp(strategy, "unique_name") == 0 ||
464468
strcmp(strategy, "field_type_hint") == 0 || strcmp(strategy, "fuzzy") == 0;
465469
}
466470

471+
bool cbm_suppress_weak_member_match(bool enabled, bool is_method, const char *strategy) {
472+
if (!enabled || !is_method) {
473+
return false;
474+
}
475+
return weak_short_name_strategy(strategy);
476+
}
477+
478+
/* Bare-call counterpart of the member guard above. A Python call `foo()` whose
479+
* callee identifier is bound as a parameter of an enclosing scope cannot be the
480+
* module-level `foo`: the parameter shadows it for the whole body. Binding such
481+
* a call to a project Function/Method by a weak short-name strategy fabricates
482+
* the edge by construction (`def _run_with_heavy_slot(run): run()` ->
483+
* SatoriLive.run).
484+
*
485+
* This is deliberately NOT keyed on the callee's spelling. A list of
486+
* "generic-looking" names (get / run / execute) asserts that certain spellings
487+
* are usually noise, which is a claim about corpus fashion rather than about
488+
* what the resolver knew — and it ages invisibly, because nothing fails when the
489+
* distribution shifts, the graph just quietly loses different edges. A parameter
490+
* binding is a fact about THIS file's scope, decidable outright.
491+
*
492+
* `enabled` is the caller's per-language gate, kept out of the helper for the
493+
* same reason as the member guard: the two call sites in pass_calls.c and
494+
* pass_parallel.c MUST enumerate the identical language set, or the sequential
495+
* and parallel resolvers diverge. Pure; unit-tested in test_registry.c. */
496+
bool cbm_suppress_weak_local_binding_call(bool enabled, bool callee_is_locally_bound,
497+
const char *strategy) {
498+
if (!enabled || !callee_is_locally_bound) {
499+
return false;
500+
}
501+
return weak_short_name_strategy(strategy);
502+
}
503+
467504
static bool js_ts_family(CBMLanguage lang) {
468505
return lang == CBM_LANG_JAVASCRIPT || lang == CBM_LANG_TYPESCRIPT || lang == CBM_LANG_TSX ||
469506
lang == CBM_LANG_ARKTS;

‎tests/test_extraction.c‎

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4621,6 +4621,85 @@ TEST(extract_python_member_call_flags_is_method) {
46214621
PASS();
46224622
}
46234623

4624+
/* Python bare-call local-binding flag (the bare-call counterpart of the
4625+
* receiver flag above). Pins BOTH directions: a callee shadowed by a parameter
4626+
* of an enclosing scope IS flagged so the resolver can suppress a weak
4627+
* short-name match, while an unshadowed callee — a genuine module-level
4628+
* function, an imported name, or a nested `def` — is NOT, so its true edge
4629+
* survives. Every parameter binding form the grammar produces is covered, since
4630+
* a form the extractor silently missed would leave that shape unguarded. */
4631+
TEST(extract_python_bare_call_flags_locally_bound_callee) {
4632+
CBMFileResult *r = extract("from pkg import helper\n"
4633+
"\n"
4634+
"def outer(run, *rest, timeout=5, label: str = 'x', **opts):\n"
4635+
" def inner():\n"
4636+
" return run()\n"
4637+
" rest()\n"
4638+
" timeout()\n"
4639+
" label()\n"
4640+
" opts()\n"
4641+
" module_level()\n"
4642+
" helper()\n"
4643+
" return inner()\n"
4644+
"\n"
4645+
"def typed(cb: Callable):\n"
4646+
" return cb()\n"
4647+
"\n"
4648+
"apply_it = lambda fn: fn()\n",
4649+
CBM_LANG_PYTHON, "t", "x.py");
4650+
ASSERT_NOT_NULL(r);
4651+
ASSERT_FALSE(r->has_error);
4652+
4653+
/* callee name -> (expected flag, seen count) */
4654+
struct {
4655+
const char *callee;
4656+
bool expect_bound;
4657+
int seen;
4658+
} cases[] = {
4659+
{"run", true, 0}, /* closure over an ENCLOSING function's parameter */
4660+
{"rest", true, 0}, /* *args -> list_splat_pattern */
4661+
{"timeout", true, 0}, /* default_parameter */
4662+
{"label", true, 0}, /* typed_default_parameter (keyword-only) */
4663+
{"opts", true, 0}, /* **kwargs -> dictionary_splat_pattern */
4664+
{"cb", true, 0}, /* typed_parameter, no default */
4665+
{"fn", true, 0}, /* lambda parameter */
4666+
{"module_level", false, 0}, /* unbound: the true cross-file edge */
4667+
{"helper", false, 0}, /* imported name, not a parameter */
4668+
{"inner", false, 0}, /* nested def: a real target, keep the edge */
4669+
};
4670+
const int case_count = (int)(sizeof(cases) / sizeof(cases[0]));
4671+
4672+
for (int i = 0; i < r->calls.count; i++) {
4673+
const char *cn = r->calls.items[i].callee_name;
4674+
if (!cn) {
4675+
continue;
4676+
}
4677+
for (int c = 0; c < case_count; c++) {
4678+
if (strcmp(cn, cases[c].callee) != 0) {
4679+
continue;
4680+
}
4681+
cases[c].seen++;
4682+
if (r->calls.items[i].callee_is_locally_bound != cases[c].expect_bound) {
4683+
printf(" bare-call flag mismatch for %s(): got %d, expected %d\n", cases[c].callee,
4684+
r->calls.items[i].callee_is_locally_bound ? 1 : 0,
4685+
cases[c].expect_bound ? 1 : 0);
4686+
}
4687+
ASSERT_EQ(r->calls.items[i].callee_is_locally_bound, cases[c].expect_bound);
4688+
}
4689+
}
4690+
/* Each shape must appear exactly once, so a missed extraction cannot let the
4691+
* loop above pass vacuously. */
4692+
for (int c = 0; c < case_count; c++) {
4693+
if (cases[c].seen != 1) {
4694+
printf(" bare call %s() extracted %d times, expected 1\n", cases[c].callee,
4695+
cases[c].seen);
4696+
}
4697+
ASSERT_EQ(cases[c].seen, 1);
4698+
}
4699+
cbm_free_result(r);
4700+
PASS();
4701+
}
4702+
46244703
/* TS/JS/TSX receiver-aware flag (#592/#606; same intent as the Perl flag above).
46254704
* A member call x.foo() with a non-this/super receiver is flagged is_method so
46264705
* the resolver can suppress a weak short-name match (`re.test()` must not bind a
@@ -6300,6 +6379,7 @@ SUITE(extraction) {
63006379
RUN_TEST(extract_perl_method_call_flags_is_method);
63016380
RUN_TEST(extract_flag_exempt_method_call_not_flagged_is_method);
63026381
RUN_TEST(extract_python_member_call_flags_is_method);
6382+
RUN_TEST(extract_python_bare_call_flags_locally_bound_callee);
63036383
RUN_TEST(extract_ts_member_call_flags_is_method);
63046384
RUN_TEST(extract_ts_this_super_receiver_not_flagged);
63056385
RUN_TEST(extract_js_member_call_flags_is_method);

0 commit comments

Comments
 (0)