Skip to content

Commit b3baebb

Browse files
committed
fix(elixir): record the qualifier a reference was written under
Elixir writes every qualified reference as a `dot` node: `Keyword.get` is dot(alias "Keyword", identifier "get"). The usage extractor kept only the identifier half. The qualifier was destroyed before resolution ever saw it, so the usage arrived at the registry as the bare name "get", receiver_chain_admits returned true at its bare-name early return, and every receiver-chain protection was bypassed by construction. That is how an unrelated local `get/1` became the target of a `Keyword.get` reference, and how adding one unrelated file could re-point an already-resolved edge: with one candidate the bare name resolved by unique_name, and a second same-named candidate anywhere in the tree flipped the strategy to proximity scoring, which picks by path distance. is_member_access already recorded that a selector was there; it never recorded what it named. CBMUsage gains `receiver`, NULL when the reference is unqualified. The pipeline composes "<receiver>.<ref_name>" before resolving, and the edge property still reports ref_name, so nothing downstream sees a renamed reference. Scoped to Elixir on purpose. A receiver fed to the registry changes which candidate a usage binds to, and selector shape is a per-grammar claim. result_compact.c relocates the new string with the rest. Every arena-owned string a CBMUsage points at has to be relocated: the pipeline composes a qualified name from `receiver` before resolving, so a stale pointer corrupts silently. It resolves a reference built from whatever bytes the old arena's memory now holds, and the result is a valid-looking USAGE edge onto an arbitrary same-suffixed function, different on every run of the same input. Measured by building a product binary from the incremental re-link commit and one from this commit, indexing the same 970-file Elixir lib tree with each into its own sandbox (`cli index_repository --mode full`), and diffing the `edges` rows out of the two SQLite stores. A USAGE site is keyed by (source qualified_name, callee), which is stable across both builds: USAGE edges 31,844 -> 25,664 USAGE sites present before and gone after 6,245 USAGE sites present after and not before 0 surviving sites whose target changed 1,025 Nothing new appears, 6,245 bindings that the bare name alone justified are withdrawn, and 1,025 of the survivors move to a different target. A qualified reference that named a module the project does not define now resolves to nothing instead of to a same-named local function. The incremental re-link commit's product binary is byte-identical to the resolution commit's on this toolchain, so the "before" column is equally the value at resolution. tests/test_extraction.c: elixir_usage_records_receiver pins the recorded qualifier. extract_compact_relocates_every_usage_string walks every arena-owned pointer on every CBMUsage after cbm_result_compact and asserts each one lands inside the surviving arena block, so a field added later and not relocated fails the test. Its content dependency is the resolution commit, whose receiver-chain guard is what the recorded qualifier feeds. Signed-off-by: Chad <4307099+BobbieBarker@users.noreply.github.com>
1 parent cf13b30 commit b3baebb

6 files changed

Lines changed: 186 additions & 10 deletions

File tree

‎internal/cbm/cbm.h‎

Lines changed: 18 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -309,8 +309,24 @@ typedef enum {
309309
typedef struct {
310310
const char *ref_name; // referenced identifier
311311
const char *enclosing_func_qn; // QN of enclosing function (or module QN)
312-
/* Fixed-width fields grouped so the record packs to 40 bytes (was 48; the
313-
* Go corpus holds 4.68M of these). Field meanings unchanged. */
312+
/* The qualifier the reference was written under, when the extractor can
313+
* see one: `Keyword` in `Keyword.get(opts, :reason)`. Without it a usage
314+
* arrives at the registry as the bare name "get", receiver_chain_admits
315+
* returns true at its bare-name early return, and EVERY receiver-chain
316+
* protection is bypassed by construction -- which is how an unrelated
317+
* local get/1 became the target of a Keyword.get call, and how adding one
318+
* unrelated file re-pointed an already-resolved edge. is_member_access
319+
* below records only THAT a selector was there, never what it named.
320+
* NULL when the reference is unqualified. */
321+
const char *receiver;
322+
/* Fixed-width fields grouped so the record packs with no interior padding.
323+
* `receiver` above makes that 48 bytes, up from the 40 this comment used to
324+
* report: three pointers, three uint32, two enums and four bools all pack
325+
* exactly, so an 8-byte pointer cannot ride in existing slack and the
326+
* record grows by its full width for every language. On the Go corpus this
327+
* comment already cites -- 4.68M usages -- the qualifier is ~37 MB of extra
328+
* peak extraction footprint, and reference_receiver() returns NULL for
329+
* every language but Elixir. Field meanings unchanged. */
314330
uint32_t lexical_scope_id; // extraction-local scope instance; never graph identity
315331
uint32_t site_start_byte; // exact reference-token span; end > start when present
316332
uint32_t site_end_byte; // exclusive byte offset in the source file

‎internal/cbm/extract_usages.c‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2626,6 +2626,33 @@ void cbm_extract_usages(CBMExtractCtx *ctx) {
26262626
// --- Unified handler: called once per node by the cursor walk ---
26272627
// Uses WalkState flags instead of parent-chain walks for O(1) context checks.
26282628

2629+
/* The qualifier a reference was written under, or NULL when unqualified.
2630+
*
2631+
* Elixir writes every qualified reference as a `dot` node -- `Keyword.get` is
2632+
* dot(alias "Keyword", identifier "get") -- and the usage extractor records
2633+
* only the identifier half, so the qualifier is destroyed before resolution
2634+
* ever sees it. Recovering it here is what lets receiver_chain_admits judge a
2635+
* usage the same way it already judges a call.
2636+
*
2637+
* Deliberately scoped to the languages whose selector shape is known here: a
2638+
* receiver fed to the registry changes which candidate a usage binds to, and
2639+
* that is a per-grammar claim, not a universal one. */
2640+
static const char *reference_receiver(CBMExtractCtx *ctx, TSNode node) {
2641+
if (ctx->language != CBM_LANG_ELIXIR) {
2642+
return NULL;
2643+
}
2644+
TSNode parent = ts_node_parent(node);
2645+
if (ts_node_is_null(parent) || strcmp(ts_node_type(parent), "dot") != 0) {
2646+
return NULL;
2647+
}
2648+
TSNode lhs = ts_node_child(parent, 0);
2649+
if (ts_node_is_null(lhs) || ts_node_eq(lhs, node)) {
2650+
return NULL;
2651+
}
2652+
char *text = cbm_node_text(ctx->arena, lhs, ctx->source);
2653+
return (text && text[0]) ? text : NULL;
2654+
}
2655+
26292656
void handle_usages(CBMExtractCtx *ctx, TSNode node, const CBMLangSpec *spec, WalkState *state) {
26302657
if (emit_direct_perl_coderef_usage(ctx, node, state->enclosing_func_qn,
26312658
active_lexical_scope_id(state))) {
@@ -2728,6 +2755,7 @@ void handle_usages(CBMExtractCtx *ctx, TSNode node, const CBMLangSpec *spec, Wal
27282755
* (field_identifier); record that shape — the name alone cannot carry
27292756
* it, and the Go Field guard keys on it (#1962). */
27302757
usage.is_member_access = strcmp(ts_node_type(node), "field_identifier") == 0;
2758+
usage.receiver = reference_receiver(ctx, node);
27312759
stamp_usage_site(ctx, &usage, node, name, state);
27322760
cbm_usages_push(&ctx->result->usages, ctx->arena, usage);
27332761
}

‎internal/cbm/result_compact.c‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -315,6 +315,13 @@ static void cr_walk(cr_ctx_t *c, CBMFileResult *r) {
315315
for (int i = 0; i < r->usages.count && r->usages.items; i++) {
316316
cr_str(c, &r->usages.items[i].ref_name);
317317
cr_str(c, &r->usages.items[i].enclosing_func_qn);
318+
/* Every arena-owned string a CBMUsage points at has to be relocated
319+
* here. `receiver` is one: the pipeline composes "<receiver>.<ref_name>"
320+
* before resolving, so a stale pointer does not crash, it resolves a
321+
* qualified reference built from whatever bytes the old arena's memory
322+
* now holds -- a valid-looking USAGE edge onto an arbitrary same-suffixed
323+
* function, different on every run of the same input. */
324+
cr_str(c, &r->usages.items[i].receiver);
318325
}
319326
cr_array(c, (void **)&r->throws.items, r->throws.count, sizeof(CBMThrow));
320327
for (int i = 0; i < r->throws.count && r->throws.items; i++) {

‎src/pipeline/pass_parallel.c‎

Lines changed: 17 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3159,15 +3159,28 @@ static void resolve_file_usages(resolve_ctx_t *rc, resolve_worker_state_t *ws,
31593159
usage->enclosing_func_qn)) {
31603160
container_qn = container_buf;
31613161
}
3162+
/* Resolve under the qualifier the reference was actually written
3163+
* with. A bare name reaches receiver_chain_admits' bare-name early
3164+
* return, so every receiver-chain protection is bypassed; the
3165+
* qualified form is what lets the guard reject an unrelated local
3166+
* of the same name and lets qualified_suffix_match find the real
3167+
* one. The EDGE property still reports ref_name. */
3168+
char qualified_ref[CBM_SZ_512];
3169+
const char *resolve_ref = usage->ref_name;
3170+
if (usage->receiver && usage->receiver[0]) {
3171+
(void)snprintf(qualified_ref, sizeof(qualified_ref), "%s.%s", usage->receiver,
3172+
usage->ref_name);
3173+
resolve_ref = qualified_ref;
3174+
}
31623175
/* A usage is a reference, not an invocation: no argument list, so
31633176
* no arity to offer. */
31643177
cbm_resolve_ctx_t rx = {.container_qn = container_qn, .arity = CBM_ARITY_NONE};
31653178
cbm_resolution_t res =
31663179
(lang == CBM_LANG_SQL)
3167-
? cbm_registry_resolve_lineage(rc->registry, usage->ref_name, module_qn,
3168-
imp_keys, imp_vals, imp_count)
3169-
: cbm_registry_resolve_ctx(rc->registry, usage->ref_name, module_qn, &rx,
3170-
imp_keys, imp_vals, imp_count);
3180+
? cbm_registry_resolve_lineage(rc->registry, resolve_ref, module_qn, imp_keys,
3181+
imp_vals, imp_count)
3182+
: cbm_registry_resolve_ctx(rc->registry, resolve_ref, module_qn, &rx, imp_keys,
3183+
imp_vals, imp_count);
31713184
if (!res.qualified_name || res.qualified_name[0] == '\0') {
31723185
continue;
31733186
}

‎src/pipeline/pass_usages.c‎

Lines changed: 17 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -205,6 +205,19 @@ static int resolve_usage_edges(cbm_pipeline_ctx_t *ctx, const CBMFileResult *res
205205
usage->enclosing_func_qn)) {
206206
container_qn = container_buf;
207207
}
208+
/* Resolve under the qualifier the reference was actually written
209+
* with. A bare name reaches receiver_chain_admits' bare-name early
210+
* return, so every receiver-chain protection is bypassed; the
211+
* qualified form is what lets the guard reject an unrelated local
212+
* of the same name and lets qualified_suffix_match find the real
213+
* one. The EDGE property still reports ref_name. */
214+
char qualified_ref[CBM_SZ_512];
215+
const char *resolve_ref = usage->ref_name;
216+
if (usage->receiver && usage->receiver[0]) {
217+
(void)snprintf(qualified_ref, sizeof(qualified_ref), "%s.%s", usage->receiver,
218+
usage->ref_name);
219+
resolve_ref = qualified_ref;
220+
}
208221
/* A usage is a reference, not an invocation: no argument list, so
209222
* no arity to offer.
210223
*
@@ -215,10 +228,10 @@ static int resolve_usage_edges(cbm_pipeline_ctx_t *ctx, const CBMFileResult *res
215228
cbm_resolve_ctx_t rx = {.container_qn = container_qn, .arity = CBM_ARITY_NONE};
216229
cbm_resolution_t res =
217230
(lang == CBM_LANG_SQL)
218-
? cbm_registry_resolve_lineage(ctx->registry, usage->ref_name, module_qn,
219-
imp_keys, imp_vals, imp_count)
220-
: cbm_registry_resolve_ctx(ctx->registry, usage->ref_name, module_qn, &rx,
221-
imp_keys, imp_vals, imp_count);
231+
? cbm_registry_resolve_lineage(ctx->registry, resolve_ref, module_qn, imp_keys,
232+
imp_vals, imp_count)
233+
: cbm_registry_resolve_ctx(ctx->registry, resolve_ref, module_qn, &rx, imp_keys,
234+
imp_vals, imp_count);
222235
if (!res.qualified_name || res.qualified_name[0] == '\0') {
223236
continue;
224237
}

‎tests/test_extraction.c‎

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2213,6 +2213,34 @@ TEST(elixir_call_scope_qn_matches_def_qn) {
22132213
PASS();
22142214
}
22152215

2216+
/* A qualified reference used to reach the registry as the BARE name: the
2217+
* extractor recorded `get` for `Keyword.get(opts, :reason)` and threw the
2218+
* qualifier away, so receiver_chain_admits returned true at its bare-name
2219+
* early return and every receiver-chain protection was bypassed. */
2220+
TEST(elixir_usage_records_receiver) {
2221+
CBMFileResult *r = extract("defmodule Fx.Store do\n"
2222+
" def run(opts) do\n"
2223+
" Keyword.get(opts, :reason)\n"
2224+
" end\n"
2225+
"end\n",
2226+
CBM_LANG_ELIXIR, "t", "lib/store.ex");
2227+
ASSERT_NOT_NULL(r);
2228+
ASSERT_FALSE(r->has_error);
2229+
int seen = 0;
2230+
for (int i = 0; i < r->usages.count; i++) {
2231+
if (!r->usages.items[i].ref_name ||
2232+
strcmp(r->usages.items[i].ref_name, "get") != 0) {
2233+
continue;
2234+
}
2235+
seen = 1;
2236+
ASSERT_NOT_NULL(r->usages.items[i].receiver);
2237+
ASSERT_STR_EQ("Keyword", r->usages.items[i].receiver);
2238+
}
2239+
ASSERT_EQ(1, seen);
2240+
cbm_free_result(r);
2241+
PASS();
2242+
}
2243+
22162244
/* --- Haskell --- */
22172245
TEST(haskell_function) {
22182246
CBMFileResult *r = extract("add :: Int -> Int -> Int\nadd x y = x + y\n\nmultiply :: Int -> "
@@ -8871,6 +8899,75 @@ static size_t arena_capacity(const CBMArena *a) {
88718899
return total;
88728900
}
88738901

8902+
/* True when `p` points inside one of the arena's own blocks. After
8903+
* cbm_result_compact there is exactly one, so this is "did compaction relocate
8904+
* this string, or is it still pointing at the freed pre-compaction arena". */
8905+
static bool in_arena(const CBMArena *a, const void *p) {
8906+
for (int i = 0; i < a->nblocks; i++) {
8907+
const char *base = a->blocks[i];
8908+
if (base && (const char *)p >= base && (const char *)p < base + a->block_sizes[i]) {
8909+
return true;
8910+
}
8911+
}
8912+
return false;
8913+
}
8914+
8915+
/* cbm_result_compact copies a CBMFileResult into one exact arena block and
8916+
* frees the old one, so EVERY arena-owned pointer the result holds has to be
8917+
* named in cr_walk. A field that is not stays pointing into freed memory, and
8918+
* the failure is silent: nothing crashes, the bytes still parse as a string,
8919+
* and the pipeline builds a real-looking edge out of whatever the recycled
8920+
* memory now holds -- a different edge on every run of the same input.
8921+
*
8922+
* The sibling test above uses a Python fixture, where a usage's `receiver` is
8923+
* always NULL, so it cannot see an unrelocated receiver at all. This one uses
8924+
* a qualified Elixir reference, which is the shape that sets it, and asserts
8925+
* containment rather than string equality: a stale pointer very often still
8926+
* reads the right characters, which is exactly why this defect survives a
8927+
* value comparison. */
8928+
TEST(extract_compact_relocates_every_usage_string) {
8929+
CBMFileResult *r = extract("defmodule Fx.Runner do\n"
8930+
" def run(opts) do\n"
8931+
" Keyword.get(opts, :reason)\n"
8932+
" end\n"
8933+
"end\n",
8934+
CBM_LANG_ELIXIR, "t", "lib/runner.ex");
8935+
ASSERT_NOT_NULL(r);
8936+
ASSERT_FALSE(r->has_error);
8937+
8938+
/* The fixture has to actually produce the field under test. */
8939+
int with_receiver = 0;
8940+
for (int i = 0; i < r->usages.count; i++) {
8941+
if (r->usages.items[i].receiver) {
8942+
with_receiver++;
8943+
ASSERT_STR_EQ("Keyword", r->usages.items[i].receiver);
8944+
}
8945+
}
8946+
ASSERT_EQ(1, with_receiver);
8947+
8948+
cbm_result_compact(r);
8949+
ASSERT_EQ(r->arena.nblocks, 1);
8950+
8951+
with_receiver = 0;
8952+
for (int i = 0; i < r->usages.count; i++) {
8953+
const CBMUsage *u = &r->usages.items[i];
8954+
if (u->ref_name && u->ref_name[0]) {
8955+
ASSERT(in_arena(&r->arena, u->ref_name));
8956+
}
8957+
if (u->enclosing_func_qn && u->enclosing_func_qn[0]) {
8958+
ASSERT(in_arena(&r->arena, u->enclosing_func_qn));
8959+
}
8960+
if (u->receiver) {
8961+
with_receiver++;
8962+
ASSERT(in_arena(&r->arena, u->receiver));
8963+
ASSERT_STR_EQ("Keyword", u->receiver);
8964+
}
8965+
}
8966+
ASSERT_EQ(1, with_receiver);
8967+
cbm_free_result(r);
8968+
PASS();
8969+
}
8970+
88748971
TEST(extract_compact_keeps_every_field_and_shrinks_the_arena) {
88758972
CBMFileResult *r = extract(COMPACT_PY_SRC, CBM_LANG_PYTHON, "t", "svc.py");
88768973
CBMFileResult *ref = extract(COMPACT_PY_SRC, CBM_LANG_PYTHON, "t", "svc.py");
@@ -9198,6 +9295,7 @@ SUITE(extraction) {
91989295
RUN_TEST(extract_spill_round_trip_keeps_every_field);
91999296
RUN_TEST(extract_lsp_skipped_file_gets_no_walk);
92009297
RUN_TEST(extract_walk_truncated_when_a_node_budget_is_set);
9298+
RUN_TEST(extract_compact_relocates_every_usage_string);
92019299
/* Initialize extraction library */
92029300
cbm_init();
92039301

@@ -9342,6 +9440,7 @@ SUITE(extraction) {
93429440
RUN_TEST(elixir_defs_inside_macro_blocks_are_extracted);
93439441
RUN_TEST(elixir_multi_clause_function_stays_one_identity);
93449442
RUN_TEST(elixir_call_scope_qn_matches_def_qn);
9443+
RUN_TEST(elixir_usage_records_receiver);
93459444
RUN_TEST(elixir_call_string_argument);
93469445
RUN_TEST(extract_elixir_guarded_def_head_scope);
93479446
RUN_TEST(extract_elixir_guarded_def_head_guard_usages);

0 commit comments

Comments
 (0)