Skip to content

Commit 4d21c3b

Browse files
committed
fix(elixir): stop walking @SPEC and @type subtrees as code
An Elixir module attribute parses as `unary_operator(@, call(<attr>, args))`, so the typespec family reaches the unified walk in exactly the shape real code has. `@spec foo(t) :: t` puts `foo(t)` in the same `call` node an invocation would occupy, and `@type entry :: String.t()` does the same to a type name. The walk then reads a declaration as code and mints reference edges from it, sourced at the enclosing Module. Measured on a fixture of three typespec lines (`@type state :: map()`, `@spec fetch(opts) :: state`, `@spec load(User.t()) :: :ok`) beside `def state`, `def opts` and a remote `MyApp.User.t/0`: seven edges, every one phantom and every one landing on a Function. CALLS onto each specified function; CALLS + USAGE + WRITES onto `state`, the WRITES asserting a mutation that does not exist; and USAGE onto `opts` and onto `MyApp.User.t/0`, because a bare or remote TYPE name resolves to the same-named FUNCTION. On a codebase whose convention is @SPEC on every public function, every specced function carries an inbound edge from its own module, so fan_in never reaches 0 and "which exported functions does nothing call" is unanswerable. Measured by indexing one 970-file Elixir lib tree before and after: every specced function loses the inbound reference its own typespec minted, and no node is added or removed. This is one of two mechanisms. It removes the phantom a typespec line mints; the def-head phantom is removed by the calls walk recognising a `when`-wrapped head as a declaration, earlier in this stack. A function carrying both an @SPEC and a `when` clause needs both, which is why this commit is ordered after that one and its test can assert 0 references onto such a function rather than pinning a remaining 1. The skip covers the whole subtree, and two costs come with that. Both were measured by indexing a fixture repo (a declaring file per attribute head plus the file each one reaches) with the pristine binary and with this one: A typespec's type references are not indexed at all. That loses nothing correct: a type declaration mints no node, so such a reference resolves onto nothing or onto a same-named FUNCTION, which is itself the pollution. `@spec build(MyApp.User.t())` where that module's `t` is a `@type` produced no edge in either build; the ones that did resolve landed on a same-named Function, cross-file included. @callback and @macrocallback lose a real relationship. The declared name is a function name, so it resolved cross-file onto the functions implementing the behaviour, and nothing replaces that edge: `@behaviour MyBehaviour` mints none of its own. It goes anyway because it is only name resolution: on a fixture with one @callback and three same-named `handle_it/1` defs, the single edge landed on the one module that implements nothing, leaving both real implementors at fan_in 0. Recording that relationship correctly means minting it from `@behaviour`, which is a separate change. A typespec head a widened span covers was also being read as a self-call, which is the other half of the cost the multi-clause span commit disclosed. Same 970-file tree, same method it used (each flagged node's own source span, counting a one-line clause's body after `, do:` as a line that names the function): self_recursive Function nodes 430 -> 389 ...with no self-call form in span 48 -> 8 The base is 129 and 6. All 8 that remain are the phantom a local variable sharing the function's name mints (`defp slug(value) do slug = ...`), which nothing in this stack touches: 6 are on the base as well, and the other 2 are guarded clauses that only have a node of their own to be flagged on because of the earlier commits here. `unquote(...)` inside a typespec does execute at compile time, so `@type t :: unquote(build_type())` loses its genuine `build_type` edge too. Every other attribute VALUE is ordinary compile-time code (`@timeout Application.compile_env(:app, :timeout)` really does call compile_env), so those subtrees are deliberately walked exactly as before, and their attribute NAME still mints one phantom onto a same-named Function. That half of the defect class needs its own fix. tests/test_extraction.c: extract_elixir_typespec_attribute_is_not_code exercises all six heads in the skip list, so deleting any one entry breaks it. It pins the two accepted costs as decisions, and it carries a control name (`without_spec`) that reads 0 on the pristine build too, so an instrument that scored a definition as a reference could not make the other zeroes vacuous. extract_elixir_declaration_head_mints_no_reference_of_any_kind tightens from a relation to 0: with both mechanisms present, a function with an @SPEC and a guard carries no reference of any kind. Depends on the guarded call-scope commit, for the ordering above. It touches no file the earlier commits in this stack touch except extract_unified.c, and none of their lines. Signed-off-by: Chad <4307099+BobbieBarker@users.noreply.github.com>
1 parent 198e211 commit 4d21c3b

3 files changed

Lines changed: 235 additions & 25 deletions

File tree

‎internal/cbm/extract_defs.c‎

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5477,9 +5477,18 @@ static TSNode elixir_call_args(TSNode node) {
54775477
// responsible are the typespec heads the widened span now covers -- `@spec
54785478
// f(t) :: u` puts the declared name in a `call` node, and the unified walk
54795479
// reads it as code -- a separate defect with a separate fix, not something this
5480-
// fold can close. What the fold and the head suppression together remove is
5481-
// 2,436 edges, none of them added back: see elixir_call_is_definition_role in
5482-
// extract_calls.c.
5480+
// fold can close.
5481+
//
5482+
// Skipping the typespec subtrees, later in this stack, closes that half: the
5483+
// same method then reports 389 self_recursive Function nodes and 8 with no
5484+
// self-call form, against 129 and 6 on the base. All 8 are the pre-existing
5485+
// phantom a local variable sharing the function's name mints (`defp slug(value)
5486+
// do slug = ...`), 6 of them already present on the base; the other 2 are
5487+
// guarded clauses that only have a node of their own to be flagged on because
5488+
// of this stack.
5489+
//
5490+
// What the fold and the head suppression together remove is 2,436 edges, none
5491+
// of them added back: see elixir_call_is_definition_role in extract_calls.c.
54835492
//
54845493
// A clause whose macro differs (`def` foo/1 beside `defp` foo/2) still folds,
54855494
// because the arity-free QN already puts both on one node; is_exported is then

‎internal/cbm/extract_unified.c‎

Lines changed: 108 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2041,6 +2041,111 @@ static bool is_unified_trivia_node(TSNode node) {
20412041
return ts_node_is_extra(node);
20422042
}
20432043

2044+
static bool unified_node_text_equals(const CBMExtractCtx *ctx, TSNode node, const char *expected) {
2045+
if (ts_node_is_null(node) || !ctx->source || !expected) {
2046+
return false;
2047+
}
2048+
uint32_t start = ts_node_start_byte(node);
2049+
uint32_t end = ts_node_end_byte(node);
2050+
size_t len = strlen(expected);
2051+
return end >= start && (size_t)(end - start) == len &&
2052+
memcmp(ctx->source + start, expected, len) == 0;
2053+
}
2054+
2055+
/* An Elixir module attribute parses as `unary_operator(@, call(<attr>, args))`,
2056+
* so the typespec family reaches the unified walk in the shape real code has:
2057+
* `@spec foo(t) :: t` puts `foo(t)` in the same `call` node an invocation would
2058+
* occupy, and `@type entry :: String.t()` does the same to a type name. The
2059+
* walk then reads a declaration as code and mints reference edges from it.
2060+
*
2061+
* Measured on a fixture of three typespec lines (`@type state :: map()`, `@spec
2062+
* fetch(opts) :: state`, `@spec load(User.t()) :: :ok`) beside `def state`,
2063+
* `def opts` and a remote `MyApp.User.t/0`: seven edges, every one phantom and
2064+
* every one landing on a Function. CALLS onto each specified function;
2065+
* CALLS + USAGE + WRITES onto `state`, the WRITES asserting a mutation that
2066+
* does not exist; and USAGE onto `opts` and onto `MyApp.User.t/0`, because a
2067+
* bare or remote TYPE name resolves to the same-named FUNCTION. On a codebase
2068+
* whose convention is @spec on every public function, fan_in therefore never
2069+
* reaches 0 and "which exported functions nothing calls" is unanswerable.
2070+
*
2071+
* The whole subtree is skipped, not just the specified name. What that costs is
2072+
* not uniform across the six heads, and both halves were measured by indexing
2073+
* one fixture repo -- a declaring file per head plus the file each one reaches
2074+
* -- with the pristine binary and with this one:
2075+
*
2076+
* spec, type, typep, opaque. A type reference resolves onto a FUNCTION or
2077+
* onto nothing, never onto the type it names, because a type declaration
2078+
* mints no node. `@spec build(MyApp.User.t())` where that module's `t` is a
2079+
* `@type` produced no edge in either build, and `@type ext :: Ecto.Schema.t()`
2080+
* naming a module outside the repo produced none either. The references that
2081+
* do resolve land on a same-named Function, cross-file included -- `USAGE
2082+
* lib/decl_type.ex -> Function type_target @lib/remote_type.ex`, and the
2083+
* @typep and @opaque equivalents. A Function standing in for a Type is the
2084+
* pollution rather than a record of it, so nothing correct is lost here.
2085+
*
2086+
* callback, macrocallback. The declared name IS a function name, so it
2087+
* resolves onto an implementation elsewhere in the corpus: pristine yields
2088+
* `CALLS lib/decl_callback.ex -> Function callback_fun @lib/impl_callback.ex`
2089+
* and the @macrocallback equivalent, both cross-file, and this build yields
2090+
* neither. That edge does point at a real Behaviour-to-Impl relationship, and
2091+
* nothing replaces it -- `@behaviour MyBehaviour` mints no edge of its own,
2092+
* so after this change nothing in the graph links a behaviour to its
2093+
* implementations. It is deleted anyway because it is name resolution and not
2094+
* a behaviour model: on a fixture with one @callback and three same-named
2095+
* `handle_it/1` defs, the single edge landed on the one module that
2096+
* implements nothing, leaving both real implementors at fan_in 0. Recording
2097+
* that relationship correctly means minting it from `@behaviour`, which is a
2098+
* separate change and not a reason to keep an arbitrary edge.
2099+
*
2100+
* The other accepted cost: `unquote(...)` inside a typespec does execute at
2101+
* compile time, so `@type t :: unquote(build_type())` loses its genuine
2102+
* `build_type` call edge along with the phantoms.
2103+
*
2104+
* Only this family is skipped. Every other attribute VALUE is ordinary
2105+
* compile-time code -- `@timeout Application.compile_env(:app, :timeout)`
2106+
* really does call compile_env -- so those subtrees are walked exactly as
2107+
* before, and their attribute NAME is still emitted as a callee: `@behaviour
2108+
* GenServer`, `@timeout 5_000` and `@doc "x"` each still mint one phantom CALLS
2109+
* onto a same-named Function. That half of the defect class is untouched here
2110+
* and needs its own fix.
2111+
*
2112+
* This subtree skip is one of two mechanisms, and neither subsumes the other.
2113+
* It removes the phantom a typespec LINE mints; the def-head phantom is removed
2114+
* separately, by the calls walk recognising a `when`-wrapped head as a
2115+
* declaration (elixir_call_is_definition_role, extract_calls.c). A function
2116+
* carrying both an @spec and a `when` clause needs both: with only this one it
2117+
* keeps the def-head reference, with only the other it keeps the @spec one.
2118+
* `extract_elixir_typespec_attribute_is_not_code` pins that by asserting 0
2119+
* references onto such a function, so removing either mechanism fails it.
2120+
*
2121+
* Measured over forge-symphony-graph/lib (970 .ex files) by classifying the
2122+
* source line every CALLS/USAGE/WRITES edge records: a build without this skip
2123+
* anchors 4,565 CALLS edges on a typespec-attribute line, and this build
2124+
* anchors 0 -- with no USAGE or WRITES appearing in their place, which is what
2125+
* a relabel rather than a removal would look like. Total USAGE edges fall from
2126+
* 25,393 to 23,044 over the same corpus; they do not rise. */
2127+
static bool is_elixir_typespec_attribute(const CBMExtractCtx *ctx, TSNode node) {
2128+
static const char *const typespec_heads[] = {
2129+
"spec", "callback", "macrocallback", "type", "typep", "opaque", NULL};
2130+
if (ctx->language != CBM_LANG_ELIXIR || strcmp(ts_node_type(node), "unary_operator") != 0 ||
2131+
!unified_node_text_equals(ctx, ts_node_child_by_field_name(node, TS_FIELD("operator")),
2132+
"@")) {
2133+
return false;
2134+
}
2135+
TSNode operand = ts_node_child_by_field_name(node, TS_FIELD("operand"));
2136+
if (ts_node_is_null(operand) || strcmp(ts_node_type(operand), "call") != 0 ||
2137+
ts_node_child_count(operand) == 0) {
2138+
return false;
2139+
}
2140+
TSNode head = ts_node_child(operand, 0);
2141+
for (const char *const *name = typespec_heads; *name; name++) {
2142+
if (unified_node_text_equals(ctx, head, *name)) {
2143+
return true;
2144+
}
2145+
}
2146+
return false;
2147+
}
2148+
20442149
// JS/TS `export_statement` appears in import_node_types so re-exports
20452150
// (`export { X } from './m'`) are treated as an import boundary. But it also
20462151
// wraps exported *declarations* (`export function f(cfg: Config) {}`), and
@@ -2631,7 +2736,8 @@ void cbm_extract_unified(CBMExtractCtx *ctx) {
26312736
break;
26322737
}
26332738
bool trivia = is_unified_trivia_node(node);
2634-
if (!trivia) {
2739+
bool typespec = is_elixir_typespec_attribute(ctx, node);
2740+
if (!trivia && !typespec) {
26352741
/* Trivia consumes no semantic state. Scope expiry may be deferred
26362742
* until the next code-bearing node; pop restores the displaced
26372743
* tuple and push applies the new frame's effect, both O(1) --
@@ -2661,7 +2767,7 @@ void cbm_extract_unified(CBMExtractCtx *ctx) {
26612767
* asking the cursor to construct a child iterator for every one of
26622768
* hundreds of thousands of flat comment siblings. Structured extras
26632769
* still descend normally. */
2664-
if ((!trivia || ts_node_child_count(node) > 0) &&
2770+
if (!typespec && (!trivia || ts_node_child_count(node) > 0) &&
26652771
ts_tree_cursor_goto_first_child(&cursor)) {
26662772
depth++;
26672773
continue;

‎tests/test_extraction.c‎

Lines changed: 115 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -1847,24 +1847,22 @@ static int elixir_any_ref_count(CBMFileResult *r, const char *name) {
18471847
}
18481848

18491849
/* The head suppression has to REMOVE the phantom, not relabel it. What makes
1850-
* that hold for a def head is not the calls walk alone: is_elixir_def_binding
1851-
* in extract_usages.c already treats everything inside a def's first argument
1852-
* as a binding occurrence rather than a reference, so the identifier
1853-
* handle_calls declines is declined by handle_usages too. That agreement is the
1854-
* invariant under test, and it is why the counts below are over every extractor
1855-
* rather than over r->calls.
1850+
* that hold for a def head is not the calls walk alone: is_elixir_def_binding in
1851+
* extract_usages.c already treats everything inside a def's first argument as
1852+
* a binding occurrence rather than a reference, so the identifier handle_calls
1853+
* declines is declined by handle_usages too. That agreement is the invariant
1854+
* under test, and it is why the counts below are over every extractor rather
1855+
* than over r->calls.
18561856
*
1857-
* `spec_subject` pins the boundary of this patch, and pins it as a RELATION
1858-
* rather than a count, because the count cannot see the defect. A typespec
1859-
* subject is the same shape as a def head and is NOT suppressed here: nothing
1860-
* in extract_usages.c treats it as a binding, so declining it in handle_calls
1861-
* only relabels its phantom as a USAGE. At extraction that relabel is
1862-
* invisible -- one reference before, one after -- and it becomes a defect
1863-
* downstream, where CALLS and USAGE are separate rows under
1864-
* UNIQUE(source,target,type) and the relabelled phantom stops colliding with
1865-
* the real call it used to hide behind. So the assertion is that every
1866-
* reference recorded for a typespec subject is a CALL and none has become a
1867-
* usage, a read/write or a type reference. */
1857+
* `spec_subject` pins the boundary between the two mechanisms. A typespec
1858+
* subject is the same shape as a def head, and suppressing it in the calls walk
1859+
* would only relabel its phantom as a USAGE, because nothing treats it as a
1860+
* binding. It is removed instead by skipping the whole `@spec` subtree before
1861+
* any extractor sees it (is_elixir_typespec_attribute, extract_unified.c), so
1862+
* the assertion is both that no reference of any kind survives AND that what
1863+
* does survive is never a relabelling: the equality holds at 0 == 0 here, and
1864+
* would hold at 1 == 1 on a build with the subtree skip removed but the calls
1865+
* suppression left in place. */
18681866
TEST(extract_elixir_declaration_head_mints_no_reference_of_any_kind) {
18691867
CBMFileResult *r = extract("defmodule Heads do\n" /* 1 */
18701868
" def guarded(x) when is_integer(x), do: x\n" /* 2 */
@@ -1886,11 +1884,19 @@ TEST(extract_elixir_declaration_head_mints_no_reference_of_any_kind) {
18861884
/* The guard predicate is a real call and survives as one. */
18871885
ASSERT_EQ(1, count_calls_named(r, "is_integer"));
18881886
ASSERT_EQ(1, elixir_any_ref_count(r, "is_integer"));
1889-
/* Declared, not suppressed here; see the comment above. Every reference
1890-
* this file records for a typespec subject is a CALL -- none of them has
1891-
* been relabelled into a usage, a read/write or a type reference. */
1887+
/* Neither the @spec line nor the def head it precedes leaves a reference,
1888+
* and nothing has been relabelled into a usage, a read/write or a type
1889+
* reference. */
1890+
ASSERT_EQ(0, count_calls_named(r, "spec_subject"));
18921891
ASSERT_EQ(count_calls_named(r, "spec_subject"),
18931892
elixir_any_ref_count(r, "spec_subject"));
1893+
1894+
ASSERT_EQ(1, count_defs_named(r, "Function", "guarded"));
1895+
const CBMDefinition *guarded = elixir_first_def(r, "guarded");
1896+
ASSERT_NOT_NULL(guarded);
1897+
ASSERT_EQ(2, (int)guarded->start_line);
1898+
ASSERT_EQ(3, (int)guarded->end_line);
1899+
ASSERT_FALSE(guarded->is_recursive);
18941900
cbm_free_result(r);
18951901
PASS();
18961902
}
@@ -1919,6 +1925,94 @@ TEST(extract_elixir_real_recursion_survives_head_suppression) {
19191925
PASS();
19201926
}
19211927

1928+
/* An Elixir module attribute parses as `unary_operator(@, call(<attr>, args))`,
1929+
* so a typespec line reaches the unified walk as ordinary code and every symbol
1930+
* it names collects a phantom inbound edge from the enclosing Module -- a CALLS
1931+
* onto the specified function, and for `@type` a USAGE plus a WRITES asserting
1932+
* a mutation that does not exist. On a codebase whose convention is @spec on
1933+
* every public function, fan_in then never reaches 0 and "which exported
1934+
* functions nothing calls" is unanswerable.
1935+
*
1936+
* All six heads in the skip list are exercised, so deleting any one entry from
1937+
* typespec_heads[] breaks this test.
1938+
*
1939+
* Two of the groups below pin a decision rather than a repair, and both are
1940+
* argued in is_elixir_typespec_attribute's comment:
1941+
*
1942+
* type_refs[] -- a typespec's type references are not indexed at all. A type
1943+
* declaration mints no node, so such a reference resolves onto nothing or
1944+
* onto a same-named FUNCTION, which is the pollution rather than a record of
1945+
* it. Restoring any of them must break this test.
1946+
*
1947+
* behaviour_contract_names[] -- accepted cost. A @callback / @macrocallback
1948+
* name is a function name, so before this change it resolved cross-file onto
1949+
* the functions implementing the behaviour. Nothing replaces that edge.
1950+
*
1951+
* The attribute NAME phantom outside the typespec family is deliberately still
1952+
* there: `@timeout` keeps its one reference, and its VALUE is ordinary
1953+
* compile-time code that still runs. `guarded` is pinned at 0 rather than 1
1954+
* because the guarded def head no longer mints a phantom either -- the calls
1955+
* walk now recognises a `when`-wrapped head as a declaration. */
1956+
TEST(extract_elixir_typespec_attribute_is_not_code) {
1957+
/* Names a typespec declares, plus the six attribute heads themselves.
1958+
* Every one carries a reference on the pristine build. */
1959+
static const char *const declared[] = {"with_spec", "entry", "secret", "handle",
1960+
"spec", "type", "typep", "opaque",
1961+
"callback", "macrocallback", NULL};
1962+
/* Accepted cost: these used to resolve onto the behaviour's implementors. */
1963+
static const char *const behaviour_contract_names[] = {"handle_it", "expand_it", NULL};
1964+
/* Pinned decision: type references are not indexed -- remote, bare-local
1965+
* and builtin alike. */
1966+
static const char *const type_refs[] = {"MyApp.User.t", "String.t", "MyApp.Vault.key",
1967+
"Macro.t", "t", "key",
1968+
"term", "integer", "reference",
1969+
"atom", NULL};
1970+
CBMFileResult *r = extract("defmodule Specced do\n"
1971+
" @timeout Application.compile_env(:app, :timeout)\n"
1972+
" @type entry :: String.t()\n"
1973+
" @typep secret :: MyApp.Vault.key()\n"
1974+
" @opaque handle :: reference()\n"
1975+
" @callback handle_it(term) :: :ok\n"
1976+
" @macrocallback expand_it(term) :: Macro.t()\n"
1977+
" @spec with_spec(MyApp.User.t()) :: integer\n"
1978+
" def with_spec(n), do: n\n"
1979+
" def without_spec(n), do: n\n"
1980+
" @spec guarded(atom) :: :ok\n"
1981+
" def guarded(x) when is_atom(x), do: :ok\n"
1982+
"end\n",
1983+
CBM_LANG_ELIXIR, "t", "specced.ex");
1984+
ASSERT_NOT_NULL(r);
1985+
ASSERT_FALSE(r->has_error);
1986+
/* The definitions themselves are a separate pass and are unaffected. */
1987+
ASSERT(has_def(r, "Function", "with_spec"));
1988+
ASSERT(has_def(r, "Function", "without_spec"));
1989+
ASSERT(has_def(r, "Function", "guarded"));
1990+
/* Control on the instrument, not a pin: `without_spec` carries no typespec
1991+
* and is never called, so it reads 0 on the pristine build too. It is here
1992+
* to show elixir_any_ref_count does not score a definition as a reference,
1993+
* which is what would make every 0 below vacuous. It cannot fail first and
1994+
* cannot catch a regression. */
1995+
ASSERT_EQ(0, elixir_any_ref_count(r, "without_spec"));
1996+
for (const char *const *name = declared; *name; name++) {
1997+
ASSERT_EQ(0, elixir_any_ref_count(r, *name));
1998+
}
1999+
for (const char *const *name = behaviour_contract_names; *name; name++) {
2000+
ASSERT_EQ(0, elixir_any_ref_count(r, *name));
2001+
}
2002+
for (const char *const *name = type_refs; *name; name++) {
2003+
ASSERT_EQ(0, elixir_any_ref_count(r, *name));
2004+
}
2005+
/* A non-typespec attribute is untouched: its value really is compile-time
2006+
* code, and its name still mints the one phantom this fix does not close. */
2007+
ASSERT_EQ(1, count_calls_named(r, "Application.compile_env"));
2008+
ASSERT_EQ(1, elixir_any_ref_count(r, "timeout"));
2009+
/* Both phantoms onto `guarded` are gone: the @spec one to the subtree skip,
2010+
* the def-head one to the calls walk recognising a guarded head. */
2011+
ASSERT_EQ(0, elixir_any_ref_count(r, "guarded"));
2012+
cbm_free_result(r);
2013+
PASS();
2014+
}
2015+
19222016
/* tree-sitter-elixir gives a call's arguments node no field name, so the
19232017
* generic `arguments` field lookup returns null and first_string_arg was never
19242018
* populated for any Elixir call — Phoenix route paths, service URLs and config
@@ -9168,6 +9262,7 @@ SUITE(extraction) {
91689262
RUN_TEST(extract_elixir_guarded_clause_head_is_not_a_self_call);
91699263
RUN_TEST(extract_elixir_declaration_head_mints_no_reference_of_any_kind);
91709264
RUN_TEST(extract_elixir_real_recursion_survives_head_suppression);
9265+
RUN_TEST(extract_elixir_typespec_attribute_is_not_code);
91719266
RUN_TEST(elixir_call_string_argument);
91729267
RUN_TEST(extract_elixir_guarded_def_head_scope);
91739268
RUN_TEST(extract_elixir_guarded_def_head_guard_usages);

0 commit comments

Comments
 (0)