Skip to content

Commit 73cfb61

Browse files
DeusDatavvenegasv
andcommitted
feat(mcp): surface edge resolution provenance behind include_evidence
The indexer already records how every CALLS edge was resolved -- pass_calls.c:355 writes {callee, confidence, strategy, candidates} into the edge properties, and cbm_store_traverse reads it back -- but no tool ever surfaced it. An agent could see THAT A calls B, never how confident the resolver was, or whether the edge came from an LSP or from a name heuristic. trace_path / trace_call_path gain include_evidence (default false). When set, each row carries two extra columns: the resolution strategy CLASS and the resolver's confidence. Two decisions worth recording: Opt-in, not default. Two extra columns on every row is exactly the inflation the tree format exists to avoid, so the default response is byte-identical to before. A closed class, not the raw strategy. Production emits ~20 internal strategy names (lsp_trait_dispatch, php_self_static, callee_suffix, ...) and the set grows with every language. Publishing them verbatim would make each internal resolver name public API by accident. cbm_mcp_edge_strategy_class maps them to a fixed vocabulary -- lsp | language_rule | heuristic | unresolved -- so adding lsp_foo_dispatch maps automatically while a genuinely new KIND of resolution fails the pinning test and forces a deliberate decision. lsp_unresolved classifies as unresolved, not lsp: the caller's question is whether the edge is trustworthy, and "we tried LSP and it did not resolve" answers no. Evidence is looked up against the PAGINATED view, so it is emitted only for the rows on the current page -- it does not bypass limit, the cursor, or the MCP_BFS_LIMIT_MAX guard. Tests: the vocabulary pin (every strategy production can emit maps to a known class), and an end-to-end test binding both halves -- absent by default, present when asked for, and the raw internal name never reaching the client. Both fail without this change. Distilled from DeusData#559 by @vvenegasv, who found that we already store this and never show it. The freshness half of that PR is not included: it is non-functional in production (both pipelines persist through the direct SQLite writer, which records a hardcoded NULL) and its ALTER TABLE migration never runs for the read-only opens the MCP server uses. Co-Authored-By: vvenegasv <vvenegasv@users.noreply.github.com> Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
1 parent f5bbfa4 commit 73cfb61

3 files changed

Lines changed: 267 additions & 7 deletions

File tree

‎src/mcp/mcp.c‎

Lines changed: 128 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -510,7 +510,11 @@ static const tool_def_t TOOLS[] = {
510510
"filtered out. When true, test nodes are included with a test column/marker.\"},"
511511
"\"format\":{\"type\":\"string\",\"enum\":[\"tree\",\"json\"],\"default\":\"tree\","
512512
"\"description\":\"Response encoding. tree (default): prefix-grouped text rows. "
513-
"json: the SAME tree model as structured JSON (groups + column-ordered row arrays).\"}},"
513+
"json: the SAME tree model as structured JSON (groups + column-ordered row arrays).\"},"
514+
"\"include_evidence\":{\"type\":\"boolean\",\"default\":false,"
515+
"\"description\":\"Add how each hop was resolved: a strategy class (lsp | language_rule | "
516+
"heuristic | unresolved) and the resolver's confidence. Off by default — it adds two "
517+
"columns per row. Use it to judge whether an edge is trustworthy, not to find edges.\"}},"
514518
"\"required\":[\"function_name\",\"project\"]}"},
515519

516520
{"get_code_snippet", "Get code snippet",
@@ -5648,6 +5652,96 @@ static const char *bfs_edge_args_for_hop(cbm_traverse_result_t *tr, int64_t hop_
56485652
return NULL;
56495653
}
56505654

5655+
/* Classify a resolver strategy into the CLOSED public vocabulary.
5656+
*
5657+
* The indexer records ~20 internal strategy names on CALLS edges
5658+
* (lsp_trait_dispatch, php_self_static, callee_suffix, ...) and the set grows
5659+
* with every language. Publishing those verbatim would make each internal
5660+
* resolver name public API by accident, so a rename would silently change a
5661+
* user-visible field. We publish the CLASS instead: adding lsp_foo_dispatch
5662+
* maps automatically, while a genuinely new KIND of resolution fails the
5663+
* pinning test in tests/test_mcp.c and forces a deliberate decision.
5664+
*
5665+
* Returns NULL only for a NULL/empty strategy — every non-empty value lands in
5666+
* a class, so an unmapped strategy can never silently vanish from output. */
5667+
const char *cbm_mcp_edge_strategy_class(const char *strategy) {
5668+
if (!strategy || !strategy[0]) {
5669+
return NULL;
5670+
}
5671+
/* Order matters: lsp_unresolved is an LSP strategy that failed, and the
5672+
* caller cares that it did NOT resolve — so it classifies as unresolved. */
5673+
if (strcmp(strategy, "lsp_unresolved") == 0 || strcmp(strategy, "unknown") == 0) {
5674+
return "unresolved";
5675+
}
5676+
if (strncmp(strategy, "lsp_", 4) == 0) {
5677+
return "lsp";
5678+
}
5679+
if (strncmp(strategy, "php_", 4) == 0 || strncmp(strategy, "perl_", 5) == 0) {
5680+
return "language_rule";
5681+
}
5682+
/* Everything else is a name/shape heuristic: callee_suffix,
5683+
* field_type_hint, service_pattern, fastapi_depends, unique_name, ... */
5684+
return "heuristic";
5685+
}
5686+
5687+
/* Find the resolution evidence on the edge that leads to the given hop node —
5688+
* the same endpoint-matching rule as bfs_edge_args_for_hop (target for an
5689+
* outbound trace, source for inbound, so both directions surface it).
5690+
*
5691+
* Reads only from tr->edges, and callers pass the PAGINATED view, so evidence
5692+
* is emitted for the rows on this page and nothing else.
5693+
*
5694+
* Returns false when the edge carries no strategy (non-CALLS edges do not). */
5695+
static bool bfs_edge_evidence_for_hop(cbm_traverse_result_t *tr, int64_t hop_node_id,
5696+
const char **class_out, double *confidence_out) {
5697+
for (int e = 0; e < tr->edge_count; e++) {
5698+
if (tr->edges[e].target_id != hop_node_id && tr->edges[e].source_id != hop_node_id) {
5699+
continue;
5700+
}
5701+
const char *pj = tr->edges[e].properties_json;
5702+
if (!pj) {
5703+
continue;
5704+
}
5705+
const char *key = strstr(pj, "\"strategy\"");
5706+
if (!key) {
5707+
continue;
5708+
}
5709+
const char *open = strchr(key + 10, '"');
5710+
if (!open) {
5711+
continue;
5712+
}
5713+
open++;
5714+
const char *close = strchr(open, '"');
5715+
if (!close || close == open) {
5716+
continue;
5717+
}
5718+
char raw[CBM_SZ_64];
5719+
size_t len = (size_t)(close - open);
5720+
if (len >= sizeof(raw)) {
5721+
len = sizeof(raw) - 1;
5722+
}
5723+
memcpy(raw, open, len);
5724+
raw[len] = '\0';
5725+
const char *cls = cbm_mcp_edge_strategy_class(raw);
5726+
if (!cls) {
5727+
continue;
5728+
}
5729+
*class_out = cls;
5730+
/* Confidence rides on the same edge; absent is reported as -1 so a
5731+
* genuine 0.0 stays distinguishable from "not recorded". */
5732+
*confidence_out = -1.0;
5733+
const char *conf = strstr(pj, "\"confidence\"");
5734+
if (conf) {
5735+
const char *colon = strchr(conf, ':');
5736+
if (colon) {
5737+
*confidence_out = strtod(colon + 1, NULL);
5738+
}
5739+
}
5740+
return true;
5741+
}
5742+
return false;
5743+
}
5744+
56515745
/* TOON table for one trace direction: callees[N]{qn,hop,...} with optional
56525746
* risk / test / args columns. `name` is omitted (it is the qn's last
56535747
* segment); the per-item JSON key envelope was 84% of the legacy payload. */
@@ -6061,7 +6155,7 @@ static int tree_hop_cmp_qn(const void *pa, const void *pb) {
60616155
}
60626156

60636157
static void bfs_to_tree_table(cbm_sb_t *sb, const char *key, cbm_traverse_result_t *tr,
6064-
bool include_tests) {
6158+
bool include_tests, bool include_evidence) {
60656159
int visible = 0;
60666160
for (int i = 0; i < tr->visited_count; i++) {
60676161
if (!include_tests && is_test_file(tr->visited[i].node.file_path)) {
@@ -6070,8 +6164,12 @@ static void bfs_to_tree_table(cbm_sb_t *sb, const char *key, cbm_traverse_result
60706164
visible++;
60716165
}
60726166
char buf[CBM_SZ_256];
6073-
snprintf(buf, sizeof(buf), "%s: %d (rows: name hop; qn = group prefix + \".\" + name)\n", key,
6074-
visible);
6167+
snprintf(buf, sizeof(buf),
6168+
include_evidence
6169+
? "%s: %d (rows: name hop strategy confidence; qn = group prefix + \".\" + "
6170+
"name)\n"
6171+
: "%s: %d (rows: name hop; qn = group prefix + \".\" + name)\n",
6172+
key, visible);
60756173
cbm_sb_append(sb, buf);
60766174
if (tr->visited_count > 1) {
60776175
qsort(tr->visited, (size_t)tr->visited_count, sizeof(cbm_node_hop_t), tree_hop_cmp_qn);
@@ -6093,7 +6191,26 @@ static void bfs_to_tree_table(cbm_sb_t *sb, const char *key, cbm_traverse_result
60936191
cbm_sb_append(sb, ":\n");
60946192
}
60956193
char row[CBM_SZ_512];
6096-
snprintf(row, sizeof(row), " %s %d\n", plen ? qn + plen + 1 : qn, tr->visited[i].hop);
6194+
const char *ev_class = NULL;
6195+
double ev_conf = -1.0;
6196+
if (include_evidence &&
6197+
bfs_edge_evidence_for_hop(tr, tr->visited[i].node.id, &ev_class, &ev_conf)) {
6198+
if (ev_conf >= 0.0) {
6199+
snprintf(row, sizeof(row), " %s %d %s %.2f\n", plen ? qn + plen + 1 : qn,
6200+
tr->visited[i].hop, ev_class, ev_conf);
6201+
} else {
6202+
snprintf(row, sizeof(row), " %s %d %s -\n", plen ? qn + plen + 1 : qn,
6203+
tr->visited[i].hop, ev_class);
6204+
}
6205+
} else if (include_evidence) {
6206+
/* The root hop has no inbound edge, and non-CALLS edges record no
6207+
* strategy. Emit placeholders so the column count stays fixed —
6208+
* a ragged table is worse to parse than an explicit "-". */
6209+
snprintf(row, sizeof(row), " %s %d - -\n", plen ? qn + plen + 1 : qn,
6210+
tr->visited[i].hop);
6211+
} else {
6212+
snprintf(row, sizeof(row), " %s %d\n", plen ? qn + plen + 1 : qn, tr->visited[i].hop);
6213+
}
60976214
cbm_sb_append(sb, row);
60986215
}
60996216
}
@@ -6137,6 +6254,10 @@ static char *handle_trace_call_path(cbm_mcp_server_t *srv, const char *args) {
61376254
}
61386255
bool risk_labels = cbm_mcp_get_bool_arg(args, "risk_labels");
61396256
bool include_tests = cbm_mcp_get_bool_arg(args, "include_tests");
6257+
/* Off by default: two extra columns on every row is exactly the kind of
6258+
* inflation the tree format exists to avoid. Opt in when you need to judge
6259+
* whether an edge is trustworthy. */
6260+
bool include_evidence = cbm_mcp_get_bool_arg(args, "include_evidence");
61406261

61416262
if (!func_name) {
61426263
free(project);
@@ -6424,15 +6545,15 @@ static char *handle_trace_call_path(cbm_mcp_server_t *srv, const char *args) {
64246545
if (flat_trace) {
64256546
bfs_to_toon_table(&sb, "callees", &view_out, risk_labels, include_tests, data_flow);
64266547
} else {
6427-
bfs_to_tree_table(&sb, "callees", &view_out, include_tests);
6548+
bfs_to_tree_table(&sb, "callees", &view_out, include_tests, include_evidence);
64286549
}
64296550
}
64306551
if (do_inbound) {
64316552
cbm_tree_scalar_int(&sb, "callers_total", in_total);
64326553
if (flat_trace) {
64336554
bfs_to_toon_table(&sb, "callers", &view_in, risk_labels, include_tests, data_flow);
64346555
} else {
6435-
bfs_to_tree_table(&sb, "callers", &view_in, include_tests);
6556+
bfs_to_tree_table(&sb, "callers", &view_in, include_tests, include_evidence);
64366557
}
64376558
}
64386559
if (more_rows) {

‎src/mcp/mcp_internal.h‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,16 @@ enum { CBM_MCP_DEFAULT_AUTO_INDEX_LIMIT = 50000 };
2828
* without retaining per-file results. A false result means the count exceeded
2929
* file_limit or could not be established before the bounded deadline; every
3030
* such failure is fail-closed because this is the memory-admission guard. */
31+
/* Map an internal resolver strategy (as recorded on a CALLS edge by
32+
* pass_calls.c) to the CLOSED public class published by trace_path's
33+
* include_evidence output: "lsp" | "language_rule" | "heuristic" |
34+
* "unresolved". NULL only for a NULL/empty strategy.
35+
*
36+
* Exposed so tests/test_mcp.c can pin every strategy production can emit to a
37+
* known class — a new resolver KIND must fail there rather than leaking an
38+
* unmapped internal name into a user-visible field. */
39+
const char *cbm_mcp_edge_strategy_class(const char *strategy);
40+
3141
bool cbm_mcp_auto_index_within_file_limit(const char *root_path, int file_limit,
3242
int *file_count_out);
3343

‎tests/test_mcp.c‎

Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2850,6 +2850,133 @@ TEST(tool_trace_call_path_prefers_definition) {
28502850
PASS();
28512851
}
28522852

2853+
/* CONTRACT PIN for the closed strategy vocabulary published by
2854+
* trace_path(include_evidence:true).
2855+
*
2856+
* The indexer records ~20 internal strategy names on CALLS edges and the set
2857+
* grows with every language added. We publish a CLASS, not the raw name, so a
2858+
* resolver rename cannot silently change a user-visible field. This test is
2859+
* what keeps that promise honest: every strategy production can emit must land
2860+
* in a known class. Adding lsp_foo_dispatch passes automatically; introducing a
2861+
* genuinely new KIND of resolution fails HERE and forces a deliberate decision
2862+
* about the public contract instead of leaking an internal name. */
2863+
TEST(trace_evidence_strategy_class_vocabulary_is_closed) {
2864+
/* Every strategy string assigned anywhere in src/ + internal/ as of this
2865+
* commit, plus the two literals pass_calls.c writes directly. */
2866+
static const char *const lsp[] = {"lsp_direct", "lsp_base_dispatch",
2867+
"lsp_embed_dispatch", "lsp_implicit_this",
2868+
"lsp_inherited_dispatch", "lsp_method_dispatch",
2869+
"lsp_proc_macro", "lsp_smart_ptr_dispatch",
2870+
"lsp_strategy_cross_file", "lsp_trait_dispatch",
2871+
"lsp_type_dispatch", "lsp_virtual_dispatch"};
2872+
for (size_t i = 0; i < sizeof(lsp) / sizeof(lsp[0]); i++) {
2873+
const char *cls = cbm_mcp_edge_strategy_class(lsp[i]);
2874+
ASSERT_NOT_NULL(cls);
2875+
ASSERT_STR_EQ(cls, "lsp");
2876+
}
2877+
static const char *const lang[] = {"php_self_static", "php_static_resolved",
2878+
"perl_method_static", "perl_method_typed"};
2879+
for (size_t i = 0; i < sizeof(lang) / sizeof(lang[0]); i++) {
2880+
const char *cls = cbm_mcp_edge_strategy_class(lang[i]);
2881+
ASSERT_NOT_NULL(cls);
2882+
ASSERT_STR_EQ(cls, "language_rule");
2883+
}
2884+
static const char *const heur[] = {"callee_suffix", "field_type_hint", "service_pattern",
2885+
"fastapi_depends"};
2886+
for (size_t i = 0; i < sizeof(heur) / sizeof(heur[0]); i++) {
2887+
const char *cls = cbm_mcp_edge_strategy_class(heur[i]);
2888+
ASSERT_NOT_NULL(cls);
2889+
ASSERT_STR_EQ(cls, "heuristic");
2890+
}
2891+
/* A failed LSP resolution is reported as unresolved, not as "lsp" — the
2892+
* caller's question is whether the edge is trustworthy, and "we tried LSP
2893+
* and it did not resolve" answers no. */
2894+
ASSERT_STR_EQ(cbm_mcp_edge_strategy_class("lsp_unresolved"), "unresolved");
2895+
ASSERT_STR_EQ(cbm_mcp_edge_strategy_class("unknown"), "unresolved");
2896+
/* Only a NULL/empty strategy is unclassified — an unmapped non-empty value
2897+
* must never silently disappear from the output. */
2898+
ASSERT_NULL(cbm_mcp_edge_strategy_class(NULL));
2899+
ASSERT_NULL(cbm_mcp_edge_strategy_class(""));
2900+
ASSERT_STR_EQ(cbm_mcp_edge_strategy_class("some_future_resolver"), "heuristic");
2901+
PASS();
2902+
}
2903+
2904+
/* Distilled from #559 (@vvenegasv). The indexer already records
2905+
* {strategy, confidence} on every CALLS edge (pass_calls.c:355) and the store
2906+
* reads it back, but no tool ever surfaced it — an agent could see THAT A->B
2907+
* exists, never HOW it was resolved.
2908+
*
2909+
* Binds two things at once: the evidence columns appear only when asked for
2910+
* (default stays lean), and the published value is the CLASS, not the raw
2911+
* internal strategy name. Fails without the production change in both
2912+
* directions — no columns at all before, and "lsp_trait_dispatch" would leak
2913+
* verbatim if the classifier were bypassed. */
2914+
TEST(tool_trace_path_evidence_is_opt_in_and_class_mapped) {
2915+
cbm_mcp_server_t *srv = cbm_mcp_server_new(NULL);
2916+
cbm_store_t *st = cbm_mcp_server_store(srv);
2917+
const char *proj = "ev-proj";
2918+
cbm_mcp_server_set_project(srv, proj);
2919+
cbm_store_upsert_project(st, proj, "/tmp/ev");
2920+
cbm_node_t caller = {.project = proj,
2921+
.label = "Function",
2922+
.name = "caller",
2923+
.qualified_name = "ev-proj.src.caller",
2924+
.file_path = "src/a.c",
2925+
.start_line = 1,
2926+
.end_line = 5};
2927+
cbm_node_t callee = {.project = proj,
2928+
.label = "Function",
2929+
.name = "target",
2930+
.qualified_name = "ev-proj.src.target",
2931+
.file_path = "src/a.c",
2932+
.start_line = 10,
2933+
.end_line = 20};
2934+
int64_t id_caller = cbm_store_upsert_node(st, &caller);
2935+
int64_t id_callee = cbm_store_upsert_node(st, &callee);
2936+
ASSERT_GT(id_caller, 0);
2937+
ASSERT_GT(id_callee, 0);
2938+
/* Exactly the shape pass_calls.c:355 writes in production. */
2939+
cbm_edge_t e = {.project = proj,
2940+
.source_id = id_caller,
2941+
.target_id = id_callee,
2942+
.type = "CALLS",
2943+
.properties_json = "{\"callee\":\"target\",\"confidence\":0.95,"
2944+
"\"strategy\":\"lsp_trait_dispatch\",\"candidates\":1}"};
2945+
ASSERT_GT(cbm_store_insert_edge(st, &e), 0);
2946+
2947+
/* Default: lean. No evidence columns, no strategy anywhere. */
2948+
char *plain = cbm_mcp_server_handle(
2949+
srv, "{\"jsonrpc\":\"2.0\",\"id\":91,\"method\":\"tools/call\","
2950+
"\"params\":{\"name\":\"trace_path\",\"arguments\":{\"function_name\":\"caller\","
2951+
"\"project\":\"ev-proj\",\"direction\":\"outbound\"}}}");
2952+
ASSERT_NOT_NULL(plain);
2953+
char *plain_txt = extract_text_content(plain);
2954+
ASSERT_NOT_NULL(plain_txt);
2955+
ASSERT_NOT_NULL(strstr(plain_txt, "target")); /* positive control: the hop IS there */
2956+
ASSERT_NULL(strstr(plain_txt, "lsp"));
2957+
ASSERT_NULL(strstr(plain_txt, "0.95"));
2958+
free(plain_txt);
2959+
free(plain);
2960+
2961+
/* Opted in: the class and the confidence appear, the raw name does not. */
2962+
char *ev = cbm_mcp_server_handle(
2963+
srv, "{\"jsonrpc\":\"2.0\",\"id\":92,\"method\":\"tools/call\","
2964+
"\"params\":{\"name\":\"trace_path\",\"arguments\":{\"function_name\":\"caller\","
2965+
"\"project\":\"ev-proj\",\"direction\":\"outbound\",\"include_evidence\":true}}}");
2966+
ASSERT_NOT_NULL(ev);
2967+
char *ev_txt = extract_text_content(ev);
2968+
ASSERT_NOT_NULL(ev_txt);
2969+
ASSERT_NOT_NULL(strstr(ev_txt, "target"));
2970+
ASSERT_NOT_NULL(strstr(ev_txt, "lsp"));
2971+
ASSERT_NOT_NULL(strstr(ev_txt, "0.95"));
2972+
/* The internal resolver name must NOT reach the client. */
2973+
ASSERT_NULL(strstr(ev_txt, "lsp_trait_dispatch"));
2974+
free(ev_txt);
2975+
free(ev);
2976+
cbm_mcp_server_free(srv);
2977+
PASS();
2978+
}
2979+
28532980
/* Reproduce-first (#887): the client-supplied `depth` on trace_call_path must be
28542981
* clamped to the MCP ceiling (cbm_mcp_max_depth(), default 15). On origin/main
28552982
* an MCP_MAX_DEPTH=15 constant was defined but never applied — `depth` flowed
@@ -9800,6 +9927,8 @@ SUITE(mcp) {
98009927
RUN_TEST(tool_trace_union_records_min_hop_across_seeds);
98019928
RUN_TEST(tool_trace_pagination_exactly_once);
98029929
RUN_TEST(tool_trace_call_path_prefers_definition);
9930+
RUN_TEST(trace_evidence_strategy_class_vocabulary_is_closed);
9931+
RUN_TEST(tool_trace_path_evidence_is_opt_in_and_class_mapped);
98039932
RUN_TEST(tool_trace_call_path_depth_clamped);
98049933
RUN_TEST(tool_trace_call_path_distinct_defs_not_over_unioned);
98059934
RUN_TEST(tool_trace_call_path_dts_stub_unions_with_impl);

0 commit comments

Comments
 (0)