Skip to content

Commit 0fd175c

Browse files
committed
feat(mcp): TOON output for trace/query/search_code/architecture + context-bomb defaults
Completes the compact-output pass across the remaining query tools (consumers are LLM agents; every response byte is context tokens): - trace_path: callees[N]{qn,hop}/callers[N]{qn,hop} tables (risk/test/ args columns per flags); the per-hop JSON key envelope was 84% of the payload. 3,589B -> 1,634B on a representative trace. - query_graph: rows[N]{cols}: table (columns double as the header); format:"json" restores columns/rows arrays. - search_code compact mode: results/raw/dirs tables; drops the field (duplicate of the qn tail). 2,488B -> 1,135B for 17 hits. - search_code full mode: per-hit source capped at a 60-line window anchored on the first match, with source_start/source_truncated markers — uncapped whole-symbol dumps ran to 142KB per response; the complete symbol stays one get_code_snippet call away. - get_architecture: default (no aspects) is now a summary (languages, packages, entry_points + always-on totals/label/type counts) with an aspects_hint — the old default rendered EVERYTHING including the full file_tree (94KB -> 4.5KB -> 2.8KB with TOON tables); all sections emit as TOON tables, aspects:[...] and ["all"] keep full access (52KB vs 94KB even for the explicit full dump). - get_code_snippet: drops the property-blob enrichment (41% of the response; signature/docstring are literally in the source; fp/sp/bt never belonged there). Metrics remain reachable via search_graph fields=[...]. - list_projects: per-project 12-field git block (mostly null) replaced by the branch name; name/root/nodes/edges/size kept. - search_graph description: in/out documented as TOTAL all-edge degree, NOT caller/callee counts (agent evals mis-read it as callers). - format:"json" escape hatch on every converted tool. Guards: tool_output_byte_budgets pins absolute byte ceilings on default search/trace outputs (a property blob sneaking back into row emission blows them immediately); snippet_enriched_properties inverted to pin the no-spill contract. Suite 5,984 green; smoke updated to the TOON contract end-to-end. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
1 parent 4843a34 commit 0fd175c

4 files changed

Lines changed: 724 additions & 109 deletions

File tree

‎scripts/smoke-test.sh‎

Lines changed: 34 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -187,7 +187,9 @@ echo "OK: search_graph found $TOTAL result(s) for 'compute'"
187187
if ! TRACE=$(cli trace_path --project "$PROJECT" --function-name compute --direction inbound --depth 1); then
188188
echo "FAIL: trace_path (flag form) exited non-zero"; cat "$CLI_STDERR"; exit 1
189189
fi
190-
CALLERS=$(echo "$TRACE" | python3 -c "import json,sys; d=json.loads(sys.stdin.read()); print(len(d.get('callers',[])))" 2>/dev/null || echo "0")
190+
# trace_path default output is TOON: the callers[N]{...} header carries the count
191+
CALLERS=$(echo "$TRACE" | sed -n 's/^callers\[\([0-9]*\)\].*/\1/p' | head -1)
192+
CALLERS=${CALLERS:-0}
191193
if [ "$CALLERS" -lt 1 ]; then
192194
echo "FAIL: trace_path found 0 callers for 'compute'"
193195
exit 1
@@ -220,7 +222,8 @@ echo "OK: $FOLDER_COUNT Folder nodes (init.py didn't clobber them)"
220222
# 3d-cypher: query_graph Cypher capabilities
221223
# #238 WITH DISTINCT — all functions share label "Function" → collapses to 1 row.
222224
CYPHER_WD=$(cli query_graph --project "$PROJECT" --query "MATCH (f:Function) WITH DISTINCT f.label AS lbl RETURN lbl")
223-
WD_ROWS=$(echo "$CYPHER_WD" | python3 -c "import json,sys; d=json.loads(sys.stdin.read()); print(len(d.get('rows',[])))" 2>/dev/null || echo "0")
225+
WD_ROWS=$(echo "$CYPHER_WD" | sed -n 's/^total: //p' | head -1)
226+
WD_ROWS=${WD_ROWS:-0}
224227
if [ "$WD_ROWS" -lt 1 ]; then
225228
echo "FAIL: query_graph WITH DISTINCT returned 0 rows"
226229
echo "$CYPHER_WD"
@@ -230,7 +233,8 @@ echo "OK: query_graph WITH DISTINCT returned $WD_ROWS row(s)"
230233

231234
# #241 WHERE label test — f:Function is true for every Function node.
232235
CYPHER_LBL=$(cli query_graph --project "$PROJECT" --query "MATCH (f:Function) WHERE f:Function RETURN f.name")
233-
LBL_ROWS=$(echo "$CYPHER_LBL" | python3 -c "import json,sys; d=json.loads(sys.stdin.read()); print(len(d.get('rows',[])))" 2>/dev/null || echo "0")
236+
LBL_ROWS=$(echo "$CYPHER_LBL" | sed -n 's/^total: //p' | head -1)
237+
LBL_ROWS=${LBL_ROWS:-0}
234238
if [ "$LBL_ROWS" -lt 1 ]; then
235239
echo "FAIL: query_graph WHERE label-test returned 0 rows"
236240
echo "$CYPHER_LBL"
@@ -240,7 +244,8 @@ echo "OK: query_graph WHERE f:Function returned $LBL_ROWS row(s)"
240244

241245
# #242 label alternation — (n:Function|Module) seeds either label.
242246
CYPHER_ALT=$(cli query_graph --project "$PROJECT" --query "MATCH (n:Function|Module) RETURN n.name")
243-
ALT_ROWS=$(echo "$CYPHER_ALT" | python3 -c "import json,sys; d=json.loads(sys.stdin.read()); print(len(d.get('rows',[])))" 2>/dev/null || echo "0")
247+
ALT_ROWS=$(echo "$CYPHER_ALT" | sed -n 's/^total: //p' | head -1)
248+
ALT_ROWS=${ALT_ROWS:-0}
244249
if [ "$ALT_ROWS" -lt 1 ]; then
245250
echo "FAIL: query_graph label alternation returned 0 rows"
246251
echo "$CYPHER_ALT"
@@ -250,7 +255,8 @@ echo "OK: query_graph (n:Function|Module) returned $ALT_ROWS row(s)"
250255

251256
# #239 count(DISTINCT) — must parse and return a single aggregate row.
252257
CYPHER_CD=$(cli query_graph --project "$PROJECT" --query "MATCH (f:Function) RETURN count(DISTINCT f.label)")
253-
CD_ROWS=$(echo "$CYPHER_CD" | python3 -c "import json,sys; d=json.loads(sys.stdin.read()); print(len(d.get('rows',[])))" 2>/dev/null || echo "0")
258+
CD_ROWS=$(echo "$CYPHER_CD" | sed -n 's/^total: //p' | head -1)
259+
CD_ROWS=${CD_ROWS:-0}
254260
if [ "$CD_ROWS" -ne 1 ]; then
255261
echo "FAIL: query_graph count(DISTINCT) expected 1 row, got $CD_ROWS"
256262
echo "$CYPHER_CD"
@@ -264,7 +270,7 @@ cyp_first_cell() {
264270
# argv token, so string-literal args (e.g. replace(f.name,"a","A")) and Cypher
265271
# metacharacters {}|=~<>" need no JSON escaping.
266272
cli query_graph --project "$PROJECT" --query "$1" |
267-
python3 -c "import json,sys; d=json.loads(sys.stdin.read()); rows=d.get('rows',[]); print(rows[0][0] if rows and rows[0] else '')" 2>/dev/null || echo ""
273+
sed -n '/^rows\[/{n;p;}' | sed 's/^ //' | sed 's/^"//;s/"$//;s/\\"/"/g'
268274
}
269275

270276
# labels(n) → JSON list like ["Function"]
@@ -319,15 +325,17 @@ echo "OK: query_graph coalesce(f.nonesuch, f.name) = $COALV"
319325

320326
# EXISTS { } pattern predicate (edge-type-specific existence)
321327
CYPHER_EX=$(cli query_graph --project "$PROJECT" --query "MATCH (f:Function) WHERE EXISTS { (f)-[:CALLS]->() } RETURN f.name")
322-
EX_ROWS=$(echo "$CYPHER_EX" | python3 -c "import json,sys; d=json.loads(sys.stdin.read()); print(len(d.get('rows',[])))" 2>/dev/null || echo "0")
328+
EX_ROWS=$(echo "$CYPHER_EX" | sed -n 's/^total: //p' | head -1)
329+
EX_ROWS=${EX_ROWS:-0}
323330
if [ "$EX_ROWS" -lt 1 ]; then
324331
echo "FAIL: query_graph EXISTS{} predicate returned 0 rows"; echo "$CYPHER_EX"; exit 1
325332
fi
326333
echo "OK: query_graph EXISTS { (f)-[:CALLS]->() } returned $EX_ROWS row(s)"
327334

328335
# =~ regex match in WHERE
329336
CYPHER_RX=$(cli query_graph --project "$PROJECT" --query 'MATCH (f:Function) WHERE f.name =~ ".+" RETURN f.name')
330-
RX_ROWS=$(echo "$CYPHER_RX" | python3 -c "import json,sys; d=json.loads(sys.stdin.read()); print(len(d.get('rows',[])))" 2>/dev/null || echo "0")
337+
RX_ROWS=$(echo "$CYPHER_RX" | sed -n 's/^total: //p' | head -1)
338+
RX_ROWS=${RX_ROWS:-0}
331339
if [ "$RX_ROWS" -lt 1 ]; then
332340
echo "FAIL: query_graph WHERE =~ regex returned 0 rows"; echo "$CYPHER_RX"; exit 1
333341
fi
@@ -350,7 +358,7 @@ LEFTV=$(cyp_first_cell 'MATCH (f:Function) RETURN left(f.name, 3) AS l LIMIT 1')
350358

351359
# NOT EXISTS dead-code query (functions with no caller)
352360
CYPHER_NX=$(cli query_graph --project "$PROJECT" --query "MATCH (f:Function) WHERE NOT EXISTS { (f)<-[:CALLS]-() } RETURN f.name")
353-
NX_OK=$(echo "$CYPHER_NX" | python3 -c "import json,sys; d=json.loads(sys.stdin.read()); print('rows' in d)" 2>/dev/null || echo "False")
361+
NX_OK=$(echo "$CYPHER_NX" | grep -qE '^rows\[[0-9]+\]\{' && echo "True" || echo "False")
354362
[ "$NX_OK" = "True" ] && echo "OK: query_graph NOT EXISTS dead-code query executed" || { echo "FAIL: NOT EXISTS query"; echo "$CYPHER_NX" | head -c 300; exit 1; }
355363

356364
# CASE expression in RETURN
@@ -373,19 +381,33 @@ esac
373381
if ! ARCH=$(cli get_architecture --project "$PROJECT" --aspects clusters); then
374382
echo "FAIL: get_architecture (flag form) exited non-zero"; cat "$CLI_STDERR"; exit 1
375383
fi
376-
NCLUST=$(echo "$ARCH" | python3 -c "import json,sys; d=json.loads(sys.stdin.read()); print(len(d.get('clusters',[])))" 2>/dev/null || echo "0")
384+
# get_architecture default output is TOON: clusters[N]{...} header carries the count
385+
NCLUST=$(echo "$ARCH" | sed -n 's/^clusters\[\([0-9]*\)\].*/\1/p' | head -1)
386+
NCLUST=${NCLUST:-0}
377387
if [ "$NCLUST" -lt 1 ]; then
378388
echo "FAIL: get_architecture returned 0 community clusters"; echo "$ARCH" | head -c 400; exit 1
379389
fi
380390
echo "OK: get_architecture returned $NCLUST community cluster(s)"
381391

382392
# 3g: search_code — basic search reports elapsed_ms + matches
383393
SC=$(cli search_code --project "$PROJECT" --pattern cbm_ --mode compact --limit 5)
384-
echo "$SC" | python3 -c "import json,sys; d=json.loads(sys.stdin.read()); assert 'elapsed_ms' in d; print('OK: search_code elapsed_ms='+str(d['elapsed_ms'])+' total_grep_matches='+str(d.get('total_grep_matches')))" 2>/dev/null || { echo "FAIL: search_code basic / no elapsed_ms"; echo "$SC" | head -c 400; exit 1; }
394+
# compact mode emits TOON scalars: `elapsed_ms: N` + `total_grep_matches: N`
395+
SC_ELAPSED=$(echo "$SC" | sed -n 's/^elapsed_ms: //p' | head -1)
396+
SC_GREPM=$(echo "$SC" | sed -n 's/^total_grep_matches: //p' | head -1)
397+
if [ -n "$SC_ELAPSED" ]; then
398+
echo "OK: search_code elapsed_ms=$SC_ELAPSED total_grep_matches=${SC_GREPM:-0}"
399+
else
400+
echo "FAIL: search_code basic / no elapsed_ms"; echo "$SC" | head -c 400; exit 1
401+
fi
385402

386403
# 3g: search_code — literal '|' under regex=false must surface a warning (#282)
387404
SCW=$(cli search_code --project "$PROJECT" --pattern "cbm_init|cbm_nope" --regex false --limit 5)
388-
echo "$SCW" | python3 -c "import json,sys; d=json.loads(sys.stdin.read()); w=' '.join(d.get('warnings',[])); assert 'regex=true' in w; print('OK: search_code literal-| warning surfaced')" 2>/dev/null || { echo "FAIL: search_code literal-| warning missing"; echo "$SCW" | head -c 400; exit 1; }
405+
# TOON scalar `warning: ... regex=true ...`
406+
if echo "$SCW" | grep -q "regex=true"; then
407+
echo "OK: search_code literal-| warning surfaced"
408+
else
409+
echo "FAIL: search_code literal-| warning missing"; echo "$SCW" | head -c 400; exit 1
410+
fi
389411

390412
# 3g: search_code — '&' in file_pattern accepted, not rejected as invalid (#272)
391413
SCA=$(cli search_code --project "$PROJECT" --pattern cbm_ --file-pattern "*R&D*.c" --limit 5)

0 commit comments

Comments
 (0)