Skip to content

refactor(code-review): decouple review agents from codebase-memory-mcp - #195

Merged
wongk merged 4 commits into
mainfrom
refactor/code-review-substrate-agnostic
Sep 10, 2026
Merged

wongk merged 4 commits into
mainfrom
refactor/code-review-substrate-agnostic

Conversation

@wongk

@wongk wongk commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

Why

The reviewer fleet was hard-bound to one MCP server. agents/code-review-worker-graph.md declared an allowlist of six mcp__codebase-memory-mcp__* tools, and an allowlist is exhaustive — no other server could ever reach a reviewer, no matter what the operator had connected. Unresolvable entries were silently dropped rather than erroring, which is why the coupling went unnoticed on machines without that server: reviews just quietly ran grep-only forever.

This makes the plugin substrate-agnostic. Reviewers now use whatever code-intelligence MCP server the operator's session provides — codebase-memory-mcp, LemonCrow, closedloop-graph, or something that does not exist yet — with no plugin edit required.

Approach

Access. The graph worker drops tools: entirely and inherits the spawning session, with disallowedTools: Bash, Edit, NotebookEdit as harness-level hardening. code-review-worker keeps its explicit four-tool allowlist, so the verifier fleet, Bug Hunter A, the domain critics, and the singleton prompts still inherit nothing — the adversarial trust boundary is unchanged.

Protocol. The codebase-memory-mcp block in shared_prompt.txt becomes OPTIONAL — CODE INTELLIGENCE: four capabilities (C1 symbol lookup, C2 usage/caller enumeration, C3 snippet read, C4 structure) that the reviewer binds to whatever it holds, discovering its own roster and loading deferred MCP schemas with ToolSearch. Same rewrite across the impact-analyzer, design-critic, verifier prompts and the four suffixes in spawn-reviewers.

Orchestrator. GRAPH_PROJECT collapses to one computed boolean, CODE_INTEL_ALLOWED, false whenever review_root is set. That preserves the stale-index kill switch — substrate-independent, since any index covers the operator checkout rather than the PR head — and drops the list_projects handshake that substituted a server-returned project name into the agents' trusted instruction zone.

Schema: comments only. discovery: "graph" was already substrate-neutral, so no schema change and no fixture churn.

Verified at runtime, not from docs

Each claim below was checked by running the real plugin through headless claude -p --plugin-dir and inspecting the spawned worker's actual tool roster.

Check Result
Baseline (before) Graph worker got exactly Read, Write, Grep, Globzero MCP. The allowlist was genuinely gating.
Unresolvable entries Agent still spawns; missing MCP tools silently dropped. Confirms the failure was invisible.
After Worker inherits 139 MCP tools; Impact Analyzer bound C2 to mcp__lc__relations — a server this plugin never names — and cross-checked with mcp__lc__code_query.
Kill switch CODE_INTEL_ALLOWED=false → zero MCP calls, silent degradation, finding still emitted.
disallowedTools Bash/Edit/NotebookEdit removed; MCP inheritance survives intact.
tools: mcp__* Silently dropped — wildcards are denylist-only, so allowlist and inheritance are mutually exclusive. Worth knowing before anyone tries to have both.

A fail-open this surfaced

Inheriting tools means inheriting the parent's absence of them. In a session without Grep/Glob, the reviewer still emitted grep_query_used: "_parse_agent_name\\("a query it never ran — which the verifier replays as its fabrication check. That is a fail-open in the exact mechanism designed to catch hallucination.

Fixed in both prompts: text search is a capability, not a tool name, and any recorded search must describe a query actually executed. With no text-search tool the analyzer now emits grep_query_used: null, external_usages_found: [], and tags entries discovery: "graph", routing them to the per-entry file-read and content-match audit — which verifier_prompt.txt already handles as the all-graph case, so no verifier change was needed.

Re-spiked after the fix and asserted the replay-gate invariant against the real vocabulary:

discovery value in vocab: True -> graph
grep_query_used: None
external_usages_found: []
grep-discovery entries: 0
REPLAY-GATE INVARIANT HOLDS: True

Verification

  • pytest plugins/ — 2107 passed, 3 skipped
  • uv run ruff check . — clean
  • uv run pyright — 0 errors
  • Zero references to codebase-memory-mcp or GRAPH_PROJECT remain outside historical CHANGELOG entries
  • README re-verified against disk: accurate, no changes needed beyond the agent line

Notes for review

  • Expected one-time cache miss. shared_prompt.txt and verifier_prompt.txt bytes feed prompt_hash, so the first run after merge re-pays the BHA and verification namespaces. Correct behavior — cached findings were produced under the old protocol.
  • Known gap: disallowedTools cannot reach shell-equivalent MCP tools (e.g. mcp__lc__bash). Addressed with one clause in the agent prompt rather than new machinery, on the principle that the harness owns permissions.
  • Version 3.7.1 → 3.8.0 (MINOR: new capability, no breaking interface change).

🤖 Generated with Claude Code

https://claude.ai/code/session_011QjUMmrqfGU4QNXDDRLPrN

@shafty023

Copy link
Copy Markdown
Collaborator

Beginning a code-review. Please allow time for me to finish before merging the PR

Comment thread plugins/code-review/agents/code-review-worker-graph.md Outdated
Comment thread plugins/code-review/tools/prompts/impact_analyzer_prompt.txt
wongk and others added 3 commits September 10, 2026 11:58
The graph worker declared an allowlist of six mcp__codebase-memory-mcp__*
tools, so only that server could ever reach a reviewer. It now declares no
tools: allowlist and inherits the spawning session, with disallowedTools
for Bash/Edit/NotebookEdit. The generic worker keeps its four-tool
allowlist so the verifier fleet still inherits nothing.

The knowledge-graph protocol becomes a capability contract (C1-C4) that
reviewers bind to whatever tools they hold; GRAPH_PROJECT collapses to a
CODE_INTEL_ALLOWED boolean and the orchestrator makes no graph calls.

Also stops the Impact Analyzer emitting a grep_query_used it never ran
when the session provides no text-search tool.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011QjUMmrqfGU4QNXDDRLPrN
The impact-analyzer requirements block and Step 2 still stated the
grep_query_used requirement categorically, 15 lines before the new
no-text-search exception. An analyzer without a search tool read the
rejection threat first and the exception second, which pushes it back
toward inventing a query. Both statements are now qualified.

spawn-reviewers claimed both worker types "end up with the core Read,
Write, Grep, Glob". That is false for the inheriting worker and
contradicted the permission-inheritance warning five lines above.

Widens the documented meaning of discovery: "graph" to cover the
no-text-search branch this change introduced. Vocabulary unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011QjUMmrqfGU4QNXDDRLPrN
The suffix injected into the Impact Analyzer's task prompt still stated
the grep contract unconditionally -- "Always run grep too", "populated
external_impact[] and grep_query_used", "If grep returns zero external
usages" -- so the no-text-search fallback was contradicted at the point
of injection even after the prompt file itself was qualified.

Adds coverage for the shape that fallback emits: an all-graph finding
with grep_query_used null, and with the field absent.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011QjUMmrqfGU4QNXDDRLPrN
@wongk
wongk force-pushed the refactor/code-review-substrate-agnostic branch from b6cb836 to fd21741 Compare September 10, 2026 17:12
The changelog said the denylist "keeps the read-only-reviewer boundary"
and the agent comment said reviewers "never shell out or mutate source".
Both overstate it: disallowedTools matches only whole servers or every
MCP tool, so a write-shaped inherited MCP tool is covered by prompt
instruction, not by the denylist.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011QjUMmrqfGU4QNXDDRLPrN
@wongk
wongk merged commit 9e97927 into main Sep 10, 2026
5 checks passed
@wongk
wongk deleted the refactor/code-review-substrate-agnostic branch September 10, 2026 17:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants