Skip to content

[ci] hand the review subagents the PR diff as a file - #658

Open
tiankongdeguiji wants to merge 1 commit into
alibaba:masterfrom
tiankongdeguiji:ci/review-pr-subagent-diff
Open

[ci] hand the review subagents the PR diff as a file#658
tiankongdeguiji wants to merge 1 commit into
alibaba:masterfrom
tiankongdeguiji:ci/review-pr-subagent-diff

Conversation

@tiankongdeguiji

Copy link
Copy Markdown
Collaborator

Problem

The five reviewer subagents are declared tools: Glob, Grep, Read — no shell — and the PR checkout uses fetch-depth: 1, so there is no merge base on disk for any git diff to resolve against. gh pr diff is the only source of the change, and it is main-agent-only.

That left the subagents' view of the change entirely up to what the dispatcher managed to put in their prompts, and since facfbd00 added --permission-mode manual the dispatcher can no longer create the file itself: shell redirection is refused (Output redirection to '…' was blocked), and Write is not in --allowedTools. The one channel still working was accidental — when gh pr diff output exceeds the harness inline limit, Claude Code spills it to a tool-results/*.txt file and the dispatcher can pass that path along. Under the limit, the output stays in the dispatcher's context and the subagents get a prose paraphrase instead, so they read files at HEAD and infer what is new.

The perverse part is the direction: large PRs get the real diff, small ones do not.

Fix

A workflow step saves the diff before the model starts:

- name: Save PR diff for the reviewers
  working-directory: pr-code
  run: gh pr diff ${{ github.event.pull_request.number }} --repo ${{ github.repository }} > pr.diff

The file then exists for every PR at every size, with no write permission involved and no dependence on --add-dir / or on the dispatcher noticing a persisted-output notice. review-pr.md points the dispatcher at pr.diff and states what the subagents actually have, and each agent carries a one-line **Scope:** paragraph telling it to read the diff first rather than reconstruct the change from HEAD — so the fix holds even when the dispatcher writes a sloppy prompt.

The step runs after use_trusted_agent_files.sh, so pr.diff stays untracked and the trusted-swap commit is unchanged; it also lands after the PR checkout, so a PR cannot pre-seed the file.

Both jobs get the step. The codex job needs it for the same reason and more strictly: its subagents run sandbox_mode = "read-only" with no network, so gh pr diff is unreachable from inside a review there.

Bash(gh pr diff:*) stays in the allowlist as a fallback; the instructions simply stop asking for it.

Test Plan

Rehearsed the CI step sequence locally against a shallow clone (--depth 1), running the real use_trusted_agent_files.sh followed by the new step:

  • The trusted swap still works — a PR-authored root CLAUDE.md and a nested tzrec/.claude/ are stripped and replaced, and pr.diff lands untracked (?? pr.diff) so the setup commit is unaffected.
  • git diff master...HEAD in that checkout still fails with unknown revision, confirming the file is genuinely the only channel available to the reviewers.
  • All five trusted agent copies carry the **Scope:** line.

Ran the diff command against a small PR and a large one (91 lines / 3 files and 1256 lines / 11 files) — both produce a well-formed unified diff as a file, including the small case that previously fell back to prose.

Confirmed the harness resolves a relative Read path against the session working directory, so a subagent finds pr.diff whether it is handed an absolute or a relative path.

scripts/ci/gen_codex_review.py regenerated: all five .codex/agents/*.toml carry the scope instruction and the emitted prompt carries the pr.diff contract. Workflow YAML parses and both jobs show the new step in order. pre-commit run --files passes on all seven changed files.

🤖 Generated with Claude Code

https://claude.ai/code/session_013KX52EswZ1UUySg2CN2nEg

The reviewer subagents have Read but no shell, and the PR checkout is
fetch-depth: 1, so no local git command can produce the change. Their only
view of it came from whatever the dispatcher wrote into their prompts, and
since --permission-mode manual denies both shell redirection and Write, the
dispatcher could only pass a diff along when `gh pr diff` happened to exceed
the harness inline limit and got spilled to disk; on smaller PRs it fell back
to paraphrasing, and the agents reviewed HEAD instead of the change.

A workflow step now saves the diff to pr.diff in the review checkout before
the model starts, so the file always exists and needs no write permission.
review-pr.md points the dispatcher at it and states what the subagents can
do, and each agent is told to read it first rather than infer the change
from HEAD.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013KX52EswZ1UUySg2CN2nEg
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant