From 87e8f8888fa8d5006a884ed31f18d876eaf90a76 Mon Sep 17 00:00:00 2001 From: "daniel.ochoa" Date: Fri, 20 Mar 2026 10:33:43 -0500 Subject: [PATCH 01/12] feat(code): add plan-with-codex debate loop for iterative plan refinement Introduces a Claude + Codex debate workflow where Claude drafts implementation plans and Codex reviews them in iterative rounds until approval or max rounds are reached. New components: - plan-agent: opus-model architect agent for plan creation/revision - plan-with-codex command: orchestrates the debate loop - codex-review skill: structured Codex review with verdict tokens - debate-loop.sh: standalone shell harness for the debate - plan-review.sh hook: triggers Codex review on ExitPlanMode --- CHANGELOG.md | 9 + plugins/code/.claude-plugin/plugin.json | 2 +- plugins/code/README.md | 39 ++ plugins/code/agents/plan-agent.md | 94 +++ plugins/code/commands/plan-with-codex.md | 262 ++++++++ plugins/code/hooks/plan-review.sh | 76 +++ plugins/code/scripts/debate-loop.sh | 608 ++++++++++++++++++ plugins/code/skills/codex-review/SKILL.md | 86 +++ .../codex-review/scripts/run_codex_review.sh | 231 +++++++ 9 files changed, 1406 insertions(+), 1 deletion(-) create mode 100644 plugins/code/agents/plan-agent.md create mode 100644 plugins/code/commands/plan-with-codex.md create mode 100755 plugins/code/hooks/plan-review.sh create mode 100755 plugins/code/scripts/debate-loop.sh create mode 100644 plugins/code/skills/codex-review/SKILL.md create mode 100755 plugins/code/skills/codex-review/scripts/run_codex_review.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index e2ac31f4..8a26fca5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,15 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ## [Unreleased] +### code v1.2.0 + +#### Added +- New `plan-agent` agent for creating and revising implementation plans via codebase exploration +- New `plan-with-codex` command for iterative plan refinement through Claude + Codex debate loops +- New `codex-review` skill to run Codex plan reviews and return structured verdict feedback +- New `debate-loop.sh` script providing standalone CLI for Claude + Codex debate orchestration +- New `plan-review.sh` hook that triggers Codex review when Claude exits plan mode + ### code-review v1.2.0 #### Added diff --git a/plugins/code/.claude-plugin/plugin.json b/plugins/code/.claude-plugin/plugin.json index 9d5cf0ae..2548bb66 100644 --- a/plugins/code/.claude-plugin/plugin.json +++ b/plugins/code/.claude-plugin/plugin.json @@ -1,7 +1,7 @@ { "name": "code", "description": "Code and planning framework plugin", - "version": "1.1.4", + "version": "1.2.0", "author": { "name": "ClosedLoop", "email": "support@closedloop.ai" diff --git a/plugins/code/README.md b/plugins/code/README.md index 62aac776..e42af853 100644 --- a/plugins/code/README.md +++ b/plugins/code/README.md @@ -102,6 +102,30 @@ Conversation state is persisted across GitHub workflow runs via `amend_state.py` Checks for `.claude/closedloop-loop.local.md`, reads the current iteration, removes the file, and reports the cancellation. Hidden from the slash command picker. +### `/code:plan-with-codex` + +**Description:** Iterative plan refinement debate between Claude and Codex. + +**Usage:** +``` +/code:plan-with-codex [--max-rounds N] [--plan-file PATH] [--codex-model MODEL] +``` + +- `--max-rounds N`: Maximum debate rounds (default: 15) +- `--plan-file PATH`: Output plan file (default: `./debate-plan.md`) +- `--codex-model MODEL`: Codex model for reviews (default: `gpt-5.4`) +- ``: Description of what to plan (required; can also be read from a `{stem}.prompt` sidecar file) + +**What it does:** + +1. Launches `code:plan-agent` to create an initial plan from the prompt, writing it to `--plan-file` +2. Presents open questions to the user and resolves them via the plan-agent before proceeding +3. Presents the plan for user review and applies any requested changes +4. Enters a fully automated debate loop: Codex reviews the plan via the `code:codex-review` skill, Claude revises per feedback, and the loop continues until `VERDICT:APPROVED` or max rounds are reached +5. Reports the final outcome and cleans up sidecar state files + +Supports resuming mid-session via a `{stem}.state` sidecar file that tracks the current round and Codex session ID. + --- ## Agents @@ -174,6 +198,9 @@ Extracts actionable plan amendments from unstructured input (meeting notes, Slac **`code-review-worker`** / **`code-review-guidelines`** Supporting agents for the code review workflow. `code-review-guidelines` provides language-specific review patterns and edge case guidance. `code-review-worker` handles individual file review tasks within the reviewer workflow. +**`plan-agent`** (model: opus) +Software architect agent for creating and revising implementation plans. Explores the codebase, designs a plan grounded in existing patterns, and writes it directly to a specified file on disk. Used by the `plan-with-codex` command; resumed across debate rounds via `SendMessage` to apply Codex feedback without losing architectural context. + --- ## Skills @@ -224,6 +251,10 @@ Dynamically locates files within the Claude Code plugins cache directory (`~/.cl A 4-phase protocol for orchestrators to refine subagent queries through follow-up questions. Phases: Initial Dispatch, Sufficiency Evaluation (4-question checklist), Refinement Request (resume with targeted follow-ups), and Loop (up to 3 cycles). Used when initial subagent responses may miss important adjacent context. +### `codex-review` + +Runs Codex to review a plan file and returns structured feedback with a verdict. Called once per debate round by the `plan-with-codex` command via `debate-loop.sh`. Supports session resume across rounds using a Codex thread ID. Returns `VERDICT:APPROVED` or `VERDICT:NEEDS_CHANGES` plus a `CODEX_SESSION` token. Emits `CODEX_FAILED` or `CODEX_EMPTY` tokens on error so the orchestrator can ask the user to retry or abort. + --- ## Hooks @@ -269,6 +300,10 @@ Implements the validation loop for agents registered in `loop-agents.json`. When Injects tool-specific learnings just before tool execution. Filters `org-patterns.toon` by tool type (Bash patterns get build/test tags; Write/Edit patterns get language-specific tags based on file extension). Injects up to 10 matching patterns as `additionalContext`. Also auto-allows tool calls targeting `.closedloop-ai/` workspace paths without prompting. +### `plan-review.sh` (PostToolUse on ExitPlanMode) + +Triggers when Claude exits plan mode to get a second opinion via Codex. Extracts plan content from `tool_response.plan`, sends it to Codex (`gpt-5.3-codex-spark`) for review, and injects the feedback as `additionalContext` so Claude sees Codex's concerns inline. Exits silently if no plan content is present or Codex returns empty. Debug logs kept in `.closedloop-ai/plan-review-logs/` (max 15 files). + ### `validate-plan.sh` (validation script, not a hook directly) Used by `loop-stop-hook.sh` as the validation script for `plan-draft-writer` and `plan-writer` loop agents. Checks: plan.json exists, contains a valid `content` field, has an `## Open Questions` section, uses checkbox task format, has no TODO/TBD placeholders, and warns on unjustified new file creation or code duplication patterns. Outputs `VALIDATION: PASS` or `VALIDATION: FAIL` with issue details. @@ -326,6 +361,10 @@ Discovers peer repositories for cross-repo orchestration by checking the `CLAUDE Installs required Python dependencies for the plugin's tools. +### `debate-loop.sh` + +Shell harness for the Codex debate loop used by the `plan-with-codex` command. Drives the round-by-round interaction between the `code:plan-agent` and Codex, maintaining state in sidecar files alongside the plan file. + ### `loop-agents.json` Configuration file (not a script) defining which agents participate in the validation loop. Each entry specifies: `validation_script`, `max_iterations`, `promise` (expected completion string), `state_file_suffix`, and `verification_criteria`. Also defines `learning_agents` — the subset of agents that must both acknowledge injected learnings and capture new learnings before stopping. diff --git a/plugins/code/agents/plan-agent.md b/plugins/code/agents/plan-agent.md new file mode 100644 index 00000000..0db21173 --- /dev/null +++ b/plugins/code/agents/plan-agent.md @@ -0,0 +1,94 @@ +--- +name: plan-agent +description: Software architect agent for creating and revising implementation plans. Explores codebases, designs plans, and writes them directly to disk. Used by plan-with-codex for iterative plan refinement. +model: opus +tools: Read, Write, Edit, Glob, Grep, Bash +--- + +# Plan Agent + +You are a software architect and planning specialist. Your role is to explore codebases and create or revise detailed implementation plans. + +## Your Process + +1. **Understand Requirements**: Focus on the requirements provided and any feedback from reviewers. + +2. **Explore Thoroughly**: + - Read any files provided to you in the prompt + - Find existing patterns and conventions using `Glob`, `Grep`, and `Read` + - Understand the current architecture + - Identify similar features as reference + - Trace through relevant code paths + - Use `Bash` for read-only exploration (ls, git status, git log, git diff, find, cat, head, tail) + +3. **Reuse Before Creating**: + - Before proposing any new function, utility, or abstraction, search the codebase for existing implementations (`Grep`, `Glob`) + - If similar logic exists in a shared module (lib/, utils/, helpers/), extend or reuse it + - Never propose a new helper for a one-time operation + +4. **Design Solution**: + - Create an implementation approach grounded in the actual codebase + - Consider trade-offs and architectural decisions + - Follow existing patterns where appropriate + - Choose the simplest approach that fully solves the problem -- avoid unnecessary abstractions, configuration layers, or indirection + +5. **Detail the Plan**: + - Provide step-by-step implementation strategy + - Identify dependencies and sequencing + - Anticipate potential challenges + - Include test tasks (unit and/or integration) for any new logic, endpoints, or behaviors + +6. **Self-Check Before Writing**: + - **Goal alignment**: Re-read the original request. Does your plan fully accomplish it? Would executing every task actually deliver the feature, fix the bug, or achieve the objective? + - **Scope discipline**: Remove any task that was not requested. Do not add "while we're at it" improvements, refactors, or nice-to-haves beyond what the request requires. + - **Simplicity**: For each abstraction or new file in the plan, ask: "Is there a simpler way?" If three lines of inline code would work, do not propose a helper function. + +7. **Write the Plan**: + - Write the complete plan directly to the file path specified in your task + - Use the `Write` tool to save the plan -- do not return it as text output + +## Plan Structure + +Structure plans with these sections: + +```markdown +# Implementation Plan: [Feature Name] + +## Summary +[2-3 sentences describing what will be implemented] + +## Architecture Decisions +| Decision | Options | Chosen | Rationale | +|----------|---------|--------|-----------| +| [Decision] | [A, B, C] | [A] | [Why] | + +## Tasks + +### Phase 1: [Phase Name] +- [ ] **T-1.1**: [Task description] +- [ ] **T-1.2**: [Task description] + +### Phase 2: [Phase Name] +- [ ] **T-2.1**: [Task description] + +## Open Questions +- [ ] Q-001: [Question] **[Recommended: answer]** + +## Risks +- [Risk description and mitigation] + +## Critical Files for Implementation +- path/to/file1 - [Brief reason] +- path/to/file2 - [Brief reason] +- path/to/file3 - [Brief reason] +``` + +## When Revising a Plan + +When given feedback to address: + +1. Read the current plan file and the feedback file +2. Address ALL concerns raised in the feedback +3. If you disagree with a point, note your reasoning in the plan but still consider the underlying concern +4. If the reviewer proposed a concrete fix, adopt it directly unless you have a strong reason not to +5. Write the updated plan back to the same file path using the `Write` tool diff --git a/plugins/code/commands/plan-with-codex.md b/plugins/code/commands/plan-with-codex.md new file mode 100644 index 00000000..76834942 --- /dev/null +++ b/plugins/code/commands/plan-with-codex.md @@ -0,0 +1,262 @@ +--- +description: "Iterative plan refinement debate between Claude and Codex" +argument-hint: [--max-rounds N] [--plan-file PATH] [--codex-model MODEL] +allowed-tools: Bash, Read, Write, Glob, Grep, TodoWrite, Task, AskUserQuestion, SendMessage, ToolSearch +skills: code:codex-review +--- + +# Debate Loop -- Claude + Codex Plan Refinement + +You are the orchestrator for an iterative plan refinement workflow. Claude (via `code:plan-agent`) creates a plan, Codex reviews it, and you coordinate revisions until Codex approves or max rounds are reached. + +## Step 0: Parse Arguments + +Parse from `$ARGUMENTS`: + +Arguments: $ARGUMENTS + +| Flag | Default | Description | +|------|---------|-------------| +| `--max-rounds N` | 15 | Maximum debate rounds | +| `--plan-file PATH` | `./debate-plan.md` | Output plan file (resolve to absolute path) | +| `--codex-model MODEL` | `gpt-5.4` | Codex model for reviews | +| Remaining text | (required) | The prompt describing what to plan | + +Derive sidecar paths from the plan file stem (e.g., for `debate-plan.md`): +- `{stem}.feedback` -- Codex feedback text +- `{stem}.state` -- phase/round/session state +- `{stem}.prompt` -- original prompt (plain text) + +**Prompt resolution**: CLI argument > `{stem}.prompt` sidecar. Only abort when neither exists. + +Initialize TodoWrite: +``` +TodoWrite([ + {"content": "Parse arguments and check for resume", "status": "in_progress"}, + {"content": "Create plan with plan-agent", "status": "pending"}, + {"content": "User review of plan", "status": "pending"}, + {"content": "Codex debate loop", "status": "pending"}, + {"content": "Final report", "status": "pending"} +]) +``` + +**Preload SendMessage tool** (required for agent resume later): +``` +ToolSearch(query="select:SendMessage") +``` + +Error if no prompt is resolvable (no CLI argument and no `{stem}.prompt` sidecar). + +## Step 0.5: Check for Resume + +Check if `{stem}.state` exists via Bash (`test -f`). If yes, read `ROUND`, `PHASE`, `CODEX_SESSION_ID` via: +```bash +grep "^ROUND=" {state_file} | cut -d= -f2- +grep "^PHASE=" {state_file} | cut -d= -f2- +grep "^CODEX_SESSION_ID=" {state_file} | cut -d= -f2- +``` + +**Validate preconditions before resuming:** + +| Phase | Required files | +|-------|---------------| +| `user_review` | plan file + prompt sidecar | +| `codex_review` | plan file + prompt sidecar | +| `claude_revision` | plan file + feedback file + prompt sidecar | + +If preconditions fail: delete stale state file. A fresh start is still possible if a prompt is available (CLI argument or `{stem}.prompt` sidecar). Only abort when neither exists. + +If all preconditions pass, announce "Resuming debate at round {N}, phase: {PHASE}" and jump to: +- `user_review` -> Step 1.5 +- `codex_review` -> Step 2a at stored ROUND +- `claude_revision` -> Step 2f at stored ROUND + +If no state file: fresh start at Step 1. + +## Step 1: Create the Plan + +Announce: "Creating plan with plan-agent..." + +Launch the plan-agent: +``` +Agent( + subagent_type="code:plan-agent", + name="plan-agent", + mode="acceptEdits", + run_in_background=false, + description="Create implementation plan", + prompt=". Write the plan to {plan-file-abs}." +) +``` + +**Store the returned agent_id** -- you will need it to resume the plan-agent in later rounds. Resume via `SendMessage(to="", message="...", summary="...")`. The ToolSearch preload in Step 0 ensures the schema is available. + +Verify the plan file exists and is non-empty (Read it). + +Announce: "Plan created ({byte_count} bytes) at {plan-file-abs}" + +Write the original prompt to `{stem}.prompt` (plain text, via Write tool). + +Write state via Bash: +```bash +printf 'ROUND=%s\nPHASE=%s\nCODEX_SESSION_ID=%s\n' '1' 'user_review' '' > {state_file} +``` + +Update TodoWrite: mark "Create plan" completed, "User review" in_progress. + +## Step 1.5: User Checkpoint + +Read the plan file. Check for an "Open Questions" section (lines matching `Q-` or `- [ ] Q-`). If open questions exist, present them to the user before anything else using AskUserQuestion: + +> The plan has open questions that need your input before proceeding: +> +> 1. **Q-001**: [question text] +> - **a) [recommended answer]** (recommended) +> - b) [alternative] +> 2. **Q-002**: [question text] +> - **a) [recommended answer]** (recommended) +> - b) [alternative] +> +> Reply with your choices (e.g., "1a, 2b") or provide your own answers. + +After the user answers, resume the plan-agent with their answers so it can update the plan and resolve the open questions. Then re-read the plan and check again -- repeat until no open questions remain. + +Once open questions are resolved (or if there were none), present the plan to the user: + +> Plan created at `{plan-file-abs}`. Review it and let me know when you're ready to start the Codex debate, or share any changes you'd like made first. + +**If the user requests changes:** + +Resume the plan-agent with their feedback: +``` +SendMessage( + to="", + message="User feedback: . Read the current plan at {plan-file-abs}, revise it, and write the updated plan back to {plan-file-abs}.", + summary="Revise plan per user feedback" +) +``` + +If no resumable agent exists (cross-session resume), launch a fresh one: +``` +Agent( + subagent_type="code:plan-agent", + name="plan-agent", + mode="acceptEdits", + run_in_background=false, + description="Revise plan per user feedback", + prompt="Read the current plan at {plan-file-abs} and revise it based on user feedback. Original request: . User feedback: . Write the updated plan back to {plan-file-abs}." +) +``` +Store the new agent_id for subsequent rounds. + +Loop back to this checkpoint until the user says to proceed. + +**When the user confirms** (e.g., "start", "go", "looks good", "proceed"): + +Update state: +```bash +printf 'ROUND=%s\nPHASE=%s\nCODEX_SESSION_ID=%s\n' '1' 'codex_review' '' > {state_file} +``` + +Proceed to Step 2. + +## Step 2: Debate Loop + +Repeat for round 1 to max-rounds: + +### 2a. Codex Review + +Update TodoWrite: "Round {N}/{max}: Codex reviewing..." + +Activate the `code:codex-review` skill and run via Bash: +```bash +bash /scripts/run_codex_review.sh \ + --plan-file {plan-file-abs} \ + --feedback-file {feedback-file-abs} \ + --round {N} \ + --codex-model {codex-model} \ + [--session-id {codex_session_id}] +``` + +Parse stdout tokens: +- `VERDICT:APPROVED` or `VERDICT:NEEDS_CHANGES` +- `CODEX_SESSION:` -- save for next round + +Update state with new CODEX_SESSION_ID. + +### 2b. Handle Failures (do NOT increment round) + +- `CODEX_FAILED:`: Announce the warning. Ask the user: "Codex failed: {reason}. Retry or abort?" On retry: re-run 2a. On abort: go to Step 3. +- `CODEX_EMPTY`: Announce "Codex returned empty response." Same retry/abort handling. + +### 2c. Display Feedback + +Read the feedback file and display the full Codex feedback to the user. + +### 2d. Check Verdict + +- **VERDICT:APPROVED**: Announce "Plan approved by Codex after {N} round(s)." Go to Step 3. +- **Last round, not approved**: Announce "Max rounds ({max}) reached without approval." Go to Step 3. +- **VERDICT:NEEDS_CHANGES**: Continue to 2e. + +### 2e. Proceed to Revision (automated) + +The Codex/Claude loop is fully automated after the user approved the plan in Step 1.5. Do NOT ask the user for confirmation between rounds -- proceed directly to revision. + +Update state: +```bash +printf 'ROUND=%s\nPHASE=%s\nCODEX_SESSION_ID=%s\n' '{N}' 'claude_revision' '{codex_session_id}' > {state_file} +``` + +Proceed to 2f. + +### 2f. Claude Revision + +Update TodoWrite: "Round {N}/{max}: Revising plan..." + +Resume the plan-agent: +``` +SendMessage( + to="", + message="Revise the plan at {plan-file-abs} based on feedback at {feedback-file-abs}. Address ALL concerns raised.", + summary="Revise plan per Codex feedback" +) +``` + +If resume fails (cross-session -- agent gone), launch fresh: +``` +Agent( + subagent_type="code:plan-agent", + name="plan-agent", + mode="acceptEdits", + run_in_background=false, + description="Revise plan based on Codex feedback", + prompt="Revise the plan at {plan-file-abs} based on feedback at {feedback-file-abs}. The original request was: . Read the current plan and feedback files to understand context. Write the updated plan back to {plan-file-abs}." +) +``` +Store the new agent_id for subsequent rounds. + +Verify the plan file was updated. + +Update state: +```bash +printf 'ROUND=%s\nPHASE=%s\nCODEX_SESSION_ID=%s\n' '{N+1}' 'codex_review' '{codex_session_id}' > {state_file} +``` + +Continue to next round. + +## Step 3: Final Report + +Report the outcome: +- If approved: "Plan approved by Codex. File: {plan-file-abs}" +- If max rounds: "Plan not approved after {max} rounds. File: {plan-file-abs}" +- If aborted: "Debate aborted. Partial plan at: {plan-file-abs}" + +Clean up ALL sidecar files: +```bash +rm -f {state_file} {feedback_file} {prompt_file} +``` + +The prompt sidecar is intentionally deleted on completion to prevent stale intent from silently reusing on future runs against the default `./debate-plan.md` path. + +Update TodoWrite: mark all remaining items completed. diff --git a/plugins/code/hooks/plan-review.sh b/plugins/code/hooks/plan-review.sh new file mode 100755 index 00000000..c98c1118 --- /dev/null +++ b/plugins/code/hooks/plan-review.sh @@ -0,0 +1,76 @@ +#!/bin/bash +# Codex Plan Review Hook +# Triggers when Claude exits plan mode to get a second opinion on the plan +# The plan content is available directly from tool_response.plan + +# Read hook input from stdin (must happen before CWD extraction) +INPUT=$(cat) + +# Debug logging — keeps at most 15 log files in .closedloop-ai/plan-review-logs/ +CWD=$(echo "$INPUT" | jq -r '.cwd // empty' 2>/dev/null) +CWD="${CWD:-$PWD}" +LOG_DIR="$CWD/.closedloop-ai/plan-review-logs" +mkdir -p "$LOG_DIR" +LOG_FILE="$LOG_DIR/$(date +%Y%m%d-%H%M%S).log" + +log() { echo "$(date): $*" >> "$LOG_FILE"; } + +# Prune old logs — keep only the 14 most recent (this run makes 15) +ls -1t "$LOG_DIR"/*.log 2>/dev/null | tail -n +15 | xargs rm -f 2>/dev/null + +log "Plan review hook started" + +# Extract the plan content directly from the tool response +PLAN_CONTENT=$(echo "$INPUT" | jq -r '.tool_response.plan // empty') + +# Exit silently if no plan content +if [ -z "$PLAN_CONTENT" ]; then + log "No plan content found in tool_response, exiting" + exit 0 +fi + +log "Plan content received (${#PLAN_CONTENT} chars)" + +# Write prompt to a temp file to avoid shell injection +TMPFILE=$(mktemp) +trap 'rm -f "$TMPFILE"' EXIT +cat > "$TMPFILE" </dev/null) + +# If codex failed, exit silently +if [ -z "$REVIEW" ]; then + log "Codex returned empty response, exiting" + exit 0 +fi + +log "Codex review received (${#REVIEW} chars)" +log "Review content: $REVIEW" + +# Output as JSON so Claude Code injects it as context +REVIEW_JSON=$(jq -Rs . <<< "$REVIEW") +OUTPUT=$(jq -n --argjson review "$REVIEW_JSON" \ + '{"hookSpecificOutput": {"hookEventName": "PostToolUse", "additionalContext": $review}}') + +log "Output JSON: $OUTPUT" +echo "$OUTPUT" + +exit 0 diff --git a/plugins/code/scripts/debate-loop.sh b/plugins/code/scripts/debate-loop.sh new file mode 100755 index 00000000..678f8a4b --- /dev/null +++ b/plugins/code/scripts/debate-loop.sh @@ -0,0 +1,608 @@ +#!/bin/bash + +# debate-loop.sh - Iterative plan refinement via Claude + Codex debate +# +# Creates a plan with Claude, has Codex review it, and iterates until +# Codex approves or max rounds are reached. +# +# Usage: +# debate-loop.sh "Build a REST API for user management" [options] +# +# Options: +# --max-rounds N Maximum debate rounds (default: 15) +# --plan-file PATH Output plan file path (default: ./debate-plan.md) +# --codex-model MODEL Codex model to use (default: gpt-5.4) +# -h, --help Show this help +# +# Claude sessions are resumed across rounds for context continuity. +# Falls back to fresh context if session resume fails. + +set -euo pipefail + +SCRIPTS_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +FORMATTER="$SCRIPTS_DIR/../tools/python/stream_formatter.py" + +# Colors +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +CYAN='\033[0;36m' +NC='\033[0m' + +# Defaults +MAX_ROUNDS=15 +PLAN_FILE="" +CODEX_MODEL="gpt-5.4" +CLAUDE_MODEL="opus[1m]" +PROMPT="" +VERBOSE=false +ADD_DIRS=() + +# Temp files tracked for cleanup +TMPFILES=() +# PID of the currently-running child process (claude or codex); 0 = none +CURRENT_CHILD_PID=0 + +cleanup() { + if [[ ${#TMPFILES[@]} -gt 0 ]]; then + rm -f "${TMPFILES[@]}" 2>/dev/null || true + fi +} + +on_interrupt() { + echo "" + echo -e "${YELLOW}Interrupted by user${NC}" + if [[ $CURRENT_CHILD_PID -ne 0 ]]; then + kill "$CURRENT_CHILD_PID" 2>/dev/null || true + fi + if [[ -n "${PLAN_FILE_ABS:-}" ]] && [[ -f "$PLAN_FILE_ABS" ]]; then + echo -e "${YELLOW}Partial plan available at: $PLAN_FILE_ABS${NC}" + fi + exit 130 +} + +trap cleanup EXIT +trap on_interrupt INT TERM + +# Run claude, streaming output only when --verbose is set. +# Writes stderr to $CLAUDE_STDERR. Returns claude's exit code. +# Runs claude in the background so the INT trap fires immediately on Ctrl+C. +run_claude() { + local -a claude_args=("$@" --model "$CLAUDE_MODEL") + if [[ ${#ADD_DIR_ARGS[@]} -gt 0 ]]; then + claude_args+=("${ADD_DIR_ARGS[@]}") + fi + local exit_code + if [[ "$VERBOSE" == "true" ]]; then + local claude_exit_file + claude_exit_file=$(mktemp) + TMPFILES+=("$claude_exit_file") + { claude "${claude_args[@]}" --output-format stream-json --verbose 2>"$CLAUDE_STDERR"; echo $? > "$claude_exit_file"; } \ + | python3 "$FORMATTER" & + CURRENT_CHILD_PID=$! + wait $CURRENT_CHILD_PID || true + CURRENT_CHILD_PID=0 + exit_code=$(cat "$claude_exit_file" 2>/dev/null || echo "1") + else + claude "${claude_args[@]}" --output-format text > /dev/null 2>"$CLAUDE_STDERR" & + CURRENT_CHILD_PID=$! + wait $CURRENT_CHILD_PID && exit_code=0 || exit_code=$? + CURRENT_CHILD_PID=0 + fi + return "$exit_code" +} + +save_state() { + printf 'SESSION_ID=%s\nCODEX_SESSION_ID=%s\nROUND=%s\nPHASE=%s\n' \ + "$SESSION_ID" "$CODEX_SESSION_ID" "$round" "$PHASE" > "$STATE_FILE" +} + +load_state() { + SESSION_ID=$(grep "^SESSION_ID=" "$STATE_FILE" | cut -d= -f2-) || true + CODEX_SESSION_ID=$(grep "^CODEX_SESSION_ID=" "$STATE_FILE" | cut -d= -f2-) || true + round=$(grep "^ROUND=" "$STATE_FILE" | cut -d= -f2-) || true + round="${round:-1}" + PHASE=$(grep "^PHASE=" "$STATE_FILE" | cut -d= -f2-) || true + PHASE="${PHASE:-codex_review}" +} + +# Shared Codex review script (extracted to avoid duplication with the native +# debate-loop.md slash command, which uses the same script via the codex-review skill). +CODEX_REVIEW_SCRIPT="$SCRIPTS_DIR/../skills/codex-review/scripts/run_codex_review.sh" + +# Run the shared Codex review script in the background so the INT trap fires +# immediately on Ctrl+C. Parses stdout tokens into CODEX_VERDICT and +# CODEX_SESSION_ID. Writes full feedback text to FEEDBACK_FILE. +# Sets CODEX_REVIEW_EXIT in the caller's scope. +run_codex_review() { + if [[ ! -f "$CODEX_REVIEW_SCRIPT" ]]; then + echo -e "${RED}Error: Codex review script not found at $CODEX_REVIEW_SCRIPT${NC}" >&2 + exit 1 + fi + + local review_args=( + --plan-file "$PLAN_FILE_ABS" + --feedback-file "$FEEDBACK_FILE" + --round "$round" + --codex-model "$CODEX_MODEL" + ) + if [[ -n "$CODEX_SESSION_ID" ]]; then + review_args+=(--session-id "$CODEX_SESSION_ID") + fi + + local review_output_file review_stderr_file + review_output_file=$(mktemp) + review_stderr_file=$(mktemp) + TMPFILES+=("$review_output_file" "$review_stderr_file") + + bash "$CODEX_REVIEW_SCRIPT" "${review_args[@]}" > "$review_output_file" 2>"$review_stderr_file" & + CURRENT_CHILD_PID=$! + wait $CURRENT_CHILD_PID && CODEX_REVIEW_EXIT=0 || CODEX_REVIEW_EXIT=$? + CURRENT_CHILD_PID=0 + + # Fail fast if the script itself crashed (no tokens parsed) + if [[ $CODEX_REVIEW_EXIT -ne 0 ]] && [[ ! -s "$review_output_file" ]]; then + echo -e "${RED}Error: Codex review script failed (exit $CODEX_REVIEW_EXIT)${NC}" >&2 + if [[ -s "$review_stderr_file" ]]; then + cat "$review_stderr_file" >&2 + fi + exit 1 + fi + + # Parse structured tokens from stdout + CODEX_VERDICT="" + local line + while IFS= read -r line; do + case "$line" in + VERDICT:*) CODEX_VERDICT="${line#VERDICT:}" ;; + CODEX_SESSION:*) CODEX_SESSION_ID="${line#CODEX_SESSION:}"; [[ "$CODEX_SESSION_ID" == "none" ]] && CODEX_SESSION_ID="" ;; + CODEX_FAILED:*) CODEX_VERDICT="FAILED:${line#CODEX_FAILED:}" ;; + CODEX_EMPTY) CODEX_VERDICT="EMPTY" ;; + esac + done < "$review_output_file" + + # If no token was parsed at all, treat as a script failure + if [[ -z "$CODEX_VERDICT" ]]; then + echo -e "${RED}Error: Codex review script produced no verdict token${NC}" >&2 + if [[ -s "$review_stderr_file" ]]; then + cat "$review_stderr_file" >&2 + fi + exit 1 + fi +} + +show_help() { + cat <<'EOF' +debate-loop.sh - Iterative plan refinement via Claude + Codex debate + +USAGE: + debate-loop.sh [options] "" + +OPTIONS: + --max-rounds N Maximum debate rounds (default: 15) + --plan-file PATH Output plan file path (default: ./debate-plan.md) + --codex-model MODEL Codex model to use (default: gpt-5.4) + --model MODEL Claude model to use (default: opus[1m]) + --add-dir DIR... Additional directories to allow Claude tool access to + --verbose Stream Claude output in real-time (default: off) + -h, --help Show this help + +DESCRIPTION: + 1. Claude creates an implementation plan from your prompt + 2. Codex reviews the plan and provides feedback + 3. Claude addresses the feedback and updates the plan + 4. Repeat until Codex approves or max rounds reached + +EXAMPLES: + debate-loop.sh "Build a REST API for user management" + debate-loop.sh --plan-file auth-plan.md --max-rounds 10 "Refactor the auth system" + debate-loop.sh --codex-model gpt-5.4 "Add caching layer" +EOF +} + +# Parse arguments +while [[ $# -gt 0 ]]; do + case $1 in + -h|--help) + show_help + exit 0 + ;; + --max-rounds) + if [[ -z "${2:-}" ]] || ! [[ "$2" =~ ^[0-9]+$ ]] || [[ "$2" -eq 0 ]]; then + echo -e "${RED}Error: --max-rounds requires a positive integer${NC}" >&2 + exit 1 + fi + MAX_ROUNDS="$2" + shift 2 + ;; + --plan-file) + if [[ -z "${2:-}" ]]; then + echo -e "${RED}Error: --plan-file requires a path${NC}" >&2 + exit 1 + fi + PLAN_FILE="$2" + shift 2 + ;; + --codex-model) + if [[ -z "${2:-}" ]]; then + echo -e "${RED}Error: --codex-model requires a model name${NC}" >&2 + exit 1 + fi + CODEX_MODEL="$2" + shift 2 + ;; + --model) + if [[ -z "${2:-}" ]]; then + echo -e "${RED}Error: --model requires a model name${NC}" >&2 + exit 1 + fi + CLAUDE_MODEL="$2" + shift 2 + ;; + --verbose) + VERBOSE=true + shift + ;; + --add-dir) + if [[ -z "${2:-}" ]]; then + echo -e "${RED}Error: --add-dir requires at least one directory${NC}" >&2 + exit 1 + fi + shift + while [[ $# -gt 0 ]] && [[ "${1:0:1}" != "-" ]]; do + ADD_DIRS+=("$1") + shift + done + ;; + -*) + echo -e "${RED}Error: Unknown option: $1${NC}" >&2 + echo "Use --help for usage information" >&2 + exit 1 + ;; + *) + if [[ -n "$PROMPT" ]]; then + echo -e "${RED}Error: Multiple prompts provided. Wrap your prompt in quotes.${NC}" >&2 + exit 1 + fi + PROMPT="$1" + shift + ;; + esac +done + +# Validate prompt +if [[ -z "$PROMPT" ]]; then + echo -e "${RED}Error: No prompt provided${NC}" >&2 + echo "Usage: debate-loop.sh [options] \"\"" >&2 + exit 1 +fi + +# Validate --add-dir paths and build args array +ADD_DIR_ARGS=() +if [[ ${#ADD_DIRS[@]} -gt 0 ]]; then + for dir in "${ADD_DIRS[@]}"; do + if [[ ! -d "$dir" ]]; then + echo -e "${RED}Error: --add-dir path is not a directory: $dir${NC}" >&2 + exit 1 + fi + ADD_DIR_ARGS+=(--add-dir "$dir") + done +fi + +# Check dependencies +for cmd in claude codex; do + if ! command -v "$cmd" &> /dev/null; then + echo -e "${RED}Error: $cmd is required but not found${NC}" >&2 + exit 1 + fi +done + +# Generate session ID (UUID required by --session-id) +generate_session_id() { + uuidgen 2>/dev/null | tr '[:upper:]' '[:lower:]' \ + || cat /proc/sys/kernel/random/uuid 2>/dev/null \ + || python3 -c "import uuid; print(uuid.uuid4())" 2>/dev/null \ + || echo "$(date +%s)-$(head -c 8 /dev/urandom | xxd -p)" +} + +# Resolve plan file to absolute path +PLAN_FILE="${PLAN_FILE:-./debate-plan.md}" +PLAN_DIR="$(dirname "$PLAN_FILE")" + +# Validate parent directory exists +if [[ ! -d "$PLAN_DIR" ]]; then + echo -e "${RED}Error: Parent directory does not exist: $PLAN_DIR${NC}" >&2 + echo "Create it first or use a different --plan-file path" >&2 + exit 1 +fi + +PLAN_DIR="$(cd "$PLAN_DIR" && pwd)" +PLAN_FILE_ABS="${PLAN_DIR}/$(basename "$PLAN_FILE")" + +STATE_FILE="${PLAN_FILE_ABS%.md}.state" +FEEDBACK_FILE="${PLAN_FILE_ABS%.md}.feedback" +RESUMING=false +CODEX_SESSION_ID="" +PHASE=codex_review + +if [[ -f "$STATE_FILE" ]] && [[ -f "$PLAN_FILE_ABS" ]]; then + load_state + RESUMING=true +fi + +if [[ "$RESUMING" == "false" ]]; then + SESSION_ID=$(generate_session_id) +fi + +# Warn if plan file already exists (skip when resuming -- file is intentionally kept) +if [[ -f "$PLAN_FILE_ABS" ]] && [[ "$RESUMING" == "false" ]]; then + echo -e "${YELLOW}Warning: Plan file already exists at $PLAN_FILE_ABS -- it will be overwritten${NC}" +fi + +# Create temp files for codex I/O +CLAUDE_STDERR=$(mktemp) +TMPFILES+=("$CLAUDE_STDERR") + +echo -e "${BLUE}================================================================${NC}" +echo -e "${BLUE} Debate Loop -- Claude + Codex Plan Refinement${NC}" +echo -e "${BLUE}================================================================${NC}" +echo "" +echo -e "Prompt: ${GREEN}${PROMPT}${NC}" +echo -e "Plan file: ${GREEN}${PLAN_FILE_ABS}${NC}" +echo -e "Max rounds: ${YELLOW}${MAX_ROUNDS}${NC}" +echo -e "Claude model: ${CYAN}${CLAUDE_MODEL}${NC}" +echo -e "Codex model: ${CYAN}${CODEX_MODEL}${NC}" +echo -e "Session: ${CYAN}${SESSION_ID}${NC}" +echo -e "Verbose: ${CYAN}${VERBOSE}${NC}" +if [[ ${#ADD_DIRS[@]} -gt 0 ]]; then + echo -e "Extra dirs: ${CYAN}${ADD_DIRS[*]}${NC}" +fi +echo "" +echo -e "${YELLOW}Press Ctrl+C to stop at any time${NC}" +echo "" + +if [[ "$RESUMING" == "true" ]]; then + echo -e "${YELLOW}Resuming from round ${round} (session: ${SESSION_ID})${NC}" + echo "" +fi + +# ── Phase 1: Claude creates the plan ──────────────────────────────────────── + +if [[ "$RESUMING" == "false" ]]; then + round=1 +fi +approved=false + +if [[ "$RESUMING" == "false" ]]; then + echo -e "${BLUE}--- Phase 1: Claude creating plan ---${NC}" + echo "" + + set +e + run_claude \ + -p "Use the Agent tool to invoke the code:plan-agent with the following task: + +'$PROMPT + +Write the plan to $PLAN_FILE_ABS.' + +Take note of the plan agent's session ID -- you will need to resume it in subsequent rounds to pass it updated feedback." \ + --session-id "$SESSION_ID" \ + --permission-mode acceptEdits \ + --allowed-tools "Read,Write,Glob,Grep,Agent(code:plan-agent)" + CLAUDE_EXIT=$? + set -e + + echo "" + + if [[ $CLAUDE_EXIT -ne 0 ]]; then + echo -e "${RED}Error: Claude exited with code $CLAUDE_EXIT${NC}" >&2 + if [[ -s "$CLAUDE_STDERR" ]]; then + echo -e "${RED}stderr:${NC}" >&2 + cat "$CLAUDE_STDERR" >&2 + fi + exit 1 + fi + + # Verify plan was created + if [[ ! -f "$PLAN_FILE_ABS" ]]; then + echo -e "${RED}Error: Claude did not create plan file at $PLAN_FILE_ABS${NC}" >&2 + exit 1 + fi + + PLAN_SIZE=$(wc -c < "$PLAN_FILE_ABS" | tr -d ' ') + echo -e "${GREEN}Plan created (${PLAN_SIZE} bytes)${NC}" + echo "" + PHASE=user_review + save_state +else + PLAN_SIZE=$(wc -c < "$PLAN_FILE_ABS" | tr -d ' ') + echo -e "${GREEN}Resuming with existing plan (${PLAN_SIZE} bytes)${NC}" + echo "" +fi + +# ── Phase 1.5: Interactive user review ──────────────────────────────────────── + +if [[ "$PHASE" == "user_review" ]]; then + echo -e "${BLUE}================================================================${NC}" + echo -e "${BLUE} Plan Review -- Interactive Mode${NC}" + echo -e "${BLUE}================================================================${NC}" + echo "" + echo -e "Plan file: ${GREEN}${PLAN_FILE_ABS}${NC}" + echo "" + echo -e "You can now chat with Claude to review and refine the plan." + echo -e "Type ${CYAN}/exit${NC} when you are done reviewing." + echo "" + + while true; do + # Launch interactive Claude session with instruction to proxy through plan-agent + local_claude_args=(--model "$CLAUDE_MODEL" -r "$SESSION_ID" -p "The user would like to iterate on the plan interactively. Resume the code:plan-agent from your conversation history. For every message the user sends, pass it to the plan agent so it can respond and make edits to the plan at ${PLAN_FILE_ABS}. Act as a transparent proxy -- do not answer plan questions yourself, always delegate to the plan agent." --permission-mode acceptEdits --allowed-tools "Read,Write,Glob,Grep,Agent(code:plan-agent)") + if [[ ${#ADD_DIR_ARGS[@]} -gt 0 ]]; then + local_claude_args+=("${ADD_DIR_ARGS[@]}") + fi + claude "${local_claude_args[@]}" 2>/dev/null + INTERACTIVE_EXIT=$? + + echo "" + echo -e "${YELLOW}Ready to start the Codex debate loop? (Y/n)${NC}" + read -r USER_RESPONSE + case "${USER_RESPONSE,,}" in + n|no) + echo -e "${CYAN}Returning to interactive review...${NC}" + echo "" + continue + ;; + *) + echo -e "${GREEN}Starting debate loop...${NC}" + echo "" + break + ;; + esac + done + + PHASE=codex_review + save_state +fi + +# ── Phase 2: Debate loop ──────────────────────────────────────────────────── + +while [[ $round -le $MAX_ROUNDS ]]; do + + # ── Codex review (skip if resuming mid-round into Claude's turn) ────────── + + if [[ "$PHASE" != "claude_revision" ]]; then + echo -e "${BLUE}--- Round ${round}/${MAX_ROUNDS}: Codex reviewing plan ---${NC}" + echo "" + + # Run the shared Codex review script + set +e + run_codex_review + set -e + + save_state + + # Handle failures and empty responses + if [[ "$CODEX_VERDICT" == FAILED:* ]]; then + echo -e "${RED}Error: Codex failed: ${CODEX_VERDICT#FAILED:}${NC}" >&2 + exit 1 + fi + + if [[ "$CODEX_VERDICT" == "EMPTY" ]]; then + echo -e "${YELLOW}Warning: Codex returned empty response, skipping round${NC}" + echo "" + round=$((round + 1)) + continue + fi + + # Read and display feedback + FEEDBACK=$(cat "$FEEDBACK_FILE" 2>/dev/null || echo "") + + echo -e "${CYAN}Codex feedback:${NC}" + echo "────────────────────────────────────────" + echo "$FEEDBACK" + echo "────────────────────────────────────────" + echo "" + + # Check verdict + if [[ "$CODEX_VERDICT" == "APPROVED" ]]; then + echo -e "${GREEN}================================================================${NC}" + echo -e "${GREEN} Plan approved by Codex after ${round} round(s)${NC}" + echo -e "${GREEN}================================================================${NC}" + echo "" + approved=true + break + fi + + # If this was the last round, don't bother with another Claude revision + if [[ $round -ge $MAX_ROUNDS ]]; then + echo -e "${YELLOW}================================================================${NC}" + echo -e "${YELLOW} Max rounds (${MAX_ROUNDS}) reached without approval${NC}" + echo -e "${YELLOW}================================================================${NC}" + echo "" + break + fi + + PHASE=claude_revision + save_state + + else + # Resuming mid-round: Codex already reviewed, load saved feedback + echo -e "${GREEN}Resuming at Claude revision (round ${round}, Codex feedback saved)${NC}" + FEEDBACK=$(cat "$FEEDBACK_FILE" 2>/dev/null || echo "") + echo -e "${CYAN}Codex feedback (saved):${NC}" + echo "────────────────────────────────────────" + echo "$FEEDBACK" + echo "────────────────────────────────────────" + echo "" + fi + + # ── Claude revision ─────────────────────────────────────────────────────── + + echo -e "${BLUE}--- Round ${round}/${MAX_ROUNDS}: Claude addressing feedback ---${NC}" + echo "" + + # Resume the Claude session so it retains full context of previous rounds. + set +e + run_claude \ + -p "Resume the plan agent from your conversation history and pass it this task: + +'Revise the plan at ${PLAN_FILE_ABS} based on feedback at ${FEEDBACK_FILE}. Write the updated plan back to ${PLAN_FILE_ABS}.' + +Do not spawn a new plan agent -- resume the existing session." \ + -r "$SESSION_ID" \ + --permission-mode acceptEdits \ + --allowed-tools "Read,Write,Glob,Grep,Agent(code:plan-agent)" + RESUME_EXIT=$? + set -e + + # If resume failed, fall back to a fresh session -- same structure, no prior context + if [[ $RESUME_EXIT -ne 0 ]]; then + if grep -qi "session\|not found\|invalid\|resume" "$CLAUDE_STDERR" 2>/dev/null; then + echo -e "${YELLOW}Session resume failed, falling back to fresh session${NC}" + set +e + run_claude \ + -p "Use the Agent tool to invoke the code:plan-agent with the following task: + +'Revise the plan at ${PLAN_FILE_ABS} based on feedback at ${FEEDBACK_FILE}. Write the updated plan back to ${PLAN_FILE_ABS}. + +The original request was: ${PROMPT}'" \ + --permission-mode acceptEdits \ + --allowed-tools "Read,Write,Glob,Grep,Agent(code:plan-agent)" + FALLBACK_EXIT=$? + set -e + + if [[ $FALLBACK_EXIT -ne 0 ]]; then + echo -e "${RED}Error: Claude failed with code $FALLBACK_EXIT${NC}" >&2 + if [[ -s "$CLAUDE_STDERR" ]]; then + cat "$CLAUDE_STDERR" >&2 + fi + exit 1 + fi + else + echo -e "${RED}Error: Claude failed with code $RESUME_EXIT${NC}" >&2 + if [[ -s "$CLAUDE_STDERR" ]]; then + cat "$CLAUDE_STDERR" >&2 + fi + exit 1 + fi + fi + + echo "" + PLAN_SIZE=$(wc -c < "$PLAN_FILE_ABS" | tr -d ' ') + echo -e "${GREEN}Plan updated (${PLAN_SIZE} bytes)${NC}" + echo "" + + PHASE=codex_review + save_state + round=$((round + 1)) +done + +# ── Done ───────────────────────────────────────────────────────────────────── + +rm -f "$STATE_FILE" "$FEEDBACK_FILE" 2>/dev/null || true + +if [[ "$approved" == "true" ]]; then + echo -e "Plan file: ${GREEN}${PLAN_FILE_ABS}${NC}" +else + echo -e "Plan file (unapproved): ${YELLOW}${PLAN_FILE_ABS}${NC}" +fi diff --git a/plugins/code/skills/codex-review/SKILL.md b/plugins/code/skills/codex-review/SKILL.md new file mode 100644 index 00000000..f7d50eca --- /dev/null +++ b/plugins/code/skills/codex-review/SKILL.md @@ -0,0 +1,86 @@ +--- +name: codex-review +description: | + Run Codex to review a plan file and return structured feedback with a verdict. + Triggers on: debate loop Codex review rounds. + Returns VERDICT:APPROVED or VERDICT:NEEDS_CHANGES plus CODEX_SESSION token. +context: fork +allowed-tools: Bash +--- + +# Codex Review + +Call Codex to review an implementation plan and return structured feedback with an approval verdict. + +## When to Use + +Activated once per debate round in the `/plan-with-codex` command, before Claude revision. The orchestrator calls this skill to get Codex's assessment of the current plan. + +## Usage + +The `scripts/` directory is relative to this skill's base directory (shown above as "Base directory for this skill"). + +```bash +bash /scripts/run_codex_review.sh \ + --plan-file \ + --feedback-file \ + --round \ + --codex-model \ + [--session-id ] +``` + +| Argument | Required | Default | Description | +|----------|----------|---------|-------------| +| `--plan-file` | Yes | -- | Absolute path to the plan file Codex should review | +| `--feedback-file` | Yes | -- | Path where full feedback text will be written | +| `--round` | No | 1 | Current debate round (affects review prompt intro) | +| `--codex-model` | No | gpt-5.4 | Codex model to use | +| `--session-id` | No | -- | Thread ID from a previous round for session resume | + +## Interpreting Output + +The script prints structured tokens to stdout. Parse these to control the debate loop. + +### Approval + +``` +VERDICT:APPROVED +CODEX_SESSION:abc-123-def +``` + +**Action:** Announce approval. Clean up sidecar files. Stop the debate loop. + +### Changes Requested + +``` +VERDICT:NEEDS_CHANGES +CODEX_SESSION:abc-123-def +``` + +**Action:** Read the feedback file for full details. Pass to plan-agent for revision. + +### Codex Failed + +``` +CODEX_FAILED: +CODEX_SESSION:abc-123-def +``` + +**Action:** Announce the failure reason. Ask the user to retry or abort. Do NOT increment the round counter. + +### Empty Response + +``` +CODEX_EMPTY +CODEX_SESSION:abc-123-def +``` + +**Action:** Announce empty response. Ask the user to retry or abort. Do NOT increment the round counter. + +## How It Works + +1. Builds a review prompt asking Codex to analyze the plan for technical soundness, missing steps, architectural issues, security/performance risks, and unclear descriptions +2. If `--session-id` is provided, attempts `codex exec resume` for context continuity; falls back to a fresh session if resume fails +3. Parses the Codex JSON stream for `thread.started` (thread ID) and `item.completed`/`agent_message` (feedback text) +4. If session resume succeeds but no new `thread.started` event appears, the input session ID is preserved and re-emitted (prevents losing session continuity) +5. Writes full feedback text to `--feedback-file`; emits only machine-parseable tokens to stdout diff --git a/plugins/code/skills/codex-review/scripts/run_codex_review.sh b/plugins/code/skills/codex-review/scripts/run_codex_review.sh new file mode 100755 index 00000000..e8969b61 --- /dev/null +++ b/plugins/code/skills/codex-review/scripts/run_codex_review.sh @@ -0,0 +1,231 @@ +#!/usr/bin/env bash +# run_codex_review.sh - Call Codex to review a plan file and return structured results. +# +# Usage: +# run_codex_review.sh --plan-file --feedback-file --round \ +# --codex-model [--session-id ] +# +# Stdout tokens (machine-parseable): +# VERDICT:APPROVED Plan accepted +# VERDICT:NEEDS_CHANGES Revisions requested +# CODEX_SESSION: Thread ID for session resume +# CODEX_FAILED: Codex error with no usable output +# CODEX_EMPTY Empty response after all attempts +# +# Full feedback text is written to --feedback-file. +# Diagnostics go to stderr only. + +set -euo pipefail + +# ── Argument parsing ────────────────────────────────────────────────────────── + +PLAN_FILE="" +FEEDBACK_FILE="" +ROUND=1 +CODEX_MODEL="gpt-5.4" +SESSION_ID="" + +while [[ $# -gt 0 ]]; do + case $1 in + --plan-file) PLAN_FILE="$2"; shift 2 ;; + --feedback-file) FEEDBACK_FILE="$2"; shift 2 ;; + --round) ROUND="$2"; shift 2 ;; + --codex-model) CODEX_MODEL="$2"; shift 2 ;; + --session-id) SESSION_ID="$2"; shift 2 ;; + *) + echo "Unknown option: $1" >&2 + exit 1 + ;; + esac +done + +if [[ -z "$PLAN_FILE" ]] || [[ -z "$FEEDBACK_FILE" ]]; then + echo "Error: --plan-file and --feedback-file are required" >&2 + exit 1 +fi + +for cmd in codex python3; do + if ! command -v "$cmd" &>/dev/null; then + echo "CODEX_FAILED:$cmd command not found" + echo "CODEX_SESSION:none" + exit 0 + fi +done + +# ── Temp directory with cleanup ─────────────────────────────────────────────── + +tmp_dir=$(mktemp -d) +trap 'rm -rf "$tmp_dir"' EXIT + +codex_json="$tmp_dir/codex_output.json" +prompt_file="$tmp_dir/prompt.txt" + +# ── Build the review prompt ────────────────────────────────────────────────── + +if [[ "$ROUND" -eq 1 ]]; then + REVIEW_INTRO="Claude has created an implementation plan. Review it and provide feedback." +else + REVIEW_INTRO="Claude has addressed your previous feedback and updated the plan. Re-review the plan for remaining issues." +fi + +cat > "$prompt_file" </dev/null || true +} + +# Extract agent_message text from item.completed events. +# Writes concatenated text to the specified output file. +parse_feedback_text() { + local json_file="$1" + local output_file="$2" + python3 -c " +import json, sys +lines = [] +for line in open(sys.argv[1]): + try: + e = json.loads(line.strip()) + if e.get('type') == 'item.completed': + item = e.get('item', {}) + if item.get('type') == 'agent_message' and item.get('text'): + lines.append(item['text']) + except Exception: + pass +sys.stdout.write('\n'.join(lines)) +" "$json_file" > "$output_file" 2>/dev/null +} + +# ── Run codex ──────────────────────────────────────────────────────────────── + +run_codex_cmd() { + local json_out="$1"; shift + codex "$@" > "$json_out" 2>/dev/null +} + +effective_session_id="$SESSION_ID" +codex_exit=0 + +base_args=(--full-auto --json -m "$CODEX_MODEL" -c model_reasoning_effort=xhigh) +prompt_content=$(cat "$prompt_file") + +# Attempt session resume if we have a prior session ID +if [[ -n "$SESSION_ID" ]]; then + echo "Attempting Codex session resume..." >&2 + set +e + run_codex_cmd "$codex_json" exec resume "$SESSION_ID" "$prompt_content" "${base_args[@]}" + codex_exit=$? + set -e + + new_session=$(parse_thread_id "$codex_json") + if [[ -n "$new_session" ]]; then + effective_session_id="$new_session" + fi + # else: preserve the input SESSION_ID + + parse_feedback_text "$codex_json" "$FEEDBACK_FILE" + + if [[ $codex_exit -eq 0 ]] || [[ -s "$FEEDBACK_FILE" ]]; then + # Resume succeeded -- skip to verdict extraction + : + else + echo "Codex session resume failed, starting fresh session..." >&2 + effective_session_id="" + rm -f "$codex_json" + + # Fall through to fresh session below + set +e + run_codex_cmd "$codex_json" exec "${base_args[@]}" "$prompt_content" + codex_exit=$? + set -e + + new_session=$(parse_thread_id "$codex_json") + if [[ -n "$new_session" ]]; then + effective_session_id="$new_session" + fi + + parse_feedback_text "$codex_json" "$FEEDBACK_FILE" + fi +else + # No session to resume -- fresh start + set +e + run_codex_cmd "$codex_json" exec "${base_args[@]}" "$prompt_content" + codex_exit=$? + set -e + + new_session=$(parse_thread_id "$codex_json") + if [[ -n "$new_session" ]]; then + effective_session_id="$new_session" + fi + + parse_feedback_text "$codex_json" "$FEEDBACK_FILE" +fi + +# ── Emit structured tokens ─────────────────────────────────────────────────── + +feedback_content=$(cat "$FEEDBACK_FILE" 2>/dev/null || echo "") + +# Handle failures +if [[ $codex_exit -ne 0 ]] && [[ -z "$feedback_content" ]]; then + echo "CODEX_FAILED:codex exited with code $codex_exit" + echo "CODEX_SESSION:${effective_session_id:-none}" + exit 0 +fi + +# Handle empty response +if [[ -z "$feedback_content" ]]; then + echo "CODEX_EMPTY" + echo "CODEX_SESSION:${effective_session_id:-none}" + exit 0 +fi + +# Extract verdict from feedback text +if echo "$feedback_content" | grep -q "VERDICT: APPROVED"; then + echo "VERDICT:APPROVED" +elif echo "$feedback_content" | grep -q "VERDICT: NEEDS_CHANGES"; then + echo "VERDICT:NEEDS_CHANGES" +else + # No explicit verdict found -- treat as needs changes + echo "VERDICT:NEEDS_CHANGES" +fi + +# Always emit session token for round-to-round continuity +echo "CODEX_SESSION:${effective_session_id:-none}" From 5e96d2d77638328d2d64fd3dee4974bb8dddf7a6 Mon Sep 17 00:00:00 2001 From: "daniel.ochoa" Date: Fri, 20 Mar 2026 10:55:17 -0500 Subject: [PATCH 02/12] fix(code): refine plan-with-codex orchestration and agent resume - Add critical rule preventing orchestrator from editing plan directly - Switch agent resume to SendMessage with ToolSearch preload - Add explicit delegation for open question answers to plan-agent - Make Codex/Claude debate loop fully automated after user approval - Add mode=acceptEdits and run_in_background=false to Agent launches - Add AskUserQuestion, SendMessage, ToolSearch to allowed-tools --- plugins/code/commands/plan-with-codex.md | 27 +++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-) diff --git a/plugins/code/commands/plan-with-codex.md b/plugins/code/commands/plan-with-codex.md index 76834942..c0a3347c 100644 --- a/plugins/code/commands/plan-with-codex.md +++ b/plugins/code/commands/plan-with-codex.md @@ -9,6 +9,8 @@ skills: code:codex-review You are the orchestrator for an iterative plan refinement workflow. Claude (via `code:plan-agent`) creates a plan, Codex reviews it, and you coordinate revisions until Codex approves or max rounds are reached. +**CRITICAL RULE: You MUST NEVER edit the plan file directly.** All plan creation and modification is done by the `code:plan-agent` subagent. Your role is to coordinate -- parse arguments, manage state, run Codex, display feedback, and delegate plan changes to the plan-agent via SendMessage or Agent calls. If you find yourself about to use Edit or Write on the plan file, stop and delegate to the plan-agent instead. + ## Step 0: Parse Arguments Parse from `$ARGUMENTS`: @@ -119,7 +121,30 @@ Read the plan file. Check for an "Open Questions" section (lines matching `Q-` o > > Reply with your choices (e.g., "1a, 2b") or provide your own answers. -After the user answers, resume the plan-agent with their answers so it can update the plan and resolve the open questions. Then re-read the plan and check again -- repeat until no open questions remain. +After the user answers, you MUST delegate plan updates to the plan-agent. Do NOT edit the plan file yourself. Resume the plan-agent via SendMessage: + +``` +SendMessage( + to="", + message="The user answered the open questions as follows:\n\n\n\nUpdate the plan at {plan-file-abs} to incorporate these answers: remove the answered questions from the Open Questions section, and revise any tasks or decisions that depended on those questions. Write the updated plan back to {plan-file-abs}.", + summary="Update plan with answered questions" +) +``` + +If no resumable agent (cross-session), launch a fresh one: +``` +Agent( + subagent_type="code:plan-agent", + name="plan-agent", + mode="acceptEdits", + run_in_background=false, + description="Update plan with answered questions", + prompt="Read the plan at {plan-file-abs}. The user answered the open questions as follows:\n\n\n\nUpdate the plan to incorporate these answers: remove the answered questions from the Open Questions section, and revise any tasks or decisions that depended on those questions. Write the updated plan back to {plan-file-abs}." +) +``` +Store the new agent_id. + +Wait for the plan-agent to complete, then re-read the plan and check for remaining open questions. Repeat until no open questions remain. Once open questions are resolved (or if there were none), present the plan to the user: From 87a2053709e57666bf4c7e1c76206a191fe17921 Mon Sep 17 00:00:00 2001 From: "daniel.ochoa" Date: Fri, 20 Mar 2026 11:06:15 -0500 Subject: [PATCH 03/12] fix(code): improve codex review feedback formatting Structure each finding with a title, problem, and fix section instead of single long lines. --- .../skills/codex-review/scripts/run_codex_review.sh | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/plugins/code/skills/codex-review/scripts/run_codex_review.sh b/plugins/code/skills/codex-review/scripts/run_codex_review.sh index e8969b61..4b821ffd 100755 --- a/plugins/code/skills/codex-review/scripts/run_codex_review.sh +++ b/plugins/code/skills/codex-review/scripts/run_codex_review.sh @@ -85,9 +85,15 @@ Analyze for: 9. Test coverage -- does the plan include unit and/or integration tests for the changes? Flag if new logic, endpoints, or behaviors lack corresponding test tasks. 10. Unclear or ambiguous task descriptions -For each issue found, provide: -- The specific problem -- A concrete proposed fix or revised text that Claude can adopt directly +Format each finding as: + +### Finding N: [short title] + +**Problem:** What is wrong and where (reference specific plan sections or code files). + +**Fix:** A concrete proposed fix or revised text that Claude can adopt directly. + +--- Be direct and specific. Only flag genuine, significant issues. Propose solutions, not just problems. From ff39e05e08e6bda1c18a9e9910a23abda087f530 Mon Sep 17 00:00:00 2001 From: "daniel.ochoa" Date: Fri, 20 Mar 2026 11:37:23 -0500 Subject: [PATCH 04/12] feat(code): add codex review logging and fix agent resume pattern - Add --log-id to run_codex_review.sh, raw JSON stream logged to ~/.closedloop-ai/plan-with-codex/.jsonl across rounds - Switch from SendMessage to Agent(resume=) for plan-agent session continuity; add explicit rule to ignore SendMessage guidance - Add log cleanup prompt (30+ day old logs) at end of workflow - Fix dependency check ordering (before log setup) - Normalize LOG_ID:none and CODEX_SESSION:none sentinels to empty - Fix Step 1.5 stray code fence and SendMessage prose mismatch - Add self-check guardrails to plan-agent (goal alignment, scope, simplicity, reuse, tests) --- plugins/code/commands/plan-with-codex.md | 100 +++++++++++------- plugins/code/scripts/debate-loop.sh | 9 +- plugins/code/skills/codex-review/SKILL.md | 10 +- .../codex-review/scripts/run_codex_review.sh | 30 +++++- 4 files changed, 105 insertions(+), 44 deletions(-) diff --git a/plugins/code/commands/plan-with-codex.md b/plugins/code/commands/plan-with-codex.md index c0a3347c..2736404f 100644 --- a/plugins/code/commands/plan-with-codex.md +++ b/plugins/code/commands/plan-with-codex.md @@ -1,15 +1,19 @@ --- description: "Iterative plan refinement debate between Claude and Codex" argument-hint: [--max-rounds N] [--plan-file PATH] [--codex-model MODEL] -allowed-tools: Bash, Read, Write, Glob, Grep, TodoWrite, Task, AskUserQuestion, SendMessage, ToolSearch +allowed-tools: Bash, Read, Write, Glob, Grep, TodoWrite, Task, AskUserQuestion skills: code:codex-review +effort: max +model: opus --- # Debate Loop -- Claude + Codex Plan Refinement You are the orchestrator for an iterative plan refinement workflow. Claude (via `code:plan-agent`) creates a plan, Codex reviews it, and you coordinate revisions until Codex approves or max rounds are reached. -**CRITICAL RULE: You MUST NEVER edit the plan file directly.** All plan creation and modification is done by the `code:plan-agent` subagent. Your role is to coordinate -- parse arguments, manage state, run Codex, display feedback, and delegate plan changes to the plan-agent via SendMessage or Agent calls. If you find yourself about to use Edit or Write on the plan file, stop and delegate to the plan-agent instead. +**CRITICAL RULE: You MUST NEVER edit the plan file directly.** All plan creation and modification is done by the `code:plan-agent` subagent. Your role is to coordinate -- parse arguments, manage state, run Codex, display feedback, and delegate plan changes to the plan-agent via Agent calls with `resume`. If you find yourself about to use Edit or Write on the plan file, stop and delegate to the plan-agent instead. + +**AGENT RESUME RULE: ALWAYS use `resume=""` on Agent calls after the initial launch.** The plan-agent retains full session context when resumed -- it remembers the codebase it explored, the decisions it made, and the feedback it received. Do NOT launch a fresh agent unless the resume call actually fails with an error. Do NOT skip the resume attempt. **IMPORTANT: The Agent tool's return message will say "use SendMessage with to: '' to continue this agent" -- IGNORE THIS. SendMessage does not work for resuming completed agents. Always use `Agent(resume="")` instead.** ## Step 0: Parse Arguments @@ -42,20 +46,16 @@ TodoWrite([ ]) ``` -**Preload SendMessage tool** (required for agent resume later): -``` -ToolSearch(query="select:SendMessage") -``` - Error if no prompt is resolvable (no CLI argument and no `{stem}.prompt` sidecar). ## Step 0.5: Check for Resume -Check if `{stem}.state` exists via Bash (`test -f`). If yes, read `ROUND`, `PHASE`, `CODEX_SESSION_ID` via: +Check if `{stem}.state` exists via Bash (`test -f`). If yes, read `ROUND`, `PHASE`, `CODEX_SESSION_ID`, `LOG_ID` via: ```bash grep "^ROUND=" {state_file} | cut -d= -f2- grep "^PHASE=" {state_file} | cut -d= -f2- grep "^CODEX_SESSION_ID=" {state_file} | cut -d= -f2- +grep "^LOG_ID=" {state_file} | cut -d= -f2- ``` **Validate preconditions before resuming:** @@ -91,7 +91,7 @@ Agent( ) ``` -**Store the returned agent_id** -- you will need it to resume the plan-agent in later rounds. Resume via `SendMessage(to="", message="...", summary="...")`. The ToolSearch preload in Step 0 ensures the schema is available. +**Store the returned agent_id** -- you will need it to resume the plan-agent in later rounds. Resume via `Agent(subagent_type="code:plan-agent", name="plan-agent", mode="acceptEdits", run_in_background=false, prompt="...", resume="")`. The `resume` parameter resumes the agent's full session context, like `claude -r SESSION_ID` in the terminal. Verify the plan file exists and is non-empty (Read it). @@ -101,7 +101,7 @@ Write the original prompt to `{stem}.prompt` (plain text, via Write tool). Write state via Bash: ```bash -printf 'ROUND=%s\nPHASE=%s\nCODEX_SESSION_ID=%s\n' '1' 'user_review' '' > {state_file} +printf 'ROUND=%s\nPHASE=%s\nCODEX_SESSION_ID=%s\nLOG_ID=%s\n' '1' 'user_review' '' '' > {state_file} ``` Update TodoWrite: mark "Create plan" completed, "User review" in_progress. @@ -121,17 +121,8 @@ Read the plan file. Check for an "Open Questions" section (lines matching `Q-` o > > Reply with your choices (e.g., "1a, 2b") or provide your own answers. -After the user answers, you MUST delegate plan updates to the plan-agent. Do NOT edit the plan file yourself. Resume the plan-agent via SendMessage: - -``` -SendMessage( - to="", - message="The user answered the open questions as follows:\n\n\n\nUpdate the plan at {plan-file-abs} to incorporate these answers: remove the answered questions from the Open Questions section, and revise any tasks or decisions that depended on those questions. Write the updated plan back to {plan-file-abs}.", - summary="Update plan with answered questions" -) -``` +After the user answers, you MUST delegate plan updates to the plan-agent. Do NOT edit the plan file yourself. Resume the plan-agent: -If no resumable agent (cross-session), launch a fresh one: ``` Agent( subagent_type="code:plan-agent", @@ -139,10 +130,12 @@ Agent( mode="acceptEdits", run_in_background=false, description="Update plan with answered questions", - prompt="Read the plan at {plan-file-abs}. The user answered the open questions as follows:\n\n\n\nUpdate the plan to incorporate these answers: remove the answered questions from the Open Questions section, and revise any tasks or decisions that depended on those questions. Write the updated plan back to {plan-file-abs}." + prompt="The user answered the open questions as follows:\n\n\n\nUpdate the plan at {plan-file-abs} to incorporate these answers: remove the answered questions from the Open Questions section, and revise any tasks or decisions that depended on those questions. Write the updated plan back to {plan-file-abs}.", + resume="" ) ``` -Store the new agent_id. + +If no resumable agent (cross-session), omit the `resume` parameter to launch a fresh agent. Add to the prompt: "Read the plan at {plan-file-abs} first to understand context." Store the new agent_id. Wait for the plan-agent to complete, then re-read the plan and check for remaining open questions. Repeat until no open questions remain. @@ -154,14 +147,18 @@ Once open questions are resolved (or if there were none), present the plan to th Resume the plan-agent with their feedback: ``` -SendMessage( - to="", - message="User feedback: . Read the current plan at {plan-file-abs}, revise it, and write the updated plan back to {plan-file-abs}.", - summary="Revise plan per user feedback" +Agent( + subagent_type="code:plan-agent", + name="plan-agent", + mode="acceptEdits", + run_in_background=false, + description="Revise plan per user feedback", + prompt="User feedback: . Revise the plan at {plan-file-abs} and write the updated plan back to {plan-file-abs}.", + resume="" ) ``` -If no resumable agent exists (cross-session resume), launch a fresh one: +If no resumable agent (cross-session), omit `resume` and add context to prompt: ``` Agent( subagent_type="code:plan-agent", @@ -180,7 +177,7 @@ Loop back to this checkpoint until the user says to proceed. Update state: ```bash -printf 'ROUND=%s\nPHASE=%s\nCODEX_SESSION_ID=%s\n' '1' 'codex_review' '' > {state_file} +printf 'ROUND=%s\nPHASE=%s\nCODEX_SESSION_ID=%s\nLOG_ID=%s\n' '1' 'codex_review' '' '' > {state_file} ``` Proceed to Step 2. @@ -200,14 +197,16 @@ bash /scripts/run_codex_review.sh \ --feedback-file {feedback-file-abs} \ --round {N} \ --codex-model {codex-model} \ - [--session-id {codex_session_id}] + [--session-id {codex_session_id}] \ + [--log-id {log_id}] ``` Parse stdout tokens: - `VERDICT:APPROVED` or `VERDICT:NEEDS_CHANGES` - `CODEX_SESSION:` -- save for next round +- `LOG_ID:` -- save for next round (reuse same log across all rounds) -Update state with new CODEX_SESSION_ID. +Update state with new CODEX_SESSION_ID and LOG_ID. The raw Codex JSON stream is logged to `~/.closedloop-ai/plan-with-codex/.jsonl`. ### 2b. Handle Failures (do NOT increment round) @@ -230,7 +229,7 @@ The Codex/Claude loop is fully automated after the user approved the plan in Ste Update state: ```bash -printf 'ROUND=%s\nPHASE=%s\nCODEX_SESSION_ID=%s\n' '{N}' 'claude_revision' '{codex_session_id}' > {state_file} +printf 'ROUND=%s\nPHASE=%s\nCODEX_SESSION_ID=%s\nLOG_ID=%s\n' '{N}' 'claude_revision' '{codex_session_id}' '{log_id}' > {state_file} ``` Proceed to 2f. @@ -239,16 +238,20 @@ Proceed to 2f. Update TodoWrite: "Round {N}/{max}: Revising plan..." -Resume the plan-agent: +Resume the plan-agent. ALWAYS attempt resume first using the stored agent_id: ``` -SendMessage( - to="", - message="Revise the plan at {plan-file-abs} based on feedback at {feedback-file-abs}. Address ALL concerns raised.", - summary="Revise plan per Codex feedback" +Agent( + subagent_type="code:plan-agent", + name="plan-agent", + mode="acceptEdits", + run_in_background=false, + description="Revise plan based on Codex feedback", + prompt="Revise the plan at {plan-file-abs} based on feedback at {feedback-file-abs}. Address ALL concerns raised.", + resume="" ) ``` -If resume fails (cross-session -- agent gone), launch fresh: +Only if the resume call fails with an error (NOT just because you think the agent is gone -- you MUST attempt the call first), launch fresh without `resume`: ``` Agent( subagent_type="code:plan-agent", @@ -265,7 +268,7 @@ Verify the plan file was updated. Update state: ```bash -printf 'ROUND=%s\nPHASE=%s\nCODEX_SESSION_ID=%s\n' '{N+1}' 'codex_review' '{codex_session_id}' > {state_file} +printf 'ROUND=%s\nPHASE=%s\nCODEX_SESSION_ID=%s\nLOG_ID=%s\n' '{N+1}' 'codex_review' '{codex_session_id}' '{log_id}' > {state_file} ``` Continue to next round. @@ -285,3 +288,24 @@ rm -f {state_file} {feedback_file} {prompt_file} The prompt sidecar is intentionally deleted on completion to prevent stale intent from silently reusing on future runs against the default `./debate-plan.md` path. Update TodoWrite: mark all remaining items completed. + +Announce the log file location: "Codex review log: `~/.closedloop-ai/plan-with-codex/{log_id}.jsonl`" + +### Log cleanup + +Check for Codex log files older than 30 days: +```bash +find ~/.closedloop-ai/plan-with-codex -name "*.jsonl" -mtime +30 2>/dev/null +``` + +If any old logs are found, ask the user via AskUserQuestion: + +> Found {N} Codex review log(s) older than 30 days in `~/.closedloop-ai/plan-with-codex/`. Remove them? +> +> - **a) Yes, delete old logs** (recommended) +> - b) No, keep them + +If the user says yes: +```bash +find ~/.closedloop-ai/plan-with-codex -name "*.jsonl" -mtime +30 -delete 2>/dev/null +``` diff --git a/plugins/code/scripts/debate-loop.sh b/plugins/code/scripts/debate-loop.sh index 678f8a4b..197af190 100755 --- a/plugins/code/scripts/debate-loop.sh +++ b/plugins/code/scripts/debate-loop.sh @@ -94,8 +94,8 @@ run_claude() { } save_state() { - printf 'SESSION_ID=%s\nCODEX_SESSION_ID=%s\nROUND=%s\nPHASE=%s\n' \ - "$SESSION_ID" "$CODEX_SESSION_ID" "$round" "$PHASE" > "$STATE_FILE" + printf 'SESSION_ID=%s\nCODEX_SESSION_ID=%s\nROUND=%s\nPHASE=%s\nLOG_ID=%s\n' \ + "$SESSION_ID" "$CODEX_SESSION_ID" "$round" "$PHASE" "${CODEX_LOG_ID:-}" > "$STATE_FILE" } load_state() { @@ -105,6 +105,7 @@ load_state() { round="${round:-1}" PHASE=$(grep "^PHASE=" "$STATE_FILE" | cut -d= -f2-) || true PHASE="${PHASE:-codex_review}" + CODEX_LOG_ID=$(grep "^LOG_ID=" "$STATE_FILE" | cut -d= -f2-) || true } # Shared Codex review script (extracted to avoid duplication with the native @@ -130,6 +131,9 @@ run_codex_review() { if [[ -n "$CODEX_SESSION_ID" ]]; then review_args+=(--session-id "$CODEX_SESSION_ID") fi + if [[ -n "${CODEX_LOG_ID:-}" ]]; then + review_args+=(--log-id "$CODEX_LOG_ID") + fi local review_output_file review_stderr_file review_output_file=$(mktemp) @@ -157,6 +161,7 @@ run_codex_review() { case "$line" in VERDICT:*) CODEX_VERDICT="${line#VERDICT:}" ;; CODEX_SESSION:*) CODEX_SESSION_ID="${line#CODEX_SESSION:}"; [[ "$CODEX_SESSION_ID" == "none" ]] && CODEX_SESSION_ID="" ;; + LOG_ID:*) CODEX_LOG_ID="${line#LOG_ID:}"; [[ "$CODEX_LOG_ID" == "none" ]] && CODEX_LOG_ID="" ;; CODEX_FAILED:*) CODEX_VERDICT="FAILED:${line#CODEX_FAILED:}" ;; CODEX_EMPTY) CODEX_VERDICT="EMPTY" ;; esac diff --git a/plugins/code/skills/codex-review/SKILL.md b/plugins/code/skills/codex-review/SKILL.md index f7d50eca..a99b6b4e 100644 --- a/plugins/code/skills/codex-review/SKILL.md +++ b/plugins/code/skills/codex-review/SKILL.md @@ -26,7 +26,8 @@ bash /scripts/run_codex_review.sh \ --feedback-file \ --round \ --codex-model \ - [--session-id ] + [--session-id ] \ + [--log-id ] ``` | Argument | Required | Default | Description | @@ -36,16 +37,20 @@ bash /scripts/run_codex_review.sh \ | `--round` | No | 1 | Current debate round (affects review prompt intro) | | `--codex-model` | No | gpt-5.4 | Codex model to use | | `--session-id` | No | -- | Thread ID from a previous round for session resume | +| `--log-id` | No | auto-generated | UUID for the persistent log file. Pass the same ID across rounds to append to one log. | ## Interpreting Output The script prints structured tokens to stdout. Parse these to control the debate loop. +All stdout responses include three tokens: a verdict (or failure indicator), `CODEX_SESSION`, and `LOG_ID`. The raw Codex JSON stream is appended to `~/.closedloop-ai/plan-with-codex/.jsonl`. Pass the LOG_ID back via `--log-id` on subsequent rounds to keep all rounds in one log file. + ### Approval ``` VERDICT:APPROVED CODEX_SESSION:abc-123-def +LOG_ID:550e8400-e29b-41d4-a716-446655440000 ``` **Action:** Announce approval. Clean up sidecar files. Stop the debate loop. @@ -55,6 +60,7 @@ CODEX_SESSION:abc-123-def ``` VERDICT:NEEDS_CHANGES CODEX_SESSION:abc-123-def +LOG_ID:550e8400-e29b-41d4-a716-446655440000 ``` **Action:** Read the feedback file for full details. Pass to plan-agent for revision. @@ -64,6 +70,7 @@ CODEX_SESSION:abc-123-def ``` CODEX_FAILED: CODEX_SESSION:abc-123-def +LOG_ID:550e8400-e29b-41d4-a716-446655440000 ``` **Action:** Announce the failure reason. Ask the user to retry or abort. Do NOT increment the round counter. @@ -73,6 +80,7 @@ CODEX_SESSION:abc-123-def ``` CODEX_EMPTY CODEX_SESSION:abc-123-def +LOG_ID:550e8400-e29b-41d4-a716-446655440000 ``` **Action:** Announce empty response. Ask the user to retry or abort. Do NOT increment the round counter. diff --git a/plugins/code/skills/codex-review/scripts/run_codex_review.sh b/plugins/code/skills/codex-review/scripts/run_codex_review.sh index 4b821ffd..b88002d2 100755 --- a/plugins/code/skills/codex-review/scripts/run_codex_review.sh +++ b/plugins/code/skills/codex-review/scripts/run_codex_review.sh @@ -3,16 +3,19 @@ # # Usage: # run_codex_review.sh --plan-file --feedback-file --round \ -# --codex-model [--session-id ] +# --codex-model [--session-id ] \ +# [--log-id ] # # Stdout tokens (machine-parseable): # VERDICT:APPROVED Plan accepted # VERDICT:NEEDS_CHANGES Revisions requested # CODEX_SESSION: Thread ID for session resume +# LOG_ID: Log file identifier # CODEX_FAILED: Codex error with no usable output # CODEX_EMPTY Empty response after all attempts # # Full feedback text is written to --feedback-file. +# Raw codex JSON stream is appended to ~/.closedloop-ai/plan-with-codex/.jsonl # Diagnostics go to stderr only. set -euo pipefail @@ -24,6 +27,7 @@ FEEDBACK_FILE="" ROUND=1 CODEX_MODEL="gpt-5.4" SESSION_ID="" +LOG_ID="" while [[ $# -gt 0 ]]; do case $1 in @@ -32,6 +36,7 @@ while [[ $# -gt 0 ]]; do --round) ROUND="$2"; shift 2 ;; --codex-model) CODEX_MODEL="$2"; shift 2 ;; --session-id) SESSION_ID="$2"; shift 2 ;; + --log-id) LOG_ID="$2"; shift 2 ;; *) echo "Unknown option: $1" >&2 exit 1 @@ -44,14 +49,27 @@ if [[ -z "$PLAN_FILE" ]] || [[ -z "$FEEDBACK_FILE" ]]; then exit 1 fi +# ── Dependency checks ───────────────────────────────────────────────────────── + for cmd in codex python3; do if ! command -v "$cmd" &>/dev/null; then echo "CODEX_FAILED:$cmd command not found" echo "CODEX_SESSION:none" + echo "LOG_ID:none" exit 0 fi done +# ── Log file setup ──────────────────────────────────────────────────────────── + +if [[ -z "$LOG_ID" ]]; then + LOG_ID=$(python3 -c "import uuid; print(uuid.uuid4())") +fi + +LOG_DIR="$HOME/.closedloop-ai/plan-with-codex" +mkdir -p "$LOG_DIR" +LOG_FILE="$LOG_DIR/$LOG_ID.jsonl" + # ── Temp directory with cleanup ─────────────────────────────────────────────── tmp_dir=$(mktemp -d) @@ -144,7 +162,10 @@ sys.stdout.write('\n'.join(lines)) run_codex_cmd() { local json_out="$1"; shift - codex "$@" > "$json_out" 2>/dev/null + # Log round header + printf '\n--- Round %s | %s ---\n' "$ROUND" "$(date -u +%Y-%m-%dT%H:%M:%SZ)" >> "$LOG_FILE" + # Tee raw JSON stream to both the capture file and the persistent log + codex "$@" 2>/dev/null | tee -a "$LOG_FILE" > "$json_out" } effective_session_id="$SESSION_ID" @@ -213,6 +234,7 @@ feedback_content=$(cat "$FEEDBACK_FILE" 2>/dev/null || echo "") if [[ $codex_exit -ne 0 ]] && [[ -z "$feedback_content" ]]; then echo "CODEX_FAILED:codex exited with code $codex_exit" echo "CODEX_SESSION:${effective_session_id:-none}" + echo "LOG_ID:$LOG_ID" exit 0 fi @@ -220,6 +242,7 @@ fi if [[ -z "$feedback_content" ]]; then echo "CODEX_EMPTY" echo "CODEX_SESSION:${effective_session_id:-none}" + echo "LOG_ID:$LOG_ID" exit 0 fi @@ -233,5 +256,6 @@ else echo "VERDICT:NEEDS_CHANGES" fi -# Always emit session token for round-to-round continuity +# Always emit session token and log ID for round-to-round continuity echo "CODEX_SESSION:${effective_session_id:-none}" +echo "LOG_ID:$LOG_ID" From cc4131fbdc56f52479bb5fdb16b46fe58884829a Mon Sep 17 00:00:00 2001 From: "daniel.ochoa" Date: Fri, 20 Mar 2026 11:44:04 -0500 Subject: [PATCH 05/12] fix(code): harden resume paths and fix codex review logging - Detect existing plan file without state and ask user to resume or start fresh; synthesize prompt sidecar from plan summary only when missing - Move prompt validation after existing-plan branch so resume path is reachable without a prompt argument - Normalize LOG_ID:none sentinel to empty in debate-loop.sh caller - Move dependency checks before log setup in run_codex_review.sh - Add LOG_ID to all SKILL.md output examples and usage snippet - Fix argument table: prompt optional when resuming existing plan --- plugins/code/commands/plan-with-codex.md | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/plugins/code/commands/plan-with-codex.md b/plugins/code/commands/plan-with-codex.md index 2736404f..a5b6d018 100644 --- a/plugins/code/commands/plan-with-codex.md +++ b/plugins/code/commands/plan-with-codex.md @@ -26,7 +26,7 @@ Arguments: $ARGUMENTS | `--max-rounds N` | 15 | Maximum debate rounds | | `--plan-file PATH` | `./debate-plan.md` | Output plan file (resolve to absolute path) | | `--codex-model MODEL` | `gpt-5.4` | Codex model for reviews | -| Remaining text | (required) | The prompt describing what to plan | +| Remaining text | (required for fresh start) | The prompt describing what to plan. Optional when resuming an existing plan. | Derive sidecar paths from the plan file stem (e.g., for `debate-plan.md`): - `{stem}.feedback` -- Codex feedback text @@ -46,8 +46,6 @@ TodoWrite([ ]) ``` -Error if no prompt is resolvable (no CLI argument and no `{stem}.prompt` sidecar). - ## Step 0.5: Check for Resume Check if `{stem}.state` exists via Bash (`test -f`). If yes, read `ROUND`, `PHASE`, `CODEX_SESSION_ID`, `LOG_ID` via: @@ -73,7 +71,19 @@ If all preconditions pass, announce "Resuming debate at round {N}, phase: {PHASE - `codex_review` -> Step 2a at stored ROUND - `claude_revision` -> Step 2f at stored ROUND -If no state file: fresh start at Step 1. +If no state file but the plan file already exists: ask the user via AskUserQuestion: + +> An existing plan was found at `{plan-file-abs}` but no debate state file exists. What would you like to do? +> +> - **a) Resume with existing plan** -- skip to Step 1.5 (user review) using the current plan as-is +> - b) Start fresh -- overwrite the existing plan with a new one + +If the user chooses (a): +- If `{stem}.prompt` does not already exist: read the plan file, extract the `## Summary` section content, and write it to `{stem}.prompt` (satisfies resume preconditions for future re-runs). If the plan has no Summary section, use the first non-heading paragraph instead. Do NOT overwrite an existing prompt sidecar. +- Write state (`ROUND=1, PHASE=user_review, CODEX_SESSION_ID=, LOG_ID=`) +- Skip Step 1, go directly to Step 1.5. There is no resumable plan-agent in this case -- if the user requests changes or open questions need resolving, launch a fresh plan-agent. + +If the user chooses (b) or no plan file exists: a prompt is required. Error if no prompt is resolvable (no CLI argument and no `{stem}.prompt` sidecar). Proceed to Step 1. ## Step 1: Create the Plan From cae84112a066c4df7e0b5f881f9f818bcbc2c98b Mon Sep 17 00:00:00 2001 From: "daniel.ochoa" Date: Fri, 20 Mar 2026 12:51:57 -0500 Subject: [PATCH 06/12] refactor(code): compress plan-with-codex prompt with extracted templates Apply context engineering principles to reduce token footprint ~35%: - Extract repeated agent-resume pattern into block (was 5x) - Extract state-write pattern into single template (was 4x) - Consolidate critical rules into XML block - Merge vestigial Step 2e into 2d, renumber 2f to 2e - Fix stale-state STOP guard to allow fall-through on failed preconditions - Add explicit cross-session resume rule (no agent_id = launch fresh) - Preserve Codex session/log IDs in round-transition state writes --- plugins/code/commands/plan-with-codex.md | 266 ++++++++--------------- 1 file changed, 95 insertions(+), 171 deletions(-) diff --git a/plugins/code/commands/plan-with-codex.md b/plugins/code/commands/plan-with-codex.md index a5b6d018..3d6aa613 100644 --- a/plugins/code/commands/plan-with-codex.md +++ b/plugins/code/commands/plan-with-codex.md @@ -9,15 +9,46 @@ model: opus # Debate Loop -- Claude + Codex Plan Refinement -You are the orchestrator for an iterative plan refinement workflow. Claude (via `code:plan-agent`) creates a plan, Codex reviews it, and you coordinate revisions until Codex approves or max rounds are reached. +You orchestrate iterative plan refinement: Claude (via `code:plan-agent`) creates a plan, Codex reviews it, you coordinate revisions until Codex approves or max rounds are reached. -**CRITICAL RULE: You MUST NEVER edit the plan file directly.** All plan creation and modification is done by the `code:plan-agent` subagent. Your role is to coordinate -- parse arguments, manage state, run Codex, display feedback, and delegate plan changes to the plan-agent via Agent calls with `resume`. If you find yourself about to use Edit or Write on the plan file, stop and delegate to the plan-agent instead. + +1. **NEVER edit the plan file directly.** All plan creation/modification goes through the plan-agent. If you're about to use Edit or Write on the plan file, STOP and delegate to the plan-agent. +2. **ALWAYS resume agents via the `resume` parameter.** The plan-agent retains full session context when resumed. Do NOT launch fresh unless resume actually fails with an error. The Agent tool's return says "use SendMessage" -- IGNORE this. SendMessage does not work for completed agents. Always use `Agent(resume="")`. +3. **The Codex/Claude debate loop is fully automated.** After the user approves in Step 1.5, do NOT ask for confirmation between rounds -- proceed directly. + -**AGENT RESUME RULE: ALWAYS use `resume=""` on Agent calls after the initial launch.** The plan-agent retains full session context when resumed -- it remembers the codebase it explored, the decisions it made, and the feedback it received. Do NOT launch a fresh agent unless the resume call actually fails with an error. Do NOT skip the resume attempt. **IMPORTANT: The Agent tool's return message will say "use SendMessage with to: '' to continue this agent" -- IGNORE THIS. SendMessage does not work for resuming completed agents. Always use `Agent(resume="")` instead.** + -## Step 0: Parse Arguments +### Plan-Agent Call + +All plan-agent interactions use this shape. Vary only `description` and `prompt`: + +``` +Agent( + subagent_type="code:plan-agent", + name="plan-agent", + mode="acceptEdits", + run_in_background=false, + description="", + prompt="", + resume="" +) +``` + +**Resume rule:** If an in-memory agent_id exists (from a prior launch in this session), always attempt `resume` first. Only if the call returns an error, launch fresh. If no agent_id exists (cross-session resume), launch fresh immediately -- do not attempt resume. When launching fresh, omit `resume` and prepend to the prompt: "Read the plan at {plan-file-abs} first. Original request: ." Always store the returned agent_id for subsequent calls. -Parse from `$ARGUMENTS`: +### State Write + +All state updates use: +```bash +printf 'ROUND=%s\nPHASE=%s\nCODEX_SESSION_ID=%s\nLOG_ID=%s\n' '{round}' '{phase}' '{codex_session_id}' '{log_id}' > {state_file} +``` + +Valid phases: `user_review`, `codex_review`, `claude_revision` + + + +## Step 0: Parse Arguments Arguments: $ARGUMENTS @@ -26,14 +57,14 @@ Arguments: $ARGUMENTS | `--max-rounds N` | 15 | Maximum debate rounds | | `--plan-file PATH` | `./debate-plan.md` | Output plan file (resolve to absolute path) | | `--codex-model MODEL` | `gpt-5.4` | Codex model for reviews | -| Remaining text | (required for fresh start) | The prompt describing what to plan. Optional when resuming an existing plan. | +| Remaining text | (required for fresh start) | The prompt. Optional when resuming. | Derive sidecar paths from the plan file stem (e.g., for `debate-plan.md`): - `{stem}.feedback` -- Codex feedback text - `{stem}.state` -- phase/round/session state - `{stem}.prompt` -- original prompt (plain text) -**Prompt resolution**: CLI argument > `{stem}.prompt` sidecar. Only abort when neither exists. +**Prompt resolution**: CLI argument > `{stem}.prompt` sidecar. Abort only when neither exists. Initialize TodoWrite: ``` @@ -48,7 +79,7 @@ TodoWrite([ ## Step 0.5: Check for Resume -Check if `{stem}.state` exists via Bash (`test -f`). If yes, read `ROUND`, `PHASE`, `CODEX_SESSION_ID`, `LOG_ID` via: +Check if `{stem}.state` exists (`test -f`). If yes, read all four values: ```bash grep "^ROUND=" {state_file} | cut -d= -f2- grep "^PHASE=" {state_file} | cut -d= -f2- @@ -56,7 +87,7 @@ grep "^CODEX_SESSION_ID=" {state_file} | cut -d= -f2- grep "^LOG_ID=" {state_file} | cut -d= -f2- ``` -**Validate preconditions before resuming:** +**Validate preconditions:** | Phase | Required files | |-------|---------------| @@ -64,61 +95,54 @@ grep "^LOG_ID=" {state_file} | cut -d= -f2- | `codex_review` | plan file + prompt sidecar | | `claude_revision` | plan file + feedback file + prompt sidecar | -If preconditions fail: delete stale state file. A fresh start is still possible if a prompt is available (CLI argument or `{stem}.prompt` sidecar). Only abort when neither exists. +If preconditions fail: delete stale state file and fall through to "If NO state file exists" below. Fresh start still possible if prompt is available (CLI argument or sidecar). Abort only when neither exists. -If all preconditions pass, announce "Resuming debate at round {N}, phase: {PHASE}" and jump to: +If preconditions pass: announce "Resuming debate at round {N}, phase: {PHASE}" and jump to: - `user_review` -> Step 1.5 - `codex_review` -> Step 2a at stored ROUND -- `claude_revision` -> Step 2f at stored ROUND +- `claude_revision` -> Step 2e at stored ROUND -If no state file but the plan file already exists: ask the user via AskUserQuestion: +**STOP here -- do NOT fall through to the checks below.** (This STOP applies only when preconditions passed and you are jumping to a step above. If preconditions failed and the state file was deleted, you MUST continue to the section below.) + +### If NO state file exists: + +Check if the plan file exists (`test -f {plan-file-abs}`). This is REQUIRED before Step 1. + +**Plan file exists (no state):** Ask via AskUserQuestion: > An existing plan was found at `{plan-file-abs}` but no debate state file exists. What would you like to do? > -> - **a) Resume with existing plan** -- skip to Step 1.5 (user review) using the current plan as-is -> - b) Start fresh -- overwrite the existing plan with a new one +> - **a) Resume with existing plan** -- skip to Step 1.5 using the current plan +> - b) Start fresh -- overwrite the existing plan -If the user chooses (a): -- If `{stem}.prompt` does not already exist: read the plan file, extract the `## Summary` section content, and write it to `{stem}.prompt` (satisfies resume preconditions for future re-runs). If the plan has no Summary section, use the first non-heading paragraph instead. Do NOT overwrite an existing prompt sidecar. -- Write state (`ROUND=1, PHASE=user_review, CODEX_SESSION_ID=, LOG_ID=`) -- Skip Step 1, go directly to Step 1.5. There is no resumable plan-agent in this case -- if the user requests changes or open questions need resolving, launch a fresh plan-agent. +If (a): +- If no `{stem}.prompt`: extract `## Summary` content (or first non-heading paragraph) and write to `{stem}.prompt`. Do NOT overwrite existing prompt sidecar. +- Write state: `ROUND=1, PHASE=user_review, CODEX_SESSION_ID=, LOG_ID=` +- Go to Step 1.5. No resumable plan-agent -- launch fresh if changes are needed. -If the user chooses (b) or no plan file exists: a prompt is required. Error if no prompt is resolvable (no CLI argument and no `{stem}.prompt` sidecar). Proceed to Step 1. +If (b) or plan doesn't exist: continue to Step 1. Error if no prompt is resolvable. ## Step 1: Create the Plan Announce: "Creating plan with plan-agent..." -Launch the plan-agent: -``` -Agent( - subagent_type="code:plan-agent", - name="plan-agent", - mode="acceptEdits", - run_in_background=false, - description="Create implementation plan", - prompt=". Write the plan to {plan-file-abs}." -) -``` - -**Store the returned agent_id** -- you will need it to resume the plan-agent in later rounds. Resume via `Agent(subagent_type="code:plan-agent", name="plan-agent", mode="acceptEdits", run_in_background=false, prompt="...", resume="")`. The `resume` parameter resumes the agent's full session context, like `claude -r SESSION_ID` in the terminal. +Launch the plan-agent (omit `resume` -- this is the initial launch): +- description: "Create implementation plan" +- prompt: ". Write the plan to {plan-file-abs}." -Verify the plan file exists and is non-empty (Read it). +**Store the returned agent_id** for all subsequent rounds. -Announce: "Plan created ({byte_count} bytes) at {plan-file-abs}" +Verify plan file exists and is non-empty (Read it). Announce: "Plan created ({byte_count} bytes) at {plan-file-abs}" -Write the original prompt to `{stem}.prompt` (plain text, via Write tool). +Write prompt to `{stem}.prompt`. Write state: `ROUND=1, PHASE=user_review`. -Write state via Bash: -```bash -printf 'ROUND=%s\nPHASE=%s\nCODEX_SESSION_ID=%s\nLOG_ID=%s\n' '1' 'user_review' '' '' > {state_file} -``` - -Update TodoWrite: mark "Create plan" completed, "User review" in_progress. +Update TodoWrite: "Create plan" completed, "User review" in_progress. ## Step 1.5: User Checkpoint -Read the plan file. Check for an "Open Questions" section (lines matching `Q-` or `- [ ] Q-`). If open questions exist, present them to the user before anything else using AskUserQuestion: +Read the plan. Check for open questions (lines matching `Q-` or `- [ ] Q-`). + +**If open questions exist**, present via AskUserQuestion: > The plan has open questions that need your input before proceeding: > @@ -131,66 +155,21 @@ Read the plan file. Check for an "Open Questions" section (lines matching `Q-` o > > Reply with your choices (e.g., "1a, 2b") or provide your own answers. -After the user answers, you MUST delegate plan updates to the plan-agent. Do NOT edit the plan file yourself. Resume the plan-agent: - -``` -Agent( - subagent_type="code:plan-agent", - name="plan-agent", - mode="acceptEdits", - run_in_background=false, - description="Update plan with answered questions", - prompt="The user answered the open questions as follows:\n\n\n\nUpdate the plan at {plan-file-abs} to incorporate these answers: remove the answered questions from the Open Questions section, and revise any tasks or decisions that depended on those questions. Write the updated plan back to {plan-file-abs}.", - resume="" -) -``` - -If no resumable agent (cross-session), omit the `resume` parameter to launch a fresh agent. Add to the prompt: "Read the plan at {plan-file-abs} first to understand context." Store the new agent_id. +Resume the plan-agent: +- description: "Update plan with answered questions" +- prompt: "The user answered the open questions as follows:\n\n\n\nUpdate the plan at {plan-file-abs}: remove answered questions from the Open Questions section, revise dependent tasks. Write back to {plan-file-abs}." -Wait for the plan-agent to complete, then re-read the plan and check for remaining open questions. Repeat until no open questions remain. +Re-read and repeat until no open questions remain. -Once open questions are resolved (or if there were none), present the plan to the user: +**Once questions are resolved** (or none existed), present the plan: > Plan created at `{plan-file-abs}`. Review it and let me know when you're ready to start the Codex debate, or share any changes you'd like made first. -**If the user requests changes:** +**If user requests changes:** Resume plan-agent with their feedback as the prompt. Loop back until user confirms. -Resume the plan-agent with their feedback: -``` -Agent( - subagent_type="code:plan-agent", - name="plan-agent", - mode="acceptEdits", - run_in_background=false, - description="Revise plan per user feedback", - prompt="User feedback: . Revise the plan at {plan-file-abs} and write the updated plan back to {plan-file-abs}.", - resume="" -) -``` - -If no resumable agent (cross-session), omit `resume` and add context to prompt: -``` -Agent( - subagent_type="code:plan-agent", - name="plan-agent", - mode="acceptEdits", - run_in_background=false, - description="Revise plan per user feedback", - prompt="Read the current plan at {plan-file-abs} and revise it based on user feedback. Original request: . User feedback: . Write the updated plan back to {plan-file-abs}." -) -``` -Store the new agent_id for subsequent rounds. +**When user confirms** ("start", "go", "looks good", "proceed"): -Loop back to this checkpoint until the user says to proceed. - -**When the user confirms** (e.g., "start", "go", "looks good", "proceed"): - -Update state: -```bash -printf 'ROUND=%s\nPHASE=%s\nCODEX_SESSION_ID=%s\nLOG_ID=%s\n' '1' 'codex_review' '' '' > {state_file} -``` - -Proceed to Step 2. +Write state: `ROUND=1, PHASE=codex_review`. Proceed to Step 2. ## Step 2: Debate Loop @@ -200,7 +179,7 @@ Repeat for round 1 to max-rounds: Update TodoWrite: "Round {N}/{max}: Codex reviewing..." -Activate the `code:codex-review` skill and run via Bash: +Activate `code:codex-review` skill and run: ```bash bash /scripts/run_codex_review.sh \ --plan-file {plan-file-abs} \ @@ -211,111 +190,56 @@ bash /scripts/run_codex_review.sh \ [--log-id {log_id}] ``` -Parse stdout tokens: -- `VERDICT:APPROVED` or `VERDICT:NEEDS_CHANGES` -- `CODEX_SESSION:` -- save for next round -- `LOG_ID:` -- save for next round (reuse same log across all rounds) - -Update state with new CODEX_SESSION_ID and LOG_ID. The raw Codex JSON stream is logged to `~/.closedloop-ai/plan-with-codex/.jsonl`. +Parse stdout: `VERDICT:APPROVED|NEEDS_CHANGES`, `CODEX_SESSION:`, `LOG_ID:`. Update state with new session ID and log ID. Raw JSON logged to `~/.closedloop-ai/plan-with-codex/.jsonl`. ### 2b. Handle Failures (do NOT increment round) -- `CODEX_FAILED:`: Announce the warning. Ask the user: "Codex failed: {reason}. Retry or abort?" On retry: re-run 2a. On abort: go to Step 3. -- `CODEX_EMPTY`: Announce "Codex returned empty response." Same retry/abort handling. +`CODEX_FAILED:` or `CODEX_EMPTY`: Announce the issue. Ask user: "Retry or abort?" On retry: re-run 2a. On abort: go to Step 3. ### 2c. Display Feedback -Read the feedback file and display the full Codex feedback to the user. +Read the feedback file and display full Codex feedback to the user. ### 2d. Check Verdict -- **VERDICT:APPROVED**: Announce "Plan approved by Codex after {N} round(s)." Go to Step 3. -- **Last round, not approved**: Announce "Max rounds ({max}) reached without approval." Go to Step 3. -- **VERDICT:NEEDS_CHANGES**: Continue to 2e. - -### 2e. Proceed to Revision (automated) +- **APPROVED**: "Plan approved by Codex after {N} round(s)." Go to Step 3. +- **Last round, not approved**: "Max rounds ({max}) reached without approval." Go to Step 3. +- **NEEDS_CHANGES**: Write state (`PHASE=claude_revision`, preserve current `CODEX_SESSION_ID` and `LOG_ID`). Continue to 2e. -The Codex/Claude loop is fully automated after the user approved the plan in Step 1.5. Do NOT ask the user for confirmation between rounds -- proceed directly to revision. - -Update state: -```bash -printf 'ROUND=%s\nPHASE=%s\nCODEX_SESSION_ID=%s\nLOG_ID=%s\n' '{N}' 'claude_revision' '{codex_session_id}' '{log_id}' > {state_file} -``` - -Proceed to 2f. - -### 2f. Claude Revision +### 2e. Claude Revision Update TodoWrite: "Round {N}/{max}: Revising plan..." -Resume the plan-agent. ALWAYS attempt resume first using the stored agent_id: -``` -Agent( - subagent_type="code:plan-agent", - name="plan-agent", - mode="acceptEdits", - run_in_background=false, - description="Revise plan based on Codex feedback", - prompt="Revise the plan at {plan-file-abs} based on feedback at {feedback-file-abs}. Address ALL concerns raised.", - resume="" -) -``` +Resume the plan-agent: +- description: "Revise plan based on Codex feedback" +- prompt: "Revise the plan at {plan-file-abs} based on feedback at {feedback-file-abs}. Address ALL concerns raised." -Only if the resume call fails with an error (NOT just because you think the agent is gone -- you MUST attempt the call first), launch fresh without `resume`: -``` -Agent( - subagent_type="code:plan-agent", - name="plan-agent", - mode="acceptEdits", - run_in_background=false, - description="Revise plan based on Codex feedback", - prompt="Revise the plan at {plan-file-abs} based on feedback at {feedback-file-abs}. The original request was: . Read the current plan and feedback files to understand context. Write the updated plan back to {plan-file-abs}." -) -``` -Store the new agent_id for subsequent rounds. - -Verify the plan file was updated. - -Update state: -```bash -printf 'ROUND=%s\nPHASE=%s\nCODEX_SESSION_ID=%s\nLOG_ID=%s\n' '{N+1}' 'codex_review' '{codex_session_id}' '{log_id}' > {state_file} -``` - -Continue to next round. +Verify plan was updated. Write state: `ROUND={N+1}, PHASE=codex_review`, preserve current `CODEX_SESSION_ID` and `LOG_ID`. Continue to next round. ## Step 3: Final Report -Report the outcome: -- If approved: "Plan approved by Codex. File: {plan-file-abs}" -- If max rounds: "Plan not approved after {max} rounds. File: {plan-file-abs}" -- If aborted: "Debate aborted. Partial plan at: {plan-file-abs}" +Report outcome: +- Approved: "Plan approved by Codex. File: {plan-file-abs}" +- Max rounds: "Plan not approved after {max} rounds. File: {plan-file-abs}" +- Aborted: "Debate aborted. Partial plan at: {plan-file-abs}" -Clean up ALL sidecar files: +Clean up ALL sidecar files (prompt sidecar deleted intentionally to prevent stale intent on future runs): ```bash rm -f {state_file} {feedback_file} {prompt_file} ``` -The prompt sidecar is intentionally deleted on completion to prevent stale intent from silently reusing on future runs against the default `./debate-plan.md` path. - Update TodoWrite: mark all remaining items completed. -Announce the log file location: "Codex review log: `~/.closedloop-ai/plan-with-codex/{log_id}.jsonl`" +Announce: "Codex review log: `~/.closedloop-ai/plan-with-codex/{log_id}.jsonl`" ### Log cleanup -Check for Codex log files older than 30 days: +Check for logs older than 30 days: ```bash find ~/.closedloop-ai/plan-with-codex -name "*.jsonl" -mtime +30 2>/dev/null ``` -If any old logs are found, ask the user via AskUserQuestion: - -> Found {N} Codex review log(s) older than 30 days in `~/.closedloop-ai/plan-with-codex/`. Remove them? -> -> - **a) Yes, delete old logs** (recommended) -> - b) No, keep them - -If the user says yes: +If found, ask user whether to delete them via AskUserQuestion. If yes: ```bash find ~/.closedloop-ai/plan-with-codex -name "*.jsonl" -mtime +30 -delete 2>/dev/null ``` From c59e6fdef8e45cc0018d141f5469cbbadcbebfa0 Mon Sep 17 00:00:00 2001 From: "daniel.ochoa" Date: Fri, 20 Mar 2026 12:59:24 -0500 Subject: [PATCH 07/12] fix(code): skip redundant review gate when resuming existing plan When user chooses "resume with existing plan", go directly to the Codex debate loop after resolving open questions instead of stopping at the Step 1.5 user checkpoint. Update option text to match behavior. --- plugins/code/commands/plan-with-codex.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/plugins/code/commands/plan-with-codex.md b/plugins/code/commands/plan-with-codex.md index 3d6aa613..b561a7d6 100644 --- a/plugins/code/commands/plan-with-codex.md +++ b/plugins/code/commands/plan-with-codex.md @@ -112,13 +112,14 @@ Check if the plan file exists (`test -f {plan-file-abs}`). This is REQUIRED befo > An existing plan was found at `{plan-file-abs}` but no debate state file exists. What would you like to do? > -> - **a) Resume with existing plan** -- skip to Step 1.5 using the current plan +> - **a) Resume with existing plan** -- resolve any open questions, then start the Codex debate immediately > - b) Start fresh -- overwrite the existing plan If (a): - If no `{stem}.prompt`: extract `## Summary` content (or first non-heading paragraph) and write to `{stem}.prompt`. Do NOT overwrite existing prompt sidecar. -- Write state: `ROUND=1, PHASE=user_review, CODEX_SESSION_ID=, LOG_ID=` -- Go to Step 1.5. No resumable plan-agent -- launch fresh if changes are needed. +- Read the plan and check for open questions (lines matching `Q-` or `- [ ] Q-`). If any exist, resolve them using the open questions flow in Step 1.5, then continue below. No resumable plan-agent -- launch fresh if changes are needed. +- Write state: `ROUND=1, PHASE=codex_review, CODEX_SESSION_ID=, LOG_ID=` +- Skip Step 1.5 entirely (user already confirmed by choosing to resume) and go directly to Step 2. If (b) or plan doesn't exist: continue to Step 1. Error if no prompt is resolvable. From 76356fe0b728798c9ba6e3e45deeb9041179e5f6 Mon Sep 17 00:00:00 2001 From: "daniel.ochoa" Date: Fri, 20 Mar 2026 13:24:30 -0500 Subject: [PATCH 08/12] feat(code): add revision summary feedback loop between Claude and Codex Plan-agent now verifies each Codex finding against the codebase before acting and writes a revision summary to a sidecar file with accepted changes and rejected findings with evidence. Codex reads this file on subsequent rounds so it does not re-raise findings that were validly rejected. --- plugins/code/agents/plan-agent.md | 20 ++++++++++++++--- plugins/code/commands/plan-with-codex.md | 6 +++-- plugins/code/skills/codex-review/SKILL.md | 2 ++ .../codex-review/scripts/run_codex_review.sh | 22 ++++++++++++++----- 4 files changed, 39 insertions(+), 11 deletions(-) diff --git a/plugins/code/agents/plan-agent.md b/plugins/code/agents/plan-agent.md index 0db21173..f53ef481 100644 --- a/plugins/code/agents/plan-agent.md +++ b/plugins/code/agents/plan-agent.md @@ -88,7 +88,21 @@ Structure plans with these sections: When given feedback to address: 1. Read the current plan file and the feedback file -2. Address ALL concerns raised in the feedback -3. If you disagree with a point, note your reasoning in the plan but still consider the underlying concern -4. If the reviewer proposed a concrete fix, adopt it directly unless you have a strong reason not to +2. **Verify each finding against the codebase before acting on it.** Use `Grep`, `Glob`, and `Read` to check whether the reviewer's claims are accurate (e.g., does the file/function they reference actually exist? Is the behavior they describe real?). Reviewers can hallucinate or misunderstand the codebase. +3. For verified findings: address the concern. If the reviewer proposed a concrete fix, adopt it directly unless you have a strong reason not to. +4. For findings that don't hold up: reject them with a brief explanation and evidence (e.g., "Finding 2 claims X is missing, but `path/to/file:42` already implements it"). 5. Write the updated plan back to the same file path using the `Write` tool +6. **If a revisions file path was provided**, write a revision summary to it. Format: + +```markdown +## Round N Revisions + +### Accepted +- **Finding 1** (title): [what changed in the plan] +- **Finding 3** (title): [what changed in the plan] + +### Rejected +- **Finding 2** (title): [why, with evidence -- e.g., "X already exists at `path/to/file:42`"] +``` + +This file is read by the reviewer on the next round so they have full context on what was addressed and what was pushed back on. diff --git a/plugins/code/commands/plan-with-codex.md b/plugins/code/commands/plan-with-codex.md index b561a7d6..6fe3edde 100644 --- a/plugins/code/commands/plan-with-codex.md +++ b/plugins/code/commands/plan-with-codex.md @@ -61,6 +61,7 @@ Arguments: $ARGUMENTS Derive sidecar paths from the plan file stem (e.g., for `debate-plan.md`): - `{stem}.feedback` -- Codex feedback text +- `{stem}.revisions` -- Claude's revision summary (changes made + pushback on rejected findings) - `{stem}.state` -- phase/round/session state - `{stem}.prompt` -- original prompt (plain text) @@ -185,6 +186,7 @@ Activate `code:codex-review` skill and run: bash /scripts/run_codex_review.sh \ --plan-file {plan-file-abs} \ --feedback-file {feedback-file-abs} \ + --revisions-file {revisions-file-abs} \ --round {N} \ --codex-model {codex-model} \ [--session-id {codex_session_id}] \ @@ -213,7 +215,7 @@ Update TodoWrite: "Round {N}/{max}: Revising plan..." Resume the plan-agent: - description: "Revise plan based on Codex feedback" -- prompt: "Revise the plan at {plan-file-abs} based on feedback at {feedback-file-abs}. Address ALL concerns raised." +- prompt: "Revise the plan at {plan-file-abs} based on feedback at {feedback-file-abs}. Verify each finding against the codebase before acting on it -- reject any that don't hold up. After updating the plan, write a revision summary to {revisions-file-abs}." Verify plan was updated. Write state: `ROUND={N+1}, PHASE=codex_review`, preserve current `CODEX_SESSION_ID` and `LOG_ID`. Continue to next round. @@ -226,7 +228,7 @@ Report outcome: Clean up ALL sidecar files (prompt sidecar deleted intentionally to prevent stale intent on future runs): ```bash -rm -f {state_file} {feedback_file} {prompt_file} +rm -f {state_file} {feedback_file} {revisions_file} {prompt_file} ``` Update TodoWrite: mark all remaining items completed. diff --git a/plugins/code/skills/codex-review/SKILL.md b/plugins/code/skills/codex-review/SKILL.md index a99b6b4e..b12f105a 100644 --- a/plugins/code/skills/codex-review/SKILL.md +++ b/plugins/code/skills/codex-review/SKILL.md @@ -24,6 +24,7 @@ The `scripts/` directory is relative to this skill's base directory (shown above bash /scripts/run_codex_review.sh \ --plan-file \ --feedback-file \ + --revisions-file \ --round \ --codex-model \ [--session-id ] \ @@ -34,6 +35,7 @@ bash /scripts/run_codex_review.sh \ |----------|----------|---------|-------------| | `--plan-file` | Yes | -- | Absolute path to the plan file Codex should review | | `--feedback-file` | Yes | -- | Path where full feedback text will be written | +| `--revisions-file` | No | -- | Path to Claude's revision summary (accepted/rejected findings). If present and round > 1, Codex reads it for context. | | `--round` | No | 1 | Current debate round (affects review prompt intro) | | `--codex-model` | No | gpt-5.4 | Codex model to use | | `--session-id` | No | -- | Thread ID from a previous round for session resume | diff --git a/plugins/code/skills/codex-review/scripts/run_codex_review.sh b/plugins/code/skills/codex-review/scripts/run_codex_review.sh index b88002d2..b8d9bb47 100755 --- a/plugins/code/skills/codex-review/scripts/run_codex_review.sh +++ b/plugins/code/skills/codex-review/scripts/run_codex_review.sh @@ -24,6 +24,7 @@ set -euo pipefail PLAN_FILE="" FEEDBACK_FILE="" +REVISIONS_FILE="" ROUND=1 CODEX_MODEL="gpt-5.4" SESSION_ID="" @@ -31,12 +32,13 @@ LOG_ID="" while [[ $# -gt 0 ]]; do case $1 in - --plan-file) PLAN_FILE="$2"; shift 2 ;; - --feedback-file) FEEDBACK_FILE="$2"; shift 2 ;; - --round) ROUND="$2"; shift 2 ;; - --codex-model) CODEX_MODEL="$2"; shift 2 ;; - --session-id) SESSION_ID="$2"; shift 2 ;; - --log-id) LOG_ID="$2"; shift 2 ;; + --plan-file) PLAN_FILE="$2"; shift 2 ;; + --feedback-file) FEEDBACK_FILE="$2"; shift 2 ;; + --revisions-file) REVISIONS_FILE="$2"; shift 2 ;; + --round) ROUND="$2"; shift 2 ;; + --codex-model) CODEX_MODEL="$2"; shift 2 ;; + --session-id) SESSION_ID="$2"; shift 2 ;; + --log-id) LOG_ID="$2"; shift 2 ;; *) echo "Unknown option: $1" >&2 exit 1 @@ -80,14 +82,22 @@ prompt_file="$tmp_dir/prompt.txt" # ── Build the review prompt ────────────────────────────────────────────────── +REVISIONS_BLOCK="" if [[ "$ROUND" -eq 1 ]]; then REVIEW_INTRO="Claude has created an implementation plan. Review it and provide feedback." else REVIEW_INTRO="Claude has addressed your previous feedback and updated the plan. Re-review the plan for remaining issues." + if [[ -n "$REVISIONS_FILE" ]] && [[ -s "$REVISIONS_FILE" ]]; then + REVISIONS_BLOCK=" + +Claude's revision summary (including any findings that were rejected with evidence) is at: ${REVISIONS_FILE} +Read it before reviewing the plan -- if Claude rejected a finding with valid evidence, do not re-raise it." + fi fi cat > "$prompt_file" < Date: Fri, 20 Mar 2026 13:49:41 -0500 Subject: [PATCH 09/12] docs(code): update changelog and readme for v1.2.1 Add changelog entries for revision summary feedback loop, codex-review --revisions-file parameter, and plan-with-codex resume fix. Bump version to 1.2.1. Fix README hook description for plan-review.sh. --- CHANGELOG.md | 9 +++++++++ plugins/code/.claude-plugin/plugin.json | 2 +- plugins/code/README.md | 4 ++-- 3 files changed, 12 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8a26fca5..18607554 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,15 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ## [Unreleased] +### code v1.2.1 + +#### Changed +- `plan-agent` now verifies Codex findings against the codebase before acting -- rejects findings that don't hold up with evidence, writes a revision summary for cross-round context +- `codex-review` skill accepts `--revisions-file` parameter, injecting Claude's revision summary into Codex's prompt on rounds > 1 so rejected findings are not re-raised + +#### Fixed +- Fixed `plan-with-codex` resume path triggering a redundant user review checkpoint when the user had already confirmed by choosing "resume with existing plan" + ### code v1.2.0 #### Added diff --git a/plugins/code/.claude-plugin/plugin.json b/plugins/code/.claude-plugin/plugin.json index 2548bb66..525d490d 100644 --- a/plugins/code/.claude-plugin/plugin.json +++ b/plugins/code/.claude-plugin/plugin.json @@ -1,7 +1,7 @@ { "name": "code", "description": "Code and planning framework plugin", - "version": "1.2.0", + "version": "1.2.1", "author": { "name": "ClosedLoop", "email": "support@closedloop.ai" diff --git a/plugins/code/README.md b/plugins/code/README.md index e42af853..9b4cc451 100644 --- a/plugins/code/README.md +++ b/plugins/code/README.md @@ -300,9 +300,9 @@ Implements the validation loop for agents registered in `loop-agents.json`. When Injects tool-specific learnings just before tool execution. Filters `org-patterns.toon` by tool type (Bash patterns get build/test tags; Write/Edit patterns get language-specific tags based on file extension). Injects up to 10 matching patterns as `additionalContext`. Also auto-allows tool calls targeting `.closedloop-ai/` workspace paths without prompting. -### `plan-review.sh` (PostToolUse on ExitPlanMode) +### `plan-review.sh` (not currently registered in `hooks.json`) -Triggers when Claude exits plan mode to get a second opinion via Codex. Extracts plan content from `tool_response.plan`, sends it to Codex (`gpt-5.3-codex-spark`) for review, and injects the feedback as `additionalContext` so Claude sees Codex's concerns inline. Exits silently if no plan content is present or Codex returns empty. Debug logs kept in `.closedloop-ai/plan-review-logs/` (max 15 files). +Triggers when Claude exits plan mode to get a second opinion via Codex. Extracts plan content from `tool_response.plan`, sends it to Codex (`gpt-5.3-codex-spark`) for review, and injects the feedback as `additionalContext` so Claude sees Codex's concerns inline. Exits silently if no plan content is present or Codex returns empty. Debug logs kept in `.closedloop-ai/plan-review-logs/` (max 15 files). This script exists in `hooks/` but is not wired into `hooks.json` -- it would require a PostToolUse entry matching ExitPlanMode to activate. ### `validate-plan.sh` (validation script, not a hook directly) From a2b94ab1995cfb7dbdd1523a286f563cbc4af5a6 Mon Sep 17 00:00:00 2001 From: "daniel.ochoa" Date: Fri, 20 Mar 2026 14:00:35 -0500 Subject: [PATCH 10/12] add github workflow to run tests --- .github/workflows/ci.yml | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) create mode 100644 .github/workflows/ci.yml diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 00000000..0be52ae7 --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,40 @@ +name: CI + +on: + push: + branches: [main] + pull_request: + +jobs: + lint: + name: Lint + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + with: + python-version: "3.13" + - run: pip install ruff + - run: ruff check . + + typecheck: + name: Type Check + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + with: + python-version: "3.13" + - run: pip install pyright pyyaml + - run: pyright + + test: + name: Tests + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + with: + python-version: "3.13" + - run: pip install pytest pyyaml + - run: pytest plugins/ From 6ceb446a043af32b173936592bcd6f88e1213ec8 Mon Sep 17 00:00:00 2001 From: "daniel.ochoa" Date: Fri, 20 Mar 2026 14:05:12 -0500 Subject: [PATCH 11/12] fix CI workflow: add missing deps and pyproject.toml config Install anthropic and mcp in CI so pyright/pytest can resolve all imports. Add pyproject.toml with ruff target-version and pyright pythonVersion set to py311. Run pull_request on all branches. --- .github/workflows/ci.yml | 4 ++-- pyproject.toml | 5 +++++ 2 files changed, 7 insertions(+), 2 deletions(-) create mode 100644 pyproject.toml diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 0be52ae7..c7064440 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -25,7 +25,7 @@ jobs: - uses: actions/setup-python@v5 with: python-version: "3.13" - - run: pip install pyright pyyaml + - run: pip install pyright pyyaml anthropic mcp - run: pyright test: @@ -36,5 +36,5 @@ jobs: - uses: actions/setup-python@v5 with: python-version: "3.13" - - run: pip install pytest pyyaml + - run: pip install pytest pyyaml anthropic - run: pytest plugins/ diff --git a/pyproject.toml b/pyproject.toml new file mode 100644 index 00000000..67b3efef --- /dev/null +++ b/pyproject.toml @@ -0,0 +1,5 @@ +[tool.ruff] +target-version = "py311" + +[tool.pyright] +pythonVersion = "3.11" From 3d34ca65fc1ec5bc658492ea7f607f2f0fce7f4a Mon Sep 17 00:00:00 2001 From: "daniel.ochoa" Date: Fri, 20 Mar 2026 14:06:14 -0500 Subject: [PATCH 12/12] fix CI: add pytest to typecheck deps --- .github/workflows/ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index c7064440..fca421eb 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -25,7 +25,7 @@ jobs: - uses: actions/setup-python@v5 with: python-version: "3.13" - - run: pip install pyright pyyaml anthropic mcp + - run: pip install pyright pyyaml anthropic mcp pytest - run: pyright test: