fix(resolve-pr-feedback): adjudicate judgment calls before escalating (EveryInc#1765)

Author: tmchow

docs/guides/ce-resolve-pr-feedback.md

@@ -74,7 +74,9 @@ Most feedback, nitpicks included, is correct. Validation is not a separate analy
 - The change buys nothing real (the bar is "no benefit," not "minor") -> `replied`
 - Risk cannot be bounded -> de-risk with a test if possible, else `needs-human`
 - It is a question -> `replied`, or `needs-human` for a product call
-- The fix would reverse a *deliberate* design choice (positive evidence of intent, plus a real disagreement) -> `needs-human`. "The code currently does X" is not evidence of intent
+- The fix would reverse a *deliberate* design choice (positive evidence of intent, plus a real disagreement) -> adjudicated, then `needs-human` only if that cannot decide it. "The code currently does X" is not evidence of intent
+
+A divert that only needs judgment, not authority the run lacks, is adjudicated before it escalates: the agent hands the composed decision (the ask, the intent evidence, the options) to `ce-pov`, asking for its cross-model panel, and applies a positioned verdict as an ordinary fix, decline, or reply with the reasoning and the concurring peers on the thread. Security, billing, data, product calls, and anything outside the inherited envelope skip adjudication and go straight to `needs-human`, as does a blocked or unavailable `ce-pov`.
 
 "I'm uneasy" is not a tripwire. Source does not matter. A bot can be right; a human can be wrong.
 
@@ -99,7 +101,7 @@ When the same invariant applies to other sites this PR introduced, those sites b
 | `declined` | Suggested fix would make the code worse | Reply citing harm + resolve |
 | `needs-human` | Cannot determine the right action | Reply with `decision_context`, leave open |
 
-`needs-human` is rare. It includes what the reviewer said, what was investigated, why a decision is needed, and options with tradeoffs. Escalations never block the rest of the run. That is what lets `/ce-babysit-pr` call this skill unattended.
+`needs-human` is rare, and rarer now that judgment-only diverts are adjudicated first. It includes what the reviewer said, what was investigated, why a decision is needed, and options with tradeoffs. Escalations never block the rest of the run. That is what lets `/ce-babysit-pr` call this skill unattended.
 
 ---
 
@@ -180,7 +182,7 @@ No. Reading the code is the same work either way. An "it's a bot, so ignore it"
 Overlapping files serialize before dispatch. If a fix expands to callers in another file, combined validation and the verify pass catch the breakage, and those agents re-run sequentially.
 
 **What does `needs-human` mean?**
-The agent investigated and still cannot choose. The thread stays open. The summary includes `decision_context`: quoted feedback, findings, options, and a lean if any.
+The agent investigated, adjudicated through `ce-pov` where the call only needed judgment, and still cannot choose, or the call needs authority the run lacks. The thread stays open. The summary includes `decision_context`: quoted feedback, findings, options, and a lean if any.
 
 **What if the loop never converges?**
 After two fix-verify cycles, it stops and escalates the recurring pattern as `needs-human`. It does not retry forever.

skills/ce-resolve-pr-feedback/SKILL.md

@@ -9,7 +9,7 @@ allowed-tools: Bash(gh *), Bash(git *), Read
 
 Evaluate and fix PR review feedback, then reply and resolve threads. You, as the orchestrator, judge every item centrally, deciding whether each one is legitimate. Then you dispatch generic subagents, each seeded with the fixer prompt bundled in this skill, only for the items you approved for a fix.
 
-**Escalations never block.** `needs-human` is how you escalate: leave the thread open with a natural reply and report the structured `decision_context`. Never pause mid-run to ask. That is what lets an autonomous caller — `ce-babysit-pr` running unattended, for example — loop this skill. Items that need a human decision come back as `needs-human` results for the caller to show the user, rather than stalling the run; that includes a fix that would change behavior the author chose deliberately (see the rubric).
+**Escalations never block.** `needs-human` is how you escalate: leave the thread open with a natural reply and report the structured `decision_context`. Never pause mid-run to ask. That is what lets an autonomous caller — `ce-babysit-pr` running unattended, for example — loop this skill. Items that need a human decision come back as `needs-human` results for the caller to show the user, rather than stalling the run. A decision that only needs judgment, not authority the run lacks, is adjudicated through `ce-pov` before it escalates; the rubric's "Adjudicate before escalating" section draws that line, so a deliberate choice a reviewer wants reversed reaches the human only when adjudication cannot decide it.
 
 **`mode:pipeline`** (set by an orchestrator like `ce-babysit-pr` or `lfg`): the run is unattended, so **never call the blocking-question tool for any reason**, and read `references/pipeline-mode.md` before acting. It defines the two things ordinary mode leaves open. First, the open thread is the record of the escalation, so never write a PR-body residual section of your own. Second, the caller may pass a `trajectory` (`unresolved_trend`, `new_threads_this_tick`, `invariant_rounds`); when it shows that the feedback is not converging, or `invariant_rounds[].rounds >= 2` for a key this pass would continue (the next fix would be that key's third round) and that key's escalation is unanswered, answer with one approach-level `needs-human` rather than fixing nit after nit — an answered escalation authorizes the next action instead. On a fix outcome, return a stable `invariant_key` per fixed root; do not run `pr-snapshot`.
 

skills/ce-resolve-pr-feedback/references/evaluation-rubric.md

@@ -54,7 +54,7 @@ Divert from fixing only on a concrete signal:
   1. **Positive evidence of intent** -- a concrete artifact showing the current behavior is a choice, not an accident: a comment/docstring stating it, a test asserting it, a PR/commit rationale, or a sentinel/guard that only makes sense as a decision. "The code currently does X" is **not** evidence (all code does something). If you cannot point to a specific artifact, there is no deliberate choice to protect -> **fix it**.
   2. **Genuine disagreement** -- a competent reviewer could reasonably have chosen the other way; it is a judgment/product call, not a clear improvement the author simply missed.
 
-  When both hold -> `needs-human` with `decision_context` contrasting the reviewer's ask, the intent artifact, and the tradeoff. When they don't, it is an ordinary fix. **Still fix these (they do NOT meet this condition):** renames, a guard the reviewer pinpoints, dead-code removal, an off-by-one, a missing null check, style, semantics-preserving perf, or correcting a choice that the evidence shows was a *mistake* rather than a decision. This guard exists so an autonomous caller (`ce-babysit-pr`) never silently reverses an intended behavior -- it is **not** a license to escalate ordinary feedback. Default remains to fix; "this might be intentional" is not evidence, and uncertainty resolves to fixing (or a brief `replied` question), not escalating.
+  When both hold -> compose `decision_context` contrasting the reviewer's ask, the intent artifact, and the tradeoff, then adjudicate it (see "Adjudicate before escalating" below) before it becomes `needs-human`. When they don't, it is an ordinary fix. **Still fix these (they do NOT meet this condition):** renames, a guard the reviewer pinpoints, dead-code removal, an off-by-one, a missing null check, style, semantics-preserving perf, or correcting a choice that the evidence shows was a *mistake* rather than a decision. This guard exists so an autonomous caller (`ce-babysit-pr`) never silently reverses an intended behavior -- it is **not** a license to escalate ordinary feedback. Default remains to fix; "this might be intentional" is not evidence, and uncertainty resolves to fixing (or a brief `replied` question), not escalating.
 - **It's a question, not a change request** ("why X?", "is this intentional?") -- answerable from the code -> `replied`; depends on a product/business call you can't determine -> `needs-human`.
 
 ## Outdated threads (`isOutdated=true`)
@@ -71,6 +71,20 @@ Beyond the risk and question cases above: architectural changes that affect othe
 
 Do the investigation work before escalating. Don't punt with "this is complex." The user should be able to read your analysis and decide in under 30 seconds.
 
+## Adjudicate before escalating
+
+An item that met a divert above still splits by what the decision needs, and only one kind reaches the human.
+
+- **Authority-bound** -- the decision needs authority or knowledge this run does not hold: it touches security, auth, billing, data retention, a migration, or another system; it is a product or business call; the action it would take is outside the inherited envelope (merge, rebase, force-push, approving CI); or only the user knows the intent. This is `needs-human` as composed. Do not adjudicate it.
+- **Judgment-bound** -- competent engineers could reasonably disagree, and every fact the decision turns on is in the repository, the PR, and the review thread: a deliberate choice a reviewer wants reversed, two reviewers asking for opposite things, an unbounded-risk read that more evidence could bound, or a non-convergence root under pipeline mode. Invoke `ce-pov` on this item before it becomes `needs-human`.
+
+Pass `ce-pov` the composed `decision_context` as an approach set: the reviewer's ask, the intent artifact, and the options with their tradeoffs. Ask for its cross-model panel with `oracle` so that, when other models are reachable, the verdict rests on more than one model's judgment. `ce-pov` grounds the question and owns the verdict; it degrades to a solo take when no peer is reachable and returns blocked when the evidence is missing. Adjudicate once per root decision, not once per thread, and never for an item that met no divert; "unsure" still resolves to fixing.
+
+The verdict decides the item:
+
+- A positioned verdict whose action is inside the inherited envelope becomes the ordinary verdict it names (`fixed`, `fixed-differently`, `declined`, or `replied`). The reply on the thread carries the reasoning, names the peers that concurred or differed, and says the call was made autonomously, so the author can reverse it after the fact.
+- A blocked result, a verdict with no lean, a verdict whose action is outside the envelope, or `ce-pov` unavailable in this harness -> `needs-human`. Fold the adjudication into `decision_context` (its findings into `investigation`, its lean into `recommendation`) so the parked item is richer than the one you started with.
+
 ## Reply text for reply-list and human-list items
 
 Compose these now -- you have the evidence. Every reply is posted as the PR author. Write each one through the `ce-noslop` skill in its user's-own-writing register. Quote the specific sentence being addressed, not the whole comment if it's long.

skills/ce-resolve-pr-feedback/references/full-mode.md

@@ -82,7 +82,7 @@ Produce a verdict per item and sort into three lists:
 
 - **fix-list** — `fixed` / `fixed-differently`. These get dispatched to fixers in step 4. For each, note the file/location (and for outdated threads, the resolved location or anchor) and a one-line "what to change." **Class fix:** when the cross-item pass (rubric: "A validated finding can span sites this PR itself introduced") found equivalent same-invariant sites this PR touched, record them as **one** fix-list item that enumerates every concrete location (`file:line`) and lists every feedback ID it covers — one class item → one fixer (step 4), so the sites are edited coherently and every covered thread/comment is replied-to and resolved from that single result. Enumerate only sites whose treatment is unambiguous; a site needing its own judgment stays a separate item.
 - **reply-list** — `replied` / `not-addressing` / `declined`. No code change. Compose the reply text now per the rubric (you have the evidence) and carry it to step 7.
-- **human-list** — `needs-human`. Compose `decision_context` now; carry to steps 7 and 9.
+- **human-list** — `needs-human`. Compose `decision_context` now and run the rubric's "Adjudicate before escalating" step on each judgment-bound item; an adjudicated verdict moves the item to the list it names, and the rest carry to steps 7 and 9.
 
 Create a task list of all new items (e.g., `TaskCreate` in Claude Code, `update_plan` in Codex) tagged with their verdict, so progress is visible.
 

skills/ce-resolve-pr-feedback/references/pipeline-mode.md

@@ -16,7 +16,7 @@ Return the `needs-human` object exactly as the rubric defines it: `type: "needs-
 
 The caller may pass a `trajectory`: a rising `unresolved_trend`, `new_threads_this_tick > 0` across passes, or any `invariant_rounds[].rounds >= 2`. When it does, group the feedback by the root decision it comes from, and decide what to do about each root before fixing anything on it:
 
-- **Escalate** — raise **one** `needs-human` about the root decision itself, at the level of the approach (e.g. "regex is the wrong tool here — options: exhaustive table / a real parser / accept known limits; lean: …"), and do it **before** any fix, commit, or push. Escalate in either of two cases. First, the root's feedback is *demonstrably* not converging: several nits share one root, "your regex misses case X" repeats for X after X, or a bot re-posts fresh nits after every commit without end. Second, a fix would begin the root's third recorded round: `invariant_rounds[].rounds >= 2` for a key this pass would continue (rounds are recorded after a fix completes).
+- **Escalate** — raise **one** `needs-human` about the root decision itself, at the level of the approach (e.g. "regex is the wrong tool here — options: exhaustive table / a real parser / accept known limits; lean: …"), and do it **before** any fix, commit, or push. A root decision is judgment-bound unless the rubric's authority-bound condition holds, so it goes through the rubric's "Adjudicate before escalating" step first; an adjudicated verdict inside the envelope is applied as that root's next fix, and the escalation is raised only when adjudication cannot decide it. Escalate in either of two cases. First, the root's feedback is *demonstrably* not converging: several nits share one root, "your regex misses case X" repeats for X after X, or a bot re-posts fresh nits after every commit without end. Second, a fix would begin the root's third recorded round: `invariant_rounds[].rounds >= 2` for a key this pass would continue (rounds are recorded after a fix completes).
 - **Execute an answered escalation** — when the open thread already carries a human's decision on the root, that answer authorizes the next action; apply it. Do not raise the same `needs-human` again; the caller's record store rejects a repeat.
 - **Otherwise fix as usual** — a normal batch of unrelated valid nits is just fixed, one pass.
 

skills/ce-resolve-pr-feedback/references/targeted-mode.md

@@ -43,6 +43,6 @@ Apply Full Mode step 7 (Reply and Resolve)'s completion check before judgment: c
 
 - **`fixed` / `fixed-differently`** — read `references/agents/pr-comment-resolver.md` and spawn a single generic subagent seeded with that fixer prompt to implement it. Do not dispatch a standalone agent by type/name. Pass the file/location fields (resolved location or anchor if outdated), the comment text, and your note on what to change and why it's valid. The fixer only implements; it does not re-judge. **When the harness exposes no way to dispatch (or the dispatch fails), apply the fix yourself in this context**, using that same prompt as your own instructions. The fixer implements a change this step already approved, so running it here costs nothing but the separate context (see Full Mode step 4, Fix).
 - **`replied` / `not-addressing` / `declined`** — no subagent. Compose the reply text per the rubric and proceed to reply/resolve.
-- **`needs-human`** — compose `decision_context` and the natural-sounding reply per the rubric, leave the thread open (don't resolve), and present the decision to the user (use the platform's blocking question tool as in Full Mode step 9). The shared reply step below posts the reply once — do not post it here.
+- **`needs-human`** — compose `decision_context`, run the rubric's "Adjudicate before escalating" step (a judgment-bound item may come back as one of the verdicts above), and for what remains compose the natural-sounding reply per the rubric, leave the thread open (don't resolve), and present the decision to the user (use the platform's blocking question tool as in Full Mode step 9). The shared reply step below posts the reply once — do not post it here.
 
 Then follow the same validate -> commit -> push -> reply -> resolve flow as Full Mode steps 5-7 (in `references/full-mode.md`). Skip validate/commit when no code changed.

tests/ce-babysit-pr-contract.test.ts

@@ -637,6 +637,28 @@ describe("ce-babysit-pr cross-skill contract parity", () => {
     expect(pipelineDelta).toMatch(/open\/claimed\/parked current currency item[^.]{0,240}residual/i)
   })
 
+  test("judgment-bound escalations are adjudicated through ce-pov before they become needs-human", async () => {
+    // The stalemate: the resolver's only exits for a contested item were fix or needs-human, so a
+    // judgment call parked forever. Adjudication runs only after a divert already fired, and an
+    // authority-bound item (security, billing, product, out-of-envelope action) never enters it.
+    const [rubric, pipeline, fullMode, targeted] = await Promise.all([
+      readRepoFile(CERESOLVE_RUBRIC),
+      readRepoFile(CERESOLVE_PIPELINE),
+      readRepoFile(CERESOLVE_FULL_MODE),
+      readRepoFile("skills/ce-resolve-pr-feedback/references/targeted-mode.md"),
+    ])
+    const section = rubric.match(/## Adjudicate before escalating[\s\S]+?(?=\n## )/)?.[0]
+    expect(section).toBeDefined()
+    expect(section).toMatch(/Authority-bound[\s\S]{0,600}Do not adjudicate/)
+    expect(section).toMatch(/Judgment-bound[\s\S]{0,600}Invoke `ce-pov`/)
+    expect(section).toMatch(/never for an item that met no divert/)
+    expect(section).toMatch(/`ce-pov` unavailable[^.]{0,80}`needs-human`/)
+    // Every judgment step that can emit needs-human routes through the section.
+    for (const [name, text] of [["pipeline", pipeline], ["full-mode", fullMode], ["targeted-mode", targeted]] as const) {
+      expect(text, `${name} must route judgment-bound escalations through adjudication`).toContain("Adjudicate before escalating")
+    }
+  })
+
   test("needs-human cannot become a successful handoff before its decision payload reaches the coordinator", async () => {
     const [resolverRubric, resolverPipeline, debugPipeline, babysitPipeline, babysitWatch, babysitTick, babysitSettle, babysitReport, commitPush, lfg] =
       await Promise.all([

tests/skill-eval-cell/catalog.test.ts

@@ -215,9 +215,12 @@ describe("skill-eval-cell catalog", () => {
         "ce-riffrec-feedback-analysis/quick-notes:references/analyzer.md",
         "ce-riffrec-feedback-analysis/quick-notes:references/quick-bug-report.md",
         "ce-riffrec-feedback-analysis/setup-before-recording:references/install-riffrec.md",
+        "ce-resolve-pr-feedback/judgment-bound-adjudicates:references/evaluation-rubric.md",
         "ce-resolve-pr-feedback/pipeline-no-merge:references/pipeline-mode.md",
         "ce-resolve-pr-feedback/pipeline-returns-complete-human-decision:references/evaluation-rubric.md",
         "ce-resolve-pr-feedback/pipeline-returns-complete-human-decision:references/pipeline-mode.md",
+        "ce-resolve-pr-feedback/pipeline-root-adjudicates-first:references/evaluation-rubric.md",
+        "ce-resolve-pr-feedback/pipeline-root-adjudicates-first:references/pipeline-mode.md",
         "ce-test-xcode/missing-mcp-stops:references/setup-and-build.md",
         "ce-test-xcode/swiftui-inline-link-fallback:references/test-and-report.md",
         "ce-work/behavior-fix-routes-to-review:references/input-triage.md",