Reviewer Zero: mandatory content-reasoning pass (Pass 5)

ClaudeR 0.12.3. The base auditing protocol was entirely tool-driven:
value reconciliation, references, cross-references, and code recompute,
with the content lenses living only in Referee Mode. A controlled
benchmark on a synthetic manuscript showed the consequence: with the
tools, a frontier model scored BELOW a native-tools baseline (16.8 vs
17.6 of 24), because the tool checklist crowded out the open critical
read that catches reasoning defects (instrument misattribution, a
reported test its data cannot support, a figure cited for evidence it
does not contain, magnitude overstatement).

Added Pass 5, a mandatory equal-weight content-reasoning pass with a
gated registry and seven checks the tools cannot do: instrument/source
attribution, test computability, figure-and-table content vs claim,
magnitude wording, convergence across studies, causal/generality
framing, and data existence for descriptive claims. Rule 15 states
plainly that the tools do not reason and Pass 5 is not optional. Final
Report and summary counts updated.

Confirmation on the same benchmark: the revised protocol scored 21,
20, 21 (mean 20.7), recovering exactly the reasoning defects the old
protocol lost, with near-zero false positives, moving ClaudeR from
below the native baseline to clearly above it.

R CMD check: Status OK. checks.R: all pass. Bridge unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Author: IMNMV

DESCRIPTION

@@ -1,6 +1,6 @@
 Package: ClaudeR
 Title: R Integration for Claude AI
-Version: 0.12.2
+Version: 0.12.3
 Authors@R: person("Nykko", "Vitali", email = "nykvt@icloud.com", role = c("aut", "cre"))
 Description: Connects RStudio with Claude AI to enable interactive coding sessions.
 License: MIT + file LICENSE

README.md

@@ -50,6 +50,8 @@ claudeAddin()
 <details>
 <summary><b>Recent Updates</b> (click to expand)</summary>
 
+- **Reviewer Zero now reasons, not just reconciles (R 0.12.3).** Added a mandatory Pass 5 (content reasoning) to the base auditing protocol. The deterministic tools (reconcile_values, verify_references, check_cross_references, probe_scripts) find numeric, reference, cross-reference, and code defects, but they do not reason about meaning, and a manuscript can clear every one of them and still be wrong. Pass 5 is a gated, equal-weight pass with seven checks the tools cannot do: instrument and source attribution, whether each reported test is computable from the data that exists, whether each cited figure or table actually contains the claimed evidence, magnitude wording, convergence across studies, causal and generality framing, and data existence for descriptive claims. In a controlled benchmark on a synthetic manuscript with a known defect set, this raised detection from below a native-tools baseline to clearly above it, recovering exactly the reasoning defects the old tool-led protocol was missing.
+
 - **Shared-connection identity, coordination visibility, and a stale-session guard (R 0.12.2 / clauder-mcp 0.14.2).** Three fixes from a live three-persona field session. (1) Personas sharing one MCP connection were renaming each other, because `set_agent_name` changes the identity of the whole connection. New `as_agent` parameter on `send_message`, `check_messages`, and `wait_for_message` acts as a named persona for one call, with a separate read cursor per name. The bridge now enforces the pattern: a second `set_agent_name` with a different name is refused unless forced, every send confirmation echoes the name it was sent as, and the agent intro states where the current identity came from. (2) Coordination messages bypass R by design, so the console and the Agents panel showed nothing while agents talked. The addin now echoes each new coordination event to the console in full (no truncation), appends it to the session log, and shows a live coordination roster with last-seen ages. (3) A bridge still pointed at a dead R session used to report "success" while writing to a coordination log no live agent reads. Coordination calls now fail loudly when no live session exists, and announce it when the connection re-binds to a different live session. Also from pilot 2: `check_cross_references` understands S-prefixed supplement numbering, and the audit protocol documents `unname()` for htest fields and author-plus-year citation matching.
 
 - **Agent identity and cross-restart history (R 0.11.0 / clauder-mcp 0.13.0).** Built from field reports of a multi-day, three-agent session. New `set_agent_name` tool: an agent sets its working name (for example "Claude-Stasis") once, and execution history, message attribution, presence, and its read cursor all carry that name. This fixes the case where several agents or personas share one MCP connection and collapse into a single random id. `get_session_history` gains `include_past`: it parses prior session log files on disk, so the audit of who ran what now survives R restarts. The coordination protocol now makes identity the first step of check-in.

inst/prompts/reviewer_zero.md

@@ -390,6 +390,91 @@ After verifying statistical claims, check that the bibliography is real.
 
 ---
 
+## Pass 5: Content reasoning (the defects no tool can surface)
+
+Passes 1-4 and their tools (`reconcile_values`, `verify_references`,
+`check_cross_references`, `probe_scripts`) find numeric, reference,
+cross-reference, and code defects. They do not reason about meaning. A
+manuscript can clear every gate above and still contain serious defects that
+only a close, skeptical read finds, and those are the defects that separate a
+real audit from a checklist.
+
+This pass is mandatory and carries equal weight with the tool passes. Do not
+treat it as optional, and do not let the tool output stand in for it: the
+tools have already told you everything they can. Read the manuscript prose
+again, slowly, with the tool output in hand, and bring outside domain
+knowledge the tools do not have. A claim can be false when every number in it
+reconciles.
+
+Build a reasoning registry so the checks are tracked and gated like every
+other pass:
+
+```r
+reasoning_registry <- data.frame(
+  check = c("instrument_attribution", "test_computability",
+            "figure_claim_match", "magnitude_wording",
+            "convergence_consistency", "causal_generality_framing",
+            "data_existence"),
+  status = "unrun",           # unrun -> clear | defect
+  finding = "",               # verbatim claim + what is wrong
+  stringsAsFactors = FALSE)
+```
+
+Run every check. For each, quote the verbatim claim, state the finding, and
+set `status` to `clear` or `defect`. A defect here is reported exactly like a
+Pass 3 discrepancy.
+
+1. **Instrument and source attribution.** For every named scale, task, or
+   measure with a citation, does the cited source match the instrument as
+   described (item count, version, authorship)? Example defect: a "22-item"
+   scale attributed to the paper that introduced the 10-item version. This
+   needs knowledge of the instrument, not a value check.
+2. **Test computability.** For every reported test or statistic, can it
+   actually be computed from the variables that exist in the data? Inspect the
+   data columns with `execute_r`. A reported p-value or group comparison for
+   which no supporting variable exists is a defect, not a number to reconcile
+   (e.g. "accuracy did not differ across conditions" when only one accuracy
+   value per participant exists).
+3. **Figure and table content versus claim.** For every figure or table cited
+   in support of a claim, does it actually contain the evidence claimed? A
+   figure cited for a relationship it does not plot (a speed-only boxplot
+   cited for a speed-accuracy trade-off) is a defect even though the
+   cross-reference resolves.
+4. **Magnitude wording.** For every qualitative magnitude word ("roughly
+   twice", "comparable", "doubled", "small"), recompute the actual
+   ratio or difference and check the word fits. "Roughly twice" for a computed
+   2.8x is a defect.
+5. **Convergence and consistency of argument.** Does any claim that results
+   "converge", "replicate", or "hold across both studies" survive when a
+   measure was collected in only one study, or a construct was operationalized
+   differently across them? Cross-check what each study actually measured.
+6. **Causal and generality framing.** Does any causal-mechanism claim rest on
+   a manipulated cause with only a measured (correlational) mediator, with no
+   mediation test? Does any generality claim ("generalize across populations
+   and contexts") exceed a single sample and a stylized task, especially where
+   the paper's own limitations contradict it?
+7. **Data existence for descriptive claims.** For every descriptive or
+   reliability claim (demographics, Cronbach's alpha, "measures collected"),
+   does the supporting data exist anywhere in the project? A claim resting on
+   data absent from the corpus must be flagged as unverifiable.
+
+### Reasoning-pass gate
+
+You cannot proceed to the Final Report until every check has been run:
+
+```r
+unrun_checks <- sum(reasoning_registry$status == "unrun")
+cat(sprintf("Reasoning pass: %d / %d checks run (%d defects, %d unrun)\n",
+    sum(reasoning_registry$status != "unrun"), nrow(reasoning_registry),
+    sum(reasoning_registry$status == "defect"), unrun_checks))
+stopifnot(unrun_checks == 0)
+```
+
+A clean result (all checks `clear`) on a sound paper is a valid outcome. Do
+not invent a reasoning defect to fill the pass.
+
+---
+
 ## Final Report
 
 After all claims and references are processed, generate a summary:
@@ -410,6 +495,9 @@ cat(sprintf("Rounding only: %d\n", sum(claim_registry$status == "rounding")))
 cat(sprintf("Discrepancies: %d\n", sum(claim_registry$status == "discrepancy")))
 cat(sprintf("Not found in code: %d\n", sum(claim_registry$status == "not_found")))
 cat(sprintf("Errors: %d\n", sum(claim_registry$status == "error")))
+cat(sprintf("Reasoning checks: %d run | %d defects\n",
+    sum(reasoning_registry$status != "unrun"),
+    sum(reasoning_registry$status == "defect")))
 ```
 
 Then print the full registry and highlight every discrepancy with:
@@ -430,6 +518,12 @@ Include an internal consistency section listing:
   elsewhere in the code, with both results shown
 - If no inconsistencies were found, state that explicitly
 
+Include a content-reasoning section (Pass 5) listing:
+- Each reasoning check and its result (clear or defect)
+- For each defect, the verbatim claim and why it fails, treated with the
+  same weight as a numeric discrepancy
+- If all checks were clear, state that explicitly
+
 ---
 
 ## Rules
@@ -463,3 +557,9 @@ Include an internal consistency section listing:
 14. Final verdicts come from clean-room recomputation
     (`probe_scripts(capture_output = TRUE)` or a fresh background session),
     never from a long-lived environment that may hold stale objects.
+15. The tools do not reason. Pass 5 is not optional and is not covered by any
+    tool: a manuscript can pass every numeric, reference, and cross-reference
+    gate and still misattribute an instrument, cite a figure for evidence it
+    does not contain, report a test its data cannot support, or overstate a
+    magnitude or a causal claim. Run every reasoning check and weight its
+    findings equally with the numeric ones.