Skip to content

code_comment failure still yields exit 0 and complete coverage; clarify automated admission semantics #1196

Description

@acoliver

Environment

  • open-code-review v1.11.6 (04284b5dc), Darwin/arm64, built 2026-09-07T07:09:04Z.
  • Provider zai-anthropic; actual review manifest model glm-5.2.
  • ocr review --audience agent --format json --concurrency 1 --from <BASE> --to <HEAD> --rule <RULE> --background-file <CONTEXT>.
  • Three changed Rust files grouped into one review group. Compact business context, no context-length warning.

Observed result

Run c82286dc-5885-4100-a219-4ac05a470941 printed:

[ocr]   ✘ code_comment failed: Error: 'comments' array is required. Got args: {}

The retained JSON reports:

{
  "status": "complete",
  "tool_calls": {
    "total": 15,
    "failure": 1,
    "failure_by_tool": {"code_comment": 1},
    "failure_details": [{
      "tool_call_number": 5,
      "tool_name": "code_comment",
      "file_path": "<fileA>,<fileB>,<fileC>",
      "arguments": "{}",
      "error": "Error: 'comments' array is required. Got args: {}"
    }]
  }
}

The process exited 0. The manifest's terminal_state is complete, all three selected files are completed, and failed, reused, and waived are empty. The run has two final findings. No warnings field is present. Tool failure is available in the JSON counters and stderr, but not reflected in the top-level completion or coverage outcome.

Our review operator interpreted complete coverage and exit0 as sufficient despite the tool error. The supervising admission check caught the raw error and blocked the delivery. This makes the distinction between completed execution and successfully submitted review findings important for automated consumers.

Requested clarification or change

Please document whether complete intentionally permits rejected comment submissions, and how a machine consumer should establish that such a failure was recovered without losing a finding. An explicit strict-admission outcome or a reconciled failure record would help distinguish recovered exploration errors from potentially lost comment submissions. This report does not assume every recoverable tool error must terminate a review.

Related grouped-path observation

A separate run of the same range, 04eaf880-f652-4df1-a827-77505ac79435, completed 3/3 with zero tool failures and empty stderr, but emitted five JSON comment_refiled warnings:

comment filed against <fileA>,<fileB>,<fileC> describes code in <fileB>; re-filed

That run had two code_comment calls and seven findings. Please clarify the contract for grouped-path submissions and whether automatic re-filing can provide explicit attribution proof rather than leaving consumers to infer it from warnings.

Evidence limits

Both are retained real-run observations, not a deterministic reproduction. We have OCR result JSON, stderr, exit markers and manifests, but not the raw provider tool-call payload. We cannot establish whether {} came from the model or OCR argument parsing, and are not claiming a parsing root cause. File names and local paths above are redacted. No credentials, reviewed source, or speculative source defects are included.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions