Skip to content

fix(predict): keep signature output fields on ReActV2 terminal predic… - #10244

Closed
jayhemnani9910 wants to merge 2 commits into
stanfordnlp:mainfrom
jayhemnani9910:fix/reactv2-terminal-output-fields
Closed

jayhemnani9910 wants to merge 2 commits into
stanfordnlp:mainfrom
jayhemnani9910:fix/reactv2-terminal-output-fields

Conversation

@jayhemnani9910

Copy link
Copy Markdown
Contributor

…tions

When the ReActV2 loop ended without a successful submit, the three exits in _forced_submit returned a Prediction carrying only history and termination_reason, so a module declared question -> answer produced no answer and pred.answer raised AttributeError.

Under dspy.Evaluate the AttributeError was caught per example and scored as zero, so a run reported a metric instead of failing and the contract break read as a model quality problem.

Route those exits through _terminal_prediction, which defaults every declared output field to None and applies history and termination_reason last so they stay authoritative. Covers the max_iters, empty_tool_calls and failed-submit paths.

…tions

When the ReActV2 loop ended without a successful `submit`, the three exits in
`_forced_submit` returned a Prediction carrying only `history` and
`termination_reason`, so a module declared `question -> answer` produced no
`answer` and `pred.answer` raised AttributeError.

Under dspy.Evaluate the AttributeError was caught per example and scored as
zero, so a run reported a metric instead of failing and the contract break read
as a model quality problem.

Route those exits through `_terminal_prediction`, which defaults every declared
output field to None and applies `history` and `termination_reason` last so they
stay authoritative. Covers the max_iters, empty_tool_calls and failed-submit
paths.
Copilot AI lite review requested due to automatic review settings August 20, 2026 08:37

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a contract break in ReActV2 where terminal exits that didn’t produce a successful submit returned a Prediction missing declared signature output fields (e.g., answer), leading to AttributeError in downstream evaluation/metrics.

Changes:

  • Added _terminal_prediction() to construct a terminal Prediction that always includes every declared output field (defaulting to None) plus history and termination_reason.
  • Routed all non-success exits in _forced_submit() through _terminal_prediction() instead of returning a minimal Prediction.
  • Added unit tests covering max-iters, empty tool calls, failed submit, and multi-output-field signatures.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
dspy/predict/react_v2.py Adds _terminal_prediction() and uses it for all _forced_submit failure/early-exit returns so output fields are always present.
tests/predict/test_react_v2.py Adds regression tests ensuring terminal predictions still expose declared output fields (set to None).

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@greptile-apps

greptile-apps Bot commented Aug 20, 2026 •

Copy link
Copy Markdown
Contributor

Greptile Summary

ReActV2 now returns all declared output fields as None when a tool loop ends without a successful submission, while preserving internal history and termination metadata.

Confidence Score: 5/5

No blocking failure remains.

No accepted blocking findings remain.

T-Rex T-Rex Logs

What T-Rex did

  • Ran uv run --frozen python trex-artifacts/react-v2-reserved-output-fields-01-before.py from /home/user/repo; the command exited successfully and observed ValueError for both termination_reason and history when declaring them as outputs.
  • Ran the final verification run from /home/user/repo; the command exited 0 and the after-run log shows both reserved names were rejected, yielding a PASS.
  • Artifacts documenting the tests were prepared for reviewer inspection, including the focused runtime script and the final after-run log.

View all artifacts

T-Rex Ran code and verified through T-Rex

Reviews (2): Last reviewed commit: "Merge upstream/main into fix/reactv2-ter..." | Re-trigger Greptile

Comment thread dspy/predict/react_v2.py
Resolves the test-file conflict with stanfordnlp#9852 (reserved output field
names): both changes append new tests, so both are kept.
@jayhemnani9910

Copy link
Copy Markdown
Contributor Author

Closing, superseded by #10355.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants