Repository navigation
fix(predict): keep signature output fields on ReActV2 terminal predic… - #10244
jayhemnani9910 wants to merge 2 commits into
Conversation
…tions When the ReActV2 loop ended without a successful `submit`, the three exits in `_forced_submit` returned a Prediction carrying only `history` and `termination_reason`, so a module declared `question -> answer` produced no `answer` and `pred.answer` raised AttributeError. Under dspy.Evaluate the AttributeError was caught per example and scored as zero, so a run reported a metric instead of failing and the contract break read as a model quality problem. Route those exits through `_terminal_prediction`, which defaults every declared output field to None and applies `history` and `termination_reason` last so they stay authoritative. Covers the max_iters, empty_tool_calls and failed-submit paths.
There was a problem hiding this comment.
Pull request overview
This PR fixes a contract break in ReActV2 where terminal exits that didn’t produce a successful submit returned a Prediction missing declared signature output fields (e.g., answer), leading to AttributeError in downstream evaluation/metrics.
Changes:
- Added
_terminal_prediction()to construct a terminalPredictionthat always includes every declared output field (defaulting toNone) plushistoryandtermination_reason. - Routed all non-success exits in
_forced_submit()through_terminal_prediction()instead of returning a minimalPrediction. - Added unit tests covering max-iters, empty tool calls, failed submit, and multi-output-field signatures.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| dspy/predict/react_v2.py | Adds _terminal_prediction() and uses it for all _forced_submit failure/early-exit returns so output fields are always present. |
| tests/predict/test_react_v2.py | Adds regression tests ensuring terminal predictions still expose declared output fields (set to None). |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Greptile SummaryReActV2 now returns all declared output fields as Confidence Score: 5/5No blocking failure remains. No accepted blocking findings remain.
What T-Rex did
Reviews (2): Last reviewed commit: "Merge upstream/main into fix/reactv2-ter..." | Re-trigger Greptile |
Resolves the test-file conflict with stanfordnlp#9852 (reserved output field names): both changes append new tests, so both are kept.
|
Closing, superseded by #10355. |
…tions
When the ReActV2 loop ended without a successful
submit, the three exits in_forced_submitreturned a Prediction carrying onlyhistoryandtermination_reason, so a module declaredquestion -> answerproduced noanswerandpred.answerraised AttributeError.Under dspy.Evaluate the AttributeError was caught per example and scored as zero, so a run reported a metric instead of failing and the contract break read as a model quality problem.
Route those exits through
_terminal_prediction, which defaults every declared output field to None and applieshistoryandtermination_reasonlast so they stay authoritative. Covers the max_iters, empty_tool_calls and failed-submit paths.