fix(dspy): record agent finish reasons for ReAct and ReActV2 by Thanatos9404 · Pull Request #885 · open-telemetry/opentelemetry-python-genai

Thanatos9404 · 2026-10-12T13:04:47Z

Description

Fixes #874. dspy.ReAct and dspy.ReActV2 agent spans now set gen_ai.response.finish_reasons from how the run ended, whether or not content capture is on. The value is stop when the agent calls finish/submit, max_iters when it runs out of iterations, and error when its loop ends early (invalid tool selection, parse failure, context overflow, or no tool calls). The output message finish_reason now uses the same value instead of always stop.

Known gaps:

max_iters is a custom value, not one of the semconv finish reasons. It follows DSPy's parameter name and the max_steps value smolagents uses.
For a forced submit, ReActV2 only reports forced_submit. The run counts as max_iters when the number of turns recorded in history reaches the limit. Other forced submits become error, even after a plain-text reply with no tool calls, because DSPy 3.4 doesn't say which cause it was.
The conformance exceptions named in the issue are only in Switch conformance tests to common runner and align scenario coverage #623. Policies on main don't flag this attribute on internal agent spans, so whichever PR merges second should remove the react/react_v2 finish_reasons entries.

Type of change

Bug fix (non-breaking change which fixes an issue)

How has this been tested?

tox -e py310-test-instrumentation-genai-dspy-oldest (DSPy 3.3.0): 68 passed, 5 skipped (ReActV2 async needs DSPy 3.4)
tox -e py314-test-instrumentation-genai-dspy-latest (DSPy 3.4.0): 73 passed
tox -e py310-test-instrumentation-genai-dspy-latest: 73 passed
tox -e precommit and tox -e typecheck (Python 3.14): pass
New tests on main: 23 of 25 fail on latest, and 20 of 20 fail on oldest. The two that pass on latest cover the DSPy 3.4 forced-submit exception path, which this change leaves as is.
-conformance was not run locally because Weaver isn't installed and the env's git dependencies hit Windows path limits. Running the react/react_v2 scenarios in-process now gives ("stop",); on main the attribute is missing.

Checklist

Followed the style guidelines of this project
Changelog updated if the change requires an entry
Unit tests added

AI-assisted (Claude Opus 5.5), disclosed with an Assisted-by: commit trailer; it was checked with a separate AI review pass and the gates above. I directed this work and will follow up on review.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

linux-foundation-easycla · 2026-10-12T13:04:57Z

The committers listed above are authorized under a signed CLA.

✅ login: Thanatos9404 / name: Yashvardhan Thanvi (4433f81, f5c9905)

Assisted-by: Claude Opus 5.5

opentelemetry-pr-dashboard · 2026-10-12T13:05:44Z

Pull request dashboard status

Waiting on reviewers · refreshed 2026-10-16 15:46 UTC

Review the latest changes.

Status above doesn't look right?

Just replied or pushed? Anything around or after the refresh time above may not be picked up yet — give it a few minutes.
Anything look wrong? Report it with what you expected; it helps us improve the dashboard.

DylanRussell · 2026-10-13T13:21:38Z

+    return None
+
+
+class _AgentRun(ABC):


camn we pass in functions isntead of using classes ? I feel like that would be simpler.. also can we improve the typing of these functions so the arguments aren't of type object but something more specific..

DylanRussell · 2026-10-13T13:26:27Z

+    if isinstance(messages, Sequence) and not isinstance(
+        messages, (str, bytes)
+    ):
+        return len(cast(Sequence[object], messages))


i dont think this cast statement is needed.

lmolkova · 2026-10-16T02:53:33Z

            role="assistant",
            parts=[TextPart(content=output_str)],
-            finish_reason="stop",
+            finish_reason=finish_reason,


OutputMessage.finish_reason is deprecated in util in favor of gen_ai.response.finish_reasons, which this PR now sets, so we could drop it here (and the matching assertion in assert_agent_finish_reason).

Suggested change

finish_reason=finish_reason,

lmolkova · 2026-10-16T02:57:09Z

+def _react_finish_reason(
+    result: Prediction, max_iters: int | None, input_history_length: int
+) -> str | None:
+    trajectory: Mapping[str, Any] | None = getattr(result, "trajectory", None)
+    if not isinstance(trajectory, Mapping):
+        return None
+    tool_names = [
+        value
+        for key, value in trajectory.items()
+        if key.startswith("tool_name_")
+    ]
+    if tool_names and tool_names[-1] == "finish":
+        return "stop"
+    if max_iters is None:
+        return None
+    # Truncation drops the oldest trajectory steps but keeps the newest.
+    if max_iters <= 0 or f"tool_name_{max_iters - 1}" in trajectory:
+        return "max_iters"
+    return "error"


Telling max_iters apart by checking whether the step count reached the limit is a weak signal. It depends on how DSPy numbers and truncates steps, and it guesses at a cause DSPy doesn't report. Not reaching finish is an abnormal end either way, which semconv covers with error:

Suggested change

def _react_finish_reason(

result: Prediction, max_iters: int | None, input_history_length: int

) -> str | None:

trajectory: Mapping[str, Any] | None = getattr(result, "trajectory", None)

if not isinstance(trajectory, Mapping):

return None

tool_names = [

value

for key, value in trajectory.items()

if key.startswith("tool_name_")

]

if tool_names and tool_names[-1] == "finish":

return "stop"

if max_iters is None:

return None

# Truncation drops the oldest trajectory steps but keeps the newest.

if max_iters <= 0 or f"tool_name_{max_iters - 1}" in trajectory:

return "max_iters"

return "error"

def _react_finish_reason(result: Prediction) -> str:

trajectory: object = getattr(result, "trajectory", None)

if isinstance(trajectory, Mapping):

tool_names = [

value

for key, value in trajectory.items()

if key.startswith("tool_name_")

]

if tool_names and tool_names[-1] == "finish":

return "stop"

return "error"

lmolkova · 2026-10-16T03:08:56Z

+def _react_v2_finish_reason(
+    result: Prediction, max_iters: int | None, input_history_length: int
+) -> str | None:
+    termination_reason: str | None = getattr(
+        result, "termination_reason", None
+    )
+    if termination_reason == "submit":
+        return "stop"
+    if termination_reason == "forced_submit":
+        history_length = _count_history_messages(
+            getattr(result, "history", None)
+        )
+        if max_iters is None or history_length is None:
+            return None
+        # Each completed turn and the forced submit add one history event.
+        turns = history_length - input_history_length - 1
+        return "max_iters" if turns >= max_iters else "error"
+    # DSPy < 3.4 returns the break reason when the forced submit fails.
+    if termination_reason == "max_iters":
+        return "max_iters"
+    if termination_reason in (
+        "parse_error",
+        "context_window_exceeded",
+        "empty_tool_calls",
+    ):
+        return "error"
+    return None


Counting history events to tell max_iters from other forced submits depends on DSPy internals. A forced submit means the agent didn't finish on its own, which semconv's error ("ended abnormally ... inferred by the instrumentation") already covers. How about mapping the known values and passing the rest through, like the anthropic and bedrock instrumentations do?

Suggested change

def _react_v2_finish_reason(

result: Prediction, max_iters: int | None, input_history_length: int

) -> str | None:

termination_reason: str | None = getattr(

result, "termination_reason", None

)

if termination_reason == "submit":

return "stop"

if termination_reason == "forced_submit":

history_length = _count_history_messages(

getattr(result, "history", None)

)

if max_iters is None or history_length is None:

return None

# Each completed turn and the forced submit add one history event.

turns = history_length - input_history_length - 1

return "max_iters" if turns >= max_iters else "error"

# DSPy < 3.4 returns the break reason when the forced submit fails.

if termination_reason == "max_iters":

return "max_iters"

if termination_reason in (

"parse_error",

"context_window_exceeded",

"empty_tool_calls",

):

return "error"

return None

def _react_v2_finish_reason(result: Prediction) -> str | None:

termination_reason: object = getattr(result, "termination_reason", None)

if termination_reason == "submit":

return "stop"

if termination_reason in ("forced_submit", "parse_error"):

return "error"

return termination_reason if isinstance(termination_reason, str) else None

This also removes _count_history_messages, _get_max_iters, and the extra arguments on _FinishReasonFn.

ReAct and ReActV2 invoke_agent spans never set gen_ai.response.finish_reasons, and the output message always claimed "stop" regardless of how the run ended. Derive the finish reason from the returned prediction, independently of content capture: "stop" when the agent calls finish/submit, "max_iters" when it uses up its iteration budget, and "error" when the loop ends early. The output message finish_reason uses the same value. Fixes open-telemetry#874 Assisted-by: Claude Opus 5.5

Assisted-by: Claude Opus 5.5

Replace the _AgentRun, _ReActRun and _ReActV2Run classes with the module-level functions _react_finish_reason and _react_v2_finish_reason, which _react_forward and _react_aforward now take as an argument. The wrappers read max_iters and the caller's history length before calling the wrapped method, since ReActV2 appends to a caller-provided History in place, and pass both to the function with the returned Prediction. Type the helpers with the DSPy types (Prediction, History, Module and Mapping[str, Any]) instead of object, and drop the cast() calls by narrowing the history messages with isinstance(..., Sized). Assisted-by: Claude Opus 5.5

OutputMessage.finish_reason is deprecated in util in favor of gen_ai.response.finish_reasons, which the agent span already sets. Stop setting it on the ReAct and ReActV2 output message, and have assert_agent_finish_reason check that it is unset instead of comparing it with the span's finish reason. Assisted-by: Claude Opus 5.5

Stop inferring max_iters from step counts and history length, and map only what DSPy reports: - ReAct: stop when the last tool is finish, error otherwise. - ReActV2: submit maps to stop, forced_submit and parse_error map to error, other termination reasons pass through, and anything else gives no finish reason. This removes _count_history_messages, _get_max_iters and the extra _FinishReasonFn arguments. Drop the tests for the max_iters inference (trajectory truncation and caller history), add unit tests for both mappings, and update the changelog fragment to match. Assisted-by: Claude Opus 5.5

Thanatos9404 requested a review from a team as a code owner October 12, 2026 13:04

Copilot AI balanced review requested due to automatic review settings October 12, 2026 13:04

github-project-automation Bot added this to GenAI Semantic Conventions and Instrumentation libraries Oct 12, 2026

Copilot AI reviewed Oct 12, 2026

Thanatos9404 added a commit to Thanatos9404/opentelemetry-python-genai that referenced this pull request Oct 12, 2026

Add changelog fragment for open-telemetry#885

4433f81

Assisted-by: Claude Opus 5.5

opentelemetry-pr-dashboard Bot mentioned this pull request Oct 12, 2026

Pull Request Dashboard #171

Open

DylanRussell reviewed Oct 13, 2026

View reviewed changes

lmolkova reviewed Oct 16, 2026

View reviewed changes

Thanatos9404 added 5 commits October 16, 2026 09:37

Add changelog fragment for open-telemetry#885

027a3ac

Assisted-by: Claude Opus 5.5

Thanatos9404 force-pushed the fix/dspy-react-finish-reasons branch from 6a21a00 to 856a86d Compare October 16, 2026 04:45

Navigation Menu

Repository navigation

fix(dspy): record agent finish reasons for ReAct and ReActV2 - #885

fix(dspy): record agent finish reasons for ReAct and ReActV2#885
Thanatos9404 wants to merge 5 commits into
open-telemetry:mainfrom
Thanatos9404:fix/dspy-react-finish-reasons

Thanatos9404 commented Oct 12, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

linux-foundation-easycla Bot commented Oct 12, 2026 •

edited

Loading

Uh oh!

opentelemetry-pr-dashboard Bot commented Oct 12, 2026 •

edited

Loading

Uh oh!

DylanRussell Oct 13, 2026

Uh oh!

DylanRussell Oct 13, 2026

Uh oh!

lmolkova Oct 16, 2026

Uh oh!

lmolkova Oct 16, 2026

Uh oh!

lmolkova Oct 16, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

Thanatos9404 commented Oct 12, 2026

Description

Type of change

How has this been tested?

Checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

linux-foundation-easycla Bot commented Oct 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

opentelemetry-pr-dashboard Bot commented Oct 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull request dashboard status

Uh oh!

DylanRussell Oct 13, 2026

Choose a reason for hiding this comment

Uh oh!

DylanRussell Oct 13, 2026

Choose a reason for hiding this comment

Uh oh!

lmolkova Oct 16, 2026

Choose a reason for hiding this comment

Uh oh!

lmolkova Oct 16, 2026

Choose a reason for hiding this comment

Uh oh!

lmolkova Oct 16, 2026

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

linux-foundation-easycla Bot commented Oct 12, 2026 •

edited

Loading

opentelemetry-pr-dashboard Bot commented Oct 12, 2026 •

edited

Loading