Skip to content

fix(sse): reconnect when a stream drops instead of reporting failure - #291

Merged
jfrench9 merged 1 commit into
mainfrom
bugfix/sse-reconnect-on-drop
Nov 1, 2026
Merged

jfrench9 merged 1 commit into
mainfrom
bugfix/sse-reconnect-on-drop

Conversation

@jfrench9

@jfrench9 jfrench9 commented Nov 1, 2026

Copy link
Copy Markdown
Member

Summary

A dropped operation stream ended monitor_operation as FAILED while the operation kept running, so a caller could retry a write that was still in progress. The client now reconnects and returns the operation's real result.

Changes

  • SSEClient and AsyncSSEClient:
    • A mid-stream drop, a clean close before the operation finishes, and a 429 or 5xx now go through the retry path. A deploy can cause any of these.
    • Other refusals, such as a 401, still emit error without retrying.
    • stream_end still stops without reconnecting.
  • Retry budget: it now resets only when a real event arrives. The server sends connected on every open, so resetting on open or on that event would let a stream that opens and ends retry forever.
  • OperatorClient: it sets max_retries=0. Its /status polling already covers a lost stream, and retrying first would delay that by about 31 seconds.
  • Connections: a reconnect now releases the previous connection first. The async client also honors a close() during backoff.
  • clients/README.md: describes the new behavior.

Worth a close look: a stream that keeps closing with no result used to raise ended before a terminal event straight away. It now retries, and when the retries run out it returns FAILED with the error, the same as other lost connections. A stream_end with no result, which is what a paused operation sends, still raises as before.

Compatibility

Fix only: no export or signature changes. Runtime behavior changes in the case above, and in max_retries_exceeded now firing for drops that used to emit error. Both correct false failures, so this fits a patch or minor, not a major.

Testing

  • The pre-commit checks passed, including tests (747 passed, 17 skipped), with new coverage for drops, clean closes, 503s, refused credentials, stream_end, the retry budget and the async client.
  • Reproduced before and after against a local mock server that drops the first connection: failed after 1 connection before, completed after 2 connections with the fix.
  • Not run against a live API or through a real deploy.

🤖 Generated with Claude Code

A dropped stream ended monitor_operation as FAILED while the operation ran on, and
a caller could then retry a write still in progress. A drop, a clean close before
the end, and a 429 or 5xx now reconnect. The retry budget resets on real events
only, since the server sends connected on every open.

OperatorClient keeps no stream retries, because its /status polling already covers
a lost stream.
@jfrench9
jfrench9 merged commit c8d81ad into main Nov 1, 2026
6 checks passed
@jfrench9
jfrench9 deleted the bugfix/sse-reconnect-on-drop branch November 1, 2026 20:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant