Skip to content

[NA] [SDK] fix: retry transient S3 attachment failures - #8136

Open
harsh181018 wants to merge 2 commits into
comet-ml:mainfrom
harsh181018:harsh/issue-8135-retry-s3-transient-errors
Open

[NA] [SDK] fix: retry transient S3 attachment failures#8136
harsh181018 wants to merge 2 commits into
comet-ml:mainfrom
harsh181018:harsh/issue-8135-retry-s3-transient-errors

Conversation

@harsh181018

@harsh181018 harsh181018 commented Sep 3, 2026

Copy link
Copy Markdown

Details

Retry transient S3 multipart attachment failures caused by remote protocol disconnects and HTTP 502, 503, and 504 responses. The S3 retry path now uses bounded jittered exponential backoff and re-raises the original failure after retry exhaustion, matching the resilience already present in the regular REST client.

Change checklist

  • User facing
  • Documentation update

Issues

AI-WATERMARK

AI-WATERMARK: yes

  • If yes:
    • Tools: OpenAI Codex
    • Model(s): GPT-5
    • Scope: Investigation, implementation, unit tests, and PR drafting
    • Human verification: Pending; this draft PR is ready for author and maintainer review.

Testing

  • PATH="$PWD/sdks/python/venv/bin:$PATH" make precommit
  • cd sdks/python && venv/bin/python -m pytest -q tests/unit/test_s3_httpx_client.py tests/unit/file_upload/s3_multipart_upload/test_s3_file_uploader.py
  • Validated successful retry after httpx.RemoteProtocolError and HTTP 500/502/503/504 responses.
  • Validated that HTTP 403 is not retried.
  • Environment: macOS, Python 3.13.11.
  • End-to-end tests were not run because this is an isolated Python SDK retry-policy change covered by unit tests.

Documentation

No documentation changes are needed; this updates internal retry behavior without changing the public API.

@github-actions github-actions Bot added python Pull requests that update Python code tests Including test files, or tests related like configuration. Python SDK 🟡 size/M labels Sep 3, 2026
@harsh181018
harsh181018 marked this pull request as ready for review September 3, 2026 17:31
@harsh181018
harsh181018 requested a review from a team as a code owner September 3, 2026 17:31
Comment thread sdks/python/src/opik/s3_httpx_client.py Outdated
Comment thread sdks/python/src/opik/s3_httpx_client.py Outdated
Comment thread sdks/python/src/opik/s3_httpx_client.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Python SDK python Pull requests that update Python code 🟡 size/M tests Including test files, or tests related like configuration.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python SDK: retry transient S3 attachment upload disconnects and gateway errors

1 participant