Repository navigation
fix(prompts): stop multiplying retries in get_prompt and media uploads - #1974
niklassemmler wants to merge 4 commits into
Conversation
|
@claude review |
There was a problem hiding this comment.
Claude Code Review
No review was started: this request came from a bot account. Manual reviews can only be requested by someone with write access to this repository. Ask a maintainer to comment @claude review, or have your automation post the comment from a user account with write access.
Tip: disable this comment in your organization's Code Review settings.
|
v5 port: #1975 (same commits cherry-picked onto 🤖 Written by Claude (an AI agent) on behalf of Niklas |
|
@claude review |
There was a problem hiding this comment.
I re-reviewed the latest push and found no new bugs. The earlier concern about retries multiplying when transport errors mix with 5xx/429 looks addressed: the API client now runs with max_retries: 0 and call_with_retries holds the only retry budget. I'm still asking for a human look because this changes retry behavior that users will notice in get_prompt and media uploads.
What was reviewed:
call_with_retriesand the delay logic: Retry-After is tried first, then X-RateLimit-Reset, capped at 60 s; with neither header it uses jittered backoff capped at 10 s. Headers are read from bothApiErrorandHTTPStatusError.- In
get_prompt,max_retries=0and the timeout are passed together. Only 408/409/429/5xx responses and network errors are retried now; other exceptions are not. - In
MediaManager,max_retries - 1keeps the old meaning of total tries, and all three API calls (get_upload_urland bothpatchcalls) turn off the client's own retries.
Extended reasoning...
The change replaces backoff with a new helper, langfuse/_utils/retry.py, and rewires the retry paths in Langfuse.get_prompt and MediaManager._request_with_backoff. It touches no auth, crypto or data-exposure code. The helper imports private parsers (_parse_retry_after, _parse_x_ratelimit_reset) from the generated http_client, which ties it to the generated code. I deferred instead of approving because users will see the behavior change: max_retries=0 now means no retries, parse errors are no longer retried, and media uploads now retry 408/409. I also can't see whether the author's replies on the two Greptile inline threads settled them.
… retries get_prompt wrapped the fetch in a constant 1 s backoff on every exception, around the generated client's own 429/5xx retries. One 429 could become 9 requests, and 401/403/404 were retried too. HTTP retries are now left to the API client (exponential, Retry-After aware, bounded by max_retries); the outer loop only retries transport errors with exponential backoff. Refs LFE-17361 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
media.get_upload_url and media.patch go through the generated client, which retries 429/5xx itself. The outer backoff retried them again, turning one 429 into 9 requests. Refs LFE-17361 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A transport error raised during the API client's internal 429/5xx retries escaped with its retry count, and the outer TransportError retry then started a fresh inner budget, so mixed failures still sent up to (max_retries+1)^2 requests. get_prompt and MediaManager now call the API client with max_retries=0 and retry in one loop (call_with_retries) that covers network errors and 408/409/429/5xx and honors Retry-After / X-RateLimit-Reset. Refs LFE-17361 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ccd2ef6 to
62d34a6
Compare
|
@claude review |
Clients sharing a rate-limited key otherwise all retry at the reset instant. Retry-After stays exact. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@claude review |
Fixes LFE-17361 and LFE-18786 (langfuse/langfuse#20340:
get_prompt(max_retries=0)still retried inside the API client).Problem
get_promptwrapped the fetch inbackoff.on_exception(backoff.constant, Exception, max_tries=max_retries + 1)around the generated API client, which already retries 408/409/429/5xx itself. With the defaultmax_retries=2, one 429 became 9 requests, and 401/403/404 were retried 3 times. The docstring promised exponential backoff.The same nesting existed in
MediaManager._request_with_backoffformedia.get_upload_url/media.patch, where a 429 or 5xx also became 9 requests.Changes
langfuse/_utils/retry.py::call_with_retriesgives one retry budget for network errors (httpx.TransportError) and 408/409/429/5xx. It honorsRetry-After/X-RateLimit-Reset, capped at 60 s like the generated client; otherwise it uses exponential backoff with full jitter, capped at 10 s.get_promptandMediaManagercall the API client withmax_retries=0and retry only throughcall_with_retries. Worst case ismax_retries + 1requests for any mix of failures. Non-retryable 4xx errors fail immediately.get_promptdocstring now matches the code.Behaviour changes
get_prompt(max_retries=0)now really means no retries. Before, the inner client still retried 429/5xx twice.get_prompt(for example, parse errors) are no longer retried.Retry-After.Verification
httpx.MockTransportand count wire requests:max_retries=3→ 4 requests (8 on the previous commit)max_retries=0→ 1 request for both a 429 and a ConnectErrortests/unit/test_retry.pycovers the delay rules (Retry-After cap, X-RateLimit-Reset, headers onHTTPStatusError, backoff cap) and the 4xx no-retry rule.uv run --frozen pytest -n auto --dist worksteal tests/unit: 905 passed, 2 skippeduv run --frozen ruff check .anduv run --frozen mypy langfuse --no-error-summary: cleanruff format --check .flagstests/unit/test_media.py, which this PR doesn't touch; that problem is already onmain.🤖 Generated with Claude Code


Confidence Score: 5/5
The PR appears safe to merge; both previous findings are fixed and no new actionable issues were found.Summary
The PR gives prompt fetches and media requests one retry budget instead of nesting two retry loops.
Diagram
%%{init: {'theme': 'neutral'}}%% flowchart TD A[Prompt fetch or media request] --> B[Shared retry loop] B --> C[API call with generated retries disabled] C --> D{Result} D -->|Success| E[Return result] D -->|Retryable error and budget remains| F[Wait] F --> B D -->|Other error or budget exhausted| G[Raise error]Reviews (2) · Last reviewed commit: "fix(retry): jitter X-RateLimit-Reset del..." · Reviewed by Greptile