Skip to content

Commit fd60067

Browse files
lesnik512claude
andcommitted
feat(retry): give up when Retry-After exceeds max_delay
Previously when `respect_retry_after=True` and the server sent `Retry-After: 120` while the client had `max_delay=5`, AsyncRetry/Retry silently clamped to 5s and retried — almost certainly hitting the same 503/429 and burning an attempt. The option name implied the header was honored. New behavior: when the header value exceeds max_delay, re-raise the last observed StatusError with a PEP 678 note `httpware: Retry-After (Ns) exceeded max_delay (Ms); giving up`. Same treatment in both AsyncRetry and Retry. `respect_retry_after=False` and raising `max_delay` to accommodate the header are the opt-outs. Replaces the existing `test_retry_after_capped_at_max_delay` (which pinned the old silent-cap) with two tests: the give-up assertion at Retry-After > max_delay, and the boundary case at Retry-After == max_delay which still retries. Sync mirrors added. Also updates `test_retry_after_seconds_honored` (sync) which implicitly tested clamping and now tests the within-budget path instead. docs/resilience.md's respect_retry_after row describes the new give-up behavior. Closes audit Low finding (retry.py:189 + :320) from planning/audit/2026-06-07-deep-audit.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 04bbdc8 commit fd60067

4 files changed

Lines changed: 98 additions & 21 deletions

File tree

‎docs/resilience.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ from httpware.middleware.resilience import AsyncRetry
2121
| `max_delay` | `5.0` (s) | Ceiling for backoff. |
2222
| `retry_status_codes` | `frozenset({408, 429, 502, 503, 504})` | Status codes considered retryable. |
2323
| `retry_methods` | `frozenset({"GET", "HEAD", "OPTIONS", "PUT", "DELETE"})` | Idempotent methods only by default. POST excluded; pass an explicit frozenset including `"POST"` to retry it. |
24-
| `respect_retry_after` | `True` | When the response carries a `Retry-After` header on a retryable status, sleep for the header value (clamped to `max_delay`) instead of the jittered backoff. |
24+
| `respect_retry_after` | `True` | When the response carries a `Retry-After` header on a retryable status, sleep for the header value instead of the jittered backoff. If the header value exceeds `max_delay`, AsyncRetry gives up and re-raises the underlying `StatusError` with a PEP 678 note `httpware: Retry-After (Ns) exceeded max_delay (Ms); giving up`. Set `max_delay` higher (or `respect_retry_after=False`) to opt out. |
2525
| `budget` | `RetryBudget()` (default-configured) | The token bucket. Pass a shared `RetryBudget` instance to apply one budget across multiple clients. |
2626

2727
For a whole-attempt wall-clock bound, use `httpx2.Timeout` on the client or
@@ -185,7 +185,7 @@ from httpware.middleware.resilience import Retry
185185
| `max_delay` | `5.0` (s) | Ceiling for backoff. |
186186
| `retry_status_codes` | `frozenset({408, 429, 502, 503, 504})` | Status codes considered retryable. |
187187
| `retry_methods` | `frozenset({"GET", "HEAD", "OPTIONS", "PUT", "DELETE"})` | Idempotent methods only by default. POST excluded; pass an explicit frozenset including `"POST"` to retry it. |
188-
| `respect_retry_after` | `True` | When the response carries a `Retry-After` header on a retryable status, sleep for the header value (clamped to `max_delay`) instead of the jittered backoff. |
188+
| `respect_retry_after` | `True` | When the response carries a `Retry-After` header on a retryable status, sleep for the header value instead of the jittered backoff. If the header value exceeds `max_delay`, Retry gives up and re-raises the underlying `StatusError` with a PEP 678 note `httpware: Retry-After (Ns) exceeded max_delay (Ms); giving up`. Set `max_delay` higher (or `respect_retry_after=False`) to opt out. |
189189
| `budget` | `RetryBudget()` (default-configured) | The token bucket. Pass a shared `RetryBudget` instance to apply one budget across multiple clients — sync, async, or both. |
190190

191191
`Retry` uses `time.sleep` between attempts. `Retry-After`, streaming-body refusal, exhaustion behavior, and `RetryBudgetExhaustedError` semantics are identical to `AsyncRetry`.

‎src/httpware/middleware/resilience/retry.py‎

Lines changed: 27 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,9 @@
4747

4848
_MAX_ATTEMPTS_INVALID = "max_attempts must be >= 1"
4949
_STREAMING_BODY_REFUSAL_NOTE = "httpware: not retrying — request body is a stream that cannot replay across attempts"
50+
_RETRY_AFTER_EXCEEDS_MAX_DELAY_NOTE = (
51+
"httpware: Retry-After ({retry_after}s) exceeded max_delay ({max_delay}s); giving up"
52+
)
5053

5154
_LOGGER = logging.getLogger("httpware.retry")
5255

@@ -185,8 +188,19 @@ async def __call__(self, request: httpx2.Request, next: AsyncNext) -> httpx2.Res
185188
if header is not None:
186189
retry_after = _parse_retry_after(header)
187190

191+
if retry_after is not None and retry_after > self.max_delay:
192+
if last_exc is None: # pragma: no cover — retry_after requires last_response which requires last_exc
193+
msg = "AsyncRetry: retry_after path reached with no last_exc"
194+
raise AssertionError(msg)
195+
last_exc.add_note(
196+
_RETRY_AFTER_EXCEEDS_MAX_DELAY_NOTE.format(
197+
retry_after=retry_after,
198+
max_delay=self.max_delay,
199+
),
200+
)
201+
raise last_exc
188202
if retry_after is not None:
189-
delay = min(retry_after, self.max_delay)
203+
delay = retry_after
190204
else:
191205
delay = full_jitter_delay(
192206
attempt,
@@ -316,8 +330,19 @@ def __call__(self, request: httpx2.Request, next: Next) -> httpx2.Response: # n
316330
if header is not None:
317331
retry_after = _parse_retry_after(header)
318332

333+
if retry_after is not None and retry_after > self.max_delay:
334+
if last_exc is None: # pragma: no cover — retry_after requires last_response which requires last_exc
335+
msg = "Retry: retry_after path reached with no last_exc"
336+
raise AssertionError(msg)
337+
last_exc.add_note(
338+
_RETRY_AFTER_EXCEEDS_MAX_DELAY_NOTE.format(
339+
retry_after=retry_after,
340+
max_delay=self.max_delay,
341+
),
342+
)
343+
raise last_exc
319344
if retry_after is not None:
320-
delay = min(retry_after, self.max_delay)
345+
delay = retry_after
321346
else:
322347
delay = full_jitter_delay(
323348
attempt,

‎tests/test_retry.py‎

Lines changed: 25 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -286,17 +286,39 @@ async def test_retry_after_http_date_overrides_backoff() -> None:
286286
assert 2.0 <= sleeper.calls[0] <= 4.0 # noqa: PLR2004 — ~3 seconds, with clock-skew tolerance
287287

288288

289-
async def test_retry_after_capped_at_max_delay() -> None:
289+
async def test_retry_after_exceeding_max_delay_raises_with_note() -> None:
290+
"""When Retry-After > max_delay, give up — don't silently retry after a too-short delay."""
290291
sleeper = _SleepRecorder()
291292
handler = _ResponseSequenceWithHeaders(
292293
[
293294
(HTTPStatus.SERVICE_UNAVAILABLE, {"Retry-After": "9999"}),
294-
(HTTPStatus.OK, {}),
295+
(HTTPStatus.OK, {}), # never reached
295296
]
296297
)
297298
client = _client(handler, retry=AsyncRetry(_sleep=sleeper, base_delay=0.01, max_delay=2.5))
299+
with pytest.raises(ServiceUnavailableError) as exc_info:
300+
await client.get("https://example.test/x")
301+
notes = getattr(exc_info.value, "__notes__", [])
302+
assert any("Retry-After" in n and "exceeded max_delay" in n for n in notes), (
303+
f"expected give-up note in __notes__; got {notes!r}"
304+
)
305+
assert sleeper.calls == [] # no sleep before give-up
306+
assert handler.calls == 1 # initial attempt only; no retry
307+
308+
309+
async def test_retry_after_equal_to_max_delay_still_retries() -> None:
310+
"""Boundary: Retry-After exactly equal to max_delay sleeps and retries — does not give up."""
311+
sleeper = _SleepRecorder()
312+
handler = _ResponseSequenceWithHeaders(
313+
[
314+
(HTTPStatus.SERVICE_UNAVAILABLE, {"Retry-After": "2"}),
315+
(HTTPStatus.OK, {}),
316+
]
317+
)
318+
client = _client(handler, retry=AsyncRetry(_sleep=sleeper, base_delay=0.01, max_delay=2.0))
298319
await client.get("https://example.test/x")
299-
assert sleeper.calls == [2.5]
320+
assert sleeper.calls == [2.0]
321+
assert handler.calls == 2 # noqa: PLR2004 — initial attempt + 1 retry
300322

301323

302324
async def test_malformed_retry_after_falls_back_to_backoff() -> None:

‎tests/test_retry_sync.py‎

Lines changed: 44 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@
1616

1717
from httpware import Client, NotFoundError, ServiceUnavailableError
1818
from httpware._internal.status import STREAMING_BODY_MARKER, _is_streaming_body_sync
19-
from httpware.errors import NetworkError, RetryBudgetExhaustedError, StatusError, TransportError
19+
from httpware.errors import NetworkError, RetryBudgetExhaustedError, TransportError
2020
from httpware.errors import TimeoutError as HttpwareTimeoutError
2121
from httpware.middleware.resilience.budget import RetryBudget
2222
from httpware.middleware.resilience.retry import (
@@ -275,20 +275,18 @@ def handler(request: httpx2.Request) -> httpx2.Response: # noqa: ARG001
275275

276276

277277
def test_retry_after_seconds_honored() -> None:
278+
"""When Retry-After fits within max_delay, it overrides the jittered backoff."""
278279
sleeper = _SleepRecorder()
279-
280-
def handler(request: httpx2.Request) -> httpx2.Response:
281-
return httpx2.Response(
282-
HTTPStatus.TOO_MANY_REQUESTS,
283-
request=request,
284-
headers={"Retry-After": "1"},
285-
)
286-
287-
client = _client(handler, retry=Retry(_sleep=sleeper, base_delay=0.01, max_delay=0.5, max_attempts=2))
288-
with pytest.raises(StatusError):
289-
client.get("https://example.test/x")
290-
# Retry-After=1 clamped to max_delay=0.5
291-
assert sleeper.calls == [0.5]
280+
handler = _ResponseSequenceWithHeaders(
281+
[
282+
(HTTPStatus.TOO_MANY_REQUESTS, {"Retry-After": "1"}),
283+
(HTTPStatus.OK, {}),
284+
]
285+
)
286+
client = _client(handler, retry=Retry(_sleep=sleeper, base_delay=0.01, max_delay=5.0, max_attempts=2))
287+
response = client.get("https://example.test/x")
288+
assert response.status_code == HTTPStatus.OK
289+
assert sleeper.calls == [1.0]
292290

293291

294292
def test_retry_after_http_date_overrides_backoff() -> None:
@@ -504,3 +502,35 @@ def test_deposit_fires_once_per_call_not_per_attempt() -> None:
504502
assert response.status_code == HTTPStatus.OK
505503
assert handler.calls == 3 # noqa: PLR2004 — "3" is intentional literal in test (max_attempts=3)
506504
assert budget.deposit_calls == 1, f"expected 1 deposit per request, got {budget.deposit_calls}"
505+
506+
507+
def test_retry_after_exceeding_max_delay_raises_with_note() -> None:
508+
"""When Retry-After > max_delay, give up — don't silently retry after a too-short delay."""
509+
sleeper = _SleepRecorder()
510+
handler = _ResponseSequenceWithHeaders(
511+
[
512+
(HTTPStatus.SERVICE_UNAVAILABLE, {"Retry-After": "9999"}),
513+
(HTTPStatus.OK, {}),
514+
]
515+
)
516+
client = _client(handler, retry=Retry(_sleep=sleeper, base_delay=0.01, max_delay=2.5))
517+
with pytest.raises(ServiceUnavailableError) as exc_info:
518+
client.get("https://example.test/x")
519+
notes = getattr(exc_info.value, "__notes__", [])
520+
assert any("Retry-After" in n and "exceeded max_delay" in n for n in notes)
521+
assert sleeper.calls == []
522+
assert handler.calls == 1
523+
524+
525+
def test_retry_after_equal_to_max_delay_still_retries() -> None:
526+
sleeper = _SleepRecorder()
527+
handler = _ResponseSequenceWithHeaders(
528+
[
529+
(HTTPStatus.SERVICE_UNAVAILABLE, {"Retry-After": "2"}),
530+
(HTTPStatus.OK, {}),
531+
]
532+
)
533+
client = _client(handler, retry=Retry(_sleep=sleeper, base_delay=0.01, max_delay=2.0))
534+
client.get("https://example.test/x")
535+
assert sleeper.calls == [2.0]
536+
assert handler.calls == 2 # noqa: PLR2004 — initial attempt + 1 retry

0 commit comments

Comments
 (0)