Skip to content

Commit 6a7f6d1

Browse files
lesnik512claude
andauthored
fix: deep-audit correctness + public-API findings (#64)
* docs(planning): audit-correctness change bundle * fix: catch OverflowError in full_jitter_delay for attempt_index >= 1024 2.0 ** attempt_index raises OverflowError in Python for attempt_index >= 1024 rather than saturating to math.inf as the old docstring claimed. Catch the OverflowError and clamp the ceiling directly to max_delay, which is the correct clamped result. Tests added to pin both the no-raise and the exact max_delay clamp behaviours. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(retry): reorder Retry-After give-up check before budget withdrawal When a server's Retry-After header exceeds max_delay, the middleware previously consumed a budget token before giving up, allowing a Retry-After flood to silently drain shared-budget capacity. Move the retry_after > max_delay guard before try_withdraw() in both AsyncRetry and Retry so the token is only spent when a real retry will happen. Adds test_retry_after_exceeds_max_delay_does_not_consume_budget_token (async) and …_sync (sync) to pin the invariant. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: strip userinfo prefix from raw netloc to prevent triple-slash form For URLs like `http://user:pass@/path` (credentials but no hostname), the previous reconstruction from `parts.hostname` yielded an empty netloc, causing `urlunsplit` to emit `http:///path`. Fix by slicing the `user:pass@` prefix directly off `parts.netloc`; when the remainder is empty (no real host), reconstruct without an authority section to produce `http:/path` instead. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(decoders): guard _contains_custom_type against NameError when msgspec absent When msgspec is not installed, `_contains_custom_type` would raise `NameError: name 'msgspec' is not defined` rather than the friendly `ImportError`. Added an early guard that raises `ImportError(MISSING_DEPENDENCY_MESSAGE)` when `import_checker.is_msgspec_installed` is False, matching the pattern used in `MsgspecDecoder.__init__`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(resilience): qualify RetryBudget asyncio-safe claim with blocking caveat The module docstring said "asyncio-safe" without noting that a sync thread holding the threading.Lock can briefly block the event-loop thread. Added clarification that the critical section is intentionally tiny to bound latency, while preserving the core safety guarantee (no torn state). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: detect sync iterables as non-replayable in _is_streaming_body_async Factor `_is_replayable_type` shared helper so both async and sync detectors use the same exclusion list. `_is_streaming_body_async` now returns True when the body has `__iter__` and is not a replayable type (bytes/bytearray/memoryview/str/dict/list/tuple), closing the gap where a sync generator passed to AsyncClient bypassed the streaming-body marker and the AsyncRetry replay guard. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(middleware): add __all__ to middleware/__init__.py to prevent star-import leakage Without __all__, `from httpware.middleware import *` would expose Awaitable, Callable, Protocol, TypeAlias, runtime_checkable, httpx2, chain, and resilience. Added explicit __all__ listing exactly the 10 public names (protocols + phase decorators). Tests added to test_public_api.py to guard this contract. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(retry): catch OverflowError in _parse_retry_after for huge integers float(int("9" * 400)) raises OverflowError (not ValueError) in Python, crashing the retry loop when a server sends a pathologically large Retry-After integer string. Widen the except clause to (ValueError, OverflowError) so oversized values fall back to jitter backoff rather than propagating unhandled. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(redaction): unify triple-slash avoidance; reconcile audit-fix gaps Follow-up reconciliation after the parallel correctness fixes: - _strip_userinfo's no-host fix only covered the no-query case; _mask_query's urlunsplit re-introduced http:///path when a sensitive query was masked. Factor a shared _reassemble() helper used by both, so the empty-authority triple-slash is avoided everywhere (normal URLs stay byte-identical). - hoist a function-body import in the msgspec test (ruff PLC0415) and use the project's # ty: ignore form. - cover the empty-netloc query/fragment branches, the @-in-path guard, and mark the never-iterated sync-generator test body no-cover. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent b9ba313 commit 6a7f6d1

14 files changed

Lines changed: 313 additions & 57 deletions

File tree

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
---
2+
status: draft
3+
date: 2026-06-14
4+
slug: audit-correctness
5+
supersedes: null
6+
superseded_by: null
7+
pr: null
8+
outcome: null
9+
---
10+
11+
# Change: Deep-audit correctness + public-API fixes
12+
13+
**Lane:** full-ish, but the spec is the [2026-06-14 deep audit](../../../audits/2026-06-14-deep-audit.md)
14+
(each finding carries root cause + suggested direction), so this bundle is a
15+
plan-light TDD sweep of the confirmed code-level findings.
16+
17+
## Goal
18+
19+
Close the confirmed **correctness** and **public-API** findings from the deep
20+
audit that #62/#63 did not already cover.
21+
22+
## Fixes
23+
24+
| # | Finding | File | Fix |
25+
|---|---------|------|-----|
26+
| L1 | RetryBudget token withdrawn before `Retry-After > max_delay` give-up | `retry.py` | evaluate the give-up guard *before* `budget.try_withdraw()`, in both `AsyncRetry` and `Retry` |
27+
| L3 | `_parse_retry_after` `OverflowError` on huge digit string crashes the loop | `retry.py` | broaden guard to `except (ValueError, OverflowError)` |
28+
| Nit1 | `full_jitter_delay` raises `OverflowError` at `attempt_index >= 1024` despite docstring claiming saturation | `_backoff.py` | clamp the exponent (or guard the `**`) so the documented `inf`-saturation/`max_delay` clamp actually holds; fix the docstring to match |
29+
| Nit3 | `_strip_userinfo` emits malformed `http:///path` when authority has creds but no host | `_internal/redaction.py` | when hostname is empty, preserve the original credential-free authority shape instead of a triple-slash URL |
30+
| Nit6 | `_contains_custom_type` uses bare `msgspec.*` at runtime → `NameError` if msgspec absent | `decoders/msgspec.py` | gate behind `is_msgspec_installed` (raise the friendly `ImportError`) so a direct call without the extra fails cleanly |
31+
| Nit2 | `_is_streaming_body_async` doesn't mark sync iterables non-replayable (async invariant rests on an undocumented httpx2 detail) | `_internal/status.py` | symmetrize the async detector to also treat sync iterables as streaming bodies, so the replay guard is explicit rather than relying on httpx2 |
32+
| L2 | `RetryBudget` docstring claims "asyncio-safe" without the blocking caveat | `resilience/budget.py` | qualify the docstring: the `threading.Lock` is correctness-safe but can briefly block the loop thread when shared sync↔async |
33+
| L8 | `middleware/__init__.py` has no `__all__`, leaking 9+ star-import symbols | `middleware/__init__.py` | add an explicit `__all__` of the ten public names (matches sibling subpackages) |
34+
35+
## Verification
36+
37+
- [ ] Each fix lands TDD-first (failing test → fix → green); reproducers from the audit drive the tests.
38+
- [ ] Sync/async parity for L1 (both `Retry` and `AsyncRetry`).
39+
- [ ] `just test` — 100% coverage; `just lint` — clean.
40+
- [ ] Grep guard: no new `httpx2._`.

‎src/httpware/_internal/redaction.py‎

Lines changed: 24 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -36,19 +36,35 @@
3636
_REDACTED = "REDACTED"
3737

3838

39+
def _reassemble(scheme: str, netloc: str, path: str, query: str, fragment: str) -> str:
40+
"""Like ``urlunsplit``, but avoid the spurious triple-slash for an empty authority.
41+
42+
``urlunsplit(("http", "", "/path", ...))`` yields ``http:///path`` for a
43+
netloc-using scheme. When userinfo stripping leaves no host (e.g.
44+
``http://user:pass@/path``) we want ``http:/path`` (scheme + path), not a
45+
triple-slash. With a non-empty netloc this delegates to ``urlunsplit``, so
46+
normal URLs are byte-identical.
47+
"""
48+
if netloc:
49+
return urlunsplit((scheme, netloc, path, query, fragment))
50+
tail = path
51+
if query:
52+
tail += "?" + query
53+
if fragment:
54+
tail += "#" + fragment
55+
return f"{scheme}:{tail}" if scheme else tail
56+
57+
3958
def _strip_userinfo(url: str) -> str:
4059
if "@" not in url or "://" not in url:
4160
return url
4261
parts = urlsplit(url)
4362
if parts.username is None and parts.password is None:
4463
return url
45-
hostname = parts.hostname or ""
46-
if ":" in hostname: # IPv6 literal — re-wrap in brackets
47-
hostname = f"[{hostname}]"
48-
netloc = hostname
49-
if parts.port is not None:
50-
netloc = f"{netloc}:{parts.port}"
51-
return urlunsplit((parts.scheme, netloc, parts.path, parts.query, parts.fragment))
64+
# Strip the "user:pass@" prefix from the raw netloc to preserve host:port
65+
# exactly (including IPv6 brackets), rather than reconstructing from parts.
66+
netloc = parts.netloc.split("@", 1)[1] if "@" in parts.netloc else parts.netloc
67+
return _reassemble(parts.scheme, netloc, parts.path, parts.query, parts.fragment)
5268

5369

5470
def _mask_component(component: str) -> tuple[str, bool]:
@@ -87,7 +103,7 @@ def _mask_query(url: str) -> str:
87103
if not changed:
88104
return url # common-path guard: nothing sensitive, leave bytes untouched
89105

90-
return urlunsplit((parts.scheme, parts.netloc, parts.path, new_query, new_fragment))
106+
return _reassemble(parts.scheme, parts.netloc, parts.path, new_query, new_fragment)
91107

92108

93109
def redact_url(url: str) -> str:

‎src/httpware/_internal/status.py‎

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -29,19 +29,24 @@ def _raise_on_status_error(response: httpx2.Response) -> None:
2929
raise exc_class(response)
3030

3131

32+
def _is_replayable_type(value: object) -> bool:
33+
"""Return True if value is a replayable type (safe to replay across retry attempts)."""
34+
return isinstance(value, (bytes, bytearray, memoryview, str, dict, list, tuple))
35+
36+
3237
def _is_streaming_body_async(value: object) -> bool:
33-
"""Return True if value is an async-iterable that cannot be safely replayed for retry."""
38+
"""Return True if value is a non-replayable body (async-iterable or sync non-replayable iterable)."""
3439
if value is None:
3540
return False
36-
if isinstance(value, (bytes, bytearray, memoryview, str, dict)):
41+
if _is_replayable_type(value):
3742
return False
38-
return hasattr(value, "__aiter__")
43+
return hasattr(value, "__aiter__") or hasattr(value, "__iter__")
3944

4045

4146
def _is_streaming_body_sync(value: object) -> bool:
4247
"""Return True if value is a sync iterable body that cannot be safely replayed for retry."""
4348
if value is None:
4449
return False
45-
if isinstance(value, (bytes, bytearray, memoryview, str, dict, list, tuple)):
50+
if _is_replayable_type(value):
4651
return False
4752
return hasattr(value, "__iter__")

‎src/httpware/decoders/msgspec.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,8 @@ def _contains_custom_type(info: "msgspec.inspect.Type") -> bool:
2626
makes the walk both correct (a Struct is a valid target) and safe against
2727
infinite recursion on self-referential struct definitions.
2828
"""
29+
if not import_checker.is_msgspec_installed:
30+
raise ImportError(MISSING_DEPENDENCY_MESSAGE)
2931
if isinstance(info, msgspec.inspect.CustomType):
3032
return True
3133
for name in dir(info):

‎src/httpware/middleware/__init__.py‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,19 @@
1111
import httpx2
1212

1313

14+
__all__ = [
15+
"AsyncMiddleware",
16+
"AsyncNext",
17+
"Middleware",
18+
"Next",
19+
"after_response",
20+
"async_after_response",
21+
"async_before_request",
22+
"async_on_error",
23+
"before_request",
24+
"on_error",
25+
]
26+
1427
AsyncNext: TypeAlias = Callable[[httpx2.Request], Awaitable[httpx2.Response]]
1528

1629

‎src/httpware/middleware/resilience/_backoff.py‎

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -17,10 +17,13 @@ def full_jitter_delay(
1717
1818
`attempt_index` is 0 for the first retry, 1 for the second, etc.
1919
20-
Uses ``2.0 **`` (float exponentiation) rather than ``2 **`` so that
21-
``attempt_index >= 1024`` saturates to ``math.inf`` and ``min`` clamps to
22-
``max_delay`` — ``2 ** 1024`` would raise ``OverflowError`` during the
23-
int→float conversion.
20+
For large ``attempt_index`` (>= 1024), ``2.0 ** attempt_index`` raises
21+
``OverflowError``. That is caught and the ceiling is clamped directly to
22+
``max_delay``, which is exactly what ``min`` would produce for an infinite
23+
exponentiation result.
2424
"""
25-
ceiling = min(max_delay, base_delay * (2.0**attempt_index))
25+
try:
26+
ceiling = min(max_delay, base_delay * (2.0**attempt_index))
27+
except OverflowError:
28+
ceiling = max_delay
2629
return _random_uniform(0.0, ceiling)

‎src/httpware/middleware/resilience/budget.py‎

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,10 +2,14 @@
22
33
See planning/specs/2026-06-05-retry-and-retry-budget-design.md for the contract.
44
5-
Thread-safe and asyncio-safe: all mutations go through a threading.Lock.
6-
A single RetryBudget instance is safe to share across threads, across
7-
coroutines on one event loop, and across (sync Client, AsyncClient) pairs
8-
in the same process.
5+
Thread-safe and asyncio-safe: all mutations go through a threading.Lock,
6+
which ensures no torn state across concurrent accesses. When a RetryBudget
7+
is shared between a sync Client (pool thread) and an AsyncClient (event-loop
8+
thread), a sync thread holding the lock can briefly block the loop thread's
9+
acquisition; the critical section (purge + append/compare) is intentionally
10+
tiny to bound this latency. Safe to share across threads, across coroutines
11+
on one event loop, and across (sync Client, AsyncClient) pairs in the same
12+
process.
913
"""
1014

1115
import math

‎src/httpware/middleware/resilience/retry.py‎

Lines changed: 37 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,7 @@ def _parse_retry_after(value: str) -> float | None:
5858
"""Parse a Retry-After header value. Returns None on malformed input."""
5959
try:
6060
return max(0.0, float(int(value))) # clamp: negative integers are malformed servers
61-
except ValueError:
61+
except (ValueError, OverflowError):
6262
pass
6363
try:
6464
parsed = email.utils.parsedate_to_datetime(value)
@@ -159,6 +159,24 @@ async def __call__(self, request: httpx2.Request, next: AsyncNext) -> httpx2.Res
159159
)
160160
raise last_exc
161161

162+
retry_after: float | None = None
163+
if self.respect_retry_after and last_response is not None:
164+
header = last_response.headers.get("Retry-After")
165+
if header is not None:
166+
retry_after = _parse_retry_after(header)
167+
168+
if retry_after is not None and retry_after > self.max_delay:
169+
if last_exc is None: # pragma: no cover — retry_after requires last_response which requires last_exc
170+
msg = "AsyncRetry: retry_after path reached with no last_exc"
171+
raise AssertionError(msg)
172+
last_exc.add_note(
173+
_RETRY_AFTER_EXCEEDS_MAX_DELAY_NOTE.format(
174+
retry_after=retry_after,
175+
max_delay=self.max_delay,
176+
),
177+
)
178+
raise last_exc
179+
162180
if not self.budget.try_withdraw():
163181
_emit_event(
164182
_LOGGER,
@@ -178,23 +196,6 @@ async def __call__(self, request: httpx2.Request, next: AsyncNext) -> httpx2.Res
178196
attempts=attempt + 1,
179197
) from last_exc
180198

181-
retry_after: float | None = None
182-
if self.respect_retry_after and last_response is not None:
183-
header = last_response.headers.get("Retry-After")
184-
if header is not None:
185-
retry_after = _parse_retry_after(header)
186-
187-
if retry_after is not None and retry_after > self.max_delay:
188-
if last_exc is None: # pragma: no cover — retry_after requires last_response which requires last_exc
189-
msg = "AsyncRetry: retry_after path reached with no last_exc"
190-
raise AssertionError(msg)
191-
last_exc.add_note(
192-
_RETRY_AFTER_EXCEEDS_MAX_DELAY_NOTE.format(
193-
retry_after=retry_after,
194-
max_delay=self.max_delay,
195-
),
196-
)
197-
raise last_exc
198199
if retry_after is not None:
199200
delay = retry_after
200201
else:
@@ -297,6 +298,24 @@ def __call__(self, request: httpx2.Request, next: Next) -> httpx2.Response: # n
297298
)
298299
raise last_exc
299300

301+
retry_after: float | None = None
302+
if self.respect_retry_after and last_response is not None:
303+
header = last_response.headers.get("Retry-After")
304+
if header is not None:
305+
retry_after = _parse_retry_after(header)
306+
307+
if retry_after is not None and retry_after > self.max_delay:
308+
if last_exc is None: # pragma: no cover — retry_after requires last_response which requires last_exc
309+
msg = "Retry: retry_after path reached with no last_exc"
310+
raise AssertionError(msg)
311+
last_exc.add_note(
312+
_RETRY_AFTER_EXCEEDS_MAX_DELAY_NOTE.format(
313+
retry_after=retry_after,
314+
max_delay=self.max_delay,
315+
),
316+
)
317+
raise last_exc
318+
300319
if not self.budget.try_withdraw():
301320
_emit_event(
302321
_LOGGER,
@@ -316,23 +335,6 @@ def __call__(self, request: httpx2.Request, next: Next) -> httpx2.Response: # n
316335
attempts=attempt + 1,
317336
) from last_exc
318337

319-
retry_after: float | None = None
320-
if self.respect_retry_after and last_response is not None:
321-
header = last_response.headers.get("Retry-After")
322-
if header is not None:
323-
retry_after = _parse_retry_after(header)
324-
325-
if retry_after is not None and retry_after > self.max_delay:
326-
if last_exc is None: # pragma: no cover — retry_after requires last_response which requires last_exc
327-
msg = "Retry: retry_after path reached with no last_exc"
328-
raise AssertionError(msg)
329-
last_exc.add_note(
330-
_RETRY_AFTER_EXCEEDS_MAX_DELAY_NOTE.format(
331-
retry_after=retry_after,
332-
max_delay=self.max_delay,
333-
),
334-
)
335-
raise last_exc
336338
if retry_after is not None:
337339
delay = retry_after
338340
else:

‎tests/test_backoff.py‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,3 +33,20 @@ def test_full_jitter_delay_uses_injected_random() -> None:
3333
_random_uniform=lambda _lo, hi: hi,
3434
)
3535
assert delay == BASE_DELAY
36+
37+
38+
def test_full_jitter_delay_large_attempt_index_does_not_raise() -> None:
39+
"""attempt_index >= 1024 must not raise OverflowError; result must be in [0, max_delay]."""
40+
delay = full_jitter_delay(2000, base_delay=BASE_DELAY, max_delay=MAX_DELAY)
41+
assert 0.0 <= delay <= MAX_DELAY
42+
43+
44+
def test_full_jitter_delay_large_attempt_index_clamped_to_max_delay() -> None:
45+
"""With a deterministic _random_uniform that returns the ceiling, it must return exactly max_delay."""
46+
delay = full_jitter_delay(
47+
2000,
48+
base_delay=BASE_DELAY,
49+
max_delay=MAX_DELAY,
50+
_random_uniform=lambda _lo, hi: hi,
51+
)
52+
assert delay == MAX_DELAY

‎tests/test_decoders_msgspec.py‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@
1212
from httpware import AsyncClient, DecodeError
1313
from httpware._internal import import_checker
1414
from httpware.decoders import ResponseDecoder
15-
from httpware.decoders.msgspec import MsgspecDecoder
15+
from httpware.decoders.msgspec import MsgspecDecoder, _contains_custom_type
1616

1717

1818
class _Item(msgspec.Struct):
@@ -230,3 +230,12 @@ def test_msgspec_can_decode_unhashable_model_does_not_raise() -> None:
230230
decoder._can_decode_results = MagicMock() # noqa: SLF001
231231
decoder._can_decode_results.get.side_effect = TypeError("unhashable type") # noqa: SLF001
232232
assert decoder.can_decode(_Item) is True
233+
234+
235+
def test_contains_custom_type_raises_import_error_when_msgspec_absent(
236+
monkeypatch: pytest.MonkeyPatch,
237+
) -> None:
238+
"""When msgspec is absent, _contains_custom_type must raise ImportError, not NameError."""
239+
monkeypatch.setattr(import_checker, "is_msgspec_installed", False)
240+
with pytest.raises(ImportError, match="MsgspecDecoder requires"):
241+
_contains_custom_type(None) # ty: ignore[invalid-argument-type] -- guard fires before isinstance checks

0 commit comments

Comments
 (0)