Skip to content

Commit a0c1a87

Browse files
authored
fix!: consistent client error paths (timeout=None, RemoteProtocolError, invalid URL, closed client, streaming bodies, async middleware) (#169)
* fix!: forward an explicit timeout=None as no timeout * fix: map httpx2 RemoteProtocolError to NetworkError * fix: map an invalid URL to TransportError when building the request * fix: raise TransportError from stream() on a closed client * fix: refuse retry for streaming bodies of requests passed to send() * fix: reject async middleware when building a sync Client * fix: read an unread async response body before decoding it * test: run every line of the client error path tests
1 parent 80bee92 commit a0c1a87

9 files changed

Lines changed: 549 additions & 46 deletions

File tree

‎docs/errors.md‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,19 @@ ClientError (catch-all for anything httpware raises)
3131
└── ResponseTooLargeError (response body exceeds max_response_body_bytes; status-agnostic)
3232
```
3333

34+
## Mapping `httpx2` exceptions
35+
36+
An exception from `httpx2` is re-raised as an httpware one, with the original on `__cause__`:
37+
38+
| `httpx2` exception | httpware exception |
39+
|---|---|
40+
| `TimeoutException` and its subclasses | `TimeoutError` |
41+
| `NetworkError` (connect, read, write and close errors) | `NetworkError` |
42+
| `RemoteProtocolError`, such as a server dropping a keep-alive connection mid-response | `NetworkError` |
43+
| any other `HTTPError`, `InvalidURL`, `CookieConflict` | `TransportError` |
44+
45+
`NetworkError` and `TimeoutError` are the transient ones: `AsyncRetry` and `Retry` retry them, and the circuit breaker counts them. An invalid URL raises `TransportError` from every method, whether `httpx2` rejects it while the request is built or while it is sent. So does a call on a client that has been closed, `stream()` included.
46+
3447
## Status-to-exception mapping
3548

3649
| Status | Exception class |

‎docs/index.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -124,7 +124,7 @@ async def main() -> None:
124124

125125
`base_url` must not contain a query string; a client built with one raises `ValueError`. Put query parameters shared by every request in `params=` instead.
126126

127-
The other keywords of `httpx2.AsyncClient` and `httpx2.Client` (`verify`, `proxy`, `http2`, `transport`, `follow_redirects`, ...) are passed through to the `httpx2` client that httpware builds and closes. Two are refused. `cert` is deprecated by `httpx2` in favour of an `ssl.SSLContext` passed as `verify`. `event_hooks` run below the middleware chain, so use [middleware](middleware.md) instead.
127+
The other keywords of `httpx2.AsyncClient` and `httpx2.Client` (`verify`, `proxy`, `http2`, `transport`, `follow_redirects`, ...) are passed through to the `httpx2` client that httpware builds and closes. An option set to `None` counts as not passed, so `httpx2` keeps its default, with one exception: `timeout=None` turns the timeout off. Leave `timeout` out to keep the `httpx2` default of 5 seconds. Two options are refused. `cert` is deprecated by `httpx2` in favour of an `ssl.SSLContext` passed as `verify`. `event_hooks` run below the middleware chain, so use [middleware](middleware.md) instead.
128128

129129
```python
130130
import ssl

‎docs/middleware.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -189,7 +189,7 @@ with Client(base_url="https://api.example.com", middleware=[add_request_id]) as
189189
client.get("/users/1")
190190
```
191191

192-
Sync and async middleware don't mix: pass `Middleware` to `Client` and `AsyncMiddleware` to `AsyncClient`.
192+
Sync and async middleware don't mix: pass `Middleware` to `Client` and `AsyncMiddleware` to `AsyncClient`. `Client` raises `TypeError` when you build it with async middleware, such as `AsyncRetry` or a function wrapped by `@async_before_request`.
193193

194194
## See also
195195

‎docs/resilience.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,8 @@ If the request body was an async iterable, `AsyncRetry` does not retry, because
5050
httpware: not retrying because the request body is a stream that cannot replay across attempts
5151
```
5252

53+
The same applies to a request you build yourself and pass to `send()` or `send_with_response()`, whether it comes from `client.build_request()` or `httpx2.Request()`. `send()` marks it as streaming when its body is not held in memory and is not a multipart upload, so `content=` with a generator is marked, while `content=b"..."`, `json=`, `data=` and `files=` are not. A body built from a list of chunks also counts as streaming here, because `send()` cannot tell it from a generator; `client.post()` and the other verb methods see the original argument and don't mark it.
54+
5355
The note, and the `retry.streaming_refused` event on `httpware.retry`, appear only when the failure would otherwise have been retried. A POST with a streaming body, for example, is re-raised without the note because POST is not retried in the first place. See [Observability](observability.md).
5456

5557
### Exhaustion behavior

‎src/httpware/_internal/exception_mapping.py‎

Lines changed: 20 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,10 @@
22
33
map_httpx2_exception is a pure function used by both Client._terminal and
44
AsyncClient._terminal, and by both stream() methods. Clause ordering:
5-
TimeoutException -> InvalidURL/CookieConflict -> NetworkError -> HTTPError
6-
(subclass before parent so the right type wins). The two context managers
7-
below wrap it for use as `with`/`async with` blocks around the httpx2 call.
5+
TimeoutException -> InvalidURL/CookieConflict -> NetworkError/RemoteProtocolError
6+
-> HTTPError (subclass before parent so the right type wins). The two context managers
7+
below wrap it for use as `with`/`async with` blocks around the httpx2 call,
8+
and also turn the RuntimeError httpx2 raises on a closed client into TransportError.
89
"""
910

1011
import contextlib
@@ -25,30 +26,41 @@ def map_httpx2_exception(exc: BaseException) -> NetworkError | TimeoutError | Tr
2526
return TimeoutError(str(exc))
2627
if isinstance(exc, (httpx2.InvalidURL, httpx2.CookieConflict)):
2728
return TransportError(str(exc))
28-
if isinstance(exc, httpx2.NetworkError):
29+
if isinstance(exc, (httpx2.NetworkError, httpx2.RemoteProtocolError)):
2930
return NetworkError(str(exc))
3031
if isinstance(exc, httpx2.HTTPError):
3132
return TransportError(str(exc))
3233
return TransportError(str(exc)) # pragma: no cover — defensive default; httpx2.HTTPError is the root
3334

3435

3536
@contextlib.asynccontextmanager
36-
async def _httpx2_exception_mapper() -> AsyncGenerator[None]:
37-
"""Map httpx2 exceptions to httpware exceptions. Shared by AsyncClient._terminal and stream()."""
37+
async def _httpx2_exception_mapper(client: httpx2.AsyncClient) -> AsyncGenerator[None]:
38+
"""Map httpx2 exceptions, and use of a closed `client`, to httpware exceptions.
39+
40+
Shared by AsyncClient._terminal and stream().
41+
"""
3842
try:
3943
yield
4044
except httpx2.HTTPError as exc:
4145
raise map_httpx2_exception(exc) from exc
4246
except (httpx2.InvalidURL, httpx2.CookieConflict) as exc:
4347
raise map_httpx2_exception(exc) from exc
48+
except RuntimeError as exc:
49+
if client.is_closed:
50+
raise TransportError(str(exc)) from exc
51+
raise
4452

4553

4654
@contextlib.contextmanager
47-
def _httpx2_exception_mapper_sync() -> Generator[None]:
48-
"""Map httpx2 exceptions to httpware exceptions. Sync sibling of _httpx2_exception_mapper."""
55+
def _httpx2_exception_mapper_sync(client: httpx2.Client) -> Generator[None]:
56+
"""Sync sibling of _httpx2_exception_mapper."""
4957
try:
5058
yield
5159
except httpx2.HTTPError as exc:
5260
raise map_httpx2_exception(exc) from exc
5361
except (httpx2.InvalidURL, httpx2.CookieConflict) as exc:
5462
raise map_httpx2_exception(exc) from exc
63+
except RuntimeError as exc:
64+
if client.is_closed:
65+
raise TransportError(str(exc)) from exc
66+
raise

‎src/httpware/_internal/status.py‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,15 @@
1818
(the consumed iterator cannot replay across attempts)."""
1919

2020

21+
def _mark_streaming_body(request: httpx2.Request) -> None:
22+
"""Set STREAMING_BODY_MARKER unless the body is in memory, multipart, or the marker is already set."""
23+
if isinstance(request.stream, httpx2.ByteStream) or STREAMING_BODY_MARKER in request.extensions:
24+
return
25+
if request.headers.get("content-type", "").startswith("multipart/form-data"):
26+
return
27+
request.extensions[STREAMING_BODY_MARKER] = True
28+
29+
2130
def _raise_on_status_error(response: httpx2.Response) -> None:
2231
"""Raise the appropriate StatusError subclass for a 4xx/5xx response. No-op for 2xx/3xx."""
2332
status = response.status_code

0 commit comments

Comments
 (0)