|
| 1 | +# Design: thin httpx2 wrapper (v0.2 pivot) |
| 2 | + |
| 3 | +**Status:** spec — awaiting review |
| 4 | +**Author:** Artur Shiriev (with Claude) |
| 5 | +**Date:** 2026-06-03 |
| 6 | +**Supersedes:** the 0.1.0 architecture documented in `docs/dev/engineering.md` (sections 2, 3, 5, 8) |
| 7 | + |
| 8 | +## 1. Intent |
| 9 | + |
| 10 | +`httpware` 0.2 is a clean rewrite. The framework becomes a **thin opinionated wrapper around `httpx2`** that adds three things and nothing else: |
| 11 | + |
| 12 | +1. **Typed response decoding** via the existing `ResponseDecoder` protocol. Pydantic and msgspec adapters ship as opt-in extras. |
| 13 | +2. **A middleware chain** that operates directly on `httpx2.Request` / `httpx2.Response`. Reserved for retry, timeout, bulkhead, and observability work in later epics. |
| 14 | +3. **A status-keyed exception tree** (`NotFoundError`, `RateLimitedError`, ...) raised automatically on 4xx and 5xx. Each exception holds the raw `httpx2.Response`. |
| 15 | + |
| 16 | +Everything else delegates to `httpx2`: `base_url`, default headers, default params, default cookies, `Limits`, `Timeout`, `Auth`, URL building, content negotiation, streaming, pool management. |
| 17 | + |
| 18 | +The 0.1.0 release is treated as a false start. The "no httpx2 leakage" CI invariant is **retired** — exposing `httpx2.Request` / `httpx2.Response` is the explicit design. |
| 19 | + |
| 20 | +## 2. Why now |
| 21 | + |
| 22 | +The 0.1.0 surface duplicates substantial logic from `httpx2`: a custom `Request` / `Response` value type with its own validation and `with_*` helpers, a `ClientConfig` that re-implements default-header / default-query merging, our own `Limits` and `Timeout` dataclasses with bespoke defaults, an auth-coercion layer that mirrors `httpx2.Auth`, a `Transport` protocol that has exactly one production implementation, and a `RecordedTransport` that duplicates `httpx2.MockTransport`. Each abstraction was justified individually but the sum costs more than it returns: more code to test, two type systems to keep in sync with `httpx2` upgrades, and a steeper learning curve for users who already know `httpx`/`httpx2`. |
| 23 | + |
| 24 | +The pivot accepts `httpx2` as the de-facto interface and concentrates `httpware`'s value on the three additions above. |
| 25 | + |
| 26 | +## 3. Architectural invariants (CI-enforced) |
| 27 | + |
| 28 | +- **No `httpx2._` private API.** Public symbols only. |
| 29 | +- **No `from __future__ import annotations`.** Python 3.11+ floor. |
| 30 | +- **No `print()`.** Library code logs, never prints. |
| 31 | +- **No global logging config** (`logging.basicConfig()`, bare `logging.getLogger()`). |
| 32 | +- **Type suppressions use `# ty: ignore[<rule>]`**, never `# type: ignore` or `# mypy: ignore`. |
| 33 | + |
| 34 | +**Retired in 0.2:** the "no `httpx2` import outside `transports/httpx2.py`" rule. `httpx2` is now part of `httpware`'s public surface and may be imported anywhere it is needed. |
| 35 | + |
| 36 | +## 4. Protocol seams (three, down from five) |
| 37 | + |
| 38 | +### Seam A: `AsyncClient ↔ Middleware` |
| 39 | + |
| 40 | +- **Where:** `src/httpware/client.py` ↔ `src/httpware/middleware/`. |
| 41 | +- **Contract:** middleware chain is composed once at `AsyncClient.__init__` and frozen for the client's lifetime. Chain bottom (the "terminal") is internal; it calls `self._httpx2_client.send(request)`, maps `httpx2` errors to `httpware` errors, and raises a `StatusError` subclass on 4xx/5xx. |
| 42 | +- **Rule:** mutating the chain after construction is not supported. Per-request behavior goes through `httpx2.Request.extensions` or through `extensions=` kwargs at call sites. |
| 43 | + |
| 44 | +### Seam B: `AsyncClient ↔ ResponseDecoder` |
| 45 | + |
| 46 | +- **Where:** `src/httpware/client.py` ↔ `src/httpware/decoders/`. |
| 47 | +- **Contract:** `decode(content: bytes, model: type[T]) -> T`. The signature is unchanged from 0.1.0. Decoder errors (`pydantic.ValidationError`, `msgspec.ValidationError`) propagate unwrapped. |
| 48 | +- **Rule:** decoder is invoked only when the caller passes `response_model=`. Single parse pass; no intermediate `dict` allocation. |
| 49 | + |
| 50 | +### Seam C: `httpware ↔ optional extras` |
| 51 | + |
| 52 | +- **Where:** `pyproject.toml` `[project.optional-dependencies]` ↔ the adapter module that imports the extra. |
| 53 | +- **Contract:** each optional dependency is imported only inside its own dedicated module. |
| 54 | +- **Rule:** never import an extra at package top-level. `import httpware` must succeed without the extras installed. |
| 55 | + |
| 56 | +The seams formerly numbered 1 (Middleware↔Transport) and 4 (Transport↔httpx2) collapse into the AsyncClient terminal — there is no transport abstraction. |
| 57 | + |
| 58 | +## 5. Module layout |
| 59 | + |
| 60 | +```text |
| 61 | +src/httpware/ |
| 62 | +├── __init__.py # public exports |
| 63 | +├── py.typed |
| 64 | +├── client.py # AsyncClient |
| 65 | +├── errors.py # status-keyed exception tree (response: httpx2.Response) |
| 66 | +├── middleware/ |
| 67 | +│ ├── __init__.py # Middleware protocol, Next type, @before_request/@after_response/@on_error |
| 68 | +│ └── chain.py # compose(middleware, terminal) -> Next |
| 69 | +├── decoders/ |
| 70 | +│ ├── __init__.py # ResponseDecoder protocol |
| 71 | +│ ├── pydantic.py # PydanticDecoder (extra: pydantic) |
| 72 | +│ └── msgspec.py # MsgspecDecoder (extra: msgspec) |
| 73 | +└── _internal/ |
| 74 | + └── import_checker.py # is_msgspec_installed, is_pydantic_installed |
| 75 | +``` |
| 76 | + |
| 77 | +**Deleted relative to 0.1.0:** |
| 78 | + |
| 79 | +- `request.py` — `Request` value type and `with_*` helpers. |
| 80 | +- `response.py` — `Response`, `StreamResponse`. |
| 81 | +- `config.py` — `Limits`, `Timeout`, `ClientConfig`. |
| 82 | +- `transports/__init__.py` — `Transport` protocol. |
| 83 | +- `transports/httpx2.py` — `Httpx2Transport`. Status→exception logic relocates into `client.py`'s terminal. |
| 84 | +- `transports/recorded.py` — `RecordedTransport`. Tests inject `httpx2.MockTransport` via `httpx2_client=`. |
| 85 | +- `_internal/auth.py` — auth coercion. Callers pass `httpx2.Auth` directly. |
| 86 | +- `_internal/chain.py` — relocates to `middleware/chain.py`. |
| 87 | + |
| 88 | +## 6. Public API |
| 89 | + |
| 90 | +### `AsyncClient.__init__` |
| 91 | + |
| 92 | +```python |
| 93 | +class AsyncClient: |
| 94 | + def __init__( |
| 95 | + self, |
| 96 | + *, |
| 97 | + # Forwarded to httpx2.AsyncClient if no httpx2_client is given. |
| 98 | + base_url: str = "", |
| 99 | + headers: dict[str, str] | None = None, |
| 100 | + params: dict[str, str] | None = None, |
| 101 | + cookies: dict[str, str] | None = None, |
| 102 | + timeout: httpx2.Timeout | float | None = None, |
| 103 | + limits: httpx2.Limits | None = None, |
| 104 | + auth: httpx2.Auth | None = None, |
| 105 | + # Caller-owned client. If given, all other forwarded kwargs above must be unset. |
| 106 | + httpx2_client: httpx2.AsyncClient | None = None, |
| 107 | + # httpware additions. |
| 108 | + decoder: ResponseDecoder | None = None, |
| 109 | + middleware: Sequence[Middleware] = (), |
| 110 | + ) -> None: ... |
| 111 | +``` |
| 112 | + |
| 113 | +Passing `httpx2_client=` together with any of the forwarded kwargs raises `TypeError`. When `httpx2_client=` is provided, `AsyncClient` does NOT close it on `__aexit__`; the caller owns the lifecycle. |
| 114 | + |
| 115 | +When `decoder` is `None`, the client falls back to `PydanticDecoder()` if the `pydantic` extra is installed, otherwise no decoder is configured and any call with `response_model=` raises a clear error. |
| 116 | + |
| 117 | +### Per-method surface |
| 118 | + |
| 119 | +Each method has two overloads — `response_model=None` returns `httpx2.Response`, `response_model=type[T]` returns `T`. |
| 120 | + |
| 121 | +```python |
| 122 | +async def get( url, *, params=None, headers=None, cookies=None, |
| 123 | + timeout=..., extensions=None, response_model=None) -> httpx2.Response | T: ... |
| 124 | +async def post( url, *, json=None, content=None, data=None, files=None, |
| 125 | + params=None, headers=None, cookies=None, |
| 126 | + timeout=..., extensions=None, response_model=None) -> httpx2.Response | T: ... |
| 127 | +# put / patch / delete / head / options / request follow the same pattern. |
| 128 | +``` |
| 129 | + |
| 130 | +Body kwargs (`json`, `content`, `data`, `files`) are passed straight through to `httpx2.AsyncClient.build_request`. `httpx2` handles content-type negotiation and serialization. |
| 131 | + |
| 132 | +### Power-user entry points |
| 133 | + |
| 134 | +```python |
| 135 | +def build_request(self, method: str, url: str, **kw) -> httpx2.Request: |
| 136 | + return self._httpx2_client.build_request(method, url, **kw) |
| 137 | + |
| 138 | +async def send( |
| 139 | + self, |
| 140 | + request: httpx2.Request, |
| 141 | + *, |
| 142 | + response_model: type[T] | None = None, |
| 143 | +) -> httpx2.Response | T: ... |
| 144 | +``` |
| 145 | + |
| 146 | +`send()` is the seam through which everything (per-method calls and user code) ultimately funnels into the middleware chain. |
| 147 | + |
| 148 | +### Lifecycle |
| 149 | + |
| 150 | +```python |
| 151 | +async def __aenter__(self) -> Self: ... |
| 152 | +async def __aexit__(self, *exc) -> None: ... # closes self._httpx2_client only if owned |
| 153 | +``` |
| 154 | + |
| 155 | +A boolean `self._owns_client` records ownership. `__aexit__` is idempotent. |
| 156 | + |
| 157 | +### Streaming |
| 158 | + |
| 159 | +Deferred to Epic 4. The eventual API is `client.stream(request)` returning an async context manager that forwards to `httpx2.AsyncClient.stream`. No `StreamResponse` type — `httpx2.Response` in stream mode is the contract. |
| 160 | + |
| 161 | +## 7. Middleware |
| 162 | + |
| 163 | +```python |
| 164 | +import httpx2 |
| 165 | +from collections.abc import Awaitable, Callable |
| 166 | +from typing import Protocol, TypeAlias, runtime_checkable |
| 167 | + |
| 168 | +type Next = Callable[[httpx2.Request], Awaitable[httpx2.Response]] |
| 169 | + |
| 170 | +@runtime_checkable |
| 171 | +class Middleware(Protocol): |
| 172 | + async def __call__(self, request: httpx2.Request, next: Next) -> httpx2.Response: ... |
| 173 | +``` |
| 174 | + |
| 175 | +Phase decorators retain their 0.1.0 names and semantics, retyped on `httpx2` types: |
| 176 | + |
| 177 | +- `@before_request` — `f: (httpx2.Request) -> Awaitable[httpx2.Request]` |
| 178 | +- `@after_response` — `f: (httpx2.Request, httpx2.Response) -> Awaitable[httpx2.Response]` |
| 179 | +- `@on_error` — `f: (httpx2.Request, Exception) -> Awaitable[httpx2.Response | None]`. Catches `Exception` (not `BaseException`), so `asyncio.CancelledError` propagates. Returning `None` re-raises. |
| 180 | + |
| 181 | +Composition lives in `middleware/chain.py`: |
| 182 | + |
| 183 | +```python |
| 184 | +def compose(middleware: Sequence[Middleware], terminal: Next) -> Next: ... |
| 185 | +``` |
| 186 | + |
| 187 | +The chain is composed once at `AsyncClient.__init__` and cached as `self._dispatch`. |
| 188 | + |
| 189 | +## 8. Errors |
| 190 | + |
| 191 | +```python |
| 192 | +class ClientError(Exception): ... |
| 193 | +class TransportError(ClientError): ... |
| 194 | +class TimeoutError(ClientError, builtins.TimeoutError): ... # shadows builtin intentionally |
| 195 | + |
| 196 | +class StatusError(ClientError): |
| 197 | + response: httpx2.Response |
| 198 | + |
| 199 | + def __init__(self, response: httpx2.Response) -> None: |
| 200 | + self.response = response |
| 201 | + super().__init__(self._summary()) |
| 202 | + |
| 203 | + def __repr__(self) -> str: ... # strips userinfo from response.request.url |
| 204 | + def _summary(self) -> str: ... # short message: "<status> <method> <sanitized url>" |
| 205 | + |
| 206 | +class ClientStatusError(StatusError): ... # base for 4xx |
| 207 | +class ServerStatusError(StatusError): ... # base for 5xx |
| 208 | + |
| 209 | +class BadRequestError(ClientStatusError): ... # 400 |
| 210 | +class UnauthorizedError(ClientStatusError): ... # 401 |
| 211 | +class ForbiddenError(ClientStatusError): ... # 403 |
| 212 | +class NotFoundError(ClientStatusError): ... # 404 |
| 213 | +class ConflictError(ClientStatusError): ... # 409 |
| 214 | +class UnprocessableEntityError(ClientStatusError): ... # 422 |
| 215 | +class RateLimitedError(ClientStatusError): ... # 429 |
| 216 | +class InternalServerError(ServerStatusError): ... # 500 |
| 217 | +class ServiceUnavailableError(ServerStatusError): ... # 503 |
| 218 | + |
| 219 | +STATUS_TO_EXCEPTION: Mapping[int, type[StatusError]] = {...} |
| 220 | +``` |
| 221 | + |
| 222 | +Notes: |
| 223 | + |
| 224 | +- `StatusError.__init__` accepts a single positional `response` argument. The 0.1.0 keyword-only signature with `status` / `body` / `headers` / `json` / `request_method` / `request_url` is dropped; all values are available via `exc.response.*`. |
| 225 | +- `StatusError.__repr__` and the summary message strip `user:pass@` userinfo from `response.request.url` to avoid leaking credentials in tracebacks. Query-string secrets are NOT stripped (`Redactor` middleware was on the 0.1 roadmap; it is dropped in this pivot — query-string sanitization is `httpx2`'s responsibility if anywhere). |
| 226 | +- `TimeoutError` continues to inherit from both `httpware.ClientError` and `builtins.TimeoutError` so `except builtins.TimeoutError` (the form `asyncio.wait_for` uses) catches httpware-raised timeouts. |
| 227 | + |
| 228 | +## 9. Data flow |
| 229 | + |
| 230 | +One `client.get(url, response_model=Foo)` call: |
| 231 | + |
| 232 | +``` |
| 233 | +client.get(url, response_model=Foo) |
| 234 | + │ |
| 235 | + ▼ |
| 236 | +build_request("GET", url, headers, params, cookies, timeout, extensions) |
| 237 | + │ delegates to self._httpx2_client.build_request → |
| 238 | + │ httpx2 merges base_url, default headers/params/cookies, applies auth |
| 239 | + ▼ |
| 240 | +self.send(httpx2.Request, response_model=Foo) |
| 241 | + │ |
| 242 | + ▼ |
| 243 | +self._dispatch(httpx2.Request) ◄── compose(user_middleware, terminal) — built once in __init__ |
| 244 | + │ |
| 245 | + ▼ |
| 246 | +[middleware chain — user middleware wraps inner middleware wraps terminal] |
| 247 | + │ |
| 248 | + ▼ |
| 249 | +terminal(httpx2.Request) → httpx2.Response |
| 250 | + │ - calls self._httpx2_client.send(request) |
| 251 | + │ - maps httpx2.TimeoutException → httpware.TimeoutError |
| 252 | + │ - maps httpx2.HTTPError + InvalidURL + CookieConflict → httpware.TransportError |
| 253 | + │ - if 400 ≤ status < 600: raise STATUS_TO_EXCEPTION.get(status, ClientStatusError|ServerStatusError)(response) |
| 254 | + ▼ |
| 255 | +back through middleware — may transform Response, may catch and translate exceptions |
| 256 | + │ |
| 257 | + ▼ |
| 258 | +self.send: if response_model is None → return Response, else decoder.decode(response.content, Foo) → Foo |
| 259 | +``` |
| 260 | + |
| 261 | +Key properties: |
| 262 | + |
| 263 | +- Status-raise lives at the **terminal**, not after the chain. Retry middleware therefore sees `ServerStatusError` and can react via `@on_error`. |
| 264 | +- Decoding happens **after** the chain. Decoder errors do not interact with middleware. |
| 265 | +- The `RuntimeError("...closed...")` that `httpx2.AsyncClient.send` raises when the client is closed externally is mapped to `TransportError`, matching the 0.1.0 special-case. |
| 266 | + |
| 267 | +## 10. Error-mapping table |
| 268 | + |
| 269 | +Single source of truth, lives in `client.py`'s terminal: |
| 270 | + |
| 271 | +| `httpx2` source | `httpware` exception | |
| 272 | +|---|---| |
| 273 | +| `httpx2.TimeoutException` (any subclass) | `httpware.TimeoutError` | |
| 274 | +| `httpx2.HTTPError` (other) | `httpware.TransportError` | |
| 275 | +| `httpx2.InvalidURL`, `httpx2.CookieConflict` | `httpware.TransportError` | |
| 276 | +| `RuntimeError` with `"closed"` in message from `client.send` | `httpware.TransportError` | |
| 277 | +| Response with status 400 ≤ status < 600 | `STATUS_TO_EXCEPTION.get(status, ClientStatusError if < 500 else ServerStatusError)(response)` | |
| 278 | + |
| 279 | +## 11. Testing pattern |
| 280 | + |
| 281 | +Transport-level tests inject `httpx2.MockTransport` via the `httpx2_client=` parameter: |
| 282 | + |
| 283 | +```python |
| 284 | +import httpx2 |
| 285 | +from httpware import AsyncClient |
| 286 | + |
| 287 | +def handler(request: httpx2.Request) -> httpx2.Response: |
| 288 | + return httpx2.Response(200, json={"ok": True}) |
| 289 | + |
| 290 | +async def test_get_returns_response(): |
| 291 | + mock = httpx2.MockTransport(handler) |
| 292 | + async with AsyncClient(httpx2_client=httpx2.AsyncClient(transport=mock)) as client: |
| 293 | + resp = await client.get("https://example.test/x") |
| 294 | + assert resp.status_code == 200 |
| 295 | +``` |
| 296 | + |
| 297 | +Property-based tests (Hypothesis) for `RetryBudget` and `Bulkhead` are unaffected — they test data structures, not request types. |
| 298 | + |
| 299 | +Coverage target remains 100% line coverage on shipped modules. |
| 300 | + |
| 301 | +## 12. Roadmap impact |
| 302 | + |
| 303 | +The 0.1 roadmap had 27 stories. After the pivot: |
| 304 | + |
| 305 | +**Deleted:** `1-8` `RecordedTransport`, `2-3` Request immutability helpers, `2-4` Auth coercion, `4-1` `StreamResponse` type, `4-2` transport stream impl, `5-3` `Redactor` middleware. |
| 306 | + |
| 307 | +**Rewritten:** `1-7` `AsyncClient` (the heart of v0.2), `2-5` wire middleware (now trivial — part of 1-7), `6-1` migration guide (extended with httpware 0.1 → 0.2 notes), `6-4` CI invariant gates (drop the "no `httpx2` leakage" rule). |
| 308 | + |
| 309 | +**Survives in shape:** `1-6` msgspec decoder, `2-1` Middleware protocol (retyped on `httpx2`), `2-2` phase decorators (retyped), `3-1` per-attempt timeout, `3-2` retry, `3-3` `RetryBudget`, `3-4` `RetryBudget` middleware integration, `3-5` `Bulkhead`, `3-6` extension-slot docs, `4-3` `AsyncClient.stream`, `5-1` Layer 1 observability, `5-2` wire into resilience middlewares, `5-4` OpenTelemetry middleware, `5-5` logging policy CI grep, `6-2` docs site, `6-3` benchmarks, `6-5` release flow. |
| 310 | + |
| 311 | +## 13. Cutover plan |
| 312 | + |
| 313 | +Per the project's clean-cutover convention, the pivot lands as a **single structural PR** that produces a working `0.2.0` baseline. Substantive follow-up (Epic 3 resilience, Epic 5 observability) lands afterward as ordinary PRs. |
| 314 | + |
| 315 | +Structural PR scope: |
| 316 | + |
| 317 | +1. Delete `request.py`, `response.py`, `config.py`, `transports/`, `_internal/auth.py`, `_internal/chain.py`. |
| 318 | +2. Rewrite `client.py`, `errors.py`, `middleware/__init__.py`. Move chain composition to `middleware/chain.py`. |
| 319 | +3. Leave `decoders/` intact. |
| 320 | +4. Rewrite tests against `httpx2.MockTransport` injection. |
| 321 | +5. Update `CLAUDE.md` (drop "no `httpx2` leakage"; update module layout) and `docs/dev/engineering.md` (rewrite sections 2, 3, 5, 8). |
| 322 | +6. Sweep `planning/deferred-work.md` — close items obsoleted by the pivot. |
| 323 | +7. Bump `pyproject.toml` version to `0.2.0`. |
| 324 | +8. Draft GitHub Release notes covering the migration from 0.1.0. |
| 325 | + |
| 326 | +CI must remain green at the end of the PR. Coverage at 100% on the new modules. |
| 327 | + |
| 328 | +## 14. Open questions |
| 329 | + |
| 330 | +None at spec-write time. If issues surface during plan-writing, this section will be updated. |
0 commit comments