This doc is the single distilled reference for httpware design rationale, protocol seams, and remaining roadmap. It complements CLAUDE.md (at the repo root): CLAUDE.md holds AI-enforced invariants and operational commands; this file holds the reasoning and the structural map.
httpware is a Python async HTTP client framework for building resilient service clients. It supersedes community-of-python/base-client and ships under the modern-python org. The framework owns the abstraction layer above the underlying HTTP client (httpx2 by default); consumers never import the transport directly.
These are non-negotiable. CI rejects PRs that violate them. The "why" exists so future contributors can judge edge cases instead of blindly following the rule.
- No
httpx2leakage outsidesrc/httpware/transports/httpx2.py. Why: the whole point of the framework is to own the abstraction above the underlying client. Any consumer that importshttpx2directly defeats the abstraction and pins us to the current transport choice. - No
httpx2private API. Why: private symbols can change between patch releases. We accept the public-API surface as the contract. - No
from __future__ import annotations. Why: Python 3.11+ floor. PEP 604/585 syntax is native; the future-import would only add noise and inconsistency. - No
print(). Why: ruff-enforced. Libraries log; they do not print to stdout. Stray prints leak into consumer applications. - No global logging config. Why:
logging.basicConfig()from a library mutates the consumer's logging tree. We only acquirelogging.getLogger("httpware")or namespaced child loggers and let consumers configure handlers. - Type suppressions use
# ty: ignore[<rule>]. Why: this project usesty, notmypy.# type: ignoreis silently accepted bytybut ambiguous;# ty: ignore[<rule>]is checked and rule-specific.
A protocol seam is a documented internal boundary. AI agents and contributors must respect it — never cross a seam except through its protocol.
- Where:
src/httpware/middleware/(chain) ↔src/httpware/transports/(anyTransportimplementation). - Contract: the chain bottom calls
transport.__call__(request) -> Response. - Rule: middleware never instantiates a transport; the
AsyncClientinjects it at construction.
- Where:
src/httpware/client.py↔src/httpware/middleware/. - Contract: the middleware chain is composed at
AsyncClient.__init__and frozen for the client's lifetime. - Rule: mutating the chain after construction is not supported. Per-request middleware is expressed via the
Requestextensions field, not by rebuilding the chain.
- Where:
src/httpware/client.py↔src/httpware/decoders/. - Contract: the decoder is invoked when the caller passes
response_model=. The protocol isdecode(content: bytes, model: type[T]) -> T. - Rule: the decoder must operate on raw bytes in a single parse pass. Two-pass decoding (
json.loadsthenvalidate_python) is rejected: a single bytes-in / typed-object-out pass avoids the redundant intermediatedictallocation and parses faster. The Pydantic adapter implements this asTypeAdapter(model).validate_json(content), with theTypeAdapteritself memoized via@functools.lru_cache(maxsize=1024)on a module-level_get_adapter(model)factory (the adapter is the expensive part to build). The msgspec adapter implements it asmsgspec.json.decode(content, type=model).
- Where:
src/httpware/transports/httpx2.pyis the only file that may importhttpx2. - Contract:
httpx2exceptions are mapped tohttpwareexceptions at this seam. - Rule: CI grep checks enforce zero
httpx2imports outside this file and zerohttpx2._private references anywhere.
- Where:
pyproject.tomlextras ([project.optional-dependencies]) ↔ the adapter modules that import them. - Contract: each optional dependency is imported only inside its own dedicated module (e.g.,
pydanticindecoders/pydantic.py;msgspecindecoders/msgspec.pywhen 1-6 lands;opentelemetryinmiddleware/observability/otel.pywhen 5-4 lands). - Rule: never import an extra at package top-level. The package must import cleanly when the extra is not installed.
All httpware HTTP exceptions are constructed with keyword arguments only. The mandatory fields on every StatusError (and its 4xx/5xx subclasses) are:
| Field | Type | Source |
|---|---|---|
status |
int |
response status code |
body |
bytes |
full response body |
headers |
Mapping[str, str] |
lowercased response headers (v0 contract) |
json |
Any | None |
parsed JSON if application/json content-type; else None |
request_method |
str |
uppercased request method |
request_url |
str |
request URL, may include userinfo (Redactor sanitizes — Story 5.3) |
The mapping table from httpx2 errors to httpware errors lives at Seam 4 (src/httpware/transports/httpx2.py). Status-keyed exceptions are looked up via the STATUS_TO_EXCEPTION table in src/httpware/errors.py.
Constructing any of these exceptions positionally is a programming error caught by ty. The keyword-only signature is enforced via __init__ definitions, not docstrings.
Current tree (post-story-1.5):
src/httpware/
├── __init__.py # public exports + __all__
├── py.typed
├── config.py # Limits, Timeout, ClientConfig
├── request.py # Request + with_* helpers
├── response.py # Response (StreamResponse pending in Epic 4)
├── errors.py # status-keyed exception hierarchy
├── decoders/
│ ├── __init__.py # ResponseDecoder protocol (Seam 3)
│ └── pydantic.py # PydanticDecoder adapter
└── transports/
├── __init__.py # Transport protocol
└── httpx2.py # Httpx2Transport adapter (Seam 4)
Planned modules (filled in as the roadmap lands):
src/httpware/
├── client.py # AsyncClient (Story 1.7)
├── decoders/msgspec.py # MsgspecDecoder via extra (Story 1.6)
├── transports/recorded.py # RecordedTransport for testing (Story 1.8)
├── middleware/ # protocols + built-in middleware (Epic 2)
│ ├── __init__.py # Middleware protocol, Next type
│ ├── chain.py # chain composition
│ ├── auth.py # auth coercion (Story 2.4)
│ ├── timeout.py # per-attempt timeout (Story 3.1)
│ ├── retry.py # retry + RetryBudget (Stories 3.2–3.4)
│ ├── bulkhead.py # concurrency limit (Story 3.5)
│ └── observability/ # Layer 1 emission + OTEL (Epic 5)
└── _internal/ # private cross-module helpers
pytest-asyncioauto mode. Async test functions do not require@pytest.mark.asyncio. The setting lives inpyproject.tomlunder[tool.pytest.ini_options].RecordedTransportfor transport mocking, notrespx. Once Story 1.8 lands, transport-level tests instantiateRecordedTransport(shipped with the library) instead of patchinghttpx2calls. This keeps tests aligned with the public seam and avoidsrespx's private-API risk.- Hypothesis property-based tests for concurrency-sensitive code:
RetryBudget,Bulkhead, retry interleaving. Files are namedtest_*_props.pyso they are easy to grep and treat separately in CI. - Performance tests are opt-in. The
perfpytest marker is registered inpyproject.toml; the defaultaddoptsline includes-m 'not perf'. Run benchmarks explicitly withpytest -m perf. - Coverage is 100% line coverage. The five merged stories ship at 100% line coverage. New code is expected to maintain this.
httpware core has a small dependency set. Capabilities that pull in heavyweight dependencies (pydantic, msgspec, opentelemetry) live behind extras declared in pyproject.toml:
[project.optional-dependencies]
pydantic = ["pydantic>=2"]
msgspec = ["msgspec>=0.18"]
otel = ["opentelemetry-api>=1.20", "opentelemetry-sdk>=1.20"]Each extra's code lives in a single dedicated module (e.g., decoders/pydantic.py, decoders/msgspec.py, middleware/observability/otel.py). The import of the extra happens inside that module — never at package top level. This way, import httpware works cleanly without the extras installed, and the seam stays observable: grep for import pydantic should return exactly one file.
Caller-facing pattern: consumers select the implementation by passing it explicitly, e.g., AsyncClient(decoder=PydanticDecoder()). There is no auto-detection or implicit registry.
Twenty-seven stories remain. Topic slugs in planning/specs/ and planning/plans/ use kebab-case descriptions, not the story IDs — these IDs are retained as a stable identifier convention from the original epic structure.
- 1-6
msgspecdecoder via extras — secondResponseDecoderadapter, opt-in. - 1-7
AsyncClientwith HTTP methods,response_model,with_options, lifecycle — the main public surface. - 1-8
RecordedTransportfor testing — ships with the library; replacesrespxfor transport-level tests.
- 2-1
Middlewareprotocol,Nexttype, chain composition. - 2-2 Phase shortcut decorators (
@before_request,@after_response,@on_error). - 2-3
Requestimmutability helpers (with_headers,with_cookie,with_extension, etc.). - 2-4 Auth coercion as middleware.
- 2-5 Wire middleware into
AsyncClient.
- 3-1 Per-attempt timeout middleware.
- 3-2 Retry middleware.
- 3-3
RetryBudgetdata structure. - 3-4
RetryBudgetmiddleware integration. - 3-5
Bulkheadmiddleware. - 3-6 Document the extension slot for custom resilience policies.
- 4-1
StreamResponsetype. - 4-2 Transport stream implementation in
Httpx2Transport. - 4-3
AsyncClient.streamcontext manager.
- 5-1 Layer 1 observability — middleware lifecycle hooks.
- 5-2 Wire emission into resilience middlewares.
- 5-3
Redactorclass and integration (closes deferred work on URL/header/userinfo sanitization). - 5-4 OpenTelemetry middleware via the
otelextra. - 5-5 Logging policy enforcement (CI grep on
logging.basicConfig,logging.getLogger()without a name).
- 6-1 Migration guide from
base-client. - 6-2 Documentation site (
mkdocs). - 6-3 Public benchmark suite.
- 6-4 CI enforcement gates (codify the invariants in Section 2 as CI jobs).
- 6-5 Release flow with Trusted Publishers + Sigstore.
When work starts on a roadmap item, it gets a spec at planning/specs/YYYY-MM-DD-<topic>-design.md and a plan at planning/plans/YYYY-MM-DD-<topic>-plan.md.
Review-surfaced items that are real but not actionable now live in planning/deferred-work.md at the repository root. Each entry cites the originating story and the file/line, and explains why the fix is deferred (cross-story dependency, scope, performance/security tradeoff, etc.). When a deferred item becomes actionable, it migrates into the spec for the story that resolves it.