|
| 1 | +# msgspec decoder via extras (design) |
| 2 | + |
| 3 | +- **Date:** 2026-05-31 |
| 4 | +- **Status:** approved, ready for plan |
| 5 | +- **Scope:** Story 1-6 (sixth story of Epic 1). Adds the second `ResponseDecoder` adapter — `MsgspecDecoder` — gated behind the `msgspec` extra. Introduces a small private `import_checker` module that future opt-in extras (otel, etc.) will reuse. Out of scope: AsyncClient wiring (Story 1-7), RecordedTransport (Story 1-8). |
| 6 | +- **Roadmap pointer:** `docs/engineering.md` §8 "Epic 1 — Make typed HTTP requests with sensible defaults". |
| 7 | + |
| 8 | +## Why |
| 9 | + |
| 10 | +Consumers with high-throughput needs want msgspec's validation speed. The `ResponseDecoder` protocol from Story 1-5 was designed for this — pluggable, single-parse-pass, model-driven. Story 1-6 ships the second adapter so when `AsyncClient` lands in 1-7, both decoders are available via `AsyncClient(decoder=PydanticDecoder())` or `AsyncClient(decoder=MsgspecDecoder())`. |
| 11 | + |
| 12 | +The pattern for handling the optional dependency is borrowed from `modern-python/lite-bootstrap`: a small `import_checker` module uses `importlib.util.find_spec` to detect the extra without importing it. The adapter module then conditionally imports the dependency at module load, and raises a hinted `ImportError` only when the decoder is constructed without the extra installed. This keeps the module itself import-safe (useful for capability-probe patterns) and honors seam #5 ("never import an extra at package top-level"). |
| 13 | + |
| 14 | +## Decisions |
| 15 | + |
| 16 | +| Decision | Choice | |
| 17 | +| --- | --- | |
| 18 | +| Import strategy | `find_spec`-based detection in `_internal/import_checker.py`; conditional `import msgspec` at the top of `decoders/msgspec.py`; `ImportError` raised at `MsgspecDecoder.__init__` if the extra is missing. Mirrors lite-bootstrap. | |
| 19 | +| Install hint location | Module-level constant `MISSING_DEPENDENCY_MESSAGE = "..."` in `decoders/msgspec.py`. Not a class attribute. | |
| 20 | +| Package-root re-export | None. Consumers import via `from httpware.decoders.msgspec import MsgspecDecoder`. Honors seam #5 (re-exporting would force eager `msgspec` load whenever the extra is installed). | |
| 21 | +| Caching layer | None. `msgspec.json.decode` is a free function; no per-model adapter object to memoize, unlike `pydantic.TypeAdapter`. | |
| 22 | +| Error propagation | `msgspec.ValidationError` and `msgspec.DecodeError` propagate unchanged. Mirrors `PydanticDecoder`'s `pydantic.ValidationError` handling. | |
| 23 | +| `__all__` in `msgspec.py` and `import_checker.py` | None. Submodules don't get `__all__` going forward (project convention shift; existing files keep theirs until a follow-up cleanup). | |
| 24 | +| `TYPE_CHECKING` block | None. The conditional `import msgspec` is sufficient for `ty`; no separate type-only import is needed. Fallback: `# ty: ignore[unresolved-reference]` on the `msgspec.json.decode` line if ty rejects, but lite-bootstrap precedent suggests it won't. | |
| 25 | +| pyproject.toml | Untouched. `msgspec = ["msgspec>=0.18"]` is already declared (Story 1-1). | |
| 26 | + |
| 27 | +## File structure |
| 28 | + |
| 29 | +**New files:** |
| 30 | +- `src/httpware/_internal/import_checker.py` — `find_spec`-based detection flags. Initial content is just `is_msgspec_installed`; future extras (otel, etc.) extend it. |
| 31 | +- `src/httpware/decoders/msgspec.py` — `MsgspecDecoder` adapter. |
| 32 | +- `tests/test_decoders_msgspec.py` — 8 behavioral tests for the decoder. |
| 33 | +- `tests/test_optional_extras_isolation.py` — subprocess-based import-time guard. Future stories (5-4 otel) extend this file with their own subprocess checks. |
| 34 | + |
| 35 | +**Modified files:** |
| 36 | +- `CHANGELOG.md` — Story 1.6 bullet under `[Unreleased]` / `### Added`. |
| 37 | + |
| 38 | +**Files NOT modified:** |
| 39 | +- `pyproject.toml` — extras declaration already in place from Story 1-1. |
| 40 | +- `src/httpware/__init__.py` — no package-root re-export of `MsgspecDecoder` (seam contract). |
| 41 | +- `src/httpware/decoders/__init__.py` — `ResponseDecoder` protocol stays as-is; no new exports. |
| 42 | + |
| 43 | +## `_internal/import_checker.py` |
| 44 | + |
| 45 | +```python |
| 46 | +"""Detect optional extras without importing them. Used by adapter modules to gate hard imports.""" |
| 47 | + |
| 48 | +from importlib.util import find_spec |
| 49 | + |
| 50 | + |
| 51 | +is_msgspec_installed = find_spec("msgspec") is not None |
| 52 | +``` |
| 53 | + |
| 54 | +Notes: |
| 55 | +- One line of state, one purpose. Future stories add more `is_X_installed` flags as additional extras land (otel in Story 5-4, etc.). |
| 56 | +- `find_spec` does NOT import the target module; it only checks whether the importer can find it. Side-effect-free. |
| 57 | +- Module-level evaluation: the flag is computed once at import time and cached for the process lifetime. Re-installing the extra after process start is not supported (acceptable — the same caveat applies to lite-bootstrap and any other find_spec-based pattern). |
| 58 | +- No `__all__`. The module exports one name; users import it via `from httpware._internal import import_checker` and access `import_checker.is_msgspec_installed`. |
| 59 | + |
| 60 | +## `decoders/msgspec.py` |
| 61 | + |
| 62 | +```python |
| 63 | +"""MsgspecDecoder — opt-in ResponseDecoder backed by msgspec.json.decode.""" |
| 64 | + |
| 65 | +from typing import TypeVar |
| 66 | + |
| 67 | +from httpware._internal import import_checker |
| 68 | + |
| 69 | + |
| 70 | +if import_checker.is_msgspec_installed: |
| 71 | + import msgspec |
| 72 | + |
| 73 | + |
| 74 | +MISSING_DEPENDENCY_MESSAGE = ( |
| 75 | + "MsgspecDecoder requires the 'msgspec' extra. " |
| 76 | + "Install with: pip install httpware[msgspec]" |
| 77 | +) |
| 78 | + |
| 79 | +T = TypeVar("T") |
| 80 | + |
| 81 | + |
| 82 | +class MsgspecDecoder: |
| 83 | + """Decode raw response bytes via `msgspec.json.decode(content, type=model)`. |
| 84 | +
|
| 85 | + Requires the `msgspec` extra: `pip install httpware[msgspec]`. Importing |
| 86 | + this module without the extra works (the `msgspec` import is guarded by a |
| 87 | + `find_spec` check), but instantiating the decoder raises `ImportError` with |
| 88 | + the install hint. |
| 89 | + """ |
| 90 | + |
| 91 | + def __init__(self) -> None: |
| 92 | + if not import_checker.is_msgspec_installed: |
| 93 | + raise ImportError(MISSING_DEPENDENCY_MESSAGE) |
| 94 | + |
| 95 | + def decode(self, content: bytes, model: type[T]) -> T: |
| 96 | + """Validate `content` as JSON against `model` in a single parse pass.""" |
| 97 | + return msgspec.json.decode(content, type=model) |
| 98 | +``` |
| 99 | + |
| 100 | +Notes: |
| 101 | +- The runtime `import msgspec` is gated by `is_msgspec_installed`; if the flag is False, the `msgspec` name is undefined at module load, but `decode` is never reachable in that case (the constructor raises first). |
| 102 | +- No `__all__`. |
| 103 | +- The class has no subclassing hooks, but subclassing is permitted; subclasses inherit the same import check. |
| 104 | +- Single-pass invariant from `engineering.md` §3 holds: `msgspec.json.decode(content, type=model)` parses bytes and constructs the model in one C-level call. |
| 105 | + |
| 106 | +## Behavior and edge cases |
| 107 | + |
| 108 | +**Decode path:** |
| 109 | +- Both `msgspec.Struct` subclasses and pydantic `BaseModel` subclasses are valid `model` arguments — msgspec's decoder accepts both natively. |
| 110 | +- Builtin types (`int`, `str`, `list[int]`, etc.) also work. |
| 111 | +- Generic containers (`list[Item]`, `dict[str, Item]`) work. |
| 112 | + |
| 113 | +**Error propagation:** |
| 114 | +- `msgspec.ValidationError` — validation failure (shape mismatch, type mismatch). |
| 115 | +- `msgspec.DecodeError` — JSON parse failure (malformed bytes). |
| 116 | +- Both surface unchanged; the user wrote `response_model=X` to validate, and the validation error is the answer. Wrapping would obscure the actual failure. |
| 117 | + |
| 118 | +**Construction:** |
| 119 | +- `MsgspecDecoder()` succeeds when the extra is installed. |
| 120 | +- Raises `ImportError(MISSING_DEPENDENCY_MESSAGE)` when missing. |
| 121 | + |
| 122 | +**Import-time guarantees:** |
| 123 | +- `import httpware` does NOT touch `msgspec` — `httpware/__init__.py` does not import from `httpware.decoders.msgspec` (seam contract). |
| 124 | +- `import httpware.decoders.msgspec` only imports the `msgspec` C extension if `is_msgspec_installed` is True. The module itself imports successfully regardless. |
| 125 | +- `from httpware.decoders.msgspec import MsgspecDecoder` returns the class even when the extra is missing — only construction fails. This enables capability-probe code. |
| 126 | + |
| 127 | +## Testing |
| 128 | + |
| 129 | +### `tests/test_decoders_msgspec.py` — 8 tests |
| 130 | + |
| 131 | +| Test | Verifies | |
| 132 | +| --- | --- | |
| 133 | +| `test_decoder_satisfies_response_decoder_protocol` | `isinstance(MsgspecDecoder(), ResponseDecoder)`. | |
| 134 | +| `test_decode_into_msgspec_struct` | Decode JSON into a `msgspec.Struct` subclass. | |
| 135 | +| `test_decode_into_pydantic_model` | Decode JSON into a pydantic `BaseModel` — verifies msgspec handles both natively. | |
| 136 | +| `test_decode_into_builtin_type` | `decode(b'42', int) == 42`. | |
| 137 | +| `test_decode_into_list_of_struct` | `decode(b'[{"name":"a","qty":1}]', list[Item])` returns `[Item(...)]`. | |
| 138 | +| `test_decode_validation_error_propagates` | Schema mismatch (`"qty"` is not an int) raises `msgspec.ValidationError` unchanged. | |
| 139 | +| `test_decode_json_parse_error_propagates` | Malformed JSON (`b'{'`) raises `msgspec.DecodeError` unchanged. | |
| 140 | +| `test_construction_raises_without_extra_via_monkeypatch` | Monkeypatch `httpware._internal.import_checker.is_msgspec_installed = False`; `MsgspecDecoder()` raises `ImportError` containing the install-hint string. | |
| 141 | + |
| 142 | +### `tests/test_optional_extras_isolation.py` — 1 test |
| 143 | + |
| 144 | +| Test | Verifies | |
| 145 | +| --- | --- | |
| 146 | +| `test_importing_httpware_does_not_import_msgspec` | Subprocess: `python -c "import httpware; import sys; sys.exit(0 if 'msgspec' not in sys.modules else 1)"`. Asserts exit code 0. | |
| 147 | + |
| 148 | +The subprocess approach is necessary because `msgspec` IS installed in the test environment (via the `[msgspec]` extra and CI's `--all-extras` flag) — an in-process check would see `msgspec` in `sys.modules` from previous tests. The subprocess gets a fresh interpreter. |
| 149 | + |
| 150 | +Future stories extend this file with their own subprocess tests as new extras land (Story 5-4 OpenTelemetry, etc.). |
| 151 | + |
| 152 | +### Coverage and constraints |
| 153 | + |
| 154 | +- **Coverage expectation:** 100% line coverage on `import_checker.py` (one line) and `decoders/msgspec.py` (all branches). |
| 155 | +- **No `httpx2` import** in either new source file or new test file. The existing `tests/test_no_httpx2_leakage.py` continues to pass. |
| 156 | +- **No `from __future__ import annotations`.** |
| 157 | +- **No `print()`, no `logging.basicConfig`.** |
| 158 | +- **No `# type: ignore`.** If `ty` rejects `msgspec.json.decode(...)` because of the conditional `import msgspec`, fallback is `# ty: ignore[unresolved-reference]` on that one line with an explanatory comment. Lite-bootstrap precedent suggests this won't be needed. |
| 159 | + |
| 160 | +## CHANGELOG entry |
| 161 | + |
| 162 | +Under `[Unreleased]` / `### Added`: |
| 163 | + |
| 164 | +```markdown |
| 165 | +- `MsgspecDecoder` opt-in `ResponseDecoder` adapter behind the `[msgspec]` extra; `msgspec.json.decode(content, type=model)` in a single C-level parse pass. Accepts `msgspec.Struct`, pydantic `BaseModel`, and builtin types as `model`. `msgspec.ValidationError` and `msgspec.DecodeError` propagate unchanged. Module import is safe without the extra (gated by `httpware._internal.import_checker.is_msgspec_installed`); only `MsgspecDecoder()` construction raises `ImportError` with an install hint when the extra is missing. `import httpware` does NOT eagerly load `msgspec` — `MsgspecDecoder` is reachable only via `from httpware.decoders.msgspec import MsgspecDecoder` (Story 1.6). |
| 166 | +``` |
| 167 | + |
| 168 | +## Constraints and invariants |
| 169 | + |
| 170 | +- Honors the five protocol seams; specifically seam #5 (optional extras isolated to their own modules; no package-root re-export). |
| 171 | +- Decoder satisfies the `ResponseDecoder` protocol structurally (no nominal inheritance required). |
| 172 | +- Module-level `import msgspec` (when reached) is the SOLE `msgspec` import in the project — the same single-seam rule that applies to `httpx2`. Future test addition could mirror `tests/test_no_httpx2_leakage.py` if msgspec confinement becomes a CI-enforced invariant; deferred for now (msgspec is not as broadly used as httpx2 across the codebase). |
| 173 | +- `import_checker.py` itself never imports the modules it checks; only `find_spec`. |
| 174 | + |
| 175 | +## Risks and mitigations |
| 176 | + |
| 177 | +| Risk | Mitigation | |
| 178 | +| --- | --- | |
| 179 | +| `ty` rejects `msgspec.json.decode(...)` because `msgspec` is imported inside a runtime `if`. | Add `# ty: ignore[unresolved-reference]` on the call site with a comment pointing at the `import_checker` guard. Decided at implementation time; not expected based on lite-bootstrap precedent. | |
| 180 | +| Subprocess isolation test is flaky on the CI runner (e.g., `python` not on PATH). | Use `sys.executable` instead of bare `python` in the subprocess invocation. Standard pytest pattern. | |
| 181 | +| A future story breaks the "no package-root re-export" invariant by adding `from httpware.decoders.msgspec import MsgspecDecoder` to `httpware/__init__.py`. | The subprocess isolation test catches this regression: `import httpware` would then transitively load `msgspec`, and the assertion would fail. | |
| 182 | +| `MsgspecDecoder()` construction inside an `AsyncClient(decoder=MsgspecDecoder())` argument expression raises `ImportError` from deep inside a config-building stack when the extra is missing. | This is the intended behavior — the user is explicitly asking for the decoder. The install-hint message names the right extra and command. AsyncClient (Story 1-7) does not need to wrap or intercept. | |
| 183 | +| msgspec API drift (`msgspec.json.decode` signature change in a future major). | `msgspec>=0.18,<1.0` is the implicit constraint via the `[msgspec]` extra. Pin upper bound explicitly in pyproject if drift becomes a concern. Deferred. | |
| 184 | + |
| 185 | +## Definition of done |
| 186 | + |
| 187 | +- `src/httpware/_internal/import_checker.py` exists with `is_msgspec_installed`. |
| 188 | +- `src/httpware/decoders/msgspec.py` exists with `MsgspecDecoder` and `MISSING_DEPENDENCY_MESSAGE`. No `__all__`. |
| 189 | +- `tests/test_decoders_msgspec.py` contains 8 passing tests; 100% line coverage on the new source files. |
| 190 | +- `tests/test_optional_extras_isolation.py` contains the subprocess-based import-isolation test; passes. |
| 191 | +- `CHANGELOG.md` has a Story 1.6 bullet under `[Unreleased]` / `### Added`. |
| 192 | +- `just test` shows the increment from the post-2.3 baseline of 198 → 207 passed, 1 deselected, 100% coverage. |
| 193 | +- `just lint-ci` clean. |
| 194 | +- `tests/test_no_httpx2_leakage.py` still passes. |
| 195 | +- `import httpware` (via subprocess) does not load `msgspec` into `sys.modules`. |
| 196 | +- Story 1-6 lands as a single PR off `main` via the branch `story/1-6-msgspec-decoder-via-extras`. |
0 commit comments