Repository navigation
Record API responses in tests for client compatibility checks - #7368
Merged
Merged
Conversation
mdegat01
force-pushed
the
client-compat-recorder
branch
from
October 14, 2026 03:38
723c1aa to
be9bf0e
Compare
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Fresh recording directories fail, and envelope-shaped Core proxy responses are incorrectly recorded.
2 open findings
What changed in this PR
Adds test-time API response recording for future aiohasupervisor compatibility checks.
Changes:
- Adds JSONL response-recording middleware.
- Registers recording across v1, v2, and app API clients.
- Tests enabled and disabled recording behavior.
| File | Description |
|---|---|
tests/common.py |
Implements response recording. |
tests/conftest.py |
Registers middleware for the primary API client. |
tests/api/conftest.py |
Registers middleware for additional API clients. |
tests/api/test_api_record.py |
Tests recorder output and opt-in behavior. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Add an aiohttp middleware to the API test clients that, when SUPERVISOR_API_RECORD_DIR is set, appends each Supervisor JSON response with its route template, params, query, status and test id to a JSONL file per xdist worker. The client library's compat checker replays these records to catch responses it cannot parse. It is a no-op when the variable is unset. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Create the record directory on first write instead of requiring callers to create it. Skip the Core API proxy and ingress passthrough routes by their route template, since Core and app responses can look like Supervisor envelopes. Drop the JSON and envelope checks so unexpected responses fail loudly in the compat check instead of being skipped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mdegat01
force-pushed
the
client-compat-recorder
branch
from
October 14, 2026 18:10
c625faa to
6d465b2
Compare
5 of 14 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Proposed change
This is step 2 of #7307: recording the responses Supervisor's API tests produce, so the
aiohasupervisorcompat checker (python-supervisor-client#431) can check that released client versions can still parse them.tests/common.pyaddsapi_record_middleware(), which is registered on theapi_client,api_client_v2andapp_api_client_with_prefixfixtures. The other API client fixtures reuse those. WhenSUPERVISOR_API_RECORD_DIRis set, every response whose body is a Supervisor JSON envelope is appended as one line to<dir>/<PYTEST_XDIST_WORKER or "main">.jsonl:{"method": "GET", "template": "/host/disks/{disk}/usage", "params": ["default"], "query": [["max_depth", "2"]], "status": 200, "body": {...}, "test": "tests/api/test_host.py::test_disk_usage_api"}templateis the matched route's canonical path, including/v2for v2 routes.paramsholds the route values in template order.queryholds the query string as[key, value]pairs. Query keys can change a response (e.g.core/stats?one_shot), and the checker matches on them.The middleware docstring documents the record format, which is the contract the client's checker reads.
tests/api/test_api_record.pycovers v1 and v2 recording, and covers the variable being unset.Step 3, the
client-compatCI job that runs the checker for the client versions in use, is still to do, so this doesn't close #7307.Expected failures when running the checker locally: if you record with this branch and run the checker from the latest client release (0.7.0), it reports two failures until the client PRs below are merged and released:
GET /addons/{addon}/infofor an app that isn't installed. Supervisor's v1 fallback returns store info, which the client can't parse asInstalledAddonComplete. This is a real client bug, fixed by Raise not found from addon_info for apps that are not installed home-assistant-libs/python-supervisor-client#439.GET /core/stats?one_shot. The checker didn't take the query string into account, so this was a false positive. Fixed by Match query keys in the compat checker home-assistant-libs/python-supervisor-client#438.With both client PRs applied, a recording of
tests/apifrom this branch checks clean: 1108 responses, 216 replayed through the client, 0 failures, 0 warnings. Supervisor's own test suite passes either way.Type of change
Additional information
Checklist
ruff format supervisor tests)If API endpoints or add-on configuration are added/changed:
🤖 Generated with Claude Code