Repository navigation
ControlPlaneStore port + embedded adapter — webhook queue/log (ADR 0010 slice 1) - #131
Merged
Merged
Conversation
…eue/log behind it (ADR 0010 slice 1) First rollout slice of ADR 0010: the webhook durable delivery queue and the delivery attempt log move behind a ControlPlaneStore port. Pure extraction — table DDL, SQL shapes, catalog-DDL-lock behavior and the dispatcher's public method signatures are byte-compatible with the pre-port code; no behavior change on the embedded (default) profile. - NEW src/serving/control_plane/: store.py (port ABC + WebhookQueueRow + get_control_plane_store resolver), embedded.py (EmbeddedControlPlaneStore over the serving DuckDB; ensure_* DDL helpers moved here verbatim, including the shared catalog_ddl_lock discipline). - Claim semantics are part of the port contract, not adapter detail: enqueue returns True only for the inserting caller (inline delivery belongs to the enqueue winner), claim_due returns rows this worker owns (embedded: degenerate single-process exclusivity; the PG adapter in slice 5 implements the same contract with FOR UPDATE SKIP LOCKED + lease). - webhook_dispatcher and routers/webhooks.py no longer reach into query_engine._conn; the single sanctioned reach is the composition seam in control_plane/store.py (documented), pinned by a structural ratchet test. The outcome state machine (delivered / backoff-pending / dead at max attempts) moved into the store; retry policy stays dispatcher config. - get_control_plane_store: lazy per-app resolution (mirrors ensure_alert_dispatcher so unit stubs keep working), eager boot-time resolution in the lifespan. AGENTFLOW_CONTROLPLANE_STORE env knob: 'embedded' (default) runs; 'postgres' raises NotImplementedError until slice 5 ships the adapter (fail-closed ratchet, pinned by test); unknown values are a boot error. - Tests: NEW tests/unit/test_control_plane_store.py (13 tests: enqueue-win, claim ordering/bounds/due-ness, outcome transitions, park, log roundtrip, resolver caching + ratchets + live-engine rebinding, structural _conn ratchet); test_webhook_dispatcher_unit / test_catalog_ddl_concurrency / integration test_webhooks rewired to the new module; the DDL-lock identity pin now covers control_plane.embedded. Verified: unit 42 (store+dispatcher+ddl) + integration webhooks 12 + full unit suite green locally; ruff check+format clean; mypy strict clean on all touched src files. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
DORA Metrics
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
ADR 0010 rollout slice 1: the webhook durable delivery queue (
webhook_delivery_queue) and the delivery attempt log (webhook_deliveries) move behind a newControlPlaneStoreport (src/serving/control_plane/). Pure extraction — no behavior change on the embedded (default) profile: table DDL, SQL shapes, the catalog-DDL-lock discipline and the dispatcher's pinned method signatures are byte-compatible with the pre-port code.Why
ADR 0010 (#130) decided control-plane state externalizes to PostgreSQL behind a port, rolled out in slices with the embedded DuckDB store staying the zero-dependency single-replica default. This is the first cut: it creates the seam and moves the subsystem with the sharpest delivery semantics, so slices 2–4 (alerts, outbox/dead-letter, usage) follow an established pattern and slice 5 (the PG adapter) implements an already-pinned contract.
Key design points
enqueue_webhook_deliveryreturnsTrueonly for the inserting caller — inline delivery belongs to the enqueue winner, so a second scanner (or replica) never re-POSTs a queued delivery;claim_due_webhook_deliveriesreturns rows the worker owns (embedded: degenerate single-process exclusivity; the slice-5 PG adapter implements the same contract withFOR UPDATE SKIP LOCKED+ lease).delivered/ backoff re-schedule /deadat max attempts) because on PostgreSQL it must be one transaction; retry policy (max attempts, backoff curve) stays dispatcher configuration.query_engine._connin the webhook path. The single sanctioned reach is the composition seam incontrol_plane/store.py(binding the embedded adapter to the serving DuckDB), and a structural ratchet test pins thatwebhook_dispatcher.pyandrouters/webhooks.pynever re-grow direct reaches.AGENTFLOW_CONTROLPLANE_STORE=postgresraisesNotImplementedErroruntil slice 5 ships (a scale profile must never silently fall back to the split-brain embedded store); unknown values fail the boot. Resolution is lazy per-app (mirrorsensure_alert_dispatcher, so existing unit stubs work unchanged) plus eager in the lifespan so misconfiguration fails at startup.Verification
tests/unit/test_control_plane_store.py— 13 tests: enqueue-win, claim ordering/bounds/due-ness, outcome transitions (pending→delivered / backoff / dead-at-max), park-on-removed-webhook, log roundtrip + isolation, resolver caching, the postgres/unknown ratchets, live-engine rebinding, and the structural_connratchet.test_webhook_dispatcher_unit(29),test_catalog_ddl_concurrency(DDL-lock identity pin now coverscontrol_plane.embedded), integrationtest_webhooks(12, incl. the durable re-drive and restart-survival pins).Scope
Slice 1 of 6 (see the ADR's rollout section). Slices 2–4 migrate alerts, outbox/dead-letter and usage behind this port; slice 5 ships
PostgresControlPlaneStore+ live verification; slice 6 wires the Helm scale profile.🤖 Generated with Claude Code