|
| 1 | +# Control plane on PostgreSQL — live verification (ADR 0010 rollout slice 5) |
| 2 | + |
| 3 | +**Date:** 2026-07-03 |
| 4 | +**Environment:** standalone PostgreSQL 17.5 (EDB windows-x64 binaries, no |
| 5 | +Docker, no service install: `initdb` + `pg_ctl`, port 55433, trust auth, |
| 6 | +user/db `agentflow`) — the same no-Docker standalone-PG recipe that verified |
| 7 | +the vault governance layer |
| 8 | +(`vault-pii-governance-pg-verify-2026-07-02.md`). Adapter under test: |
| 9 | +`src/serving/control_plane/postgres.py` (`PostgresControlPlaneStore`), |
| 10 | +driven by the probe suite the ADR names for this slice: |
| 11 | +`tests/integration/test_control_plane_postgres_live.py` with |
| 12 | +`AGENTFLOW_TEST_PG_DSN=postgresql://agentflow@127.0.0.1:55433/agentflow`. |
| 13 | + |
| 14 | +**Result: 31/31 probes passed** (`pytest`, 19.45s; psycopg 3.3.4). The same |
| 15 | +suite runs in CI against the `postgres:17` service container added to the |
| 16 | +`test-integration` job (it self-skips when the DSN env var is absent, the |
| 17 | +`test_clickhouse_backend_live.py` pattern). |
| 18 | + |
| 19 | +## The ADR's named probes |
| 20 | + |
| 21 | +| Probe (ADR 0010 § Rollout 5) | Test | Result | |
| 22 | +| --- | --- | --- | |
| 23 | +| Enqueue-win uniqueness | 8 threads race `enqueue_webhook_delivery` on one (webhook, event) → exactly **1** `True`, 1 row | passed | |
| 24 | +| Parallel claim exclusivity | 4 threads claim 10 due rows concurrently → no row handed out twice, none lost (`FOR UPDATE SKIP LOCKED`) | passed | |
| 25 | +| Lease-expiry re-drive | row claimed with a 0.4s lease, owner "crashes" → invisible while leased, claimable again after expiry; outcome writes clear the lease so backoff alone governs | passed | |
| 26 | +| Restart re-drive | pending row enqueued by one store instance is claimed by a **fresh** instance, canonical body verbatim | passed | |
| 27 | +| Outbox↔dead-letter atomicity (invariant 8) | `mark_outbox_sent` flips both rows in one transaction; with `dead_letter_events` dropped mid-scenario the outbox flip **rolls back** (row stays `pending`); `enqueue_outbox_replay` rolls back symmetrically | passed | |
| 28 | +| Alert-tick single-flight (§2) | second claimant loses; `complete_alert_tick` persists the advanced record and releases in one transaction; a stale claim self-expires; a concurrent CRUD full-set save does **not** release an in-flight claim | passed | |
| 29 | + |
| 30 | +## Contract parity sweep |
| 31 | + |
| 32 | +Every port method exercised against live PostgreSQL with the same assertions |
| 33 | +the embedded adapter's unit pins make: webhook outcome state machine |
| 34 | +(backoff → `dead` at max, success → `delivered`, park), oldest-first claim |
| 35 | +ordering under `limit`, attempt-log and alert-history roundtrips (newest |
| 36 | +first, JSON payload decoded), webhook-registration and alert-rule |
| 37 | +repositories (order preserved via a `position` column, full-set save = |
| 38 | +YAML-replace semantics, ids required), dead-letter reads (tenant scoping, |
| 39 | +reason filter, pagination, stats + trend), usage accounting (per tenant / |
| 40 | +per key / old-key-slot hour window), session analytics (idempotent |
| 41 | +insert-or-replace on `request_id`, usage/top-queries/top-entities/latency |
| 42 | +percentiles/anomalies/QPS shapes, malformed window → `ValueError`, QPS |
| 43 | +degrades to 0.0 on an unreachable server). |
| 44 | + |
| 45 | +## End to end: the app itself on the postgres profile |
| 46 | + |
| 47 | +`test_app_on_postgres_profile_shares_state_across_boots` boots the real |
| 48 | +FastAPI app twice with `AGENTFLOW_CONTROLPLANE_STORE=postgres`: |
| 49 | + |
| 50 | +- boot #1 resolves `PostgresControlPlaneStore` in the lifespan, |
| 51 | + `AuthManager.store` **is** the shared app-wide store (slice 5 injection in |
| 52 | + `main.py`), and `POST /v1/webhooks` registers a webhook; |
| 53 | +- boot #2 (a second pod, in production terms) sees that registration through |
| 54 | + `GET /v1/webhooks` — the class-5 per-pod YAML split-brain is gone; |
| 55 | +- `api_usage` rows for the authenticated tenant landed in PostgreSQL, not in |
| 56 | + a local DuckDB file. |
| 57 | + |
| 58 | +## Notes |
| 59 | + |
| 60 | +- Claim leases default to 300s (`AGENTFLOW_CONTROLPLANE_LEASE_SECONDS` |
| 61 | + overrides); alert-tick leases are 120s (`AlertDispatcher.tick_lease_seconds`). |
| 62 | +- Payload columns are TEXT holding JSON strings — callers see the same |
| 63 | + "string, you decode it" shape as on the embedded adapter. |
| 64 | +- One connection per method call, no pooling (out of ADR 0010 scope, |
| 65 | + recorded follow-up). |
0 commit comments