You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit ea66fbb
Browse filesBrowse the repository at this point in the historyBrowse files
3.**Storage**: Valid events land in Iceberg tables; production uses AWS Glue as the catalog over object storage
46
46
4.**Quality**: Pre-storage gates check schema + semantic rules. Failures → dead letter topic
47
-
5.**Serving**: Agent API reads from the configured serving backend; the checked-in paths are DuckDB by default and optional ClickHouse for higher read concurrency, while Iceberg remains the lakehouse storage layer
47
+
5.**Bridge**: A consumer of `events.validated` applies each event to the serving store, idempotently by `event_id` — this is what makes the serving store move when a real Kafka event happens. After a successful apply the bridge also **pushes** metric-cache invalidation (Redis channel + in-process callback); a journal-scan fallback covers writers that do not push. See [Serving Bridge](serving-bridge.md#cache-invalidation-s7).
48
+
6.**Serving**: Agent API reads from the configured serving backend; the checked-in paths are DuckDB by default and optional ClickHouse for higher read concurrency, while Iceberg remains the lakehouse storage layer
48
49
49
50
For CDC sources, Debezium/Kafka Connect handles source capture while a shared normalizer converts Postgres/MySQL envelopes into one canonical AgentFlow CDC contract before validation. See [ADR 0005](decisions/0005-cdc-ingestion-strategy.md).
requests bypass the metric cache when multi-tenancy is configured) and
18
27
served through the production `QueryCache` (Redis semantics via in-process
19
28
`fakeredis`; localhost RTT to a real Redis is sub-millisecond and
20
-
excluded). Cache invalidation is driven by the webhook dispatcher poll
21
-
loop seeing new pipeline events — the same wiring as
22
-
`src/serving/api/main.py` production defaults.
29
+
excluded). At measurement time, cache invalidation was driven by the webhook
30
+
dispatcher poll loop seeing new pipeline events.
23
31
24
32
## System Under Test
25
33
@@ -39,7 +47,7 @@ loop seeing new pipeline events — the same wiring as
39
47
40
48
## Reading the numbers
41
49
42
-
-`event_driven`is the production configuration: staleness is bounded by the 2 s dispatcher poll, not the 30 s cache TTL.
50
+
-`event_driven`was the production configuration at measurement time: staleness was bounded by the 2 s dispatcher poll, not the 30 s cache TTL. Post-S7 the production path is push-driven (see note above); these numbers are the pre-S7 baseline.
43
51
- A TTL-only cache at the production TTL (30 s) would sit at ~U(0, TTL) staleness — p50 ≈ 15 s (linear scaling of the measured ttl_only arm: 2.75 s at TTL 5 s ⇒ ≈ 16.51 s at TTL 30 s). Event-driven invalidation measured 1.06 s p50 / 1.99 s p95 on the same pipeline.
44
52
-`fast_poll` shows the same wiring with the dispatcher poll tuned to 0.25 s — freshness is a configuration knob, not an architectural ceiling.
45
53
-`no_cache` is the floor: with the cache off every read hits the store directly, so the measured delay collapses to this script's 25 ms read granularity plus one query.
0 commit comments