Skip to content

Commit 0240751

Browse files
lesnik512claude
andauthored
docs(releases): 0.11.0 release notes (#136)
Curated notes for the 0.11.0 minor release: producer NOTIFY dedup (#135), opt-in batched terminal flush (#131/#132), outbox_autovacuum_ddl helper + cost knobs and check_autovacuum probe (#133/#134), benchmark harness (#130). Required by release.yml before the 0.11.0 tag can publish. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 2afc0ac commit 0240751

1 file changed

Lines changed: 133 additions & 0 deletions

File tree

‎planning/releases/0.11.0.md‎

Lines changed: 133 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,133 @@
1+
# faststream-outbox 0.11.0 — a performance pass: write-path, terminal-path, and autovacuum tooling
2+
3+
**Minor release.** A throughput-focused release built from a performance review
4+
of the transactional-outbox transport: the producer stops emitting redundant
5+
`NOTIFY`s, the subscriber gains an opt-in batched terminal flush, and a new
6+
`outbox_autovacuum_ddl` helper (plus an opt-in `validate_schema` probe) makes the
7+
recommended table hygiene one function call. A reproducible benchmark harness
8+
underpins all of it. **Backward-compatible by default** — every new knob defaults
9+
to today's behavior, so an in-place upgrade changes nothing until you opt in.
10+
11+
## Performance
12+
13+
- **Producer `NOTIFY` dedup — one `pg_notify` per `(transaction, queue)`, not per
14+
publish.** `broker.publish` / `publish_batch` ran a `SELECT pg_notify(channel,
15+
queue)` on every call, so N publishes to one queue in one transaction emitted N
16+
`NOTIFY` statements — N−1 of them pure round-trip waste, since Postgres already
17+
coalesces identical `(channel, payload)` `NOTIFY`s per transaction at delivery. A
18+
transaction-scoped memo (a `WeakKeyDictionary` keyed on the innermost active
19+
transaction) now emits exactly one `pg_notify` per distinct queue per
20+
transaction. **Behavior-preserving** — the subscriber still gets the same
21+
"≥1 wake per queue per commit" it did before, fired inline at the first publish;
22+
delivery, ordering, and the transactional producer contract are unchanged. On a
23+
bulk publish (5000 rows, one queue, one transaction) the write path measured
24+
**~1.9× throughput / ~46% less wall-clock** on loopback Postgres, and the
25+
absolute saving grows with database round-trip latency. Single-publish-per-
26+
transaction workloads are unaffected (one row still emits one `NOTIFY`). Default-on,
27+
no knob. (#135)
28+
29+
## New features
30+
31+
- **Opt-in batched terminal flush — `terminal_flush_batch_size` (default `1`).**
32+
The subscriber deletes/DLQs each terminally-handled row individually by default
33+
(`terminal_flush_batch_size=1`, identical to prior behavior). Set it higher to
34+
batch terminal writes into one statement per N rows, cutting the terminal-path
35+
statement count under high throughput. This **widens the at-least-once
36+
redelivery window** — a crash between handler success and the batched flush
37+
redelivers up to `N−1` already-processed rows on recovery — so it is opt-in and
38+
off by default; only raise it when your handlers are idempotent. Available on the
39+
broker subscriber, `OutboxRouter`, and the FastAPI router. (#131, #132)
40+
41+
- **`outbox_autovacuum_ddl(...)` — the recommended per-table autovacuum settings
42+
as one call.** A new top-level export that renders the
43+
`ALTER TABLE … SET (autovacuum_*)` statement for an outbox table — drop it into
44+
an Alembic migration (`op.execute(outbox_autovacuum_ddl("outbox"))`) or run it via
45+
psql. It sets `autovacuum_vacuum_scale_factor = 0` plus size-independent dead-tuple
46+
/ insert thresholds (eligibility), breaking the stale-`reltuples` death-spiral that
47+
makes a high-churn queue table bloat. Signature:
48+
49+
```python
50+
outbox_autovacuum_ddl(
51+
table_name="outbox", *, schema=None,
52+
vacuum_threshold=1000, insert_threshold=1000,
53+
vacuum_cost_delay=None, vacuum_cost_limit=None,
54+
) -> str
55+
```
56+
57+
`vacuum_cost_delay` / `vacuum_cost_limit` are the vacuum **throughput** knobs
58+
(off by default → cluster default emitted). A churn measurement showed throughput
59+
— not eligibility — is the binding constraint under heavy sustained load, so these
60+
let vacuum keep pace; `vacuum_cost_delay=0` runs it unthrottled (fast, but
61+
I/O-heavy on a shared cluster — opt in deliberately). (#133, #134)
62+
63+
- **`validate_schema(check_autovacuum=True)` — opt-in autovacuum probe.** The
64+
schema validator gains a `check_autovacuum` flag (default `False`) that asserts the
65+
live table carries the structural `scale_factor = 0` + threshold settings, so a CI
66+
gate can catch a table created without the recommended hygiene. The situational
67+
throughput knobs (`cost_delay` / `cost_limit`) are **not** enforced — they are
68+
tuning, not a structural requirement. Default `False` keeps `validate_schema()`
69+
identical to 0.10.x. (#133)
70+
71+
## Bug fixes
72+
73+
- **`validate_schema()` now reflects a table in a named schema correctly.** The
74+
validator's Alembic reflection omitted `include_schemas=True`, so a table created
75+
under a non-default schema (`make_outbox_table(..., schema="app")`) failed
76+
validation spuriously. Reflection now filters by the target schema, and an
77+
explicit-default-schema table (`MetaData(schema="public")`) is normalized against
78+
the connection's `default_schema_name` so it validates as unqualified. (#133)
79+
80+
## Internals / tooling
81+
82+
- **Benchmark harness for the outbox transport (`benchmarks/`, `just bench` /
83+
`just bench-check`).** A reproducible producer/consumer workload sweep that records
84+
per-message DB counters via `pg_stat_statements` (split by leading SQL keyword) and
85+
gates the deterministic, machine-independent counts against
86+
`benchmarks/baseline.json` in CI. This is contributor tooling — it drove and now
87+
locks the wins above (the producer `NOTIFY` dedup shows as `select_calls` 5000 → 1;
88+
the batched terminal flush as fewer `delete_calls`). Not part of the installed
89+
wheel. (#130)
90+
- **Honest autovacuum docs.** The operations and architecture docs were rewritten to
91+
scope the benefit precisely: the shipped `scale_factor=0` + threshold settings
92+
control vacuum *eligibility* (necessary, not sufficient); under heavy churn,
93+
*throughput* (`cost_delay`/`cost_limit`) is the binding lever. Framed as standard
94+
queue-table hygiene and insurance, not a measured silver bullet. (#134)
95+
96+
## Breaking changes
97+
98+
None. Every new parameter defaults to prior behavior:
99+
`terminal_flush_batch_size=1` (per-message flush), `check_autovacuum=False`, and the
100+
`outbox_autovacuum_ddl` cost knobs default to `None` (cluster default). The
101+
`NOTIFY` dedup is behavior-preserving. An in-place upgrade requires no code or
102+
schema change.
103+
104+
## Migration
105+
106+
- **No action required to upgrade.** Optional adoption:
107+
- Set `terminal_flush_batch_size=N` on your subscriber(s) **only if your handlers
108+
are idempotent** — it trades a wider redelivery window for terminal-path
109+
throughput.
110+
- Apply `outbox_autovacuum_ddl("<table>")` in an Alembic migration to adopt the
111+
recommended autovacuum settings; add `vacuum_cost_delay`/`vacuum_cost_limit` if
112+
you run a high-churn table and vacuum can't keep pace.
113+
- Add `check_autovacuum=True` to a `validate_schema()` CI gate to enforce the above.
114+
115+
## Touched surface
116+
117+
- `faststream_outbox/publisher/producer.py` — per-`(transaction, queue)` `NOTIFY`
118+
dedup memo. (#135)
119+
- `faststream_outbox/subscriber/` + `registrator.py`, `router.py`,
120+
`fastapi/router.py` — `terminal_flush_batch_size` plumbed through the subscriber
121+
config, factory, and all three registration entry points. (#131)
122+
- `faststream_outbox/autovacuum.py` (new) + `__init__.py` export — `outbox_autovacuum_ddl`. (#133, #134)
123+
- `faststream_outbox/client.py`, `broker.py`, `testing.py` — `validate_schema(check_autovacuum=...)` + named-schema reflection fix. (#133)
124+
- `benchmarks/` (new) + `Justfile` (`bench` / `bench-check`) + CI `bench` job. (#130)
125+
- `docs/operations/`, `architecture/` — batching, autovacuum (throughput-scoped), and producer-`NOTIFY` docs. (#132, #134, #135)
126+
127+
## See also
128+
129+
- [`planning/changes/2026-07-14.01-benchmark-harness.md`](../changes/2026-07-14.01-benchmark-harness.md) ([#130](https://github.com/modern-python/faststream-outbox/pull/130))
130+
- [`planning/changes/2026-07-15.01-batched-terminal-flush.md`](../changes/2026-07-15.01-batched-terminal-flush.md) ([#131](https://github.com/modern-python/faststream-outbox/pull/131), [#132](https://github.com/modern-python/faststream-outbox/pull/132))
131+
- [`planning/changes/2026-07-15.02-autovacuum-tuning.md`](../changes/2026-07-15.02-autovacuum-tuning.md) ([#133](https://github.com/modern-python/faststream-outbox/pull/133))
132+
- [`planning/changes/2026-07-16.01-autovacuum-cost-knobs.md`](../changes/2026-07-16.01-autovacuum-cost-knobs.md) ([#134](https://github.com/modern-python/faststream-outbox/pull/134))
133+
- [`planning/changes/2026-07-16.02-notify-dedup.md`](../changes/2026-07-16.02-notify-dedup.md) ([#135](https://github.com/modern-python/faststream-outbox/pull/135))

0 commit comments

Comments
 (0)