|
| 1 | +# ADR-0010: Bounded page rotation with generation-tagged reuse |
| 2 | + |
| 3 | +**Status:** Proposed |
| 4 | +**Date:** 2026-07-14 |
| 5 | +**Issue/PR:** pending |
| 6 | + |
| 7 | +> **Design-first ADR.** No implementation accompanies this document. Per |
| 8 | +> [architecture.md](../architecture.md) §14 and the ADR index rule, a Proposed |
| 9 | +> ADR permits only unwired prototypes, and this one intentionally lands before |
| 10 | +> any prototype so the rotation state machine is reviewed as a design, not as |
| 11 | +> a diff. ADR-0009 §"Revisit when" required exactly this stop. |
| 12 | +
|
| 13 | +## Context |
| 14 | + |
| 15 | +ADR-0009's `VolatileAdmissionChannel<N>` binds one preallocated |
| 16 | +published-prefix page (ADR-0008) to one SPSC descriptor ring (ADR-0006). When |
| 17 | +the page's byte or descriptor capacity is exhausted the lane is finished: the |
| 18 | +typed `page full` error is terminal for the channel's useful life, and the |
| 19 | +producer's only recovery is to tear down and rebuild the composite, losing the |
| 20 | +ring, sequence continuity, and the consumer binding. |
| 21 | + |
| 22 | +A production event fabric cannot run on one page per lane lifetime. It needs |
| 23 | +the producer to continue admitting into a fresh page while the consumer |
| 24 | +finishes draining the previous one — with the number of simultaneously live |
| 25 | +pages **bounded by construction**, page memory **reused** rather than |
| 26 | +reallocated, and reuse **provably safe** against stale descriptors addressing |
| 27 | +a recycled page (the ABA problem). |
| 28 | + |
| 29 | +Scope boundaries inherited from ADR-0009 remain: one producer, one consumer, |
| 30 | +volatile, process-local, unwired, non-authoritative. This ADR adds only |
| 31 | +rotation, bounded outstanding pages, and generation-tagged reuse. It does |
| 32 | +**not** add WAL durability/replay, authenticated registry lookup, NUMA pool |
| 33 | +placement policy, priority lanes, crossbeam-epoch, multi-producer or |
| 34 | +multi-consumer topology, or any protected-evidence semantics. |
| 35 | + |
| 36 | +## Decision |
| 37 | + |
| 38 | +Add a `RotatingAdmissionChannel<N, P>` in `aegis-event`: one SPSC descriptor |
| 39 | +ring bound to a **fixed pool of `P >= 2` identically-configured page slots**, |
| 40 | +all preallocated at construction. No allocation, deallocation, or `Arc` |
| 41 | +reference-count traffic occurs after `split`. |
| 42 | + |
| 43 | +### Page epochs and slot addressing |
| 44 | + |
| 45 | +Pages are identified by a monotonically increasing **page epoch** `e` |
| 46 | +(`u64`, starting at 0). Epoch `e` occupies pool slot `e mod P` and stamps its |
| 47 | +descriptors with `arena_generation = e as u32` (wrapping). The consumer |
| 48 | +recovers the slot index from a descriptor as |
| 49 | +`descriptor.arena_generation as u64 mod P` and then requires **exact |
| 50 | +generation equality** with the slot's currently-bound page before any |
| 51 | +descriptor-cell or payload access — the same identity check ADR-0007/0008 |
| 52 | +already enforce, now doing double duty as the reuse guard. |
| 53 | + |
| 54 | +Sequence space is continuous across pages: page epoch `e+1` is constructed |
| 55 | +with `first_sequence` equal to the sealed page `e`'s `next_sequence`, so ring |
| 56 | +sequence and page sequence remain a single unbroken wrapping sequence exactly |
| 57 | +as in ADR-0009. |
| 58 | + |
| 59 | +### Producer rotation algorithm |
| 60 | + |
| 61 | +Rotation is attempted only inside admission, only on the typed |
| 62 | +`page full` / `descriptor full` validation results, and **before** ring |
| 63 | +reservation — a rotated-then-admitted event keeps ADR-0009's phase discipline |
| 64 | +unchanged: |
| 65 | + |
| 66 | +```text |
| 67 | +validate payload length against the ACTIVE page |
| 68 | +on page/descriptor full: |
| 69 | + Acquire-load consumer released_epoch [A] |
| 70 | + require active_epoch + 1 <= released_epoch + P (slot free?) |
| 71 | + on failure: return typed PageQuotaExhausted — nothing mutated |
| 72 | + Release-close the active page writer (seal, ADR-0008) [S] |
| 73 | + rebind slot (active_epoch + 1) mod P: |
| 74 | + new generation = (active_epoch + 1) as u32 |
| 75 | + first_sequence = sealed page next_sequence |
| 76 | + active_epoch += 1 |
| 77 | + re-validate payload against the fresh page |
| 78 | +then the unchanged ADR-0009 admission sequence: |
| 79 | + reserve ring slot, InFlight, append + page Release [P], |
| 80 | + ring slot write, ring Release [R], Open |
| 81 | +``` |
| 82 | + |
| 83 | +Properties: |
| 84 | + |
| 85 | +- **Bounded work, no waiting.** Rotation is a seal, one Acquire load, one |
| 86 | + slot re-initialization over preallocated memory, and bookkeeping. If the |
| 87 | + successor slot is still outstanding, the producer returns |
| 88 | + `PageQuotaExhausted` with **zero page and zero ring mutation** — the caller |
| 89 | + applies its declared event-class policy (bounded retry, spool, or |
| 90 | + best-effort drop). No spin, sleep, allocation, or overflow queue. |
| 91 | +- **A payload larger than one empty page** is still the ADR-0008 typed |
| 92 | + invalid-input error; rotation never loops. |
| 93 | +- **Failure atomicity is preserved.** The quota check happens before the seal |
| 94 | + `[S]`, so a refused rotation leaves the active page open and usable for |
| 95 | + smaller payloads. An unwind during rotation follows ADR-0009's caught-unwind |
| 96 | + contract: ordered faulted closure of the page (sealed or fresh), terminal |
| 97 | + word, and ring. |
| 98 | + |
| 99 | +### Consumer release protocol |
| 100 | + |
| 101 | +The ring is FIFO and consumption is strictly in-order (ADR-0009's exact |
| 102 | +expected-sequence rule), so descriptors arrive grouped by epoch in epoch |
| 103 | +order. The consumer tracks `current_epoch` and, upon validating the first |
| 104 | +descriptor of epoch `e+1`: |
| 105 | + |
| 106 | +1. requires its per-epoch committed count to equal the sealed page `e`'s |
| 107 | + published count — a shortfall is the terminal orphaned-prefix/count |
| 108 | + mismatch of ADR-0009, now detected at the page boundary instead of only at |
| 109 | + end of stream; |
| 110 | +2. Release-stores `released_epoch = e` `[E]`; |
| 111 | +3. proceeds with ADR-0009 validation of the new epoch's descriptor against |
| 112 | + the freshly Acquire-loaded slot binding. |
| 113 | + |
| 114 | +Clean or faulted end-of-stream releases the final epoch after the ADR-0009 |
| 115 | +terminal checks, which now aggregate: total committed count must equal the |
| 116 | +sum of sealed published counts plus the final page's published count. |
| 117 | + |
| 118 | +### Reclamation proof (why no epochs are needed) |
| 119 | + |
| 120 | +The single reclamation edge is consumer → producer over `released_epoch`: |
| 121 | + |
| 122 | +```text |
| 123 | +last frame commit for epoch e, ring-tail Release [C] |
| 124 | + -> released_epoch Release-store [E] |
| 125 | + -> producer released_epoch Acquire [A] |
| 126 | + -> slot (e mod P) rebind and first cell write of epoch e+P |
| 127 | +``` |
| 128 | + |
| 129 | +- A payload lease (`AdmittedEvent`) mutably borrows the consumer, so no lease |
| 130 | + can be alive when the consumer later executes `[E]` inside `try_next` — the |
| 131 | + borrow checker, not a runtime count, proves no reader holds bytes of a page |
| 132 | + being released. This is the property crossbeam-epoch would otherwise buy, |
| 133 | + and it holds only because the topology is exactly one consumer; any |
| 134 | + multi-reader future (query snapshots, secondary indexes) must not reuse |
| 135 | + this argument and gets its own ADR. |
| 136 | +- The producer never rebinds a slot without Acquire-observing `[E]` for its |
| 137 | + previous occupant, so every cell write of epoch `e+P` happens-after every |
| 138 | + committed read of epoch `e`. |
| 139 | +- **ABA/wrap safety:** at most `P` epochs are ever live, and the ring is |
| 140 | + FIFO, so a descriptor observable by the consumer references an epoch in |
| 141 | + `[released_epoch, active_epoch]`, a window of width `<= P`. The `u32` |
| 142 | + generation tag is unambiguous while `P < 2^32`, which the type-level |
| 143 | + `P: usize` bound enforces absurdly early; the epoch counter itself is `u64` |
| 144 | + and non-wrapping for any realistic process lifetime (`2^64` rotations). |
| 145 | + A descriptor whose generation fails slot equality is the ADR-0009 terminal |
| 146 | + identity mismatch — never a skip, never a fallback read. |
| 147 | + |
| 148 | +### Terminal-state extension |
| 149 | + |
| 150 | +The composite terminal word and its `OPEN/CLEAN/FAULTED` values are unchanged. |
| 151 | +`finish` seals the **active** page before storing `CLEAN` and closing the |
| 152 | +ring. Clean end-of-stream now additionally requires every sealed epoch to have |
| 153 | +been fully committed at its boundary (checked incrementally by the release |
| 154 | +protocol) — so the aggregate clean condition remains "total committed equals |
| 155 | +total published", with page-boundary early detection as a strengthening, not |
| 156 | +a replacement, of ADR-0009's end-of-stream checks. |
| 157 | + |
| 158 | +## Public result semantics |
| 159 | + |
| 160 | +`AdmittedSequence` gains nothing: it already carries page identity, generation, |
| 161 | +and wrapping sequence. Success semantics are ADR-0009's verbatim — volatile |
| 162 | +publication only; never consumption, durability, a receipt, or authorization. |
| 163 | + |
| 164 | +New/changed typed errors: |
| 165 | + |
| 166 | +| Error | Page mutation | Ring mutation | Retry meaning | |
| 167 | +|---|---:|---:|---| |
| 168 | +| `PageQuotaExhausted` (all `P` slots outstanding) | none | none | consumer lagging; bounded retry/spool per event class | |
| 169 | +| `page full` / `descriptor full` | no longer surfaced when rotation succeeds; surfaced unchanged when the payload exceeds one empty page | none | correct input | |
| 170 | + |
| 171 | +All other ADR-0009 error rows are unchanged. |
| 172 | + |
| 173 | +## Memory and ownership layout |
| 174 | + |
| 175 | +```text |
| 176 | +RotatingAdmissionChannel<N, P> |
| 177 | + pool: [page slot 0][page slot 1]...[page slot P-1] (preallocated) |
| 178 | + ring, terminal word (as ADR-0009) |
| 179 | + released_epoch: cache-line-aligned AtomicU64 (consumer writes, |
| 180 | + producer reads) |
| 181 | +
|
| 182 | +producer: [active writer][active_epoch][pool handles][ring producer][terminal] |
| 183 | +consumer: [ring consumer][current_epoch][per-epoch committed][pool handles] |
| 184 | +``` |
| 185 | + |
| 186 | +`released_epoch` lives on its own cache line: it is written once per page |
| 187 | +lifetime, not per event, so rotation metadata adds no steady-state false |
| 188 | +sharing to the `[P]`/`[R]`/`[C]` hot lines. Slot rebinding reuses the page |
| 189 | +allocation in place; the pool never grows, shrinks, or reallocates. |
| 190 | + |
| 191 | +## Failure, overload, and security behavior |
| 192 | + |
| 193 | +- `PageQuotaExhausted` is backpressure, not data loss: nothing is admitted, |
| 194 | + nothing is dropped, and the caller's event-class policy decides. The |
| 195 | + critical/WAL lane semantics remain future work — this channel still cannot |
| 196 | + carry protected evidence. |
| 197 | +- A slow or stalled consumer bounds producer memory at exactly `P` pages plus |
| 198 | + the ring; there is no unbounded queue anywhere in the composite |
| 199 | + (architecture law §2.7). |
| 200 | +- Generation mismatch, epoch-boundary count shortfall, sequence discontinuity, |
| 201 | + CRC failure, and corrupt terminal state all remain terminal, fail-closed |
| 202 | + lane errors — rotation adds detection points, never recovery-by-skipping. |
| 203 | +- Descriptors remain non-capabilities; slot addressing via |
| 204 | + `generation mod P` selects memory already owned by this channel's bound |
| 205 | + pool and never a registry, tenant, or foreign allocation. Authenticated |
| 206 | + registry lookup remains explicitly out of scope and future work. |
| 207 | + |
| 208 | +## Progress and performance hypothesis |
| 209 | + |
| 210 | +Steady-state admission and consumption are byte-for-byte the ADR-0009 paths; |
| 211 | +rotation adds one Acquire load on the page-full branch only. The hypothesis is |
| 212 | +that amortized cost per event is unchanged and rotation cost is `O(1)` |
| 213 | +bounded, paid once per page. No measurement accompanies this ADR; the fabric |
| 214 | +remains `target` with no performance claim, and qualification requirements are |
| 215 | +unchanged from ADR-0009. |
| 216 | + |
| 217 | +## Alternatives considered |
| 218 | + |
| 219 | +- **Allocate a fresh page per rotation, drop the old one** — rejected: |
| 220 | + per-page allocation/free in the hot path, unbounded live pages under a slow |
| 221 | + consumer, and no reuse story; violates the bounded-everything law. |
| 222 | +- **crossbeam-epoch reclamation now** — rejected: the SPSC borrow-checker |
| 223 | + argument above makes epochs redundant for this topology; epochs enter with |
| 224 | + multi-reader query snapshots (LLD §epoch retirement) under their own ADR. |
| 225 | +- **Per-slot busy/free atomic flags** — rejected: a single released-epoch |
| 226 | + counter is sufficient under FIFO in-order consumption and keeps one |
| 227 | + reclamation edge to prove instead of `P`. |
| 228 | +- **Producer blocks/spins when the pool is exhausted** — rejected: hot crates |
| 229 | + admit no unbounded waits; typed backpressure lets the declared event class |
| 230 | + decide. |
| 231 | +- **Encode slot index in `arena_id`** — rejected: `arena_id` identifies the |
| 232 | + lane/arena binding and participates in identity checks across the channel; |
| 233 | + overloading it conflates lane identity with position. The epoch already |
| 234 | + determines the slot. |
| 235 | +- **Skip the page-boundary count check and rely on end-of-stream totals** — |
| 236 | + rejected: boundary checking converts a silent mid-stream orphan into an |
| 237 | + immediate terminal error while the evidence is fresh. |
| 238 | + |
| 239 | +## Verification (required before the prototype can merge) |
| 240 | + |
| 241 | +The implementing PR must provide, mirroring ADR-0009's evidence classes: |
| 242 | + |
| 243 | +- rotation at byte exhaustion and at descriptor exhaustion; sequence |
| 244 | + continuity across the seam; admission token generations advancing; |
| 245 | +- `PageQuotaExhausted` with zero page/ring mutation, then successful admission |
| 246 | + after the consumer crosses the boundary; |
| 247 | +- generation-tagged reuse: slot rebinding after release, stale-descriptor |
| 248 | + injection failing identity terminally, wrap of the `u32` tag under a small |
| 249 | + `P` fixture; |
| 250 | +- epoch-boundary committed-count shortfall detected terminally at the seam; |
| 251 | +- caught-unwind fault injection at: before quota check, after seal `[S]` |
| 252 | + before rebind, after rebind before re-validation, plus all ADR-0009 points; |
| 253 | +- differential oracle (safe sealed pages + `VecDeque`) across many rotations |
| 254 | + with variable-length payloads; |
| 255 | +- Loom models for the release/rebind race (`[C]→[E]→[A]→rebind`), quota |
| 256 | + refusal versus in-flight release, and faulted closure mid-rotation; |
| 257 | +- full Miri; ASan/TSan lanes extended to the rotation suites; zero |
| 258 | + steady-state allocations including across a rotation. |
| 259 | + |
| 260 | +## Migration and rollback |
| 261 | + |
| 262 | +While Proposed, nothing ships. The implementing PR lands |
| 263 | +`RotatingAdmissionChannel` as isolated, unwired prototype code beside the |
| 264 | +single-page composite, which remains the reviewed baseline and differential |
| 265 | +reference. Rollback deletes the rotating module and this ADR's index row; |
| 266 | +ADR-0006..0009 artifacts are untouched. Production wiring still additionally |
| 267 | +requires WAL durability/replay, authenticated registry lookup, NUMA-owner |
| 268 | +reclamation policy, priority lanes, shadow equality, qualification, and |
| 269 | +release-artifact rollback — rotation removes exactly one blocker from that |
| 270 | +list, not several. |
| 271 | + |
| 272 | +## Revisit when |
| 273 | + |
| 274 | +Revisit before adding WAL/durability classes, an authenticated page registry, |
| 275 | +NUMA placement or cross-node pools, priority lanes, crossbeam-epoch or any |
| 276 | +second reader, or any multi-producer/multi-consumer topology — each invalidates |
| 277 | +at least one proof above (most immediately the borrow-checker reclamation |
| 278 | +argument, which is single-consumer-only). |
| 279 | + |
| 280 | +## References |
| 281 | + |
| 282 | +- [Mandatory architecture law](../architecture.md) |
| 283 | +- [ADR-0007: sealed generation-tagged slab pages](0007-sealed-generation-tagged-slab-pages.md) |
| 284 | +- [ADR-0008: append-only published-prefix slab pages](0008-append-only-published-prefix-slab-pages.md) |
| 285 | +- [ADR-0009: failure-atomic single-page slab/ring admission](0009-failure-atomic-slab-ring-admission.md) |
| 286 | +- [Target LLD: epoch retirement and reclamation](../LLD.md) |
| 287 | +- [Migration matrix](../../MIGRATION_MATRIX.md#10-gap-analysis-against-target) |
| 288 | +- [Contribution and unsafe-code standard](../../CONTRIBUTING.md#lock-free-structures) |
0 commit comments