Skip to content

cache: the Store admission/serve sidecar seam (RPZ phase 4 prerequisite) - #601

Merged
semihalev merged 4 commits into
mainfrom
feat/cache-sidecar-seam
Aug 31, 2026
Merged

cache: the Store admission/serve sidecar seam (RPZ phase 4 prerequisite)#601
semihalev merged 4 commits into
mainfrom
feat/cache-sidecar-seam

Conversation

@semihalev

Copy link
Copy Markdown
Owner

The seam docs/rpz-design.md §5.6 requires before the response-IP phase: policy state carried beside cache entries, stamped at admission, judged at the byte serve. This PR is deliberately policy-free — no RPZ logic, no behavior change while unwired — so the seam's design can be reviewed on its own, as the design doc prescribes.

Design

The sidecar. CacheEntry gains one atomic pointer beside its immutable payload (the same shape as the existing prefetch claim — a mutable atomic the entry carries without being copied). The cache never reads the value. nil is load-bearing: an absent sidecar means unknown, never clean — because a fourth admission door may exist someday, and the audit indeed found one the design's callsite list didn't (below).

Admission. One evaluator, wired into the Store, stamps every entry with the result of evaluating that entry's own stored records — the truth, post-filter, never the client's copy. The design named three doors (writer, resolver's direct store, prefetch replacement); reading the code found they funnel through three construction sites, and all three stamp:

  • setFromResponseWithKey — the writer's SetFromResponseWithKey/Scoped and the resolver's SetFromResponseWithCut funnel here;
  • ReplaceIfCurrent — the prefetch CAS replacement;
  • Cache.SetSetEntryWithKey — the compatibility door for prefetch write-backs and plugin callers, which the design's list did not carry.

An entry admitted before the seam was wired carries nil; the first decoded serve evaluates the message it already built and CAS-stamps it, so such an entry pays the Msg path once, not forever.

Serve. A WireHitGate is consulted before every record-bearing byte serve, with the sidecar(s) the reply would be built from:

  • the wire-born exact hit (serveHitFromWire),
  • the Msg-born inline byte path (handleCacheHit),
  • the composed chase (serveChaseHit) — one sidecar per segment, in chain order: a whole-chain verdict on the alias entry would go stale the moment the target refreshed under it (§5.6 item 4).

A false verdict declines to the decoded path, where the policy middleware's own response writer sees a full message. Composite denial classes (subtree cuts, failure state) carry no stored records and are never gated. The gate signature is entry-only by design: per-query policy state stays in the policy middleware, which steers a query off the byte path entirely by withholding its writer's wire capability — so the gate never needs the request, and a verdict is a pure function of the stored state.

Wiring. Setup's marker pattern: SidecarPolicyProvider (rpz, in the next PR) → SidecarPolicySetter (cache), first provider wins with a warning, injected before the pipeline publishes so the atomic store is the happens-before barrier — same discipline as the existing Store/Queryer wiring.

Cost while unwired

One nil field check per hit and 8 bytes per entry. The existing zero-alloc hit-class pins (TestServeRawWireHitAllocatesNothing, the acquire pins) run unchanged and green.

Verification

  • Seam tests: every door stamps (each pinned separately); unwired leaves entries unstamped; a declining gate is consulted with the entry's own sidecar and the decoded path answers correctly; an approving gate genuinely serves bytes (fast-served counter); the chase gate sees every segment in order; the decoded serve stamps an unevaluated entry; CAS restamp semantics; autoWire injects the provider (and never calls a setter without one).
  • 8/8 guard mutations killed, one per stamp/gate/wiring point.
  • Full suite, -race on middleware + cache, lint clean.

Phase 4 proper (rpz-ip triggers, the match-list sidecar values, serve-time merge, winner-bounded counting) follows in its own PR on top of this seam.

…byte serve

The admission/serve seam RPZ's response-IP phase builds on, policy-free
by itself (docs/rpz-design.md §5.6 item 6). Three pieces:

- CacheEntry carries an atomic sidecar pointer beside its immutable
  payload, the same shape as the prefetch claim: opaque policy state a
  wired evaluator computed from the entry's own stored records. nil is
  load-bearing — unknown, never clean.
- Admission: a single evaluator wired into the Store stamps every entry
  at every door — the SetFromResponse funnel, the prefetch CAS
  replacement, and the compatibility Set (the audit found three doors
  where the design counted on the writer alone). An entry that predates
  the wiring is evaluated and stamped by the first decoded serve.
- Serve: a WireHitGate is consulted before every record-bearing byte
  serve — the wire-born exact hit, the Msg-born inline byte path, and
  the composed chase, which shows the gate one sidecar per segment in
  chain order (a whole-chain verdict on the alias would go stale when
  the target refreshed under it). A false verdict declines to the
  decoded path; composite denial classes carry no records and are not
  gated.

Wired through Setup's marker pattern (SidecarPolicyProvider ->
SidecarPolicySetter), first provider wins. Unwired, the seam is one nil
field check per hit: the existing zero-alloc hit-class pins run
unchanged, and every stamp and gate call is mutation-pinned by the seam
tests.
@codecov

codecov Bot commented Aug 31, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 81.15942% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.88%. Comparing base (5fa514e) to head (4a78fe1).

Files with missing lines Patch % Lines
middleware/cache/cache.go 80.00% 6 Missing ⚠️
middleware/cache/entry_wire_chase.go 66.66% 4 Missing ⚠️
middleware/pipeline.go 83.33% 2 Missing ⚠️
middleware/sidecar.go 85.71% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #601      +/-   ##
==========================================
- Coverage   86.89%   86.88%   -0.02%     
==========================================
  Files         179      180       +1     
  Lines       22732    22801      +69     
==========================================
+ Hits        19754    19810      +56     
- Misses       2968     2981      +13     
  Partials       10       10              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

…erve, counting after the commit

Both findings land on the same joint and one contract change closes them:
the gate's boolean became a verdict, and accounting left it entirely.

- Judge{WireHit,WireChase} return Serve, Decode, or Restamp, and are
  pure decisions over the sidecars. Decode means policy wants the full
  message and the sidecar itself was fine; Restamp means the sidecar is
  unusable — unevaluated, or stamped under a generation the gate no
  longer accepts. The decoded serve acts on Restamp by re-evaluating the
  entry's records and CASing over the judged pointer, stale and nil
  alike, so a policy reload no longer strands live entries on the
  decoded path until eviction. The verdict is judged once per hit,
  before the byte/decoded split, internal serves included — the
  Msg-path chase reaches its segments through internal sub-queries,
  which is where a stale segment gets its restamp.
- Count{WireHit,WireChase} fire once per byte-served hit, after the
  transport accepted the bytes — past the last point a serve can still
  fall back to the decoded path (writer readiness, chain mismatch,
  build, the fallback sentinel) and be counted a second time by the
  policy writer there. A transport failure after the commit counts,
  matching the response-counts-as-written parity the serve paths
  already keep.

Every placement is pinned: stale-restamp against a nil-only CAS,
fallback-after-approval counting nothing on both the Msg-born and
wire-born paths, committed serves counting exactly once on both, and
the chase count with its segments.
Moving the segment array out of the gate branch for the commit-time
count made it visible to escape analysis through the interface call, so
every ungated chase paid a heap allocation the hit-class pins forbid —
CI caught it. The slice is now made on the gated branch only and stays
in scope for the count; an ungated chase touches nothing.
…path to the ungated cost

The gated chase still allocated: the slice handed to JudgeWireChase
escaped through the interface call, so wiring the gate put a heap
allocation on every cache-contained chase hit — the zero-allocation hit
contract holds with policy live, not only without it. The chain is now a
bounded value type (SidecarChain, capacity asserted against the chase
depth at compile time) passed by value through Judge and Count, which
closes the slice-escape class in the type system rather than in review.

The new pin closes the blind spot the finding named: the allocation CI
never wires a gate, so a baseline-relative AllocsPerRun test serves the
exact and chase hits against two caches — one bare, one behind an
allocation-free Serve gate — and refuses any gated surplus. Verified to
fire: an escaping allocation planted on the gated branch reads 3.0
against a 2.0 baseline.
@semihalev
semihalev merged commit 21d3218 into main Aug 31, 2026
10 checks passed
@semihalev
semihalev deleted the feat/cache-sidecar-seam branch August 31, 2026 09:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants