Skip to content

fix(registry): publish slot state only after payload commits under the seqlock [P1] - #5

Merged
benaliabderrahmane merged 1 commit into
mainfrom
fix/p1-2-registry-seqlock-publication
Jun 5, 2026
Merged

benaliabderrahmane merged 1 commit into
mainfrom
fix/p1-2-registry-seqlock-publication

Conversation

@benaliabderrahmane

Copy link
Copy Markdown
Owner

Severity: P1 — found by a full multi-agent code audit (method + findings in AUDIT_FINAL.md).

Problem

try_add_once() published slot occupancy via the state CAS (EMPTY → type) before write_slot_payload() opened the seqlock (seq → odd). That left a window where state == type, seq is still even, and the payload is all-zero. A reader in snapshot_slot() could land entirely inside it and return a bogus "valid" all-zero snapshot (pid == 0, empty socket_path/node_name).

Worse: registry_cleanup_stale() could snapshot that half-built slot, read pid == 0 (treated as reclaimable), and win the reclaim CAS — tearing down a slot a live process just claimed and unlinking its in-use socket, so the endpoint silently vanishes from discovery.

Fix

  • Claim the slot with an intermediate ENTRY_RESERVED sentinel that readers treat as empty.
  • Write the payload under the seqlock, then publish the real state with a release store, so any reader observing the real state also observes the committed payload.
  • Defense-in-depth: cleanup_stale() skips pid == 0 snapshots.

ENTRY_RESERVED lives only in the atomic state byte → no shm layout change, no REGISTRY_VERSION bump.

Files: registry.cpp, registry.hpp
Build: clean.

@benaliabderrahmane

Copy link
Copy Markdown
Owner Author

Reviewed the fix in depth -- it is a correct, surgical fix of a real cross-process P1 race, and I'm approving. I traced the memory ordering and it is sound:

  • Writer does CAS state EMPTY -> RESERVED (acq_rel, registry.cpp:243-245), seq->odd, payload memcpy, seq->even (write_slot_payload), then state.store(type, release) at registry.cpp:250.
  • A reader's acquire-load of state at registry.cpp:175 that observes the released type synchronizes-with the release store, so it also sees the committed payload and the even seq; a reader straddling publication gets s1 != s2 and retries.
  • RESERVED is rejected at registry.cpp:175, so no half-built slot is ever snapshotted, and cleanup_stale (registry.cpp:380) only ever sees a real committed type as its CAS expected.
  • ENTRY_RESERVED = 0xFF lives only in the atomic state byte (registry.hpp:35), so no shm-layout / REGISTRY_VERSION change is needed -- the static_asserts at registry.hpp:61-66 are unaffected. CI is green on jazzy/kilted/rolling.

Non-blocking follow-ups:

  1. (CLAUDE.md sec.4) No reproducing/regression test ships with this fix. The commit touches only registry.cpp and registry.hpp -- 0 test files. A deterministic test looks feasible via the test-exposed registry_slots()/RegistryEntrySlot (cf. test/test_registry.cpp:163,177): write a non-empty state byte with seq left even and the payload zeroed, then assert snapshot_slot/registry_query returns nothing AND registry_cleanup_stale does not reclaim it (fails pre-fix, passes post-fix). The existing ReadersNeverSeeTornEntries (test_registry_concurrent.cpp:112-187) is probabilistic and never exercises cleanup_stale concurrently with adds, so it does not cover the reclaim-of-a-live-slot path.

  2. (CLAUDE.md sec.3, orphan) registry.hpp:81 -- the reader-protocol comment step 2 still says if (st == EMPTY) skip, but snapshot_slot now also rejects ENTRY_RESERVED (registry.cpp:175). The writer steps were renumbered and the RESERVED enum was documented; the reader step was missed. One-line fix.

  3. (nit, optional) The pid==0 guard in registry_cleanup_stale (registry.cpp:363-367) is redundant given the RESERVED fix and that all five production callers set pid=getpid(); it changes no behavior today (pid_is_alive_or_unreachable already rejects pid<=0). It's disclosed in the body as defense-in-depth and guards a catastrophic unlink, so I'm fine keeping it -- just noting the redundancy is intentional.

None of these block. Approving.

@benaliabderrahmane
benaliabderrahmane merged commit 74f4cf0 into main Jun 5, 2026
3 checks passed
@benaliabderrahmane
benaliabderrahmane deleted the fix/p1-2-registry-seqlock-publication branch June 17, 2026 08:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant