Release v2.1.0: segments outlive their creator; add remove_shm()

Author: kzhdev

API_DIFFERENCES.md

@@ -1,7 +1,7 @@
 # API Differences: Python vs C++
 
 The Python implementation is binary-compatible with the C++ `slick::SlickStreamBuffer`
-(`slick/stream_buffer.hpp` v2.0.0) at the shared memory level — every byte, offset, and
+(`slick/stream_buffer.hpp` v2.1.0) at the shared memory level — every byte, offset, and
 memory ordering matches. The *language-level* API differs in the following ways.
 
 ## Construction
@@ -119,19 +119,28 @@ Python adds explicit lifecycle management that C++ handles via RAII:
 
 - `close()` — detach from the segment (does not delete it)
 - `unlink()` — delete the segment this instance is attached to (call once, from the owner,
-  after all users closed - including this instance, so `close()` then `unlink()` is fine)
+  after all users closed - including this instance, so `close()` then `unlink()` is fine).
+  A repeat is a no-op, so it cannot delete a segment a later run created under the same name
 - context manager support: `with SlickStreamBuffer(...) as buf: ...` (closes on exit)
-- `get_shm_name()` — the exact segment name to pass to C++ (`/`-prefixed on POSIX)
+- `get_shm_name()` — the exact segment name to pass to C++ (`/`-prefixed on POSIX); C++
+  `shm_name()` returns `""` for a local-memory buffer where Python returns `None`
+
+Neither side deletes a segment on its own, matching C++ v2.1.0: not the C++ destructor, and not
+Python's `close()`, context manager or process exit. `remove_shm()` is shared with the same
+contract: unlink this buffer's segment name, on every call, leaving open mappings valid, and
+return `False` for a local-memory buffer.
 
 `remove(name)` is the one lifecycle call both sides share, with the same contract: unlink a
 segment by name without attaching to it, returning `True` on success and, on Windows, always
 `True` because a section dies with its last handle. Python routes it through
-`_posixshmem.shm_unlink`, then drops the name from `multiprocessing`'s resource tracker if -
-and only if - this process is what registered it. A name registered elsewhere stays that
-owner's to account for: the tracker answers an unregister for a name it never saw by raising
-`KeyError` and printing a traceback from its own process.
-
-On POSIX with Python >= 3.13, attaching to an existing segment uses
-`SharedMemory(track=False)` so Python's resource tracker never unlinks a segment the
-process does not own. On older Pythons the tracker may log a harmless warning at exit
-(see `tests/run_test.py`, which filters it).
+`_posixshmem.shm_unlink` and never sends the resource tracker an unregister: the tracker answers
+one for a name it does not hold by raising `KeyError` and printing a traceback from its own
+process.
+
+Python keeps every segment it creates or attaches to out of `multiprocessing`'s resource tracker,
+which would otherwise unlink it when the process exits, and sends the tracker nothing at all:
+`SharedMemory(track=False)` on Python 3.13+, and on older POSIX Pythons a `SharedMemory` subclass
+that maps the segment without registering it. Registering and retracting at once instead would
+race between processes sharing one tracker - `multiprocessing` children share their parent's -
+and need a lock that a fork could leave held. A `SharedMemory` you open yourself is still tracked
+unless you pass `track=False` (Python 3.13+).

CHANGELOG.md

@@ -1,5 +1,63 @@
 # Changelogs
 
+## [v2.1.0] - 2026-10-17
+
+Matches C++ [slick-stream-buffer v2.1.0](https://github.com/SlickQuant/slick-stream-buffer/blob/main/CHANGELOG.md):
+no buffer removes its shared-memory segment on its own any more, and `remove_shm()` is how the
+coordinating process does. The shared memory format is unchanged (`SSB1`), so 2.0 and 2.1
+processes in either language can share a segment.
+
+### Segment lifetime (behavior change)
+- On POSIX, a segment is no longer unlinked when the process that created it exits.
+  `SharedMemory(create=True)` registered it with `multiprocessing`'s resource tracker, which
+  unlinks what is registered when the process exits, so a producer that exited first stranded
+  every consumer still attached on an orphaned mapping, while its restart silently created a
+  fresh segment under the same name that those consumers never saw. C++ v2.1.0 stopped its
+  destructor doing the same. A name now lives until `unlink()`, `remove_shm()` or
+  `remove(name)`, and a restarted process reattaches to the existing segment and keeps its
+  contents. Windows is unaffected: a section dies with its last handle.
+- The buffer sends the resource tracker nothing at all: `track=False` on Python 3.13+, and on
+  older POSIX Pythons a `SharedMemory` subclass that maps the segment without registering it.
+  The 2.0.0 bookkeeping of which names this process had registered is gone, and `remove()` no
+  longer contacts the tracker.
+- Migration: code that relied on the cleanup at exit now leaves segments in `/dev/shm`, and its
+  next run attaches to them - `own_buffer` is `False`, reading resumes from the old cursors, and
+  a creator with a different geometry raises `RuntimeError` ("geometry mismatch"). Call
+  `remove(name)` at startup for a clean start, or `unlink()` / `remove_shm()` at shutdown once
+  no peer will attach again.
+
+### Added
+- `remove_shm() -> bool`, matching C++: unlinks this buffer's segment name so the next creator
+  starts fresh, leaving open mappings - this buffer's own included - valid. It unlinks by name
+  on every call, where `unlink()` runs once, and leaves `unlink()` nothing to do afterwards, so a
+  later `unlink()` cannot delete a replacement created under the name. Returns `False` for a
+  local-memory buffer or when the name is already gone, and always `True` for a shared buffer on
+  Windows. C++ v2.1.0's
+  other addition, `shm_name()`, is the existing `get_shm_name()` (`None` rather than `""` for a
+  local-memory buffer).
+
+### Fixed
+- Before Python 3.13 on POSIX, a process that only attached to a segment unlinked it when it
+  exited, because attaching registered the name with that process's resource tracker too.
+  2.0.0 avoided this only on 3.13+, through `track=False`.
+
+### Tests
+- Segment lifetime: `unlink()` repeated, `remove_shm()` on local and shared buffers, `unlink()`
+  after `remove_shm()` sparing a replacement segment, a
+  creator's and an attacher's exit in a fresh interpreter, no resource-tracker traffic at all,
+  a child forked while another thread is constructing a buffer, and spawned children attaching
+  at once. A test that `remove()` leaves the tracker alone for this process's own segment
+  replaces the 2.0.0 one that it retracted the registration.
+- The interop tests build against C++ slick-stream-buffer `v2.1.0` (was `v2.0.0`). The two in
+  which C++ creates the segment remove it before starting and after the producer exits, since
+  C++ now leaves it in place; a rerun used to attach to the previous run's segment.
+- Tests that hand-build a raw segment open it through the library's opener, so that handle and
+  the buffer no longer both claim the name in the resource tracker.
+
+Checked against C++ slick-stream-buffer v2.1.0 with the interop tests on Windows and on Linux,
+run three times on Linux so a segment left behind would fail a rerun, and on Linux under
+Python 3.12 (the subclass) and 3.14 (`track=False`).
+
 ## [v2.0.0] - 2026-09-12
 
 Matches C++ [slick-stream-buffer v2.0.0](https://github.com/SlickQuant/slick-stream-buffer/blob/main/CHANGELOG.md).

README.md

@@ -7,7 +7,7 @@ a lock-free single-producer multi-consumer (SPMC) byte stream buffer with shared
 consumer (or vice versa) communicate seamlessly through the same shared memory segment, using
 the same `std::atomic` synchronization primitives.
 
-Tracks C++ `slick/stream_buffer.hpp` **v2.0.0** (header magic `'SSB1'`); the two carry the same
+Tracks C++ `slick/stream_buffer.hpp` **v2.1.0** (header magic `'SSB1'`); the two carry the same
 version number. The shared memory format is unchanged from v1.x, so a segment written by an
 older process is still readable.
 
@@ -171,13 +171,28 @@ On POSIX `remove()` unlinks immediately: the next creator gets a fresh segment,
 process still mapped to the old one keeps reading the orphaned copy. On Windows it is a no-op
 returning `True` - a section there disappears once the last handle closes.
 
-One Python-only caveat: on POSIX before Python 3.13, `multiprocessing`'s resource tracker
-unlinks every segment the process mapped when it exits, including ones it only attached to.
-Python 3.13+ attaches with `SharedMemory(track=False)`, which is why 3.13 or newer is
-recommended when several processes share a segment. `remove()` unlinks the segment either way,
-and retracts the tracker's registration only when this process is the one that registered it -
-a name registered by some other process, or by a `SharedMemory` you built yourself, stays that
-owner's to account for. See [API_DIFFERENCES.md](API_DIFFERENCES.md).
+No buffer deletes its segment on its own: not `close()`, not the context manager, and not
+process exit, matching C++ v2.1.0. On POSIX, a departing process that unlinked the name while a
+peer was still attached would leave that peer reading an orphaned mapping while a restarted
+producer silently created a fresh segment under the same name, so the two never met again.
+Instead, the process that coordinates the buffer's lifetime deletes the name once no peer will
+attach again, with `unlink()` or `remove_shm()` on an instance, or `remove(name)`:
+
+```python
+# Coordinator, after every peer has shut down
+buf.close()
+buf.unlink()    # once; a repeat is a no-op, so it cannot delete a later run's segment
+```
+
+`remove_shm()` is the C++-style call: it unlinks by name on every call and returns `bool`, and
+open mappings, including this instance's own, stay valid. Until the name is deleted, a restarted
+process reattaches to the existing segment and keeps its contents.
+
+Buffers keep their segments out of `multiprocessing`'s resource tracker, which would otherwise
+unlink them when the process exits - before Python 3.13, even in a process that only attached.
+They send the tracker nothing at all: `track=False` on Python 3.13+, and on older POSIX Pythons a
+`SharedMemory` subclass that maps the segment without registering it. See
+[API_DIFFERENCES.md](API_DIFFERENCES.md).
 
 ### Shared memory layout
 
@@ -213,7 +228,8 @@ Total segment size: `64 + 32 * control_size + capacity`.
 | `initial_reading_index()` | consumer | cursor for a late joiner (skips history) |
 | `loss_count()` | consumer | records skipped by this instance, counting only reads made with `count_loss` traits |
 | `reset()` | owner | clear all state (not thread-safe) |
-| `close()` / `unlink()` | lifecycle | detach / delete this instance's segment |
+| `close()` / `unlink()` | lifecycle | detach / delete this instance's segment (once, valid after `close()`) |
+| `remove_shm() -> bool` | lifecycle | unlink this instance's segment name, as C++ `remove_shm()` |
 | `SlickStreamBuffer.remove(name)` | lifecycle | unlink a segment by name - the stale-segment recovery step |
 
 See [API_DIFFERENCES.md](API_DIFFERENCES.md) for the exact deviations from the C++ API