From a6f52c1713993da36b007061050b5c1aba6b034d Mon Sep 17 00:00:00 2001 From: Abderahmane BENALI Date: Fri, 11 Sep 2026 20:12:00 +0200 Subject: [PATCH 1/3] chore(release): prepare the package for a rosdistro release bloom reads CHANGELOG.rst from next to package.xml and nowhere else, so the root CHANGELOG.md was invisible to catkin_generate_changelog and the release would have shipped an empty debian changelog. Move it into the package and convert it to rst. Two rst constraints shaped the conversion. REP-132 allows only paragraphs and lists under a version heading and catkin_pkg drops sub-sections silently, so the per-release Added/Changed/Fixed headings are bold paragraphs, not rst sub-sections; written as sub-sections, 0.4.0 and 0.4.1 parsed to an empty body. rst also has no nested inline markup, so ``code`` inside **bold** rendered the backticks literally. Also: - package.xml gains repository/bugtracker/website tags, which ROS Index renders and rosdistro reviewers look for. - release.yml builds lyrical, matching ci.yml and the set of distros we intend to open bloom tracks for. Humble is excluded deliberately: serialization.cpp uses fastcdr's CdrVersion::DDS_CDR, which is Fast CDR 2.x and absent from the 1.0.x Humble ships. - RELEASING.md records the bloom track answers, notably the v:{version} release tag our v-prefixed tags need and the per-release command sequence. - A release-metadata CI job re-checks all of the above, because a changelog that parses to an empty body fails silently rather than loudly. No source files are touched; this is release metadata only. --- .github/workflows/ci.yml | 34 ++++ .github/workflows/release.yml | 2 +- CHANGELOG.md | 251 ------------------------------ RELEASING.md | 74 +++++++++ rmw_unix_socket_cpp/CHANGELOG.rst | 243 +++++++++++++++++++++++++++++ rmw_unix_socket_cpp/package.xml | 4 + 6 files changed, 356 insertions(+), 252 deletions(-) delete mode 100644 CHANGELOG.md create mode 100644 RELEASING.md create mode 100644 rmw_unix_socket_cpp/CHANGELOG.rst diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 94e59e1..e580d61 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -101,3 +101,37 @@ jobs: name: test-logs-${{ matrix.ros_distro }} path: log/ retention-days: 7 + + release-metadata: + name: release metadata + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - name: Check package.xml and CHANGELOG.rst are bloom-ready + shell: bash + run: | + sudo apt-get update + sudo apt-get install -y --no-install-recommends python3-catkin-pkg + python3 - <<'PY' + from catkin_pkg.changelog import get_changelog_from_path + from catkin_pkg.package import parse_package + + PKG = 'rmw_unix_socket_cpp' + pkg = parse_package(PKG) + pkg.validate() + assert pkg.urls, 'package.xml has no tags' + assert pkg.maintainers[0].email, 'maintainer has no email' + + # bloom only reads CHANGELOG.rst sitting next to package.xml. + changelog = get_changelog_from_path(PKG, PKG) + assert changelog is not None, f'no CHANGELOG.rst in {PKG}/' + versions = [v for v, _, _ in changelog.foreach_version()] + assert pkg.version in versions, f'package.xml {pkg.version} has no changelog entry' + # REP-132 forbids sub-sections under a version; catkin_pkg drops them + # silently, so a version written that way ships an empty debian entry. + for version, date, content in changelog.foreach_version(): + assert date, f'{version} has no parseable date' + assert content, f'{version} body is empty (sub-sections are dropped)' + print(f'{PKG} {pkg.version}: {len(versions)} changelog entries, metadata OK') + PY diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 8b93192..9afe932 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -17,7 +17,7 @@ jobs: strategy: fail-fast: false matrix: - ros_distro: [jazzy, kilted, rolling] + ros_distro: [jazzy, kilted, rolling, lyrical] container: image: ros:${{ matrix.ros_distro }}-ros-base steps: diff --git a/CHANGELOG.md b/CHANGELOG.md deleted file mode 100644 index 9cf8f0d..0000000 --- a/CHANGELOG.md +++ /dev/null @@ -1,251 +0,0 @@ -# Changelog - -All notable changes to `rmw_unix_socket_cpp` are documented here. - -The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), -and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). - -## [0.5.0] - 2026-08-27 - -The wait/wakeup release. The 200 ms `rmw_wait` poll is gone, replaced by an -event-driven doorbell; TRANSIENT_LOCAL latched replay is rebuilt as a pull from -a per-publisher shared-memory ring; and the per-node graph guard conditions that -`rclcpp`'s `GraphListener` actually waits on are finally triggered. - -> **Upgrade together.** See [Upgrade notes](#upgrade-notes-050) — a mixed-build -> fleet can lose latched replay and registry wakeups during the rollout window. - -### Added - -- **Pull-based TRANSIENT_LOCAL replay** (#60). A latched publisher writes each - sample into a per-publisher shm ring (`tl_ring`, `qos.depth` per-record-seqlocked - slots) at publish time; a late-joining subscription reads that ring itself - inside `rmw_create_subscription`. The idle publisher is never woken, polled, or - rung. History is enqueued before the subscription handle returns, so it always - precedes live samples. Payloads over `TL_EMBED_CAP` (1 KiB) ride the existing - durable segments as 32-byte descriptors; ring bytes are capped at - `TL_RING_MAX_BYTES` (2 MiB). -- **`rmw_subscription_options_t::ignore_local_publications`** (#49). Previously - discarded at creation. Each context now draws a random 64-bit `context_id` in - `rmw_init` and embeds it in the trailing 8 bytes of every GID, so - `is_same_context()` answers from bytes already on the wire — no extra traffic, - no wire-format change. "Local" is the same `rmw_context_t`, per the rmw - contract, not merely the same process. -- **Event-driven registry wakeup (doorbell).** Each context binds a doorbell - socket at `rmw_init`; every registry mutation sends one octet to every - registered doorbell strictly *after* bumping the generation, and `rmw_wait` - drains its doorbell strictly *before* reading the generation. That ordering - pair makes a lost wakeup impossible. No new threads, no registry layout change. -- **Graph-change triggering of per-node guard conditions** (#43). -- CI: **Lyrical** added to the build & test matrix (#17); CI now also runs on - `devel` pushes and PRs. -- GitHub issue forms and a pull request template (#56). -- Tests: `test_rmw_wait.cpp` (new, +648 lines), `test_shm_transport.cpp` (+283), - substantial additions to `test_rmw_qos.cpp` (+873), `test_rmw_graph.cpp`, - `test_transport.cpp`, `test_rmw_pub_sub.cpp`, and - `test_rmw_service_client.cpp`. - -### Changed - -- **VOLATILE late joiners no longer receive latched history.** Pulled records - are filtered by durability — DDS-correct behaviour. The old push path replayed - to every subscriber on the topic regardless of its durability QoS. -- The `ENTRY_DOORBELL` slot is now registered **lazily**, on the first - `rmw_wait` holding a graph guard condition, so only graph-event consumers - (`wait_for_service`, `GraphListener`, rosbag2) pay for registry wakeups. - Doorbell-slot mutations no longer ring, which removes a K²/2 registration - mini-storm. -- Removed with the push machinery: `transient_local_pubs` (and its mutex), - `known_subscriber_paths`, `CachedMessage` / the heap message cache, - `transient_local_publish`'s replay loop, and the wait-side TL replay block. -- `registry`: `teardown_slot` now `shm_unlink`s `tl_`-prefixed slot paths, and - the orphan sweep gained a `ros2_uds_tl_` prefix pass. -- `DESIGN.md` documents the doorbell wakeup, the graph-event wiring, GID - composition and context identity, and the TL ring. - -### Fixed - -- **`rmw_wait` timeout contract.** The 200 ms poll bound made every wait return - `RMW_RET_TIMEOUT` at 200 ms regardless of the caller's deadline — a 600 ms wait - returned at 200 ms, and an infinite wait, which must never time out, returned - `TIMEOUT`. Idle processes also woke 5×/s. `RMW_RET_TIMEOUT` now surfaces only - at the caller's own deadline. -- **`rmw_wait` reported spurious timeouts** when a drain legitimately yielded - nothing (an `ignore_local_publications` drop, or a shm descriptor whose sender - is gone). -- **A wait set holding only guard conditions got a null context** and skipped - the registry check entirely — the shape a publish-only node's executor - produces. The wait set now stores its context at `rmw_create_wait_set`. -- **`rmw_wait` dispatched fds the current call never armed.** The armed-fd cache - survived across calls and was insert-only, so a guard condition owned by - another wait set could have its eventfd consumed with the trigger recorded - nowhere — a permanent lost wakeup that hung `wait_for_service` and - `GraphListener`. Failed `epoll_ctl` ADDs now degrade to polling instead of - being silently dropped. -- **Dead graph-guard trigger.** `rmw_wait` fired `ctx->graph_guard_condition`, a - context-level field that is declared and never assigned, so the branch was - dead code and the object `rcl` waits on was never triggered (#43). -- **Zero-length datagrams were never dequeued.** `recv_from` peeks to size its - buffer; a zero-length datagram made the peek return 0 and the function bailed - out. `MSG_PEEK` does not consume, so the socket stayed readable forever, - spinning every wait that polled that fd. -- **Junk datagrams are now consumed atomically.** The zero-length cleanup used a - blind 1-byte `recv` after a separate, non-atomic peek; under a - `MultiThreadedExecutor` a racing drain could consume the peeked datagram and - the 1-byte `recv` would dequeue a real message and silently discard it. -- **Truncated consumes are detected with `MSG_TRUNC` and dropped.** Pre-existing: - the consuming `recv` omitted `MSG_TRUNC`, so a concurrently-dequeued peek - followed by a larger datagram was silently truncated to the smaller buffer and - delivered as valid. -- **TL: a writer filling a slot the pull scan skipped** opened a sequence gap - with `overlapped` left false, defeating the contiguous-prefix trim. -- **TL: the replay watermark now stops at an unresolvable record.** `max_seq` was - fixed before the enqueue loop, so a record whose descriptor could no longer be - resolved was skipped while its sequence stayed under the watermark — the - sample was lost, not lapped. -- `rmw_init`'s stale sweep is now stamped, and the throttle-scope comments were - corrected. -- Docs: `TL_RING_MAX_BYTES` was documented as 1 MiB (it is 2 MiB), and the - durable-segment budgeting note understated tmpfs sizing for over-cap latched - samples (a retained 5 MiB sample costs 5 MiB, not "an inode and a couple of - pages"). Reported by Copilot on #60. - -### Performance - -- **`rmw_wait` drains only the fds `epoll` reported ready** (#57). It previously - made three passes over every entity per call — drain every socket, re-arm every - fd, drain every socket again — roughly 2150 syscalls for a 719-entity wait set, - nearly all returning `EAGAIN` or `EEXIST`. -- **The stale-slot sweep is throttled off the graph query path** (#58). - `query_all` ran `registry_cleanup_stale` before every graph query across 13 - call sites, so `rmw_get_node_names`, `rmw_count_publishers`, - `rmw_count_subscribers`, `rmw_get_topic_names_and_types` and the rest each - walked every live slot, copied 1164 bytes out of each, and `stat`ed - `/proc/`. -- **Launch arithmetic at N=200 nodes:** doorbell datagrams drop from ~100,300 - (19,900 of them from doorbell self-registration alone) to roughly the number of - graph-event consumers; wake-side registry copies go from - O(mutations × processes × slots) to zero for plain pub/sub processes. -- Latched-replay latency improves from ~20 ms (doorbell wake) to synchronous at - subscription creation. - -### Upgrade notes (0.5.0) - -- **Latched replay across a mixed-build pair is lost.** A 0.5.0 publisher no - longer push-replays and a pre-0.5.0 subscriber never pulls, so latched replay - between that pair does not work during a rolling upgrade — upgrade together - (same precedent as the shm payload flag). Old publisher + new subscriber keeps - working: an empty `socket_path` skips the pull and the old push path still - delivers. -- **Registry wakeups across a mixed-build fleet can be missed.** A build that - predates the doorbell bumps the generation but never rings. -- **Containers need `--shm-size` ≥ 1 GiB** to run the full test suite. The 64 MB - Docker default cannot hold the ~38 MB registry plus the payload rings. - -## [0.4.1] - 2026-07-23 - -### Reverted - -- `fix(rmw_wait): deliver TRANSIENT_LOCAL latched samples to late joiners of - idle publishers` (#37). The push-based approach woke the publisher process to - resend, and the doorbell broadcast it relied on did not scale — every registry - mutation rang every process, and every wake rescanned the registry, which - melted 200-node launches. Replaced in 0.5.0 by the pull-based ring (#60). - -## [0.4.0] - 2026-07-16 - -### Added - -- **Shared-memory ring transport for large topic payloads**, with CDR serialized - directly into the ring record. -- **Durable shm segments for large TRANSIENT_LOCAL messages**, including the - serialized-publish path. -- **Large service request/response payloads routed through shm.** -- `fork()`-based cross-process shm integration tests. - -### Changed - -- The inline-vs-shm send/receive decision unified into two helpers. -- `DESIGN.md` expanded into a fuller architecture document (#16); doc/comment - drift from the merge-readiness audit fixed. -- Repository prepared for open-source release (#15). - -### Fixed - -- Contained the inline-path resize; the ring inline fallback is now tested. -- Honest TRANSIENT_LOCAL publish return value. - -### Performance - -- Shorter `cache_mutex` hold, smaller ring floor, POD-keyed reader cache. - -## [0.3.0] - 2026-06-17 - -Audit hardening & registry/fan-out performance. - -### Added - -- TRANSIENT_LOCAL replay to late-joining subscribers (#2). -- README. - -### Fixed - -- Contained `fastcdr`/`std` exceptions in `serialize()` so none cross the - `extern "C"` boundary (#4). -- Publish a registry slot's state only after its payload commits under the - seqlock (#5). -- Deep-copy `security_options` in `rmw_init_options_copy` to prevent a - double-free (#6). -- Accumulate the `rmw_wait` timeout in `int64` so `RMW_DURATION_INFINITE` keeps - blocking (#8). -- Write `rmw_take_sequence` output at the taken cursor (#7). -- Surface `EMSGSIZE` as `RMW_RET_ERROR` on publish (#3). -- Assorted P2/P3 hardening and identifier checks, plus dead-code removal (#9). -- Probe `localhost_only` via include dirs rather than linked targets, so Rolling - configure stops aborting. - -### Performance - -- Bound query/cleanup scans with a shared high-water mark (#14). -- Filter registry slots by type before the per-slot snapshot copy (#11). -- Copy-on-write subscriber-path cache; prune the stale known-subscriber set on - graph change (#12). -- Cache `rmw_service_server_is_available` on the registry generation (#13). - -## [0.2.0] - 2026-05-11 - -Observability. - -### Changed - -- Failure paths routed through `rcutils` logging. -- `ament_target_dependencies` replaced with direct `target_link_libraries`. -- `ament_lint_auto` dropped from the test suite. - -### Fixed - -- CI: install `test_msgs` explicitly so Rolling stops silently skipping it; tell - `rosdep` to install `test_depend` packages; raise `/dev/shm` in the container - to 1 GiB. - -## [0.1.0] - 2026-05-11 - -Initial release: a ROS 2 RMW implementation over `AF_UNIX` datagram sockets with -a lock-free shared-memory registry. - -### Added - -- Lock-free registry via per-slot seqlock + atomic state. -- Cached discovery lookups and scale benchmarks. -- Build & test workflows for Rolling, Jazzy, and Kilted. - -### Fixed - -- Guard `rmw_init_options_t::localhost_only` behind a CMake probe. - -[0.5.0]: https://github.com/benaliabderrahmane/rmw_unix_socket_cpp/compare/v0.4.1...v0.5.0 -[0.4.1]: https://github.com/benaliabderrahmane/rmw_unix_socket_cpp/compare/v0.4.0...v0.4.1 -[0.4.0]: https://github.com/benaliabderrahmane/rmw_unix_socket_cpp/compare/v0.3.0...v0.4.0 -[0.3.0]: https://github.com/benaliabderrahmane/rmw_unix_socket_cpp/compare/v0.2.0...v0.3.0 -[0.2.0]: https://github.com/benaliabderrahmane/rmw_unix_socket_cpp/compare/v0.1.0...v0.2.0 -[0.1.0]: https://github.com/benaliabderrahmane/rmw_unix_socket_cpp/releases/tag/v0.1.0 diff --git a/RELEASING.md b/RELEASING.md new file mode 100644 index 0000000..9feee23 --- /dev/null +++ b/RELEASING.md @@ -0,0 +1,74 @@ +# Releasing + +Binaries reach users through the ROS build farm. `bloom` generates debian +metadata into a separate `-release` repo and opens a PR against `ros/rosdistro`; +after it merges, the farm builds `ros--rmw-unix-socket-cpp`. None of this +touches source on `main`. + +```bash +sudo apt install python3-bloom python3-catkin-pkg +``` + +`bloom` also needs a GitHub token with `public_repo` scope. + +## Supported distros + +`jazzy`, `kilted`, `rolling`, `lyrical` — one bloom track each, matching the CI +and release workflow matrices. + +**Not Humble.** `src/serialization.cpp` uses +`eprosima::fastcdr::CdrVersion::DDS_CDR`, which is Fast CDR 2.x. Humble ships +Fast CDR 1.0.x, where that enum does not exist. + +## Track answers (asked once, on `--new-track`) + +| Prompt | Answer | +| --- | --- | +| Release repository url | yes — let bloom create `…/rmw_unix_socket_cpp-release` | +| Upstream repository uri | `https://github.com/benaliabderrahmane/rmw_unix_socket_cpp.git` | +| Upstream devel branch | the branch for that distro (`main` until per-distro branches exist) | +| Version | `:{auto}` | +| Release tag | `v:{version}` | + +The release tag matters: our tags are `v0.5.0` but `catkin_prepare_release` +writes `0.5.0`, so a bare `:{version}` makes bloom look for a tag that isn't +there. + +## First release + +```bash +bloom-release --new-track --rosdistro jazzy --track jazzy rmw_unix_socket_cpp +``` + +Only this first rosdistro PR gets real scrutiny. + +## Every release after that + +```bash +catkin_generate_changelog # appends unreleased commits to rmw_unix_socket_cpp/CHANGELOG.rst +$EDITOR rmw_unix_socket_cpp/CHANGELOG.rst +git commit -am "Update changelog" +catkin_prepare_release # bumps package.xml, commits, tags, pushes +bloom-release --rosdistro jazzy --track jazzy rmw_unix_socket_cpp +``` + +No `--new-track` — bloom reuses the answers above. The resulting rosdistro PR +changes one line, the `version:` field. + +- One invocation and one rosdistro PR per distro. +- Versions are independent across distros but must increase monotonically within + one. +- Rebuild with no source change: re-run `bloom-release` at the same version and + the debian increment bumps, `0.6.0-1` → `0.6.0-2`. +- Never hand-edit or delete the `-release` repo. bloom regenerates it each time + but needs its history. + +Tag first, let `release.yml` go green, then run bloom. + +## CHANGELOG.rst + +`CHANGELOG.rst` must sit next to `package.xml`; bloom reads nowhere else. +Version headings are `0.5.0 (2026-08-27)` over a `------` underline, and a +version body may contain only paragraphs and bullet lists — REP-132 forbids +sub-sections and `catkin_pkg` drops them silently, which ships an empty debian +changelog entry. The `release-metadata` CI job checks both. diff --git a/rmw_unix_socket_cpp/CHANGELOG.rst b/rmw_unix_socket_cpp/CHANGELOG.rst new file mode 100644 index 0000000..1fef109 --- /dev/null +++ b/rmw_unix_socket_cpp/CHANGELOG.rst @@ -0,0 +1,243 @@ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Changelog for package rmw_unix_socket_cpp +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +0.5.0 (2026-08-27) +------------------ +The wait/wakeup release. The 200 ms ``rmw_wait`` poll is gone, replaced by an +event-driven doorbell; TRANSIENT_LOCAL latched replay is rebuilt as a pull from +a per-publisher shared-memory ring; and the per-node graph guard conditions that +``rclcpp``'s ``GraphListener`` actually waits on are finally triggered. + +**Upgrade together.** See the upgrade notes below — a mixed-build fleet can lose +latched replay and registry wakeups during the rollout window. + +**Added** + +* **Pull-based TRANSIENT_LOCAL replay** (#60). A latched publisher writes each + sample into a per-publisher shm ring (``tl_ring``, ``qos.depth`` + per-record-seqlocked slots) at publish time; a late-joining subscription reads + that ring itself inside ``rmw_create_subscription``. The idle publisher is + never woken, polled, or rung. History is enqueued before the subscription + handle returns, so it always precedes live samples. Payloads over + ``TL_EMBED_CAP`` (1 KiB) ride the existing durable segments as 32-byte + descriptors; ring bytes are capped at ``TL_RING_MAX_BYTES`` (2 MiB). +* **rmw_subscription_options_t::ignore_local_publications** (#49). + Previously discarded at creation. Each context now draws a random 64-bit + ``context_id`` in ``rmw_init`` and embeds it in the trailing 8 bytes of every + GID, so ``is_same_context()`` answers from bytes already on the wire — no + extra traffic, no wire-format change. "Local" is the same ``rmw_context_t``, + per the rmw contract, not merely the same process. +* **Event-driven registry wakeup (doorbell).** Each context binds a doorbell + socket at ``rmw_init``; every registry mutation sends one octet to every + registered doorbell strictly *after* bumping the generation, and ``rmw_wait`` + drains its doorbell strictly *before* reading the generation. That ordering + pair makes a lost wakeup impossible. No new threads, no registry layout + change. +* **Graph-change triggering of per-node guard conditions** (#43). +* CI: **Lyrical** added to the build & test matrix (#17); CI now also runs on + ``devel`` pushes and PRs. +* GitHub issue forms and a pull request template (#56). +* Tests: ``test_rmw_wait.cpp`` (new, +648 lines), ``test_shm_transport.cpp`` + (+283), substantial additions to ``test_rmw_qos.cpp`` (+873), + ``test_rmw_graph.cpp``, ``test_transport.cpp``, ``test_rmw_pub_sub.cpp``, and + ``test_rmw_service_client.cpp``. + +**Changed** + +* **VOLATILE late joiners no longer receive latched history.** Pulled records + are filtered by durability — DDS-correct behaviour. The old push path replayed + to every subscriber on the topic regardless of its durability QoS. +* The ``ENTRY_DOORBELL`` slot is now registered **lazily**, on the first + ``rmw_wait`` holding a graph guard condition, so only graph-event consumers + (``wait_for_service``, ``GraphListener``, rosbag2) pay for registry wakeups. + Doorbell-slot mutations no longer ring, which removes a K²/2 registration + mini-storm. +* Removed with the push machinery: ``transient_local_pubs`` (and its mutex), + ``known_subscriber_paths``, ``CachedMessage`` / the heap message cache, + ``transient_local_publish``'s replay loop, and the wait-side TL replay block. +* ``registry``: ``teardown_slot`` now ``shm_unlink``\ s ``tl_``-prefixed slot + paths, and the orphan sweep gained a ``ros2_uds_tl_`` prefix pass. +* ``DESIGN.md`` documents the doorbell wakeup, the graph-event wiring, GID + composition and context identity, and the TL ring. + +**Fixed** + +* **rmw_wait timeout contract.** The 200 ms poll bound made every wait + return ``RMW_RET_TIMEOUT`` at 200 ms regardless of the caller's deadline — a + 600 ms wait returned at 200 ms, and an infinite wait, which must never time + out, returned ``TIMEOUT``. Idle processes also woke 5×/s. ``RMW_RET_TIMEOUT`` + now surfaces only at the caller's own deadline. +* **rmw_wait reported spurious timeouts** when a drain legitimately yielded + nothing (an ``ignore_local_publications`` drop, or a shm descriptor whose + sender is gone). +* **A wait set holding only guard conditions got a null context** and skipped + the registry check entirely — the shape a publish-only node's executor + produces. The wait set now stores its context at ``rmw_create_wait_set``. +* **rmw_wait dispatched fds the current call never armed.** The armed-fd + cache survived across calls and was insert-only, so a guard condition owned by + another wait set could have its eventfd consumed with the trigger recorded + nowhere — a permanent lost wakeup that hung ``wait_for_service`` and + ``GraphListener``. Failed ``epoll_ctl`` ADDs now degrade to polling instead of + being silently dropped. +* **Dead graph-guard trigger.** ``rmw_wait`` fired + ``ctx->graph_guard_condition``, a context-level field that is declared and + never assigned, so the branch was dead code and the object ``rcl`` waits on + was never triggered (#43). +* **Zero-length datagrams were never dequeued.** ``recv_from`` peeks to size its + buffer; a zero-length datagram made the peek return 0 and the function bailed + out. ``MSG_PEEK`` does not consume, so the socket stayed readable forever, + spinning every wait that polled that fd. +* **Junk datagrams are now consumed atomically.** The zero-length cleanup used a + blind 1-byte ``recv`` after a separate, non-atomic peek; under a + ``MultiThreadedExecutor`` a racing drain could consume the peeked datagram and + the 1-byte ``recv`` would dequeue a real message and silently discard it. +* **Truncated consumes are detected with MSG_TRUNC and dropped.** + Pre-existing: the consuming ``recv`` omitted ``MSG_TRUNC``, so a + concurrently-dequeued peek followed by a larger datagram was silently + truncated to the smaller buffer and delivered as valid. +* **TL: a writer filling a slot the pull scan skipped** opened a sequence gap + with ``overlapped`` left false, defeating the contiguous-prefix trim. +* **TL: the replay watermark now stops at an unresolvable record.** ``max_seq`` + was fixed before the enqueue loop, so a record whose descriptor could no + longer be resolved was skipped while its sequence stayed under the watermark — + the sample was lost, not lapped. +* ``rmw_init``'s stale sweep is now stamped, and the throttle-scope comments + were corrected. +* Docs: ``TL_RING_MAX_BYTES`` was documented as 1 MiB (it is 2 MiB), and the + durable-segment budgeting note understated tmpfs sizing for over-cap latched + samples (a retained 5 MiB sample costs 5 MiB, not "an inode and a couple of + pages"). Reported by Copilot on #60. + +**Performance** + +* **rmw_wait drains only the fds epoll reported ready** (#57). It + previously made three passes over every entity per call — drain every socket, + re-arm every fd, drain every socket again — roughly 2150 syscalls for a + 719-entity wait set, nearly all returning ``EAGAIN`` or ``EEXIST``. +* **The stale-slot sweep is throttled off the graph query path** (#58). + ``query_all`` ran ``registry_cleanup_stale`` before every graph query across + 13 call sites, so ``rmw_get_node_names``, ``rmw_count_publishers``, + ``rmw_count_subscribers``, ``rmw_get_topic_names_and_types`` and the rest each + walked every live slot, copied 1164 bytes out of each, and ``stat``\ ed + ``/proc/``. +* **Launch arithmetic at N=200 nodes:** doorbell datagrams drop from ~100,300 + (19,900 of them from doorbell self-registration alone) to roughly the number + of graph-event consumers; wake-side registry copies go from + O(mutations × processes × slots) to zero for plain pub/sub processes. +* Latched-replay latency improves from ~20 ms (doorbell wake) to synchronous at + subscription creation. + +**Upgrade notes** + +* **Latched replay across a mixed-build pair is lost.** A 0.5.0 publisher no + longer push-replays and a pre-0.5.0 subscriber never pulls, so latched replay + between that pair does not work during a rolling upgrade — upgrade together + (same precedent as the shm payload flag). Old publisher + new subscriber keeps + working: an empty ``socket_path`` skips the pull and the old push path still + delivers. +* **Registry wakeups across a mixed-build fleet can be missed.** A build that + predates the doorbell bumps the generation but never rings. +* **Containers need --shm-size ≥ 1 GiB** to run the full test suite. The + 64 MB Docker default cannot hold the ~38 MB registry plus the payload rings. + +0.4.1 (2026-07-23) +------------------ +**Reverted** + +* ``fix(rmw_wait): deliver TRANSIENT_LOCAL latched samples to late joiners of + idle publishers`` (#37). The push-based approach woke the publisher process to + resend, and the doorbell broadcast it relied on did not scale — every registry + mutation rang every process, and every wake rescanned the registry, which + melted 200-node launches. Replaced in 0.5.0 by the pull-based ring (#60). + +0.4.0 (2026-07-16) +------------------ +**Added** + +* **Shared-memory ring transport for large topic payloads**, with CDR + serialized directly into the ring record. +* **Durable shm segments for large TRANSIENT_LOCAL messages**, including the + serialized-publish path. +* **Large service request/response payloads routed through shm.** +* ``fork()``-based cross-process shm integration tests. + +**Changed** + +* The inline-vs-shm send/receive decision unified into two helpers. +* ``DESIGN.md`` expanded into a fuller architecture document (#16); doc/comment + drift from the merge-readiness audit fixed. +* Repository prepared for open-source release (#15). + +**Fixed** + +* Contained the inline-path resize; the ring inline fallback is now tested. +* Honest TRANSIENT_LOCAL publish return value. + +**Performance** + +* Shorter ``cache_mutex`` hold, smaller ring floor, POD-keyed reader cache. + +0.3.0 (2026-06-17) +------------------ +Audit hardening & registry/fan-out performance. + +**Added** + +* TRANSIENT_LOCAL replay to late-joining subscribers (#2). +* README. + +**Fixed** + +* Contained ``fastcdr``/``std`` exceptions in ``serialize()`` so none cross the + ``extern "C"`` boundary (#4). +* Publish a registry slot's state only after its payload commits under the + seqlock (#5). +* Deep-copy ``security_options`` in ``rmw_init_options_copy`` to prevent a + double-free (#6). +* Accumulate the ``rmw_wait`` timeout in ``int64`` so ``RMW_DURATION_INFINITE`` + keeps blocking (#8). +* Write ``rmw_take_sequence`` output at the taken cursor (#7). +* Surface ``EMSGSIZE`` as ``RMW_RET_ERROR`` on publish (#3). +* Assorted P2/P3 hardening and identifier checks, plus dead-code removal (#9). +* Probe ``localhost_only`` via include dirs rather than linked targets, so + Rolling configure stops aborting. + +**Performance** + +* Bound query/cleanup scans with a shared high-water mark (#14). +* Filter registry slots by type before the per-slot snapshot copy (#11). +* Copy-on-write subscriber-path cache; prune the stale known-subscriber set on + graph change (#12). +* Cache ``rmw_service_server_is_available`` on the registry generation (#13). + +0.2.0 (2026-05-11) +------------------ +Observability. + +**Changed** + +* Failure paths routed through ``rcutils`` logging. +* ``ament_target_dependencies`` replaced with direct ``target_link_libraries``. +* ``ament_lint_auto`` dropped from the test suite. + +**Fixed** + +* CI: install ``test_msgs`` explicitly so Rolling stops silently skipping it; + tell ``rosdep`` to install ``test_depend`` packages; raise ``/dev/shm`` in the + container to 1 GiB. + +0.1.0 (2026-05-11) +------------------ +Initial release: a ROS 2 RMW implementation over ``AF_UNIX`` datagram sockets +with a lock-free shared-memory registry. + +**Added** + +* Lock-free registry via per-slot seqlock + atomic state. +* Cached discovery lookups and scale benchmarks. +* Build & test workflows for Rolling, Jazzy, and Kilted. + +**Fixed** + +* Guard ``rmw_init_options_t::localhost_only`` behind a CMake probe. diff --git a/rmw_unix_socket_cpp/package.xml b/rmw_unix_socket_cpp/package.xml index 40422b3..dc9b26e 100644 --- a/rmw_unix_socket_cpp/package.xml +++ b/rmw_unix_socket_cpp/package.xml @@ -10,6 +10,10 @@ Abderahmane BENALI Apache-2.0 + https://github.com/benaliabderrahmane/rmw_unix_socket_cpp + https://github.com/benaliabderrahmane/rmw_unix_socket_cpp/issues + https://github.com/benaliabderrahmane/rmw_unix_socket_cpp#readme + ament_cmake rcutils From 8354ddc0a32210f4056d6c3bbd45170822e8da67 Mon Sep 17 00:00:00 2001 From: Abderahmane BENALI Date: Fri, 11 Sep 2026 20:40:00 +0200 Subject: [PATCH 2/3] chore(release): a CI guard for the release metadata The release-metadata job re-parses CHANGELOG.rst the way catkin_pkg does and fails on a dropped sub-section instead of shipping an empty debian changelog, and checks the version/tag pairing; release.yml uses the v tag prefix that catkin_prepare_release --tag-prefix v produces. Squashed from: - chore(release): tighten the metadata guard and fix the release tag prefix - chore(release): reject dropped changelog sub-sections structurally --- .github/workflows/ci.yml | 40 ++++++++++++++++++++++++++++++++++------ RELEASING.md | 10 ++++++++-- 2 files changed, 42 insertions(+), 8 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index e580d61..af810ca 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -114,24 +114,52 @@ jobs: sudo apt-get update sudo apt-get install -y --no-install-recommends python3-catkin-pkg python3 - <<'PY' + import re + from catkin_pkg.changelog import get_changelog_from_path from catkin_pkg.package import parse_package + from docutils import nodes + from docutils.core import publish_doctree PKG = 'rmw_unix_socket_cpp' pkg = parse_package(PKG) pkg.validate() - assert pkg.urls, 'package.xml has no tags' + required_urls = {'repository', 'bugtracker', 'website'} + missing = required_urls - {u.type for u in pkg.urls} + assert not missing, f'package.xml is missing types: {sorted(missing)}' assert pkg.maintainers[0].email, 'maintainer has no email' # bloom only reads CHANGELOG.rst sitting next to package.xml. - changelog = get_changelog_from_path(PKG, PKG) + changelog = get_changelog_from_path(PKG) assert changelog is not None, f'no CHANGELOG.rst in {PKG}/' versions = [v for v, _, _ in changelog.foreach_version()] - assert pkg.version in versions, f'package.xml {pkg.version} has no changelog entry' - # REP-132 forbids sub-sections under a version; catkin_pkg drops them - # silently, so a version written that way ships an empty debian entry. + assert pkg.version == versions[-1], \ + f'package.xml {pkg.version} is not the newest changelog entry {versions[-1]}' for version, date, content in changelog.foreach_version(): assert date, f'{version} has no parseable date' - assert content, f'{version} body is empty (sub-sections are dropped)' + assert content, f'{version} body is empty' + + # The parsed view above is not enough: REP-132 allows only paragraphs + # and lists under a version heading, and catkin_pkg drops sub-sections + # silently. A version with an intro paragraph plus sub-sections still + # parses non-empty while every bullet under them is lost, so check the + # raw rst structure too. + tree = publish_doctree( + changelog.rst, + settings_overrides={'report_level': 5, 'warning_stream': False}) + for section in tree.findall(nodes.section): + title = section.next_node(nodes.title).astext() + if not re.match(r'^\d+\.\d+\.\d+ \(', title): + continue + nested = [s.next_node(nodes.title).astext() + for s in section.findall(nodes.section) if s is not section] + assert not nested, f'{title} has rst sub-sections {nested}, which are dropped' + + # rst has no nested inline markup, so ``code`` inside **bold** emits + # the backticks literally and docutils reports nothing. + for strong in tree.findall(nodes.strong): + text = strong.astext() + assert '``' not in text, f'literal backticks in bold: {text[:60]}' + print(f'{PKG} {pkg.version}: {len(versions)} changelog entries, metadata OK') PY diff --git a/RELEASING.md b/RELEASING.md index 9feee23..3563d58 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -45,13 +45,19 @@ Only this first rosdistro PR gets real scrutiny. ## Every release after that ```bash -catkin_generate_changelog # appends unreleased commits to rmw_unix_socket_cpp/CHANGELOG.rst +catkin_generate_changelog # appends unreleased commits to rmw_unix_socket_cpp/CHANGELOG.rst $EDITOR rmw_unix_socket_cpp/CHANGELOG.rst git commit -am "Update changelog" -catkin_prepare_release # bumps package.xml, commits, tags, pushes +catkin_prepare_release --tag-prefix v # bumps package.xml, commits, tags, pushes bloom-release --rosdistro jazzy --track jazzy rmw_unix_socket_cpp ``` +`--tag-prefix v` is not optional. `catkin_prepare_release` builds its tag as +`tag_prefix + version` and the prefix defaults to empty, so without it you get +`0.6.0` — which the `v:{version}` bloom track will not find, and which +`release.yml` ignores because it only triggers on `v*`. It pushes the commit and +the tag itself; `--no-push` opts out. + No `--new-track` — bloom reuses the answers above. The resulting rosdistro PR changes one line, the `version:` field. From a8ad796addc97963652d485aa29de76f014cba18 Mon Sep 17 00:00:00 2001 From: benaliabderrahmane Date: Fri, 18 Sep 2026 01:08:15 +0200 Subject: [PATCH 3/3] docs(release): first release must cut a new tag; bloom does not create the release repo - "First release" ran bloom-release straight away. With Version :{auto} that resolves to package.xml's 0.5.0, i.e. the existing v0.5.0 tag - which predates CHANGELOG.rst and the tags this PR adds. Tag first. - bloom's "Release repository url" prompt wants the URL of an existing repository; it cannot create one. Rolling-derived distros additionally require it to live in the ros2-gbp organisation. - The token needs the workflow scope as well as public_repo. - catkin_prepare_release bumps the patch version by default; the 0.6.0 in the examples needs --bump minor. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01NUNQNo26cKRPrVXcaHZnje --- RELEASING.md | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/RELEASING.md b/RELEASING.md index 3563d58..b875f56 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -9,7 +9,7 @@ touches source on `main`. sudo apt install python3-bloom python3-catkin-pkg ``` -`bloom` also needs a GitHub token with `public_repo` scope. +`bloom` also needs a GitHub token with `public_repo` and `workflow` scopes. ## Supported distros @@ -24,18 +24,30 @@ Fast CDR 1.0.x, where that enum does not exist. | Prompt | Answer | | --- | --- | -| Release repository url | yes — let bloom create `…/rmw_unix_socket_cpp-release` | +| Release repository url | `https://github.com/benaliabderrahmane/rmw_unix_socket_cpp-release.git` | | Upstream repository uri | `https://github.com/benaliabderrahmane/rmw_unix_socket_cpp.git` | | Upstream devel branch | the branch for that distro (`main` until per-distro branches exist) | | Version | `:{auto}` | | Release tag | `v:{version}` | +bloom does not create the release repository: the URL must point at one that +already exists, empty is fine. Rolling (and every distro branched from it after +that policy, so Lyrical too) only accepts a release repository hosted in the +`ros2-gbp` organisation - request one through +`ros2-gbp/ros2-gbp-github-org` and use its URL for those tracks instead. + The release tag matters: our tags are `v0.5.0` but `catkin_prepare_release` writes `0.5.0`, so a bare `:{version}` makes bloom look for a tag that isn't there. ## First release +Cut a new tag first. `:{auto}` reads the version from `package.xml`, and the +`v0.5.0` tag that already exists predates `CHANGELOG.rst` and the `` tags - +releasing it would ship exactly the empty changelog this setup exists to avoid. +Run the tagging half of the loop below (`catkin_generate_changelog` … +`catkin_prepare_release --tag-prefix v`), then: + ```bash bloom-release --new-track --rosdistro jazzy --track jazzy rmw_unix_socket_cpp ``` @@ -48,7 +60,7 @@ Only this first rosdistro PR gets real scrutiny. catkin_generate_changelog # appends unreleased commits to rmw_unix_socket_cpp/CHANGELOG.rst $EDITOR rmw_unix_socket_cpp/CHANGELOG.rst git commit -am "Update changelog" -catkin_prepare_release --tag-prefix v # bumps package.xml, commits, tags, pushes +catkin_prepare_release --tag-prefix v # bumps the patch version (--bump minor|major for more), commits, tags, pushes bloom-release --rosdistro jazzy --track jazzy rmw_unix_socket_cpp ```