Skip to content

Commit 2dfc7ac

Browse files
benaliabderrahmanebenaliabderrahmane
authored
Merge pull request #63 from benaliabderrahmane/devel
Release v0.5.0
2 parents 6238500 + db652db commit 2dfc7ac

28 files changed

Lines changed: 3893 additions & 625 deletions

‎.github/workflows/ci.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,9 @@ name: CI
22

33
on:
44
push:
5-
branches: [main]
5+
branches: [main, devel]
66
pull_request:
7-
branches: [main]
7+
branches: [main, devel]
88
workflow_dispatch:
99

1010
jobs:

‎CHANGELOG.md‎

Lines changed: 251 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,251 @@
1+
# Changelog
2+
3+
All notable changes to `rmw_unix_socket_cpp` are documented here.
4+
5+
The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
6+
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7+
8+
## [0.5.0] - 2026-08-27
9+
10+
The wait/wakeup release. The 200 ms `rmw_wait` poll is gone, replaced by an
11+
event-driven doorbell; TRANSIENT_LOCAL latched replay is rebuilt as a pull from
12+
a per-publisher shared-memory ring; and the per-node graph guard conditions that
13+
`rclcpp`'s `GraphListener` actually waits on are finally triggered.
14+
15+
> **Upgrade together.** See [Upgrade notes](#upgrade-notes-050) — a mixed-build
16+
> fleet can lose latched replay and registry wakeups during the rollout window.
17+
18+
### Added
19+
20+
- **Pull-based TRANSIENT_LOCAL replay** (#60). A latched publisher writes each
21+
sample into a per-publisher shm ring (`tl_ring`, `qos.depth` per-record-seqlocked
22+
slots) at publish time; a late-joining subscription reads that ring itself
23+
inside `rmw_create_subscription`. The idle publisher is never woken, polled, or
24+
rung. History is enqueued before the subscription handle returns, so it always
25+
precedes live samples. Payloads over `TL_EMBED_CAP` (1 KiB) ride the existing
26+
durable segments as 32-byte descriptors; ring bytes are capped at
27+
`TL_RING_MAX_BYTES` (2 MiB).
28+
- **`rmw_subscription_options_t::ignore_local_publications`** (#49). Previously
29+
discarded at creation. Each context now draws a random 64-bit `context_id` in
30+
`rmw_init` and embeds it in the trailing 8 bytes of every GID, so
31+
`is_same_context()` answers from bytes already on the wire — no extra traffic,
32+
no wire-format change. "Local" is the same `rmw_context_t`, per the rmw
33+
contract, not merely the same process.
34+
- **Event-driven registry wakeup (doorbell).** Each context binds a doorbell
35+
socket at `rmw_init`; every registry mutation sends one octet to every
36+
registered doorbell strictly *after* bumping the generation, and `rmw_wait`
37+
drains its doorbell strictly *before* reading the generation. That ordering
38+
pair makes a lost wakeup impossible. No new threads, no registry layout change.
39+
- **Graph-change triggering of per-node guard conditions** (#43).
40+
- CI: **Lyrical** added to the build & test matrix (#17); CI now also runs on
41+
`devel` pushes and PRs.
42+
- GitHub issue forms and a pull request template (#56).
43+
- Tests: `test_rmw_wait.cpp` (new, +648 lines), `test_shm_transport.cpp` (+283),
44+
substantial additions to `test_rmw_qos.cpp` (+873), `test_rmw_graph.cpp`,
45+
`test_transport.cpp`, `test_rmw_pub_sub.cpp`, and
46+
`test_rmw_service_client.cpp`.
47+
48+
### Changed
49+
50+
- **VOLATILE late joiners no longer receive latched history.** Pulled records
51+
are filtered by durability — DDS-correct behaviour. The old push path replayed
52+
to every subscriber on the topic regardless of its durability QoS.
53+
- The `ENTRY_DOORBELL` slot is now registered **lazily**, on the first
54+
`rmw_wait` holding a graph guard condition, so only graph-event consumers
55+
(`wait_for_service`, `GraphListener`, rosbag2) pay for registry wakeups.
56+
Doorbell-slot mutations no longer ring, which removes a K²/2 registration
57+
mini-storm.
58+
- Removed with the push machinery: `transient_local_pubs` (and its mutex),
59+
`known_subscriber_paths`, `CachedMessage` / the heap message cache,
60+
`transient_local_publish`'s replay loop, and the wait-side TL replay block.
61+
- `registry`: `teardown_slot` now `shm_unlink`s `tl_`-prefixed slot paths, and
62+
the orphan sweep gained a `ros2_uds_tl_` prefix pass.
63+
- `DESIGN.md` documents the doorbell wakeup, the graph-event wiring, GID
64+
composition and context identity, and the TL ring.
65+
66+
### Fixed
67+
68+
- **`rmw_wait` timeout contract.** The 200 ms poll bound made every wait return
69+
`RMW_RET_TIMEOUT` at 200 ms regardless of the caller's deadline — a 600 ms wait
70+
returned at 200 ms, and an infinite wait, which must never time out, returned
71+
`TIMEOUT`. Idle processes also woke 5×/s. `RMW_RET_TIMEOUT` now surfaces only
72+
at the caller's own deadline.
73+
- **`rmw_wait` reported spurious timeouts** when a drain legitimately yielded
74+
nothing (an `ignore_local_publications` drop, or a shm descriptor whose sender
75+
is gone).
76+
- **A wait set holding only guard conditions got a null context** and skipped
77+
the registry check entirely — the shape a publish-only node's executor
78+
produces. The wait set now stores its context at `rmw_create_wait_set`.
79+
- **`rmw_wait` dispatched fds the current call never armed.** The armed-fd cache
80+
survived across calls and was insert-only, so a guard condition owned by
81+
another wait set could have its eventfd consumed with the trigger recorded
82+
nowhere — a permanent lost wakeup that hung `wait_for_service` and
83+
`GraphListener`. Failed `epoll_ctl` ADDs now degrade to polling instead of
84+
being silently dropped.
85+
- **Dead graph-guard trigger.** `rmw_wait` fired `ctx->graph_guard_condition`, a
86+
context-level field that is declared and never assigned, so the branch was
87+
dead code and the object `rcl` waits on was never triggered (#43).
88+
- **Zero-length datagrams were never dequeued.** `recv_from` peeks to size its
89+
buffer; a zero-length datagram made the peek return 0 and the function bailed
90+
out. `MSG_PEEK` does not consume, so the socket stayed readable forever,
91+
spinning every wait that polled that fd.
92+
- **Junk datagrams are now consumed atomically.** The zero-length cleanup used a
93+
blind 1-byte `recv` after a separate, non-atomic peek; under a
94+
`MultiThreadedExecutor` a racing drain could consume the peeked datagram and
95+
the 1-byte `recv` would dequeue a real message and silently discard it.
96+
- **Truncated consumes are detected with `MSG_TRUNC` and dropped.** Pre-existing:
97+
the consuming `recv` omitted `MSG_TRUNC`, so a concurrently-dequeued peek
98+
followed by a larger datagram was silently truncated to the smaller buffer and
99+
delivered as valid.
100+
- **TL: a writer filling a slot the pull scan skipped** opened a sequence gap
101+
with `overlapped` left false, defeating the contiguous-prefix trim.
102+
- **TL: the replay watermark now stops at an unresolvable record.** `max_seq` was
103+
fixed before the enqueue loop, so a record whose descriptor could no longer be
104+
resolved was skipped while its sequence stayed under the watermark — the
105+
sample was lost, not lapped.
106+
- `rmw_init`'s stale sweep is now stamped, and the throttle-scope comments were
107+
corrected.
108+
- Docs: `TL_RING_MAX_BYTES` was documented as 1 MiB (it is 2 MiB), and the
109+
durable-segment budgeting note understated tmpfs sizing for over-cap latched
110+
samples (a retained 5 MiB sample costs 5 MiB, not "an inode and a couple of
111+
pages"). Reported by Copilot on #60.
112+
113+
### Performance
114+
115+
- **`rmw_wait` drains only the fds `epoll` reported ready** (#57). It previously
116+
made three passes over every entity per call — drain every socket, re-arm every
117+
fd, drain every socket again — roughly 2150 syscalls for a 719-entity wait set,
118+
nearly all returning `EAGAIN` or `EEXIST`.
119+
- **The stale-slot sweep is throttled off the graph query path** (#58).
120+
`query_all` ran `registry_cleanup_stale` before every graph query across 13
121+
call sites, so `rmw_get_node_names`, `rmw_count_publishers`,
122+
`rmw_count_subscribers`, `rmw_get_topic_names_and_types` and the rest each
123+
walked every live slot, copied 1164 bytes out of each, and `stat`ed
124+
`/proc/<pid>`.
125+
- **Launch arithmetic at N=200 nodes:** doorbell datagrams drop from ~100,300
126+
(19,900 of them from doorbell self-registration alone) to roughly the number of
127+
graph-event consumers; wake-side registry copies go from
128+
O(mutations × processes × slots) to zero for plain pub/sub processes.
129+
- Latched-replay latency improves from ~20 ms (doorbell wake) to synchronous at
130+
subscription creation.
131+
132+
### Upgrade notes (0.5.0)
133+
134+
- **Latched replay across a mixed-build pair is lost.** A 0.5.0 publisher no
135+
longer push-replays and a pre-0.5.0 subscriber never pulls, so latched replay
136+
between that pair does not work during a rolling upgrade — upgrade together
137+
(same precedent as the shm payload flag). Old publisher + new subscriber keeps
138+
working: an empty `socket_path` skips the pull and the old push path still
139+
delivers.
140+
- **Registry wakeups across a mixed-build fleet can be missed.** A build that
141+
predates the doorbell bumps the generation but never rings.
142+
- **Containers need `--shm-size` ≥ 1 GiB** to run the full test suite. The 64 MB
143+
Docker default cannot hold the ~38 MB registry plus the payload rings.
144+
145+
## [0.4.1] - 2026-07-23
146+
147+
### Reverted
148+
149+
- `fix(rmw_wait): deliver TRANSIENT_LOCAL latched samples to late joiners of
150+
idle publishers` (#37). The push-based approach woke the publisher process to
151+
resend, and the doorbell broadcast it relied on did not scale — every registry
152+
mutation rang every process, and every wake rescanned the registry, which
153+
melted 200-node launches. Replaced in 0.5.0 by the pull-based ring (#60).
154+
155+
## [0.4.0] - 2026-07-16
156+
157+
### Added
158+
159+
- **Shared-memory ring transport for large topic payloads**, with CDR serialized
160+
directly into the ring record.
161+
- **Durable shm segments for large TRANSIENT_LOCAL messages**, including the
162+
serialized-publish path.
163+
- **Large service request/response payloads routed through shm.**
164+
- `fork()`-based cross-process shm integration tests.
165+
166+
### Changed
167+
168+
- The inline-vs-shm send/receive decision unified into two helpers.
169+
- `DESIGN.md` expanded into a fuller architecture document (#16); doc/comment
170+
drift from the merge-readiness audit fixed.
171+
- Repository prepared for open-source release (#15).
172+
173+
### Fixed
174+
175+
- Contained the inline-path resize; the ring inline fallback is now tested.
176+
- Honest TRANSIENT_LOCAL publish return value.
177+
178+
### Performance
179+
180+
- Shorter `cache_mutex` hold, smaller ring floor, POD-keyed reader cache.
181+
182+
## [0.3.0] - 2026-06-17
183+
184+
Audit hardening & registry/fan-out performance.
185+
186+
### Added
187+
188+
- TRANSIENT_LOCAL replay to late-joining subscribers (#2).
189+
- README.
190+
191+
### Fixed
192+
193+
- Contained `fastcdr`/`std` exceptions in `serialize()` so none cross the
194+
`extern "C"` boundary (#4).
195+
- Publish a registry slot's state only after its payload commits under the
196+
seqlock (#5).
197+
- Deep-copy `security_options` in `rmw_init_options_copy` to prevent a
198+
double-free (#6).
199+
- Accumulate the `rmw_wait` timeout in `int64` so `RMW_DURATION_INFINITE` keeps
200+
blocking (#8).
201+
- Write `rmw_take_sequence` output at the taken cursor (#7).
202+
- Surface `EMSGSIZE` as `RMW_RET_ERROR` on publish (#3).
203+
- Assorted P2/P3 hardening and identifier checks, plus dead-code removal (#9).
204+
- Probe `localhost_only` via include dirs rather than linked targets, so Rolling
205+
configure stops aborting.
206+
207+
### Performance
208+
209+
- Bound query/cleanup scans with a shared high-water mark (#14).
210+
- Filter registry slots by type before the per-slot snapshot copy (#11).
211+
- Copy-on-write subscriber-path cache; prune the stale known-subscriber set on
212+
graph change (#12).
213+
- Cache `rmw_service_server_is_available` on the registry generation (#13).
214+
215+
## [0.2.0] - 2026-05-11
216+
217+
Observability.
218+
219+
### Changed
220+
221+
- Failure paths routed through `rcutils` logging.
222+
- `ament_target_dependencies` replaced with direct `target_link_libraries`.
223+
- `ament_lint_auto` dropped from the test suite.
224+
225+
### Fixed
226+
227+
- CI: install `test_msgs` explicitly so Rolling stops silently skipping it; tell
228+
`rosdep` to install `test_depend` packages; raise `/dev/shm` in the container
229+
to 1 GiB.
230+
231+
## [0.1.0] - 2026-05-11
232+
233+
Initial release: a ROS 2 RMW implementation over `AF_UNIX` datagram sockets with
234+
a lock-free shared-memory registry.
235+
236+
### Added
237+
238+
- Lock-free registry via per-slot seqlock + atomic state.
239+
- Cached discovery lookups and scale benchmarks.
240+
- Build & test workflows for Rolling, Jazzy, and Kilted.
241+
242+
### Fixed
243+
244+
- Guard `rmw_init_options_t::localhost_only` behind a CMake probe.
245+
246+
[0.5.0]: https://github.com/benaliabderrahmane/rmw_unix_socket_cpp/compare/v0.4.1...v0.5.0
247+
[0.4.1]: https://github.com/benaliabderrahmane/rmw_unix_socket_cpp/compare/v0.4.0...v0.4.1
248+
[0.4.0]: https://github.com/benaliabderrahmane/rmw_unix_socket_cpp/compare/v0.3.0...v0.4.0
249+
[0.3.0]: https://github.com/benaliabderrahmane/rmw_unix_socket_cpp/compare/v0.2.0...v0.3.0
250+
[0.2.0]: https://github.com/benaliabderrahmane/rmw_unix_socket_cpp/compare/v0.1.0...v0.2.0
251+
[0.1.0]: https://github.com/benaliabderrahmane/rmw_unix_socket_cpp/releases/tag/v0.1.0

‎README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,9 +22,9 @@ datagram sockets; the wait set is `epoll` + `eventfd`.
2222

2323
## Status
2424

25-
- ROS 2 **Jazzy**, **Kilted**, and **Rolling** — built and tested in CI.
25+
- ROS 2 **Jazzy**, **Kilted**, **Rolling**, and **Lyrical** — built and tested in CI.
2626
- **Linux / x86_64 only** (uses `epoll`, `eventfd`, `/dev/shm`, `AF_UNIX`).
27-
- Version `0.1.0`, Apache-2.0.
27+
- Version `0.5.0`, Apache-2.0.
2828

2929
## Quick start
3030

‎rmw_unix_socket_cpp/CMakeLists.txt‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -197,8 +197,12 @@ if(BUILD_TESTING)
197197
target_link_libraries(test_rmw_wait ${_rmw_test_libs} ${_test_msg_deps})
198198
target_include_directories(test_rmw_wait PRIVATE src)
199199

200-
# RMW API: graph introspection
201-
ament_add_gtest(test_rmw_graph test/test_rmw_graph.cpp)
200+
# RMW API: graph introspection. registry.cpp is compiled in as well so the
201+
# test can seed the shared registry directly; the rmw library does not export
202+
# those internal symbols. Both copies act on the same shared memory; the only
203+
# process-local state in registry.cpp is ring_doorbells' thread_local send-fd
204+
# cache, one per copy, so the duplication is benign.
205+
ament_add_gtest(test_rmw_graph test/test_rmw_graph.cpp src/registry.cpp)
202206
target_link_libraries(test_rmw_graph ${_rmw_test_libs} ${_test_msg_deps})
203207
target_include_directories(test_rmw_graph PRIVATE src)
204208

0 commit comments

Comments
 (0)