You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 3ca379b
Browse filesBrowse the repository at this point in the historyBrowse files
fix(rmw_wait): keep the wait timeout's sub-millisecond resolution (#83)
* fix(rmw_wait): keep the wait timeout's sub-millisecond resolution
rmw_wait converted the caller's timeout to whole milliseconds for
epoll_wait: a timeout under 1 ms became 1 ms, the deadline was computed
from the truncated value, and every re-block rounded the remaining time
up. rcl passes the time to the next timer in nanoseconds, so a 1 kHz
timer woke up to a whole period late and rclcpp skipped about 6 % of its
periods, and a 1.7 ms wait returned RMW_RET_TIMEOUT after 1 ms.
Keep the timeout and the deadline in nanoseconds and block with
epoll_pwait2. Where it is unavailable (glibc < 2.35 at build time, ENOSYS
or EPERM at run time) fall back to epoll_wait with the remaining time
rounded up, so the fallback never returns before the deadline either.
Test: WaitKeepsSubMillisecondTimeoutResolution and
OneKilohertzWaitLoopStaysOnSchedule fail on devel (1.7 ms waits time out
after ~1.06 ms; 264 of 500 periods wake more than 0.5 ms late) and pass
with the fix.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Jh5PFqFmFoZnaSX5FfVv4G
* Address review: drop a redundant comment
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Jh5PFqFmFoZnaSX5FfVv4G
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: rmw_unix_socket_cpp/DESIGN.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -534,7 +534,7 @@ Each `rmw_wait()` runs the same sequence on the calling thread:
534
534
1.**Drain first.** Every subscription, service, and client socket is drained into a per-entity message queue before anything blocks. The receive sockets are `SOCK_DGRAM | SOCK_NONBLOCK`, so the drain loop calls `recv_from` repeatedly and stops cleanly on `EAGAIN` (nothing left to read). This step exists because data may have arrived between the previous `rmw_wait()` and this one; draining up front means such data is not missed.
535
535
2.**Check the graph.** The wait set carries its context (stored at `rmw_create_wait_set`), so this step runs for every wait — including a wait set that holds only guard conditions, which is exactly the shape rclcpp's GraphListener uses. The check first drains the context's doorbell socket, then reads the registry's `generation` counter and compares it against the value cached on the context. If it moved, the graph changed: every node's graph guard condition is triggered (see the next subsection). (`TRANSIENT_LOCAL` replay no longer runs here — it is pull-based, served inside `rmw_create_subscription` of the joining process.) The drain-before-read order is half of the lost-wakeup proof; the other half is on the registry's writer side.
536
536
3.**Arm the fds.** Every entity fd, guard-condition eventfd, and the context's doorbell fd is added to the epoll instance with `EPOLL_CTL_ADD`. This is idempotent across calls: a still-live fd returns `EEXIST` and is treated as already armed, and a fd number reused after its previous owner closed gets freshly armed. The kernel removes closed fds from an epoll set automatically, so there is no matching `EPOLL_CTL_DEL` and no per-call teardown.
537
-
4.**Check ready, then block only if needed.** If any queue already holds a message, or any guard-condition eventfd reads as triggered, the call skips blocking entirely and reports what is ready. Only when nothing is ready does it block in `epoll_wait()`. The thread is asleep in the kernel here, not spinning; there is no internal poll interval. A wake caused only by the doorbell is internal: the wait re-runs the graph check of step 2 and goes back to sleep for the caller's remaining time. `epoll_wait()` is retried on `EINTR` against the same deadline so a signal neither returns a false timeout nor busy-loops.
537
+
4.**Check ready, then block only if needed.** If any queue already holds a message, or any guard-condition eventfd reads as triggered, the call skips blocking entirely and reports what is ready. Only when nothing is ready does it block in `epoll_pwait2()`, which takes the caller's timeout with nanosecond resolution so a sub-millisecond wait (the remainder of a 1 kHz timer period) is neither stretched nor cut to a whole millisecond. Where `epoll_pwait2()` is unavailable (glibc < 2.35, kernel < 5.11, or a seccomp profile that blocks it) it falls back to `epoll_wait()` with the timeout rounded up to the next millisecond. The thread is asleep in the kernel here, not spinning; there is no internal poll interval. A wake caused only by the doorbell is internal: the wait re-runs the graph check of step 2 and goes back to sleep for the caller's remaining time. The wait is retried on `EINTR` against the same deadline so a signal neither returns a false timeout nor busy-loops.
538
538
5.**Drain again and report.** After waking, the sockets are drained once more, then the output arrays are pruned: entities with no pending data are set to `NULL`, and the ones that are ready are left in place for the executor to service. `RMW_RET_TIMEOUT` is returned only once the caller's own deadline has passed. A wake that drained nothing to take returns `RMW_RET_OK` with every entry `NULL` instead: a drain can legitimately yield nothing — a subscription with `ignore_local_publications` set drops its own context's messages, and a descriptor whose sender is gone is dropped — and that is a spurious wake, not a timeout. An infinite wait therefore never reports `RMW_RET_TIMEOUT`.
Copy file name to clipboardExpand all lines: rmw_unix_socket_cpp/test/README.md
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -130,6 +130,8 @@ Tests the epoll-based wait set.
130
130
|`WaitWithGuardCondition`| Pre-triggered guard condition causes immediate return with `RMW_RET_OK`| Validates the "already ready" fast path — avoids unnecessary epoll_wait when data is already available. |
131
131
|`WaitTimeoutWhenNoData`| Returns `RMW_RET_TIMEOUT` after specified duration, guard condition nulled out | Validates the timeout path. Without this, the executor would hang indefinitely when no messages arrive. |
132
132
|`WaitWithSubscription`| Publish → wait on subscription → returns ready → take succeeds | End-to-end: message arrives on socket → epoll detects it → drain into queue → subscription marked ready → take succeeds. Tests the full executor wakeup path. |
133
+
|`WaitKeepsSubMillisecondTimeoutResolution`| 0.3 ms and 1.7 ms waits return `RMW_RET_TIMEOUT` no earlier than the timeout and less than 0.5 ms after it | rcl passes the time to the next timer in nanoseconds. Rounding it to whole milliseconds made a 1.7 ms wait end after 1 ms (a false timeout) and a 0.3 ms wait take 1 ms. |
134
+
|`OneKilohertzWaitLoopStaysOnSchedule`| A 1 kHz loop of deadline-based `rmw_wait` calls wakes within 0.5 ms of each period in at least 95 % of 500 periods | This is what an executor does for a 1 kHz timer. With millisecond rounding the loop slid behind schedule and rclcpp skipped periods. |
0 commit comments