Skip to content

Commit 4acb4bc

Browse files
JanStaschulatralph-lange
authored andcommitted
Restructured document. Combined description of ROS Executor into one section.
Signed-off-by: Staschulat Jan (CR/AEX3) <jan.staschulat@de.bosch.com>
1 parent 4e758df commit 4acb4bc

1 file changed

Lines changed: 34 additions & 35 deletions

File tree

  • _docs/concepts/client_library/real-time_executor

‎_docs/concepts/client_library/real-time_executor/index.md‎

Lines changed: 34 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -8,18 +8,22 @@ permalink: /docs/concepts/client_library/real-time_executor/
88
## Table of contents
99

1010
* [Introduction](#introduction)
11-
* [ROS 2 Executor Concept](#ros-2-executor-concept)
12-
* [rcl LET-Executor](#rcl-let-executor)
11+
12+
* [ROS 2 Executor](#ros-2-executor)
1313
* [Concept](#concept)
14-
* [Example](#example)
15-
* [Download](#download)
16-
* [Analysis of Default Rclcpp Executor](#analysis-of-default-rclcpp-executor)
1714
* [Architecture](#architecture)
1815
* [Scheduling Semantics](#scheduling-semantics)
16+
17+
* [Rcl LET-Executor](#rcl-let-executor)
18+
* [Concept](#concept)
19+
* [Example](#example)
20+
* [Download](#download)
21+
1922
* [Callback-group-level Executor](#callback-group-level-executor)
2023
* [API Changes](#api-changes)
2124
* [Meta Executor Concept](#meta-executor-concept)
2225
* [Test Bench](#test-bench)
26+
2327
* [Related Work](#related-work)
2428
* [Roadmap](#roadmap)
2529
* [References](#references)
@@ -48,16 +52,39 @@ Our approach is to provide a real-time executor on two layers as described in se
4852

4953
As the first step, we propose the LET-Executor, which implements static order scheduling policy with logic execution time semantics. In this scheduling policy, all processes are executed in a pre-defined order. Logical execution time refers to the concept, that first input data is read before tasks are executed. Secondly, we developed a Callback-group-level executor, which allows to prioritize a group of callbacks. These approaches are based on the concept of executors, which have been introduced in ROS 2.
5054

55+
## ROS 2 Executor
5156

52-
## ROS 2 Executor Concept
57+
### Concept
5358

5459
ROS 2 allows to bundle multiple nodes in one operating system process. To coordinate the execution of the callbacks of the nodes of a process, the Executor concept was introduced in rclcpp (and also in rclpy).
5560

5661
The ROS 2 design defines one Executor (instance of [rclcpp::executor::Executor](https://github.com/ros2/rclcpp/blob/master/rclcpp/include/rclcpp/executor.hpp)) per process, which is typically created either in a custom main function or by the launch system. The Executor coordinates the execution of all callbacks issued by these nodes by checking for available work (timers, services, messages, subscriptions, etc.) from the DDS queue and dispatching it to one or more threads, implemented in [SingleThreadedExecutor](https://github.com/ros2/rclcpp/blob/master/rclcpp/include/rclcpp/executors/single_threaded_executor.hpp) and [MultiThreadedExecutor](https://github.com/ros2/rclcpp/blob/master/rclcpp/include/rclcpp/executors/multi_threaded_executor.hpp), respectively.
5762

5863
The dispatching mechanism resembles the ROS 1 spin thread behavior: the Executor looks up the wait queues, which notifies it of any pending callback in the DDS queue. If there are multiple pending callbacks, the ROS 2 Executor executes them in an in the order as they were registred at the Executor.
5964

60-
See also Section [Analysis of Default Rclcpp Executor](#analysis-of-default-rclcpp-executor) for a more detailed functional description and an analysis of its semantics.
65+
### Architecture
66+
67+
The following diagram depicts the relevant classes of the standard ROS 2 Executor implementation:
68+
69+
![ROS 2 Executor class diagram](executor_class_diagram.png)
70+
71+
Note that an Executor instance maintains weak pointers to the NodeBaseInterfaces of the nodes only. Therefore, nodes can be destroyed safely, without notifying the Executor.
72+
73+
Also, the Executor does not maintain an explicit callback queue, but relies on the queue mechanism of the underlying DDS implementation as illustrated in the following sequence diagram:
74+
75+
![Call sequence from executor to DDS](executor_to_dds_sequence_diagram.png)
76+
77+
The Executor concept, however, does not provide means for prioritization or categorization of the incoming callback calls. Moreover, it does not leverage the real-time characteristics of the underlying operating-system scheduler to have finer control on the order of executions. The overall implication of this behavior is that time-critical callbacks could suffer possible deadline misses and a degraded performance since they are serviced later than non-critical callbacks. Additionally, due to the FIFO mechanism, it is difficult to determine usable bounds on the worst-case latency that each callback execution may incur.
78+
79+
### Scheduling Semantics
80+
81+
In a recent paper [CB2019](#CB2019), the rclcpp executor has been analyzed in detail and a response time analysis of cause-effect chains has been proposed under reservation-based scheduling. The executor distinguishes four categories of callbacks: _timers_, which are triggered by system-level timers, _subscribers_, which are triggered by new messages on a subscribed topic, _services_, which are triggered by service requests, and _clients_, which are triggered by responses to service requests. The executor is responsible for taking messages from the input queues of the DDS layer and executing the corresponding callback. Since it executes callbacks to completion, it is a non-preemptive scheduler, However it does not consider all ready tasks for execution, but only a snapshot, called readySet. This readySet is updated when the executor is idle and in this step it interacts with the DDS layer updating the set of ready tasks. Then for every type of task, there are dedicated queues (timers, subscriptions, services, clients) which are processed sequentially. The following key issues were pointed out:
82+
83+
* The executor processes _timers_ always first. This can lead to the intrinsic effect, that in overload situations messages from the DDS queue are not processed
84+
* Messages arriving during the processing of the readySet are not considered until the next update, which depends on the execution time of all remaining callbacks. This leads to priority inversion, as lower-priority callbacks may implicitly block higher-priority callbacks by prolonging the current processing of the readySet.
85+
* The readySet contains only one task instance, For example, even if multiple messages of the same topic are available, only one instance is processed until the executor is idle again and the readySet is updated from the DDS layer. This aggravates priority inversion, as a backlogged callback might have to wait for multiple processing of readySets until it is considered for scheduling. This means that non-timer callback instances might be blocked by multiple instances of the same lower-priority callback.
86+
87+
Due to these findings, an alternative approach is presented to provide determinism and to apply well-known schedulability analyses to a ROS 2 systems. A response time analysis is described under reservation-based scheduling.
6188

6289
## Rcl LET-Executor
6390
This section describes the rcl-LET-Executor. It is a first step towards deterministic execution by providing static order scheduling with a let semantics. The abbreviation let stands for Logical-Execution-Time (LET) and is a known concept in automotive domain to simplify synchronization in process scheduling. If refers to the concept to schedule multiple ready tasks in such a way, that first all input data is read for all tasks, and then all tasks are executed. This removes any inter-dependence of input data among these ready tasks and hence input data synchronization is improved [[BP2017](#BP2017)] [[EK2018](#EK2018)].
@@ -118,34 +145,6 @@ rcle_let_executor_spin_period(&exe, 20);
118145
### Download
119146
The LET-Executor can be downloaded from the micro-ROS GitHub [rcl_executor repository](https://github.com/micro-ROS/rcl_executor). The package [rcl_executor](https://github.com/micro-ROS/rcl_executor/tree/dashing/rcl_executor) provides the LET-Executor library with a step-by-step tutorial and the package [rcl_executor_examples](https://github.com/micro-ROS/rcl_executor/tree/dashing/rcl_executor_examples) provides an example, how to use the LET-Executor.
120147

121-
## Analysis of Default Rclcpp Executor
122-
123-
In section we provide a detailed analysis of the default executor of the rclcpp, including the software architecture, scheduling semantics and a potential refinement of the API.
124-
125-
### Architecture
126-
127-
The following diagram depicts the relevant classes of the standard ROS 2 Executor implementation:
128-
129-
![ROS 2 Executor class diagram](executor_class_diagram.png)
130-
131-
Note that an Executor instance maintains weak pointers to the NodeBaseInterfaces of the nodes only. Therefore, nodes can be destroyed safely, without notifying the Executor.
132-
133-
Also, the Executor does not maintain an explicit callback queue, but relies on the queue mechanism of the underlying DDS implementation as illustrated in the following sequence diagram:
134-
135-
![Call sequence from executor to DDS](executor_to_dds_sequence_diagram.png)
136-
137-
The Executor concept, however, does not provide means for prioritization or categorization of the incoming callback calls. Moreover, it does not leverage the real-time characteristics of the underlying operating-system scheduler to have finer control on the order of executions. The overall implication of this behavior is that time-critical callbacks could suffer possible deadline misses and a degraded performance since they are serviced later than non-critical callbacks. Additionally, due to the FIFO mechanism, it is difficult to determine usable bounds on the worst-case latency that each callback execution may incur.
138-
139-
### Scheduling Semantics
140-
141-
In a recent paper [CB2019](#CB2019), the rclcpp executor has been analyzed in detail and a response time analysis of cause-effect chains has been proposed under reservation-based scheduling. The executor distinguishes four categories of callbacks: _timers_, which are triggered by system-level timers, _subscribers_, which are triggered by new messages on a subscribed topic, _services_, which are triggered by service requests, and _clients_, which are triggered by responses to service requests. The executor is responsible for taking messages from the input queues of the DDS layer and executing the corresponding callback. Since it executes callbacks to completion, it is a non-preemptive scheduler, However it does not consider all ready tasks for execution, but only a snapshot, called readySet. This readySet is updated when the executor is idle and in this step it interacts with the DDS layer updating the set of ready tasks. Then for every type of task, there are dedicated queues (timers, subscriptions, services, clients) which are processed sequentially. The following key issues were pointed out:
142-
143-
* The executor processes _timers_ always first. This can lead to the intrinsic effect, that in overload situations messages from the DDS queue are not processed
144-
* Messages arriving during the processing of the readySet are not considered until the next update, which depends on the execution time of all remaining callbacks. This leads to priority inversion, as lower-priority callbacks may implicitly block higher-priority callbacks by prolonging the current processing of the readySet.
145-
* The readySet contains only one task instance, For example, even if multiple messages of the same topic are available, only one instance is processed until the executor is idle again and the readySet is updated from the DDS layer. This aggravates priority inversion, as a backlogged callback might have to wait for multiple processing of readySets until it is considered for scheduling. This means that non-timer callback instances might be blocked by multiple instances of the same lower-priority callback.
146-
147-
Due to these findings, an alternative approach is presented to provide determinism and to apply well-known schedulability analyses to a ROS 2 systems. A response time analysis is described under reservation-based scheduling.
148-
149148
## Callback-group-level Executor
150149

151150
The Callback-group-level Executor was an early prototype for a refined rclcpp executor API developed in micro-ROS. It has been derived from the default rclcpp Executor and addresses some of the aforementioned deficits. Most important, it was used to validate that the underlying layers (rcl, rmw, rmw_adapter, DDS) allow for multiple executor instances without any negative interferences.

0 commit comments

Comments
 (0)