Skip to content

Commit

Permalink
Added user documentation for scenario topic subscription-publishing
Browse files Browse the repository at this point in the history
  • Loading branch information
CihatAltiparmak committed Aug 18, 2024
1 parent 8a88cbf commit 2594789
Show file tree
Hide file tree
Showing 3 changed files with 52 additions and 0 deletions.
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ This middleware benchmark tool aims to measure middleware effects on various sce

* [Perception Pipeline](./docs/scenarios/perception_pipeline_benchmark.md)
* [Basic Service Client Works](./docs/scenarios/basic_service_client_benchmark.md)
* [Basic Topic Subscription-Publishing](./docs/scenarios/basic_topic_sub_pub.md)

## Getting Started

Expand Down
4 changes: 4 additions & 0 deletions docs/how_to_run.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,3 +94,7 @@ For instance, the selected test_case includes 20 goal poses. These 20 goals is s
This benchmark measures the total elapsed time based on the time interval between sending the request by the client to the server and getting the response of server. This benchmark utilizes the [ros2/demos](https://github.com/ros2/demos) packages' [example server](https://github.com/ros2/demos/blob/rolling/demo_nodes_cpp/src/services/add_two_ints_server.cpp).

In this benchmark scenario, the benchmarker node only has client interface. The necessary server for this client is run in [the launch file of this benchmark scenario](../launch/scenario_basic_service_client_benchmark.launch.py). Client sends a request to server and waits for the response from server. Client sends second request to server once the client receives response of first request from client. This actions are repeated `sending_request_number` times. You can configure this `sending_request_number` parameter in [this scenario's launch file]((../launch/scenario_basic_service_client_benchmark.launch.py)).

### [Basic Topic Subscription Publishing Benchmark](scenarios/basic_topic_sub_pub.md)

This benchmark aims to measure the ROS message latency by sending some ROS message which has array section and timestamp section and receiving this message in topic listener side. Firstly, in the topic publisher side, message is created using given `pose_array_size`, `bwnchmarked_topic_name` and `benchmarked_topic_hz`. The topic listener subscribes the topic named `benchmarked_topic_name` to listen this ROS message is published. When message is received from topic publisher side, It's found the message latency by subtracting message timestamp (indicates the time message is published) inside the message callback from current time (indicates the time message is received). Finally, this message latencies are added to total elapsed time. When scenario is finished, this total elapsed time is used to compare middleware effects in scenario of topic subscription-publishing.
47 changes: 47 additions & 0 deletions docs/scenarios/basic_topic_sub_pub.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
## How To Run Basic Topic Subscription Publishing Benchmark

Firstly, source your ros version. It's suggested to test with rolling version of ROS 2.

For instance, to test with rmw_zenoh, start to zenoh router using following command in the terminal.
```sh
# go to your workspace
cd ws
# Be sure that ros2 daemon is killed.
pkill -9 -f ros && ros2 daemon stop
# Then start zenoh router
source /opt/ros/rolling/setup.bash
source install/setup.bash
export RMW_IMPLEMENTATION=rmw_zenoh_cpp
ros2 run rmw_zenoh_cpp rmw_zenohd
```

Select your rmw_implementation as `rmw_zenoh_cpp` and run the perception benchmark launch file in the another terminal.
```sh
# go to your workspace
cd ws
source /opt/ros/rolling/setup.bash
source install/setup.bash
export RMW_IMPLEMENTATION=rmw_zenoh_cpp # select your rmw_implementation to benchmark
ros2 launch moveit_middleware_benchmark scenario_basic_subscription_benchmark.launch.py
```

It will be defaultly benchmarked with 6 repetitions. It will be created the json file named `middleware_benchmark_results.json` for benchmarking results after finishing benchmark code execution. You can see the benchmark results in more detail inside this json file.

If you want to customize your benchmark arguments or select different test case, you can use below command.

```shell
ros2 launch moveit_middleware_benchmark scenario_basic_subscription_benchmark.launch.py benchmark_command_args:="--benchmark_out=middleware_benchmark_results.json --benchmark_out_format=json --benchmark_repetitions=1" pose_array_size:=1000 benchmarked_topic_name:="/dummy_benchmark_topic_name" benchmarked_topic_hz:=10 max_received_topic_number:=100
```

Let's explain some parameters used in benchmark.

| args | explanation |
| ---- | ----------- |
| `benchmark_command_args` | you can utilize this parameter to use custom benchmark arguments in your benchmark |
| `pose_array_size` | In this benchmark, [geometry_msgs/msg/PoseArray](https://docs.ros.org/en/rolling/p/geometry_msgs/interfaces/msg/PoseArray.html) is used. This parameter presents how many poses will be published from [basic_topic_publisher](../../src/scenarios/basic_topic_sub_pub/basic_topic_publisher.cpp). You can set message size sent by [basic_topic_publisher](../../src/scenarios/basic_topic_sub_pub/basic_topic_publisher.cpp) using this parameter. |
| `benchmarked_topic_hz` | This parameter presents in how many rate the messsage will be published from [basic_topic_publisher](../../src/scenarios/basic_topic_sub_pub/basic_topic_publisher.cpp). |
| `max_received_topic_number` | This parameter sets how many message to handled by [scenario_basic_subscription benchmarking module](../../src/scenarios/basic_topic_sub_pub/scenario_basic_subscription.cpp). The more message number is handled, the more reliable bechmark results is achieved in `scenario_basic_subscription` scenario. |

## How to benchmark the basic topic subscription publishing works

The main idea here is to firstly save the message timestamp when published from [basic_topic_publisher](../../src/scenarios/basic_topic_sub_pub/basic_topic_publisher.cpp) and then handle this message in [scenario_basic_subscription benchmarking module](../../src/scenarios/basic_topic_sub_pub/scenario_basic_subscription.cpp) by which this message is received. This benchmark module uses [geometry_msgs/msg/PoseArray](https://docs.ros.org/en/rolling/p/geometry_msgs/interfaces/msg/PoseArray.html) which has arrangable size and timestamp sections in message body. When message is received by [scenario_basic_subscription benchmarking module](../../src/scenarios/basic_topic_sub_pub/scenario_basic_subscription.cpp), the difference between the time message is received and the time message is sent is added to total elapsed time. In the end, this total elapsed time is saved to benchmark results.

0 comments on commit 2594789

Please sign in to comment.