diff --git a/README.md b/README.md index 69c3eed..a030c26 100644 --- a/README.md +++ b/README.md @@ -10,6 +10,7 @@ This middleware benchmark tool aims to measure middleware effects on various sce * [Perception Pipeline](./docs/scenarios/perception_pipeline_benchmark.md) * [Basic Service Client Works](./docs/scenarios/basic_service_client_benchmark.md) +* [Basic Topic Subscription-Publishing](./docs/scenarios/basic_topic_sub_pub.md) ## Getting Started diff --git a/docs/how_to_run.md b/docs/how_to_run.md index 3767bf8..9cc3d87 100644 --- a/docs/how_to_run.md +++ b/docs/how_to_run.md @@ -94,3 +94,7 @@ For instance, the selected test_case includes 20 goal poses. These 20 goals is s This benchmark measures the total elapsed time based on the time interval between sending the request by the client to the server and getting the response of server. This benchmark utilizes the [ros2/demos](https://github.com/ros2/demos) packages' [example server](https://github.com/ros2/demos/blob/rolling/demo_nodes_cpp/src/services/add_two_ints_server.cpp). In this benchmark scenario, the benchmarker node only has client interface. The necessary server for this client is run in [the launch file of this benchmark scenario](../launch/scenario_basic_service_client_benchmark.launch.py). Client sends a request to server and waits for the response from server. Client sends second request to server once the client receives response of first request from client. This actions are repeated `sending_request_number` times. You can configure this `sending_request_number` parameter in [this scenario's launch file]((../launch/scenario_basic_service_client_benchmark.launch.py)). + +### [Basic Topic Subscription Publishing Benchmark](scenarios/basic_topic_sub_pub.md) + +This benchmark aims to measure the ROS message latency by sending some ROS message which has array section and timestamp section and receiving this message in topic listener side. Firstly, in the topic publisher side, message is created using given `pose_array_size`, `bwnchmarked_topic_name` and `benchmarked_topic_hz`. The topic listener subscribes the topic named `benchmarked_topic_name` to listen this ROS message is published. When message is received from topic publisher side, It's found the message latency by subtracting message timestamp (indicates the time message is published) inside the message callback from current time (indicates the time message is received). Finally, this message latencies are added to total elapsed time. When scenario is finished, this total elapsed time is used to compare middleware effects in scenario of topic subscription-publishing. diff --git a/docs/scenarios/basic_topic_sub_pub.md b/docs/scenarios/basic_topic_sub_pub.md new file mode 100644 index 0000000..38c9a7a --- /dev/null +++ b/docs/scenarios/basic_topic_sub_pub.md @@ -0,0 +1,47 @@ +## How To Run Basic Topic Subscription Publishing Benchmark + +Firstly, source your ros version. It's suggested to test with rolling version of ROS 2. + +For instance, to test with rmw_zenoh, start to zenoh router using following command in the terminal. +```sh +# go to your workspace +cd ws +# Be sure that ros2 daemon is killed. +pkill -9 -f ros && ros2 daemon stop +# Then start zenoh router +source /opt/ros/rolling/setup.bash +source install/setup.bash +export RMW_IMPLEMENTATION=rmw_zenoh_cpp +ros2 run rmw_zenoh_cpp rmw_zenohd +``` + +Select your rmw_implementation as `rmw_zenoh_cpp` and run the perception benchmark launch file in the another terminal. +```sh +# go to your workspace +cd ws +source /opt/ros/rolling/setup.bash +source install/setup.bash +export RMW_IMPLEMENTATION=rmw_zenoh_cpp # select your rmw_implementation to benchmark +ros2 launch moveit_middleware_benchmark scenario_basic_subscription_benchmark.launch.py +``` + +It will be defaultly benchmarked with 6 repetitions. It will be created the json file named `middleware_benchmark_results.json` for benchmarking results after finishing benchmark code execution. You can see the benchmark results in more detail inside this json file. + +If you want to customize your benchmark arguments or select different test case, you can use below command. + +```shell +ros2 launch moveit_middleware_benchmark scenario_basic_subscription_benchmark.launch.py benchmark_command_args:="--benchmark_out=middleware_benchmark_results.json --benchmark_out_format=json --benchmark_repetitions=1" pose_array_size:=1000 benchmarked_topic_name:="/dummy_benchmark_topic_name" benchmarked_topic_hz:=10 max_received_topic_number:=100 +``` + +Let's explain some parameters used in benchmark. + +| args | explanation | +| ---- | ----------- | +| `benchmark_command_args` | you can utilize this parameter to use custom benchmark arguments in your benchmark | +| `pose_array_size` | In this benchmark, [geometry_msgs/msg/PoseArray](https://docs.ros.org/en/rolling/p/geometry_msgs/interfaces/msg/PoseArray.html) is used. This parameter presents how many poses will be published from [basic_topic_publisher](../../src/scenarios/basic_topic_sub_pub/basic_topic_publisher.cpp). You can set message size sent by [basic_topic_publisher](../../src/scenarios/basic_topic_sub_pub/basic_topic_publisher.cpp) using this parameter. | +| `benchmarked_topic_hz` | This parameter presents in how many rate the messsage will be published from [basic_topic_publisher](../../src/scenarios/basic_topic_sub_pub/basic_topic_publisher.cpp). | +| `max_received_topic_number` | This parameter sets how many message to handled by [scenario_basic_subscription benchmarking module](../../src/scenarios/basic_topic_sub_pub/scenario_basic_subscription.cpp). The more message number is handled, the more reliable bechmark results is achieved in `scenario_basic_subscription` scenario. | + +## How to benchmark the basic topic subscription publishing works + +The main idea here is to firstly save the message timestamp when published from [basic_topic_publisher](../../src/scenarios/basic_topic_sub_pub/basic_topic_publisher.cpp) and then handle this message in [scenario_basic_subscription benchmarking module](../../src/scenarios/basic_topic_sub_pub/scenario_basic_subscription.cpp) by which this message is received. This benchmark module uses [geometry_msgs/msg/PoseArray](https://docs.ros.org/en/rolling/p/geometry_msgs/interfaces/msg/PoseArray.html) which has arrangable size and timestamp sections in message body. When message is received by [scenario_basic_subscription benchmarking module](../../src/scenarios/basic_topic_sub_pub/scenario_basic_subscription.cpp), the difference between the time message is received and the time message is sent is added to total elapsed time. In the end, this total elapsed time is saved to benchmark results.