Allow users to specify percentiles for compare subcommand (#503)

[IanHoang]

Description

Currently, users running with compare subcommand are only seeing P100 latency and service time in the results. This PR adds to previous changes made by Peter Alfonsi and allows users to specify which percentiles they want displayed. If no percentiles are passed through, the default percentiles in the GlobalStatsCalculator is used, similar to how it is used in --latency-percentiles and --throughput-percentiles params in execute-test subcommand.

Issues Resolved

#503

Testing

• New functionality includes testing

When enough data is present, it will show all percentiles specified.

hoangia@3c22fbd0d988 opensearch-benchmark % opensearch-benchmark compare --baseline ffe95f09-e5f0-4acf-965c-ab7c88cff083 --contender ffe95f09-e5f0-4acf-965c-ab7c88cff083

   ____                  _____                      __       ____                  __                         __
  / __ \____  ___  ____ / ___/___  ____ ___________/ /_     / __ )___  ____  _____/ /_  ____ ___  ____ ______/ /__
 / / / / __ \/ _ \/ __ \\__ \/ _ \/ __ `/ ___/ ___/ __ \   / __  / _ \/ __ \/ ___/ __ \/ __ `__ \/ __ `/ ___/ //_/
/ /_/ / /_/ /  __/ / / /__/ /  __/ /_/ / /  / /__/ / / /  / /_/ /  __/ / / / /__/ / / / / / / / / /_/ / /  / ,<
\____/ .___/\___/_/ /_/____/\___/\__,_/_/   \___/_/ /_/  /_____/\___/_/ /_/\___/_/ /_/_/ /_/ /_/\__,_/_/  /_/|_|
    /_/


Comparing baseline
  TestExecution ID: ffe95f09-e5f0-4acf-965c-ab7c88cff083
  TestExecution timestamp: 2024-01-25 17:46:22
  TestProcedure: default-test-procedure
  ProvisionConfigInstance: external

with contender
  TestExecution ID: ffe95f09-e5f0-4acf-965c-ab7c88cff083
  TestExecution timestamp: 2024-01-25 17:46:22
  TestProcedure: default-test-procedure
  ProvisionConfigInstance: external

------------------------------------------------------
    _______             __   _____
   / ____(_)___  ____ _/ /  / ___/_________  ________
  / /_  / / __ \/ __ `/ /   \__ \/ ___/ __ \/ ___/ _ \
 / __/ / / / / / /_/ / /   ___/ / /__/ /_/ / /  /  __/
/_/   /_/_/ /_/\__,_/_/   /____/\___/\____/_/   \___/
------------------------------------------------------

|                                                        Metric |                     Task |    Baseline |   Contender |   Diff |   Unit |
|--------------------------------------------------------------:|-------------------------:|------------:|------------:|-------:|-------:|
|                    Cumulative indexing time of primary shards |                          |     2.63007 |     2.63007 |      0 |    min |
|             Min cumulative indexing time across primary shard |                          |           0 |           0 |      0 |    min |
|          Median cumulative indexing time across primary shard |                          | 0.000891667 | 0.000891667 |      0 |    min |
|             Max cumulative indexing time across primary shard |                          |      0.5247 |      0.5247 |      0 |    min |
|           Cumulative indexing throttle time of primary shards |                          |           0 |           0 |      0 |    min |
|    Min cumulative indexing throttle time across primary shard |                          |           0 |           0 |      0 |    min |
| Median cumulative indexing throttle time across primary shard |                          |           0 |           0 |      0 |    min |
|    Max cumulative indexing throttle time across primary shard |                          |           0 |           0 |      0 |    min |
|                       Cumulative merge time of primary shards |                          |   0.0424667 |   0.0424667 |      0 |    min |
|                      Cumulative merge count of primary shards |                          |          12 |          12 |      0 |        |
|                Min cumulative merge time across primary shard |                          |           0 |           0 |      0 |    min |
|             Median cumulative merge time across primary shard |                          |           0 |           0 |      0 |    min |
|                Max cumulative merge time across primary shard |                          |   0.0108167 |   0.0108167 |      0 |    min |
|              Cumulative merge throttle time of primary shards |                          |           0 |           0 |      0 |    min |
|       Min cumulative merge throttle time across primary shard |                          |           0 |           0 |      0 |    min |
|    Median cumulative merge throttle time across primary shard |                          |           0 |           0 |      0 |    min |
|       Max cumulative merge throttle time across primary shard |                          |           0 |           0 |      0 |    min |
|                     Cumulative refresh time of primary shards |                          |     0.25905 |     0.25905 |      0 |    min |
|                    Cumulative refresh count of primary shards |                          |        1362 |        1362 |      0 |        |
|              Min cumulative refresh time across primary shard |                          |           0 |           0 |      0 |    min |
|           Median cumulative refresh time across primary shard |                          | 0.000183333 | 0.000183333 |      0 |    min |
|              Max cumulative refresh time across primary shard |                          |      0.0419 |      0.0419 |      0 |    min |
|                       Cumulative flush time of primary shards |                          |     0.00195 |     0.00195 |      0 |    min |
|                      Cumulative flush count of primary shards |                          |          28 |          28 |      0 |        |
|                Min cumulative flush time across primary shard |                          |           0 |           0 |      0 |    min |
|             Median cumulative flush time across primary shard |                          |           0 |           0 |      0 |    min |
|                Max cumulative flush time across primary shard |                          |     0.00195 |     0.00195 |      0 |    min |
|                                       Total Young Gen GC time |                          |       0.441 |       0.441 |      0 |      s |
|                                      Total Young Gen GC count |                          |           7 |           7 |      0 |        |
|                                         Total Old Gen GC time |                          |           0 |           0 |      0 |      s |
|                                        Total Old Gen GC count |                          |           0 |           0 |      0 |        |
|                                                    Store size |                          |    0.150055 |    0.150055 |      0 |     GB |
|                                                 Translog size |                          | 1.99769e-06 | 1.99769e-06 |      0 |     GB |
|                                        Heap used for segments |                          |           0 |           0 |      0 |     MB |
|                                      Heap used for doc values |                          |           0 |           0 |      0 |     MB |
|                                           Heap used for terms |                          |           0 |           0 |      0 |     MB |
|                                           Heap used for norms |                          |           0 |           0 |      0 |     MB |
|                                          Heap used for points |                          |           0 |           0 |      0 |     MB |
|                                   Heap used for stored fields |                          |           0 |           0 |      0 |     MB |
|                                                 Segment count |                          |          77 |          77 |      0 |        |
|                                                Min Throughput |             index-append |      4439.5 |      4439.5 |      0 | docs/s |
|                                               Mean Throughput |             index-append |      4439.5 |      4439.5 |      0 | docs/s |
|                                             Median Throughput |             index-append |      4439.5 |      4439.5 |      0 | docs/s |
|                                                Max Throughput |             index-append |      4439.5 |      4439.5 |      0 | docs/s |
|                                       50th percentile latency |             index-append |      391.04 |      391.04 |      0 |     ms |
|                                      100th percentile latency |             index-append |     409.392 |     409.392 |      0 |     ms |
|                                  50th percentile service time |             index-append |      391.04 |      391.04 |      0 |     ms |
|                                 100th percentile service time |             index-append |     409.392 |     409.392 |      0 |     ms |
|                                                    error rate |             index-append |           0 |           0 |      0 |      % |
|                                                Min Throughput | wait-until-merges-finish |     4.43566 |     4.43566 |      0 |  ops/s |
|                                               Mean Throughput | wait-until-merges-finish |     4.43566 |     4.43566 |      0 |  ops/s |
|                                             Median Throughput | wait-until-merges-finish |     4.43566 |     4.43566 |      0 |  ops/s |
|                                                Max Throughput | wait-until-merges-finish |     4.43566 |     4.43566 |      0 |  ops/s |
|                                      100th percentile latency | wait-until-merges-finish |     193.708 |     193.708 |      0 |     ms |
|                                 100th percentile service time | wait-until-merges-finish |     193.708 |     193.708 |      0 |     ms |
|                                                    error rate | wait-until-merges-finish |           0 |           0 |      0 |      % |
|                                                Min Throughput |                match-all |     9.95597 |     9.95597 |      0 |  ops/s |
|                                               Mean Throughput |                match-all |     9.97532 |     9.97532 |      0 |  ops/s |
|                                             Median Throughput |                match-all |     9.97773 |     9.97773 |      0 |  ops/s |
|                                                Max Throughput |                match-all |     9.98505 |     9.98505 |      0 |  ops/s |
|                                       50th percentile latency |                match-all |      226.56 |      226.56 |      0 |     ms |
|                                       90th percentile latency |                match-all |     256.942 |     256.942 |      0 |     ms |
|                                       99th percentile latency |                match-all |     363.537 |     363.537 |      0 |     ms |
|                                      100th percentile latency |                match-all |     1845.22 |     1845.22 |      0 |     ms |
|                                  50th percentile service time |                match-all |     225.745 |     225.745 |      0 |     ms |
|                                  90th percentile service time |                match-all |     255.362 |     255.362 |      0 |     ms |
|                                  99th percentile service time |                match-all |     360.962 |     360.962 |      0 |     ms |
|                                 100th percentile service time |                match-all |     1844.55 |     1844.55 |      0 |     ms |
|                                                    error rate |                match-all |           0 |           0 |      0 |      % |


-------------------------------
[INFO] SUCCESS (took 0 seconds)
-------------------------------

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[gkamat]

Thanks for fixing this. Information on this should be added to the documentation, including the default percentiles, and the fact that the output will present numbers only if they were captured earlier.