Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix serverless public use in trivial tracks #1785

Merged
merged 1 commit into from
Sep 26, 2023

Conversation

gbanasiak
Copy link
Contributor

@gbanasiak gbanasiak commented Sep 26, 2023

Eliminates the following blocking points for Rally public use against serverless clusters:

  • The driver.Driver.prepare_benchmark() method calls client.wait_for_rest_layer() which includes _cluster/health ES API call. This is not available in serverless, but at this point we have already confirmed REST API availability in BenchmarkCoordinator.setup() through client.factory.cluster_distribution_version().
  • The on_benchmark_stop() in telemetry is incorrectly trying to stop all telemetry devices even if they were determined to be unsuitable for serverless, see on_benchmark_start().
  • The DeleteIndex runner includes a call to set_destructive_requires_name() method which relies on _cluster/settings API which is not available in serverless.

With the fixes in it's now possible to run trivial benchmarks with delete-index, create-index and bulk operations.

% esrally race --track-path=~/tracks/test_track --target-hosts=${ES_HOST}:443 --pipeline=benchmark-only --client-options="use_ssl:true,api_key:${ES_API_KEY}" --on-error=abort

    ____        ____
   / __ \____ _/ / /_  __
  / /_/ / __ `/ / / / / /
 / _, _/ /_/ / / / /_/ /
/_/ |_|\__,_/_/_/\__, /
                /____/

[INFO] Race id is [bc7d2dcc-b176-4a79-a293-8f53b97e9fd2]
[INFO] Detected Elasticsearch Serverless mode with operator=[False].
[INFO] Excluding [cluster-health] as challenge [default] is run on serverless.
[INFO] Racing on track [test_track] and car ['external'] with version [oss].

Running delete-index                                                           [100% done]
Running create-index                                                           [100% done]
Running bulk                                                                   [100% done]

------------------------------------------------------
    _______             __   _____
   / ____(_)___  ____ _/ /  / ___/_________  ________
  / /_  / / __ \/ __ `/ /   \__ \/ ___/ __ \/ ___/ _ \
 / __/ / / / / / /_/ / /   ___/ / /__/ /_/ / /  /  __/
/_/   /_/_/ /_/\__,_/_/   /____/\___/\____/_/   \___/
------------------------------------------------------
            
|                        Metric |   Task |   Value |   Unit |
|------------------------------:|-------:|--------:|-------:|
|                Min Throughput |   bulk | 1345.31 | docs/s |
|               Mean Throughput |   bulk | 1345.31 | docs/s |
|             Median Throughput |   bulk | 1345.31 | docs/s |
|                Max Throughput |   bulk | 1345.31 | docs/s |
|       50th percentile latency |   bulk | 1929.95 |     ms |
|      100th percentile latency |   bulk | 3366.24 |     ms |
|  50th percentile service time |   bulk | 1929.95 |     ms |
| 100th percentile service time |   bulk | 3366.24 |     ms |
|                    error rate |   bulk |    0    |      % |


--------------------------------
[INFO] SUCCESS (took 44 seconds)
--------------------------------

There are many index settings that will still prevent completion of benchmarks from https://github.com/elastic/rally-tracks but this will be addressed separately.

Copy link
Member

@pquentin pquentin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! LGTM.

@pquentin
Copy link
Member

I'm not sure why this is a draft, this is a good step forward, as shown by the serverless IT tests that report a different error now.

@gbanasiak gbanasiak marked this pull request as ready for review September 26, 2023 14:51
@gbanasiak
Copy link
Contributor Author

@pquentin Many thanks for a quick review. I've left it in draft mode while waiting for the tests to complete, but I wasn't planning to add anything.

@gbanasiak gbanasiak merged commit ba701e5 into elastic:master Sep 26, 2023
11 checks passed
@gbanasiak gbanasiak deleted the public-user-fixes branch September 26, 2023 15:00
@gbanasiak gbanasiak added this to the 2.10.0 milestone Sep 26, 2023
@gbanasiak gbanasiak added the enhancement Improves the status quo label Nov 6, 2023
@gbanasiak gbanasiak changed the title Fix serverless public user in trivial tracks Fix serverless public use in trivial tracks Nov 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Improves the status quo
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants