Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade ES Client to 8.x #1669

Merged
merged 73 commits into from
Mar 4, 2023
Merged
Show file tree
Hide file tree
Changes from 59 commits
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
82cd43f
Update ES Python client dependency to 8.2.0.
Jun 3, 2022
e7d83cc
Update customizations of the ES client and ensure backwards compatibi…
Jun 3, 2022
5eaf566
Update client.wait_for_rest_layer().
Jun 3, 2022
3ed00b8
The `_normalize_hosts` function has been removed from the client.
Jun 3, 2022
c7d5858
Update client unit tests.
Jun 3, 2022
99f8d2b
Update metrics code.
Jun 3, 2022
b4a63fd
Update metrics unit tests.
Jun 3, 2022
4846223
Ensure that ES client method calls in telemetry use kwargs only.
Jun 3, 2022
0228326
Update telemetry unit tests.
Jun 3, 2022
6f43b03
Update tracker: use kwargs, move away from `ElasticsearchException`.
Jun 3, 2022
1f566fb
Update tracker/corpus unit tests.
Jun 3, 2022
9bdb361
Ensure that the load driver creates ES client objects correctly.
Jun 3, 2022
c73e58c
Handle exceptions from the ES client in the load driver.
Jun 3, 2022
e4a547e
Set `request_timeout` via `es.options()` instead of API method kwarg.
Jun 3, 2022
d3c8c1b
Ensure runners use kwargs when calling ES client methods.
Jun 3, 2022
2df742c
Update driver unit tests.
Jun 3, 2022
17d03e7
Update runner unit tests.
Jun 3, 2022
ef921df
Fix linting error.
Jun 3, 2022
9627d29
Begin DRYing up EsClientFactory.
Jun 3, 2022
70393ab
Use newer ssl parameter in aiohttp client API
pquentin Nov 4, 2022
f2e7688
Merge branch 'aiohttp-ssl' into es-client-upgrade
pquentin Nov 4, 2022
4a648da
Merge remote-tracking branch 'origin/master' into es-client-upgrade
pquentin Jan 27, 2023
a4c4410
Set connection attributes after client instantiation
b-deam Feb 1, 2023
a233c01
Refactor API keys creation
b-deam Feb 2, 2023
d85604c
Get client version string for compatability headers
b-deam Feb 2, 2023
70c2e6a
Fix static_response connector
b-deam Feb 2, 2023
00ea4c3
Merge remote-tracking branch 'origin/master' into es-client-upgrade
b-deam Feb 9, 2023
93c338e
Refactor exception handling
b-deam Feb 10, 2023
808e977
Refactor client options and target hosts
b-deam Feb 10, 2023
7e117b9
Refactor client creation to pop 'distro' arg before calling parent cl…
b-deam Feb 10, 2023
86aed78
Disable pylint on client version test
b-deam Feb 10, 2023
4ba9650
Convert create snapshot repo runner to kwargs
b-deam Feb 13, 2023
568f481
Remove 'ClientResponse' class override because 'RallyAiohttpHttpNode'…
b-deam Feb 13, 2023
71d536b
Refactor 'static_responses' to always return encoded responses
b-deam Feb 13, 2023
8a6aab6
Run isort on common.py
b-deam Feb 13, 2023
b62d793
Make ILM client BWC
b-deam Feb 14, 2023
54fe814
Add new 'elastic_transport' logger to configuration file
b-deam Feb 14, 2023
c85a5ac
Remove 'trace_configs' getters and setters
b-deam Feb 14, 2023
b99b773
Remove 'enable_cleanup_closed' getters and setters
b-deam Feb 14, 2023
d1a911f
Simplify '_client_major_version_to_str'
b-deam Feb 14, 2023
2d3270c
Remove 'host_string' 'elasticsearch-py' comment
b-deam Feb 14, 2023
32b5f9a
Make EQL custom runner BWC
b-deam Feb 15, 2023
e72a13b
Linting and spelling
b-deam Feb 15, 2023
f50ae06
Merge remote-tracking branch 'origin/master' into es-client-upgrade
b-deam Feb 15, 2023
071d886
Bump 'elasticsearch[async]' to 8.6.1 and 'elastic-transport' to 8.4.0
b-deam Feb 16, 2023
3c665a9
Make searchable snapshots client backwards compatible
b-deam Feb 16, 2023
5057f00
Refactor 'DiskUsageStats' to use 'disk_usage' client method
b-deam Feb 16, 2023
92ce7ba
Revert "Make searchable snapshots client backwards compatible"
b-deam Feb 16, 2023
99f7731
Revert "Make EQL custom runner BWC"
b-deam Feb 16, 2023
c14b0e9
Merge remote-tracking branch 'origin/master' into es-client-upgrade
pquentin Feb 16, 2023
34936a5
Fix 'migrate.rst' wording on logger change
b-deam Feb 21, 2023
80623af
Add trailing comma to 'test_create_async_client_with_api_key_auth_ove…
b-deam Feb 21, 2023
f204084
Linting
b-deam Feb 22, 2023
98ac188
Silence aiohttp SSL cert verify warnings via es client constructor
b-deam Feb 23, 2023
199e557
Remove 'urllib3.disable_warnings()' from client factory as it's now s…
b-deam Feb 24, 2023
5650661
Tighten up import of 'elastic_transport' to avoid Thespian's double f…
b-deam Feb 24, 2023
d7e4c61
Further tightening of 'elasticsearch' and 'elastic_transport' imports
b-deam Feb 27, 2023
2a15ab5
Add 'elastic_transport' logger IT test
b-deam Feb 27, 2023
09206b4
New test linting
b-deam Feb 27, 2023
e4144bf
Update docs/migrate.rst
b-deam Feb 28, 2023
14d209f
Make '_normalize_hosts' class method
b-deam Feb 28, 2023
55a2e1f
Rename 'distro' to 'distribution_version'
b-deam Feb 28, 2023
f0a5693
Move product verification prior to first request
b-deam Feb 28, 2023
f594218
Comment 'RallyIlmClient' method signature changes
b-deam Feb 28, 2023
a16450d
Update esrally/log.py
b-deam Feb 28, 2023
c3936a1
Update esrally/log.py
b-deam Feb 28, 2023
5da718a
Revert "Update esrally/log.py"
b-deam Feb 28, 2023
527b44e
Rename 'missing_keys' to '_missing_loggers'
b-deam Feb 28, 2023
cb52f45
Re-open 'target' in write mode to avoid tricky code
b-deam Feb 28, 2023
57c8bfb
Add '6.8' TODOs for all runner fallbacks
b-deam Feb 28, 2023
758c95c
Refactor exception handling in client factory's 'wait_for_rest_layer'…
b-deam Mar 1, 2023
b9808fc
Use 'es'snapshot.get' for 'WaitForCurrentSnapshotsCreate' runner
b-deam Mar 2, 2023
407b5f5
Link to 'elastic_transport' repo in docs
b-deam Mar 2, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 12 additions & 17 deletions docs/command_line_reference.rst
Original file line number Diff line number Diff line change
Expand Up @@ -765,34 +765,31 @@ Define a JSON file containing a list of objects with the following properties:

* ``path``: A path or path pattern that should be matched. Only leading and trailing wildcards (``*``) are supported. A path containing only a wildcard acts matches any path.
* ``body``: The respective response body.
* ``body-encoding``: Either ``raw`` or ``json``. Use ``json`` by default and ``raw`` for the operation-type ``bulk`` and ``search``.

Here we define the necessary responses for a track that bulk-indexes data::

[
{
"path": "/_cluster/settings",
"body": {
"transient": {
"action.destructive_requires_name": "true"
}
}
},
{
"path": "*/_bulk",
"body": {
"errors": false,
"took": 1
},
"body-encoding": "raw"
}
},
{
"path": "/_cluster/health*",
"body": {
"status": "green",
"relocating_shards": 0
},
"body-encoding": "json"
},
{
"path": "/_cluster/settings",
"body": {
"persistent": {},
"transient": {}
},
"body-encoding": "json"
}
},
{
"path": "/_all/_stats/_all",
Expand All @@ -804,13 +801,11 @@ Here we define the necessary responses for a track that bulk-indexes data::
}
}
}
},
"body-encoding": "json"
}
},
{
"path": "*",
"body": {},
"body-encoding": "json"
"body": {}
}
]

Expand Down
4 changes: 4 additions & 0 deletions docs/migrate.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ Migration Guide
Migrating to Rally 2.7.1
------------------------

Elasticsearch client logs are now captured by the ``elastic_transport`` logger
b-deam marked this conversation as resolved.
Show resolved Hide resolved
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The 8.x version of the ``elasticsearch-py`` client Rally uses has moved the underlying network requests into a new module (and subsequently a new logger) named ``elastic_transport``. Both new and existing installations of Rally will automatically be configured to use this logger.
b-deam marked this conversation as resolved.
Show resolved Hide resolved

Snapshot repository plugins are no longer built from source
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Expand Down
Loading