-
Notifications
You must be signed in to change notification settings - Fork 313
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Upgrade Elasticsearch client to 8.0.0 #1350
Comments
I think it's fine to continue using |
The first pre-release of 8.0.0 is now available on PyPI: https://github.com/elastic/elasticsearch-py/releases/tag/v8.0.0a1 |
As discussed, I've closed the linked PR due to size and am instead going to break it into a sequence of smaller PRs, while trying to limit the potential for breakage as much as possible. This is a little tricky. Here's my thinking: First, we organize the client-related code to make it more readable and separate the different concerns at play. I've issued a PR that does this by creating a new Assuming that looks good and we merge it, we could do a few small PRs that add in utility code that we'll eventually need (e.g. the Now, we'd be at the point where things are more challenging, as we'd need to introduce a dependency on the new client version and cut over to using it. I see two ways that we could do this cutover gradually. Option 1: Install two client versions side-by-side temporarily We could phase most of the changes required for the upgrade by installing v This would allow us to merge the core additions we need for the upgrade (e.g. our own subclasses of
Note: We could go as far as to do this behind a feature flag. Could be overkill, but could also make testing easier. Option 2: Feature branch Another option could be to have a feature branch in Rally for the client upgrade and issue PRs against that branch gradually before doing one large merge to master once we're happy with the state of the feature branch.
In either case, we'd have more atomic changes that should be easier to review and present less risk overall. What do you think? |
Hmm, those options are interesting, thanks. My preference here is to merge code to master that gets used immediately, like the rally and rally-tracks PRs we merged yesterday, they were validated in our nightly benchmarks. And yes, this means at some point we'll have a bigger PR that makes the switch, but that's OK. (And I'm now more familiar with the problem so reviewing will be easier.) So basically I would suggest the following PRs:
|
By the way thinking about this more I don't think we should have the 7.17 step, let's not increase the scope too much. Regarding the version check, since we already disable it in the async client that is used for the actual benchmarking measures, I think we should just check once when Rally starts and then disable it everywhere. |
As mentioned above, since #1510 was quite big, we issued a few pull requests to reduce the scope:
That said, #1510 was itself quite reasonable, so I merged master into it after the above work, fixing many conflicts. The result is in https://github.com/elastic/rally/tree/es-client-upgrade, see https://github.com/elastic/rally/compare/es-client-upgrade?expand=1 for the diff. If we want to upgrade to 7.17 first (which is the officially recommended path), I backported a fix in the client (elastic/elasticsearch-py#2102) that would allow us to revert #1580, further simplifying the actual upgrade to 8.x. |
I've picked this up. For now, my high level plan is to continue on working on the
Once I'm comfortable there's no obvious performance regressions, I'll open a PR and we can work through the feedback then. |
Note: This is not actionable at the moment but acts as a reference of things to consider when upgrading the Elasticsearch client to 8.0.0.
Looking at the Elasticsearch client roadmap a few items stand out but specifically elastic/elasticsearch-py#1680 will be tricky: The client will not allow to pass a
body
but rather requires to specify the body in a more structured manner. Initially usingbody
will only be deprecated but we need to think about alternatives.The text was updated successfully, but these errors were encountered: