Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[2.x] Upgrade AD's bwc baseline version to 1.3.2 to resolve cluster join issue #1029

Merged
merged 1 commit into from
Sep 8, 2023

Conversation

kaituo
Copy link
Collaborator

@kaituo kaituo commented Sep 8, 2023

Description

Issue:

  • In versions between 1.1 and 2.10, the task :adBwcCluster#twoThirdsUpgradedClusterTask fails.
  • Symptoms mirror those in OpenSearch Issue #5076.
  • Logs show nodes from old and new versions failing to join as a cluster, resulting in a "master not found" exception.
  • Similar things happened between 1.1 and 2.9/2.8 when task :adBwcCluster#mixedClusterTask executes.

Resolution:

This PR also backports retry logic in #1017 to 2.x.

Testing:

  • Executed ./gradlew bwcTestSuite -Dtests.security.manager=false and all tests passed.

Issues Resolved

#964

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Issue:
- In versions between 1.1 and 2.10, the task :adBwcCluster#twoThirdsUpgradedClusterTask fails.
- Symptoms mirror those in [OpenSearch Issue #5076](opensearch-project/OpenSearch#5076).
- Logs show nodes from old and new versions failing to join as a cluster, resulting in a "master not found" exception.

Resolution:
- Upgraded the bwc version to 1.3.2, aligning with other plugins which use 1.3+ as their baseline.
  - [Cross-Cluster Replication PR opensearch-project#469](opensearch-project/cross-cluster-replication#469)
  - [Security PR #2253](opensearch-project/security#2253)
  - [ML Commons PR opensearch-project#681](opensearch-project/ml-commons#681)
- Post-upgrade, the twoThirdsUpgradedClusterTask runs successfully, suggesting potential incompatibility between versions 1.1 and 2.10.

Testing:
- Executed `./gradlew bwcTestSuite -Dtests.security.manager=false` and all tests passed.

Signed-off-by: Kaituo Li <[email protected]>
@codecov
Copy link

codecov bot commented Sep 8, 2023

Codecov Report

Merging #1029 (bea81c8) into 2.x (f05924b) will decrease coverage by 0.05%.
The diff coverage is n/a.

❗ Current head bea81c8 differs from pull request most recent head b30fed2. Consider uploading reports for the commit b30fed2 to get more accurate results

Impacted file tree graph

@@             Coverage Diff              @@
##                2.x    #1029      +/-   ##
============================================
- Coverage     79.39%   79.34%   -0.05%     
+ Complexity     4318     4315       -3     
============================================
  Files           307      307              
  Lines         18159    18157       -2     
  Branches       1909     1909              
============================================
- Hits          14417    14407      -10     
- Misses         2822     2828       +6     
- Partials        920      922       +2     
Flag Coverage Δ
plugin 79.34% <ø> (-0.05%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

see 12 files with indirect coverage changes

📢 Have feedback on the report? Share it here.

@owaiskazi19
Copy link
Member

This looks good @kaituo! Any reason why we are not making the changes to main and then backporting to 2.x and 2.10?

@kaituo
Copy link
Collaborator Author

kaituo commented Sep 8, 2023

This looks good @kaituo! Any reason why we are not making the changes to main and then backporting to 2.x and 2.10?

The baseline version of main is 2.10. So I am not making the changes from main.

@joshpalis
Copy link
Member

@kaituo out of curiosity, should other plugins take a proactive step and upgrade the BWC test versions for 2.x from 1.1 to 1.3? Job Scheduler 2.x for instance uses 1.13.0 here in the sample job scheduler extension plugin that is used to execute BWC tests. I have not observed any recent BWC test failures though

@jackiehanyang jackiehanyang merged commit 3c1a830 into opensearch-project:2.x Sep 8, 2023
14 checks passed
opensearch-trigger-bot bot pushed a commit that referenced this pull request Sep 8, 2023
…sue (#1029)

Issue:
- In versions between 1.1 and 2.10, the task :adBwcCluster#twoThirdsUpgradedClusterTask fails.
- Symptoms mirror those in [OpenSearch Issue #5076](opensearch-project/OpenSearch#5076).
- Logs show nodes from old and new versions failing to join as a cluster, resulting in a "master not found" exception.

Resolution:
- Upgraded the bwc version to 1.3.2, aligning with other plugins which use 1.3+ as their baseline.
  - [Cross-Cluster Replication PR #469](opensearch-project/cross-cluster-replication#469)
  - [Security PR #2253](opensearch-project/security#2253)
  - [ML Commons PR #681](opensearch-project/ml-commons#681)
- Post-upgrade, the twoThirdsUpgradedClusterTask runs successfully, suggesting potential incompatibility between versions 1.1 and 2.10.

Testing:
- Executed `./gradlew bwcTestSuite -Dtests.security.manager=false` and all tests passed.

Signed-off-by: Kaituo Li <[email protected]>
(cherry picked from commit 3c1a830)
jackiehanyang pushed a commit that referenced this pull request Sep 8, 2023
…sue (#1029) (#1030)

Issue:
- In versions between 1.1 and 2.10, the task :adBwcCluster#twoThirdsUpgradedClusterTask fails.
- Symptoms mirror those in [OpenSearch Issue #5076](opensearch-project/OpenSearch#5076).
- Logs show nodes from old and new versions failing to join as a cluster, resulting in a "master not found" exception.

Resolution:
- Upgraded the bwc version to 1.3.2, aligning with other plugins which use 1.3+ as their baseline.
  - [Cross-Cluster Replication PR #469](opensearch-project/cross-cluster-replication#469)
  - [Security PR #2253](opensearch-project/security#2253)
  - [ML Commons PR #681](opensearch-project/ml-commons#681)
- Post-upgrade, the twoThirdsUpgradedClusterTask runs successfully, suggesting potential incompatibility between versions 1.1 and 2.10.

Testing:
- Executed `./gradlew bwcTestSuite -Dtests.security.manager=false` and all tests passed.

Signed-off-by: Kaituo Li <[email protected]>
(cherry picked from commit 3c1a830)

Co-authored-by: Kaituo Li <[email protected]>
@owaiskazi19 owaiskazi19 changed the title Upgrade AD's bwc baseline version to 1.3.2 to resolve cluster join issue [2.x] Upgrade AD's bwc baseline version to 1.3.2 to resolve cluster join issue Sep 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants