Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Backport 2.x] Support centralize snapshot creation #15569

Merged

Conversation

anshu1106
Copy link
Contributor

@anshu1106 anshu1106 commented Sep 2, 2024

Description

This PR backports

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

  • Functionality includes testing.
  • API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@anshu1106 anshu1106 changed the base branch from main to 2.x September 2, 2024 10:12
@anshu1106 anshu1106 changed the title [2.x Backport] Support centralize snapshot creation [Backport 2.x] Support centralize snapshot creation Sep 2, 2024
Copy link
Contributor

github-actions bot commented Sep 2, 2024

❌ Gradle check result for 477f274: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

…#15124)

* Initial Commit to support centralize snapshot creation and implicit locking mechanism

Signed-off-by: Anshu Agarwal <[email protected]>

* Fix deserilization error

Signed-off-by: Anshu Agarwal <[email protected]>

* Fix gradle spotless check

Signed-off-by: Anshu Agarwal <[email protected]>

* Fix listener

Signed-off-by: Anshu Agarwal <[email protected]>

* Fix test

Signed-off-by: Anshu Agarwal <[email protected]>

* Fix snapshot generation

Signed-off-by: Anshu Agarwal <[email protected]>

* Modify cluster setting name

Signed-off-by: Anshu Agarwal <[email protected]>

* Add more tests

Signed-off-by: Anshu Agarwal <[email protected]>

* Uncomment pin timestamp code

Signed-off-by: Anshu Agarwal <[email protected]>

* Modify log messages

Signed-off-by: Anshu Agarwal <[email protected]>

* Add spotless check failure fix

Signed-off-by: Anshu Agarwal <[email protected]>

* Fix completion listener for snapshot v2

Signed-off-by: Anshu Agarwal <[email protected]>

* Elevate cluster state update priority for repository metadata update task

Signed-off-by: Anshu Agarwal <[email protected]>

* Add more integ tests

Signed-off-by: Anshu Agarwal <[email protected]>

* Add priority as IMMEDIATE for cluster state repo update task only for v2 snapshots

Signed-off-by: Anshu Agarwal <[email protected]>

* Fix build error

Signed-off-by: Anshu Agarwal <[email protected]>

* Fix spotless error

Signed-off-by: Anshu Agarwal <[email protected]>

* Add repository setting for snapshot v2

Signed-off-by: Anshu Agarwal <[email protected]>

* Address review comments

Signed-off-by: Anshu Agarwal <[email protected]>

* Add integ test to verify snapshot creation if shallow copy repo setting is disabled

Signed-off-by: Anshu Agarwal <[email protected]>

* Fix spotless vilation error

Signed-off-by: Anshu Agarwal <[email protected]>

* Address review comment

Signed-off-by: Anshu Agarwal <[email protected]>

* Address review comments

Signed-off-by: Anshu Agarwal <[email protected]>

* Add min version check for backward compatibility

Signed-off-by: Anshu Agarwal <[email protected]>

* address review comments

Signed-off-by: Anshu Agarwal <[email protected]>

* add integ test for master failover scenario

Signed-off-by: Anshu Agarwal <[email protected]>

* Add more integ tests

Signed-off-by: Anshu Agarwal <[email protected]>

* refactor code

Signed-off-by: Anshu Agarwal <[email protected]>

* add changelog

Signed-off-by: Anshu Agarwal <[email protected]>

* Add pinned timestamp setting in integ tests

Signed-off-by: Anshu Agarwal <[email protected]>

---------

Signed-off-by: Anshu Agarwal <[email protected]>
Signed-off-by: Anshu Agarwal <[email protected]>
Co-authored-by: Anshu Agarwal <[email protected]>
(cherry picked from commit 23cba28)
Signed-off-by: Anshu Agarwal <[email protected]>
Copy link
Contributor

github-actions bot commented Sep 2, 2024

❌ Gradle check result for 296b0c9: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

github-actions bot commented Sep 2, 2024

❌ Gradle check result for 53c119c: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Anshu Agarwal <[email protected]>
Copy link
Contributor

github-actions bot commented Sep 2, 2024

✅ Gradle check result for 4c108b2: SUCCESS

Copy link

codecov bot commented Sep 2, 2024

Codecov Report

Attention: Patch coverage is 23.24324% with 142 lines in your changes missing coverage. Please review.

Project coverage is 71.52%. Comparing base (ed675f8) to head (a5679b2).
Report is 11 commits behind head on 2.x.

Files with missing lines Patch % Lines
...ava/org/opensearch/snapshots/SnapshotsService.java 17.03% 107 Missing and 5 partials ⚠️
...ch/repositories/blobstore/BlobStoreRepository.java 17.64% 14 Missing ⚠️
...in/java/org/opensearch/snapshots/SnapshotInfo.java 51.85% 8 Missing and 5 partials ⚠️
.../org/opensearch/repositories/FilterRepository.java 0.00% 2 Missing ⚠️
...napshots/create/TransportCreateSnapshotAction.java 75.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##                2.x   #15569      +/-   ##
============================================
- Coverage     71.60%   71.52%   -0.09%     
+ Complexity    63801    63789      -12     
============================================
  Files          5240     5240              
  Lines        298603   298753     +150     
  Branches      43475    43495      +20     
============================================
- Hits         213819   213686     -133     
- Misses        66920    67191     +271     
- Partials      17864    17876      +12     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Anshu Agarwal <[email protected]>
Copy link
Contributor

github-actions bot commented Sep 3, 2024

❌ Gradle check result for 282e451: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Anshu Agarwal <[email protected]>
Copy link
Contributor

github-actions bot commented Sep 3, 2024

✅ Gradle check result for a5679b2: SUCCESS

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants