[Remote Store] Add extra buffer before deleting older generations of translog #10817
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Gradle Check (Jenkins) Run Completed with:
|
Compatibility status:Checks if related components are compatible with change c411a25 Incompatible componentsIncompatible components: [https://github.com/opensearch-project/cross-cluster-replication.git] Skipped componentsCompatible componentsCompatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git] |
gbbafna
force-pushed
the
extra-gen
branch
2 times, most recently
from
dd548fb to
8d45cd2
Compare
Gradle Check (Jenkins) Run Completed with:
|
Codecov Report
@@ Coverage Diff @@
## main #10817 +/- ##
============================================
- Coverage 71.31% 71.27% -0.05%
- Complexity 58671 58706 +35
============================================
Files 4860 4869 +9
Lines 276335 276450 +115
Branches 40198 40198
============================================
- Hits 197068 197032 -36
- Misses 62803 62991 +188
+ Partials 16464 16427 -37
|
Gradle Check (Jenkins) Run Completed with:
|
Gradle Check (Jenkins) Run Completed with:
|
gbbafna
changed the title
|
Lets create an issue or tag an existing one around using an approach where we can prevent deletion of translog from remote only for the peer recovery case. |
ashking94
requested changes
Oct 22, 2023
server/src/main/java/org/opensearch/index/translog/RemoteFsTranslog.java
Outdated
Show resolved
Hide resolved
Gradle Check (Jenkins) Run Completed with:
|
gbbafna
requested review from
abbashus,
anasalkouz,
andrross,
Bukhtawar,
CEHENKLE,
dblock,
dbwiddis,
dreamer-89,
kartg,
kotwanikunal,
mch2,
msfroh,
nknize and
owaiskazi19
as code owners
gbbafna
requested review from
ryanbogan,
sachinpkale,
saratvemulapalli,
setiah,
shwetathareja,
sohami and
VachaShah
as code owners
Gradle Check (Jenkins) Run Completed with:
|
ashking94
reviewed
Oct 22, 2023
server/src/test/java/org/opensearch/index/translog/RemoteFsTranslogTests.java
Outdated
Show resolved
Hide resolved
server/src/test/java/org/opensearch/index/translog/RemoteFsTranslogTests.java
Show resolved
Hide resolved
server/src/test/java/org/opensearch/index/translog/RemoteFsTranslogTests.java
Show resolved
Hide resolved
Gradle Check (Jenkins) Run Completed with:
|
Gradle Check (Jenkins) Run Completed with:
|
Gradle Check (Jenkins) Run Completed with:
|
sachinpkale
approved these changes
Oct 23, 2023
Gradle Check (Jenkins) Run Completed with:
|
sachinpkale
changed the title
… translog files Signed-off-by: Gaurav Bafna <[email protected]>
Signed-off-by: Gaurav Bafna <[email protected]>
Signed-off-by: Gaurav Bafna <[email protected]>
|
@gbbafna @sachinpkale I have rebased and forced push to this branch for build to succeed. |
Gradle Check (Jenkins) Run Completed with:
|
opensearch-trigger-bot bot
pushed a commit
that referenced
this pull request
Oct 23, 2023
…translog (#10817) --------- Signed-off-by: Gaurav Bafna <[email protected]> (cherry picked from commit 218a2ef) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
sachinpkale
pushed a commit
that referenced
this pull request
Oct 23, 2023
…translog (#10817) (#10850) --------- (cherry picked from commit 218a2ef) Signed-off-by: Gaurav Bafna <[email protected]> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
shiv0408
pushed a commit
to Gaurav614/OpenSearch
that referenced
this pull request
Apr 25, 2024
…translog (opensearch-project#10817) --------- Signed-off-by: Gaurav Bafna <[email protected]> Signed-off-by: Shivansh Arora <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Even after #9191 , we are seeing recovery failures on primary relocations.
This is due to the fact that older primary is continuously uploading and deleting from remote translog. So it can happen that even with retries , newer primary is not able to complete the download of all the files.
This PR adds some buffer before deleting the files from remote translog .
Related Issues
Resolves #[Issue number to be closed when this PR is merged]
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.