Increase segrep pressure checkpoint default limit to 30 #16577
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gbbafna
force-pushed
the
segrep_bp_default
branch
from
4f8107d to
965595e
Compare
|
❌ Gradle check result for 965595e: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
gbbafna
force-pushed
the
segrep_bp_default
branch
from
965595e to
b14c7c8
Compare
gbbafna
requested review from
anasalkouz,
andrross,
ashking94,
Bukhtawar,
CEHENKLE,
dblock,
dbwiddis,
jed326,
kotwanikunal,
mch2,
msfroh,
nknize,
owaiskazi19,
reta,
Rishikesh1159,
sachinpkale,
saratvemulapalli,
shwetathareja,
sohami,
VachaShah,
jainankitk and
linuxpi
as code owners
|
❌ Gradle check result for b14c7c8: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
gbbafna
force-pushed
the
segrep_bp_default
branch
from
b14c7c8 to
3fef8c0
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #16577 +/- ##
============================================
- Coverage 72.15% 72.07% -0.08%
+ Complexity 65128 65084 -44
============================================
Files 5315 5315
Lines 303573 303573
Branches 43925 43925
============================================
- Hits 219036 218807 -229
- Misses 66565 66773 +208
- Partials 17972 17993 +21 ☔ View full report in Codecov by Sentry. |
kotwanikunal
approved these changes
Nov 8, 2024
There was a problem hiding this comment.
@gbbafna - Users who upgrade to this release will have the default value updated without them noticing. Do you think it makes sense to document this as a part of release highlights?
linuxpi
reviewed
Nov 11, 2024
server/src/main/java/org/opensearch/index/SegmentReplicationPressureService.java
Outdated
Show resolved
Hide resolved
linuxpi
reviewed
Nov 11, 2024
server/src/test/java/org/opensearch/index/SegmentReplicationPressureServiceTests.java
Show resolved
Hide resolved
ashking94
approved these changes
Nov 11, 2024
gbbafna
force-pushed
the
segrep_bp_default
branch
from
3fef8c0 to
67bdeaf
Compare
gbbafna
changed the title
ashking94
reviewed
Nov 11, 2024
Signed-off-by: Gaurav Bafna <[email protected]>
gbbafna
force-pushed
the
segrep_bp_default
branch
from
67bdeaf to
b2ccfeb
Compare
opensearch-trigger-bot bot
pushed a commit
that referenced
this pull request
Nov 11, 2024
Signed-off-by: Gaurav Bafna <[email protected]> (cherry picked from commit 10873f1) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
gbbafna
pushed a commit
that referenced
this pull request
Nov 12, 2024
) (cherry picked from commit 10873f1) Signed-off-by: Gaurav Bafna <[email protected]> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
The default
segrep.pressure.time.limitis 5 mins , while defaultsegrep.pressure.checkpoint.limitis just 4 which only allows for 40 sec lag given the refresh interval of 10 secs . Though the default refresh interval is 1sec, for remote backed clusters, we advice >10 sec due to prohibitive costs and high resource consumption of segment upload every sec. This PR increasessegrep.pressure.checkpoint.limitto 10 so that both checkpoint limit and time limit equates to same 300 sec and we don't give unnecessary rejections to the user .This relaxation is due to the fact that we don't have multipart download or block level fetches in download part, hence the observed lag on clusters are on a higher side (100s of seconds) during force merges .
Check List
API changes companion pull request created, if applicable.Public documentation issue/PR created, if applicable.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.