Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix flaky test SegmentReplicationStatsIT.testMultipleIndices #12070

Merged

Conversation

Poojita-Raj
Copy link
Contributor

@Poojita-Raj Poojita-Raj commented Jan 29, 2024

Description

Fixes flaky test SegmentReplicationStatsIT.testMultipleIndices. The test was failing an assertion error on the replica stats after the segmentReplicationStatsResponse was received. We fix this by adding in an assertBusy on the stats request call.

Ran the test 1000 times and an additional 200 times as part of the SegmentReplicationStatsIT test suite.

image

Related Issues

Resolves #11454

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)
  • Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link
Contributor

❌ Gradle check result for 636a546: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

github-actions bot commented Jan 29, 2024

Compatibility status:

Checks if related components are compatible with change 7f020c4

Incompatible components

Incompatible components: [https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/performance-analyzer-rca.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/alerting.git]

@Poojita-Raj
Copy link
Contributor Author

❌ Gradle check result for 636a546: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

image No discernable failures.

Copy link
Contributor

❌ Gradle check result for 636a546: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@Rishikesh1159
Copy link
Member

❌ Gradle check result for 636a546: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

FAILURE: Build failed with an exception.

  • What went wrong:
    Execution failed for task ':qa:verify-version-constants:v2.12.0#integTest'.

There were failing tests. See the report at: file:///var/jenkins/workspace/gradle-check/search/qa/verify-version-constants/build/reports/tests/v2.12.0%23integTest/index.html

@Rishikesh1159
Copy link
Member

@Poojita-Raj can you please rebase your branch with latest changes in main.

@Poojita-Raj Poojita-Raj force-pushed the testMultipleIndicesFix branch from 636a546 to 7f020c4 Compare January 30, 2024 20:55
Copy link
Contributor

❌ Gradle check result for 7f020c4: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@Poojita-Raj
Copy link
Contributor Author

❌ Gradle check result for 7f020c4: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

#12095

@kkmr
Copy link
Contributor

kkmr commented Jan 30, 2024

Could you please update your PR so only the lines that changed show up in the diff?

Copy link
Contributor

❌ Gradle check result for 7f020c4:

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❕ Gradle check result for 7f020c4: UNSTABLE

  • TEST FAILURES:
      1 org.opensearch.repositories.azure.AzureBlobContainerRetriesTests.testWriteBlobWithRetries
      1 org.opensearch.remotestore.RemoteIndexPrimaryRelocationIT.testPrimaryRelocationWhileIndexing
      1 org.opensearch.cluster.coordination.AwarenessAttributeDecommissionIT.testConcurrentDecommissionAction

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

Copy link

codecov bot commented Jan 30, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (8ed66ae) 71.31% compared to head (7f020c4) 71.42%.
Report is 4 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main   #12070      +/-   ##
============================================
+ Coverage     71.31%   71.42%   +0.11%     
- Complexity    59423    59509      +86     
============================================
  Files          4925     4925              
  Lines        279513   279520       +7     
  Branches      40643    40645       +2     
============================================
+ Hits         199329   199661     +332     
+ Misses        63594    63221     -373     
- Partials      16590    16638      +48     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Rishikesh1159 Rishikesh1159 merged commit 2f8d267 into opensearch-project:main Jan 31, 2024
30 checks passed
@Poojita-Raj Poojita-Raj added the backport 2.x Backport to 2.x branch label Jan 31, 2024
opensearch-trigger-bot bot pushed a commit that referenced this pull request Jan 31, 2024
Signed-off-by: Poojita Raj <[email protected]>
(cherry picked from commit 2f8d267)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Rishikesh1159 pushed a commit that referenced this pull request Jan 31, 2024
…#12100)

(cherry picked from commit 2f8d267)

Signed-off-by: Poojita Raj <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
peteralfonsi pushed a commit to peteralfonsi/OpenSearch that referenced this pull request Mar 1, 2024
rayshrey pushed a commit to rayshrey/OpenSearch that referenced this pull request Mar 18, 2024
shiv0408 pushed a commit to Gaurav614/OpenSearch that referenced this pull request Apr 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch bug Something isn't working flaky-test Random test failure that succeeds on second run skip-changelog
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] The org.opensearch.indices.replication.SegmentReplicationStatsIT.testMultipleIndices is flaky
3 participants