Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix a flaky test that relies on terminate_after for an exact count ma… #12179

Merged
merged 3 commits into from
Feb 15, 2024

Conversation

austintlee
Copy link
Contributor

@austintlee austintlee commented Feb 6, 2024

…tch. (Issue # 10435)

Description

[Describe what this change achieves]

Related Issues

Resolves #10435

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)
  • Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@github-actions github-actions bot added bug Something isn't working Search Search query, autocomplete ...etc labels Feb 6, 2024
Copy link
Contributor

github-actions bot commented Feb 6, 2024

Compatibility status:

Checks if related components are compatible with change 44a6bde

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/flow-framework.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/performance-analyzer.git]

Copy link
Contributor

github-actions bot commented Feb 6, 2024

❕ Gradle check result for 0acb641: UNSTABLE

  • TEST FAILURES:
      2 org.opensearch.common.util.concurrent.QueueResizableOpenSearchThreadPoolExecutorTests.classMethod
      1 org.opensearch.search.SearchWeightedRoutingIT.testMultiGetWithNetworkDisruption_FailOpenEnabled
      1 org.opensearch.remotestore.RemoteIndexPrimaryRelocationIT.testPrimaryRelocationWhileIndexing
      1 org.opensearch.common.util.concurrent.QueueResizableOpenSearchThreadPoolExecutorTests.testResizeQueueDown

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

Copy link

codecov bot commented Feb 6, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (03e415e) 71.45% compared to head (44a6bde) 71.35%.

Additional details and impacted files
@@             Coverage Diff              @@
##               main   #12179      +/-   ##
============================================
- Coverage     71.45%   71.35%   -0.10%     
+ Complexity    59831    59754      -77     
============================================
  Files          4959     4959              
  Lines        281129   281129              
  Branches      40857    40857              
============================================
- Hits         200892   200611     -281     
- Misses        63580    63800     +220     
- Partials      16657    16718      +61     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Collaborator

@msfroh msfroh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing this @austintlee, and sorry for the delay on reviewing it. I was just working with @sandeshkr419 on another issue and he ran into this same problem and told me about your PR.

Can you please rebase onto the latest main? Let's get this merged.

@msfroh msfroh added skip-changelog flaky-test Random test failure that succeeds on second run labels Feb 8, 2024
@msfroh
Copy link
Collaborator

msfroh commented Feb 8, 2024

Oh -- also the very helpful comments need to use HTML entities:

/home/runner/work/OpenSearch/OpenSearch/server/src/internalClusterTest/java/org/opensearch/search/simple/SimpleSearchIT.java:344: error: bad use of '>'
> Task :modules:repository-url:compileTestJava
     *   But if track_total_hits_up_to >= terminate_after, relation can be EQ _or_ GTE.

                                       ^
> Task :modules:repository-url:copyPluginPropertiesTemplate
/home/runner/work/OpenSearch/OpenSearch/server/src/internalClusterTest/java/org/opensearch/search/simple/SimpleSearchIT.java:346: error: bad use of '>'

     *   or totalHits > track_total_hits_up_to and SearchPhaseController sets totalHits = track_total_hits_up_to when returning results
> Task :server:compileInternalClusterTestJava

Copy link
Contributor

github-actions bot commented Feb 8, 2024

❌ Gradle check result for 84e9d34: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@andrross
Copy link
Member

I just rebased against main, which should fix the bwc failure from the last gradle check

Copy link
Contributor

❌ Gradle check result for bdfdb0c: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for bdfdb0c: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

✅ Gradle check result for 44a6bde: SUCCESS

@reta reta merged commit 8fe15e6 into opensearch-project:main Feb 15, 2024
30 checks passed
@reta reta added the backport 2.x Backport to 2.x branch label Feb 15, 2024
opensearch-trigger-bot bot pushed a commit that referenced this pull request Feb 15, 2024
#12179)

* Fix a flaky test that relies on terminate_after for an exact count match.  (Issue # 10435)

Signed-off-by: Austin Lee <[email protected]>

* Make comments HTML Javadoc friendly.

Signed-off-by: Austin Lee <[email protected]>

* Fix javadoc syntax

Signed-off-by: Andrew Ross <[email protected]>

---------

Signed-off-by: Austin Lee <[email protected]>
Signed-off-by: Andrew Ross <[email protected]>
Co-authored-by: Andrew Ross <[email protected]>
(cherry picked from commit 8fe15e6)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
reta pushed a commit that referenced this pull request Feb 16, 2024
#12179) (#12343)

* Fix a flaky test that relies on terminate_after for an exact count match.  (Issue # 10435)



* Make comments HTML Javadoc friendly.



* Fix javadoc syntax



---------




(cherry picked from commit 8fe15e6)

Signed-off-by: Austin Lee <[email protected]>
Signed-off-by: Andrew Ross <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Andrew Ross <[email protected]>
peteralfonsi pushed a commit to peteralfonsi/OpenSearch that referenced this pull request Mar 1, 2024
opensearch-project#12179)

* Fix a flaky test that relies on terminate_after for an exact count match.  (Issue # 10435)

Signed-off-by: Austin Lee <[email protected]>

* Make comments HTML Javadoc friendly.

Signed-off-by: Austin Lee <[email protected]>

* Fix javadoc syntax

Signed-off-by: Andrew Ross <[email protected]>

---------

Signed-off-by: Austin Lee <[email protected]>
Signed-off-by: Andrew Ross <[email protected]>
Co-authored-by: Andrew Ross <[email protected]>
rayshrey pushed a commit to rayshrey/OpenSearch that referenced this pull request Mar 18, 2024
opensearch-project#12179)

* Fix a flaky test that relies on terminate_after for an exact count match.  (Issue # 10435)

Signed-off-by: Austin Lee <[email protected]>

* Make comments HTML Javadoc friendly.

Signed-off-by: Austin Lee <[email protected]>

* Fix javadoc syntax

Signed-off-by: Andrew Ross <[email protected]>

---------

Signed-off-by: Austin Lee <[email protected]>
Signed-off-by: Andrew Ross <[email protected]>
Co-authored-by: Andrew Ross <[email protected]>
shiv0408 pushed a commit to Gaurav614/OpenSearch that referenced this pull request Apr 25, 2024
opensearch-project#12179)

* Fix a flaky test that relies on terminate_after for an exact count match.  (Issue # 10435)

Signed-off-by: Austin Lee <[email protected]>

* Make comments HTML Javadoc friendly.

Signed-off-by: Austin Lee <[email protected]>

* Fix javadoc syntax

Signed-off-by: Andrew Ross <[email protected]>

---------

Signed-off-by: Austin Lee <[email protected]>
Signed-off-by: Andrew Ross <[email protected]>
Co-authored-by: Andrew Ross <[email protected]>
Signed-off-by: Shivansh Arora <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch bug Something isn't working flaky-test Random test failure that succeeds on second run Search Search query, autocomplete ...etc skip-changelog
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] Search requests with size=0 & terminate_after sometimes return incorrectly hit count
5 participants