Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix testHAProxyModeConnectionWorks is flaky #16087

Merged
merged 3 commits into from
Sep 29, 2024

Conversation

gaobinlong
Copy link
Collaborator

@gaobinlong gaobinlong commented Sep 26, 2024

Description

This PR fixes the flaky test testHAProxyModeConnectionWorks in RemoteClustersIT, from the gradle check log, we can see that the test case testProxyModeConnectionWorks always succeeds but the case testHAProxyModeConnectionWorks fails, the haproxy service haproxy:9600 provides a reversed proxy for opensearch-2:9300, opensearch-2:9300 is OK but haproxy:9600 cannot be accessed, so the only reason is that the haproxy service is not available at that time, maybe caused by that haproxy starts before opensearch-2, and the domain opensearch-2 is not ready, so haproxy exits with error:

2024-09-26 14:03:42 [ALERT] 269/060342 (1) : parsing [/usr/local/etc/haproxy/haproxy.cfg:22] : 'server default' : could not resolve address 'opensearch-2'.
2024-09-26 14:03:42 [ALERT] 269/060342 (1) : Failed to initialize server(s) addr.

Since haproxy depends on opensearch-2, so we should add a depends_on attribute for the haproxy service in the docker-compose.yml, and also add healthcheck for it to make sure the haproxy service is ready before the tests execute.

Related Issues

#14599

Check List

- [ ] Functionality includes testing.
- [ ] API changes companion pull request created, if applicable.
- [ ] Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link
Contributor

✅ Gradle check result for def25bd: SUCCESS

Copy link

codecov bot commented Sep 26, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 71.96%. Comparing base (d510b12) to head (df2e1fe).
Report is 1 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main   #16087      +/-   ##
============================================
+ Coverage     71.91%   71.96%   +0.04%     
- Complexity    64420    64424       +4     
============================================
  Files          5281     5281              
  Lines        301228   301228              
  Branches      43519    43519              
============================================
+ Hits         216643   216788     +145     
+ Misses        66835    66602     -233     
- Partials      17750    17838      +88     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Gao Binlong <[email protected]>
Copy link
Contributor

❌ Gradle check result for ea53d79: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for ea53d79: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for ea53d79: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@reta
Copy link
Collaborator

reta commented Sep 29, 2024

Thanks @gaobinlong , I think you may need to rebase against main

Copy link
Contributor

✅ Gradle check result for df2e1fe: SUCCESS

@reta reta merged commit 031d8ae into opensearch-project:main Sep 29, 2024
33 of 34 checks passed
opensearch-trigger-bot bot pushed a commit that referenced this pull request Sep 29, 2024
* Fix testHAProxyModeConnectionWorks is flaky

Signed-off-by: Gao Binlong <[email protected]>

* Optimize health check method

Signed-off-by: Gao Binlong <[email protected]>

---------

Signed-off-by: Gao Binlong <[email protected]>
(cherry picked from commit 031d8ae)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
reta pushed a commit that referenced this pull request Sep 29, 2024
* Fix testHAProxyModeConnectionWorks is flaky



* Optimize health check method



---------


(cherry picked from commit 031d8ae)

Signed-off-by: Gao Binlong <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
hainenber pushed a commit to hainenber/OpenSearch that referenced this pull request Oct 1, 2024
* Fix testHAProxyModeConnectionWorks is flaky

Signed-off-by: Gao Binlong <[email protected]>

* Optimize health check method

Signed-off-by: Gao Binlong <[email protected]>

---------

Signed-off-by: Gao Binlong <[email protected]>
ruai0511 pushed a commit to ruai0511/OpenSearch that referenced this pull request Oct 4, 2024
* Fix testHAProxyModeConnectionWorks is flaky

Signed-off-by: Gao Binlong <[email protected]>

* Optimize health check method

Signed-off-by: Gao Binlong <[email protected]>

---------

Signed-off-by: Gao Binlong <[email protected]>
dk2k pushed a commit to dk2k/OpenSearch that referenced this pull request Oct 16, 2024
* Fix testHAProxyModeConnectionWorks is flaky

Signed-off-by: Gao Binlong <[email protected]>

* Optimize health check method

Signed-off-by: Gao Binlong <[email protected]>

---------

Signed-off-by: Gao Binlong <[email protected]>
dk2k pushed a commit to dk2k/OpenSearch that referenced this pull request Oct 17, 2024
* Fix testHAProxyModeConnectionWorks is flaky

Signed-off-by: Gao Binlong <[email protected]>

* Optimize health check method

Signed-off-by: Gao Binlong <[email protected]>

---------

Signed-off-by: Gao Binlong <[email protected]>
dk2k pushed a commit to dk2k/OpenSearch that referenced this pull request Oct 21, 2024
* Fix testHAProxyModeConnectionWorks is flaky

Signed-off-by: Gao Binlong <[email protected]>

* Optimize health check method

Signed-off-by: Gao Binlong <[email protected]>

---------

Signed-off-by: Gao Binlong <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch skip-changelog
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants