Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] org.opensearch.http.SearchRestCancellationIT.testAutomaticCancellationMultiSearchDuringQueryPhase test is flaky #13674

Closed
shiv0408 opened this issue May 14, 2024 · 3 comments
Closed

[BUG] org.opensearch.http.SearchRestCancellationIT.testAutomaticCancellationMultiSearchDuringQueryPhase test is flaky #13674

shiv0408 opened this issue May 14, 2024 · 3 comments
Assignees
@kaushalmahi12
Labels
bug Something isn't working flaky-test Random test failure that succeeds on second run Search Search query, autocomplete ...etc

Comments

@shiv0408
Copy link
Member

Describe the bug

Test org.opensearch.http.SearchRestCancellationIT.testAutomaticCancellationMultiSearchDuringQueryPhase is flaky

Stack trace:

java.lang.AssertionError
	at org.junit.Assert.fail(Assert.java:87)
	at org.junit.Assert.assertTrue(Assert.java:42)
	at org.junit.Assert.assertTrue(Assert.java:53)
	at org.opensearch.http.SearchRestCancellationIT.lambda$ensureSearchTaskIsCancelled$0(SearchRestCancellationIT.java:222)
	at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1128)
	at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1101)
	at org.opensearch.http.SearchRestCancellationIT.ensureSearchTaskIsCancelled(SearchRestCancellationIT.java:218)
	at org.opensearch.http.SearchRestCancellationIT.verifyCancellationDuringQueryPhase(SearchRestCancellationIT.java:140)
	at org.opensearch.http.SearchRestCancellationIT.testAutomaticCancellationMultiSearchDuringQueryPhase(SearchRestCancellationIT.java:114)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
	at org.opensearch.test.OpenSearchTestClusterRule$1.evaluate(OpenSearchTestClusterRule.java:369)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:1583)
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.http.SearchRestCancellationIT.lambda$ensureSearchTaskIsCancelled$0(SearchRestCancellationIT.java:222)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1116)
		... 43 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.http.SearchRestCancellationIT.lambda$ensureSearchTaskIsCancelled$0(SearchRestCancellationIT.java:222)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1116)
		... 43 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.http.SearchRestCancellationIT.lambda$ensureSearchTaskIsCancelled$0(SearchRestCancellationIT.java:222)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1116)
		... 43 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.http.SearchRestCancellationIT.lambda$ensureSearchTaskIsCancelled$0(SearchRestCancellationIT.java:222)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1116)
		... 43 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.http.SearchRestCancellationIT.lambda$ensureSearchTaskIsCancelled$0(SearchRestCancellationIT.java:222)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1116)
		... 43 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.http.SearchRestCancellationIT.lambda$ensureSearchTaskIsCancelled$0(SearchRestCancellationIT.java:222)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1116)
		... 43 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.http.SearchRestCancellationIT.lambda$ensureSearchTaskIsCancelled$0(SearchRestCancellationIT.java:222)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1116)
		... 43 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.http.SearchRestCancellationIT.lambda$ensureSearchTaskIsCancelled$0(SearchRestCancellationIT.java:222)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1116)
		... 43 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.http.SearchRestCancellationIT.lambda$ensureSearchTaskIsCancelled$0(SearchRestCancellationIT.java:222)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1116)
		... 43 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.http.SearchRestCancellationIT.lambda$ensureSearchTaskIsCancelled$0(SearchRestCancellationIT.java:222)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1116)
		... 43 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.http.SearchRestCancellationIT.lambda$ensureSearchTaskIsCancelled$0(SearchRestCancellationIT.java:222)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1116)
		... 43 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.http.SearchRestCancellationIT.lambda$ensureSearchTaskIsCancelled$0(SearchRestCancellationIT.java:222)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1116)
		... 43 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.http.SearchRestCancellationIT.lambda$ensureSearchTaskIsCancelled$0(SearchRestCancellationIT.java:222)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1116)
		... 43 more

Related component

Search

To Reproduce

./gradlew ':qa:smoke-test-http:integTest' --tests "org.opensearch.http.SearchRestCancellationIT.testAutomaticCancellationMultiSearchDuringQueryPhase" -Dtests.seed=8BE32D75FF21B14A -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=mk-MK -Dtests.timezone=Asia/Phnom_Penh -Druntime.java=21

Expected behavior

Test should always pass

Additional Details

CI failing in unrelated change: https://build.ci.opensearch.org/job/gradle-check/38710/testReport/org.opensearch.http/SearchRestCancellationIT/testAutomaticCancellationMultiSearchDuringQueryPhase_2/
@shiv0408 shiv0408 added bug Something isn't working untriaged labels May 14, 2024
@github-actions github-actions bot added the Search Search query, autocomplete ...etc label May 14, 2024
@shiv0408 shiv0408 mentioned this issue May 14, 2024
Merged
7 tasks
@peternied peternied added flaky-test Random test failure that succeeds on second run and removed untriaged labels May 15, 2024
@peternied
Copy link
Member

[Triage - attendees 1 2 3 4 5 6 7 8]
@shiv0408 Thanks for creating this issue to track this flaky test case

@getsaurabh02
Copy link
Member

@kaushalmahi12 can we try to reproduce this and see if its flakiness or any functional issue with the cancellation.

@akolarkunnu akolarkunnu mentioned this issue Jun 14, 2024
Merged
9 tasks
@kkewwei kkewwei mentioned this issue Jun 18, 2024
Merged
3 tasks
@reta
Copy link
Collaborator

reta commented Jun 19, 2024

Closing in favour of #14311

@reta reta closed this as completed Jun 19, 2024
@github-project-automation github-project-automation bot moved this from 🆕 New to ✅ Done in Search Project Board Jun 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kaushalmahi12 kaushalmahi12

Labels
bug Something isn't working flaky-test Random test failure that succeeds on second run Search Search query, autocomplete ...etc
Projects
Search Project Board
Archived in project
Milestone
No milestone
Development

No branches or pull requests
5 participants
@reta @peternied @getsaurabh02 @kaushalmahi12 @shiv0408