[BUG] test failure ':sandbox:plugins:concurrent-search:test' QueryProfilePhaseTests.testTerminateAfterEarlyTermination

[tlfeng]

Describe the bug

> Task :sandbox:plugins:concurrent-search:test

REPRODUCE WITH: ./gradlew ':sandbox:plugins:concurrent-search:test' --tests "org.opensearch.search.query.QueryProfilePhaseTests" -Dtests.method="testTerminateAfterEarlyTermination {p0=5 p1=org.opensearch.search.query.ConcurrentQueryPhaseSearcher@2c4eae94}" -Dtests.seed=CDA03188B38F0C0D -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=ar-TN -Dtests.timezone=America/Noronha -Druntime.java=17

org.opensearch.search.query.QueryProfilePhaseTests > testTerminateAfterEarlyTermination {p0=5 p1=org.opensearch.search.query.ConcurrentQueryPhaseSearcher@2c4eae94} FAILED
    java.lang.AssertionError: 
    Expected: a value greater than <0L>
         but: <0L> was equal to <0L>
        at __randomizedtesting.SeedInfo.seed([CDA03188B38F0C0D:5B04881FBABD7064]:0)
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
        at org.junit.Assert.assertThat(Assert.java:964)
        at org.junit.Assert.assertThat(Assert.java:930)
        at org.opensearch.search.query.QueryProfilePhaseTests.lambda$testTerminateAfterEarlyTermination$26(QueryProfilePhaseTests.java:553)
        at org.opensearch.search.query.QueryProfilePhaseTests.lambda$assertProfileData$55(QueryProfilePhaseTests.java:1072)
        at org.opensearch.search.query.QueryProfilePhaseTests.assertProfileData(QueryProfilePhaseTests.java:1123)
        at org.opensearch.search.query.QueryProfilePhaseTests.assertProfileData(QueryProfilePhaseTests.java:1078)
        at org.opensearch.search.query.QueryProfilePhaseTests.assertProfileData(QueryProfilePhaseTests.java:1069)
        at org.opensearch.search.query.QueryProfilePhaseTests.testTerminateAfterEarlyTermination(QueryProfilePhaseTests.java:534)

REPRODUCE WITH: ./gradlew ':sandbox:plugins:concurrent-search:test' --tests "org.opensearch.search.query.QueryProfilePhaseTests" -Dtests.method="testTerminateAfterEarlyTermination {p0=0 p1=org.opensearch.search.query.QueryPhase$DefaultQueryPhaseSearcher@6ad198c9}" -Dtests.seed=CDA03188B38F0C0D -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=ar-TN -Dtests.timezone=America/Noronha -Druntime.java=17

org.opensearch.search.query.QueryProfilePhaseTests > testTerminateAfterEarlyTermination {p0=0 p1=org.opensearch.search.query.QueryPhase$DefaultQueryPhaseSearcher@6ad198c9} FAILED
    java.lang.AssertionError: 
    Expected: a value greater than <0L>
         but: <0L> was equal to <0L>
... (same as above)

* What went wrong:
Execution failed for task ':sandbox:plugins:concurrent-search:test'.
> org.gradle.test-retry was unable to retry the following test methods, which is unexpected. Please file a bug report at https://github.com/gradle/test-retry-gradle-plugin/issues
     org.opensearch.search.query.QueryProfilePhaseTests#testTerminateAfterEarlyTermination {p0=5 p1=org.opensearch.search.query.ConcurrentQueryPhaseSearcher@2c4eae94}
     org.opensearch.search.query.QueryProfilePhaseTests#testTerminateAfterEarlyTermination {p0=0 p1=org.opensearch.search.query.QueryPhase$DefaultQueryPhaseSearcher@6ad198c9}

Comes from the grale check run in #2680 (comment)
Log 4209
Reports 4209

To Reproduce
It's reproducible in main and 2.x branch: ./gradlew ':sandbox:plugins:concurrent-search:test' --tests "org.opensearch.search.query.QueryProfilePhaseTests" -Dtests.method="testTerminateAfterEarlyTermination" -Dtests.seed=CDA03188B38F0C0D

Expected behavior
test passes.

Plugins
n/a

Screenshots
n/a

Host/Environment (please complete the following information):
n/a

Additional context
The test is introduced by commit b6ca0d1

[kartg]

Unable to repro this failure on either main or 2.x using the command above:

./gradlew ':sandbox:plugins:concurrent-search:test' --tests "org.opensearch.search.query.QueryProfilePhaseTests" -Dtests.method="testTerminateAfterEarlyTermination" -Dtests.seed=CDA03188B38F0C0D

@tlfeng i'm going to close this issue since the test seems to be succeeding now. Feel free to reopen if you see a reason to continue tracking this.

[shwetathareja]

It is reproed again here - https://build.ci.opensearch.org/job/gradle-check/28588/
#10782 (comment)

[ashking94]

#10830 (comment)

[jed326]

@ticheng-aws would you mind taking a look at this? Thanks!