Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] ProcessNodeTest is flaky on CI #50

Closed
owaiskazi19 opened this issue Sep 22, 2023 · 2 comments · Fixed by #47
Closed

[BUG] ProcessNodeTest is flaky on CI #50

owaiskazi19 opened this issue Sep 22, 2023 · 2 comments · Fixed by #47
Assignees
Labels
bug Something isn't working

Comments

@owaiskazi19
Copy link
Member

owaiskazi19 commented Sep 22, 2023

What is the bug?

The below test passes on local but fails on CI.

org.opensearch.flowframework.template.ProcessNodeTests > testNode FAILED
    java.lang.AssertionError
        at __randomizedtesting.SeedInfo.seed([9B313DC0540DBCB3:C8934CDC5B151A88]:0)
        at org.junit.Assert.fail(Assert.java:87)
        at org.junit.Assert.assertTrue(Assert.java:42)
        at org.junit.Assert.assertTrue(Assert.java:53)
        at org.opensearch.flowframework.template.ProcessNodeTests.testNode(ProcessNodeTests.java:56)
  1> [2023-09-22T00:46:59,576][INFO ][o.o.f.t.ProcessNodeTests ] [testNode] before test
  1> [2023-09-22T00:46:59,767][INFO ][o.o.f.t.ProcessNode      ] [[Thread-6]] >>> Starting A.
  1> [2023-09-22T00:46:59,805][INFO ][o.o.f.t.ProcessNodeTests ] [testNode] after test
        at __randomizedtesting.SeedInfo.seed([9B313DC0540DBCB3:C8934CDC5B151A88]:0)

This fails on CI when checking whether future is complete on

        assertTrue(f.isDone());

How can one reproduce the bug?

  1. Run tests on Github CI
./gradlew ':test' --tests "org.opensearch.flowframework.template.ProcessNodeTests.testNode" -Dtests.seed=9B313DC0540DBCB3 -Dtests.security.manager=false -Dtests.locale=en-US -Dtests.timezone=Europe/Moscow -Druntime.java=17

What is the expected behavior?

Test should pass

@owaiskazi19 owaiskazi19 added bug Something isn't working untriaged labels Sep 22, 2023
@dbwiddis
Copy link
Member

This test should probably be removed from ProcessNode. As noted in comments, it is more properly tested by a IT.

Also, I expect I will be able to make the test pass when #42 is implemented, which requires one of the multiple PRs implementing createComponents() to get merged.

Recommend commenting it out and tagging either this issue or #42 to re-enable it.

@dbwiddis
Copy link
Member

So implementing #42 didn't fix this, but it turns out using orTimeout() on a future in the test class is somehow incompatible with the randomized testing we use.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants