Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Finished fixing flaky test #378

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Finished fixing flaky test #378

wants to merge 1 commit into from

Conversation

wuh3
Copy link

@wuh3 wuh3 commented Nov 13, 2023

Changes proposed

Testcom.jakduk.api.GalleryOnListTest.자바배열객체를_JSON문자열로변환 was found flaky by an open-source research tool NonDex, which will shuffle implementation-dependent operations. The error was thrown by: assertEquals(expectResult, result); at line 179. The flakiness was resulted from the random order of the fields stored inside the Hashmaps image01 and image02. The expectedResult used in the unit test was defined using a string with fixed order, and it will not match the result generated by mapper.writeValueAsString(images);. The error messages:

org.opentest4j.AssertionFailedError: expected: <[{"uid":"a1","name":"test01","size":100},{"uid":"a2","name":"test02","size":200}]> but was: <[{"uid":"a1","size":100,"name":"test01"},{"uid":"a2","size":200,"name":"test02"}]>
        at org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:55)
        at org.junit.jupiter.api.AssertionUtils.failNotEqual(AssertionUtils.java:62)
        at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:182)
        at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:177)
        at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:1124)
        at com.jakduk.api.GalleryOnListTest.자바배열객체를_JSON문자열로변환(GalleryOnListTest.java:179)

Fix of the problem

The fix was rather simple. Since it was totally due to the property of HashMap where its fields are stored in random orders. I changed the data structures of ``images01andimages02``` from HashSet to LinkedHashSet. Besides, I modified the order of the ```expectedResult``` to match the sorted values of the LinkedHashMaps.

Result of the fix

The test successed with Nondex runs for 3, 10, and 100 times. It can be safely concluded that the flakiness of this test is fixed

Test Environment:

Gradle 7.2
java OpenJDK 1.8.0_382
Linux 5.4.0-166-generic amd64

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant