Fixing flaky test in LogbackLoggerOverridesTest #499
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This change is meant to fix a flaky test -
testSave_writtenInLogbackXMLFormat()
inLogbackLoggerOverridesTest.java
Analysis
This test presents flakiness because the ordering of
loggerLevels
in LogbackLoggerOverrides.java is non-deterministic.This can be reproduced by running the following command using the nondex plugin.
mvn -pl components/nexus-base edu.illinois:nondex-maven-plugin:2.1.7:nondex -Dtest="LogbackLoggerOverridesTest"
throwing the error
Since the above test is converting these into XMLs and comparing them as Strings, it is failing. Since the order of elements in an XML matters, converting these Strings into XMLs will not fix the test.
Fix
In order to make the ordering of
loggerLevels
deterministic, this fix is converting it's type fromHashMap.java
toLinkedHashMap.java
, which retains insertion order. Furthermore this fix also changes the ordering of loglevels in getExpectedXml(boolean afterReset) in order to maintain a consistent loggerLevel ordering with the tests in the class.I would love to get feedback on this Pull Request. Do let me know if you'd like to see any more changes!
Thanks