Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixing flaky configuration tests #198

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

mihirgune
Copy link

Description

This Pull Request is intend to fix flakiness in the following tests in org.mariadb.jdbc.unit.util.ConfigurationTest

  1. org.mariadb.jdbc.unit.util.ConfigurationTest#testBuild
  2. org.mariadb.jdbc.unit.util.ConfigurationTest#builder
  3. org.mariadb.jdbc.unit.util.ConfigurationTest#toConf

These tests fail intermittently when executed with the NonDex plugin due to non-deterministic ordering of properties. The failure logs are attached for reference:

Cause of the Issue:

The failures occur because the tests compare expected strings with actual strings generated from the Configuration object. The ordering of query parameters and non-mapped options in these strings is non-deterministic due to the use of java.util.Properties, which does not guarantee order. This leads to mismatches in the expected and actual outputs during assertions.

Fix

All three tests (testBuild, builder, and toConf) fail because they perform assertions comparing expected strings to actual strings generated from the Configuration object. However, the ordering of query parameters in the connection string is non-deterministic due to the use of java.util.Properties, which does not guarantee the order of its elements. This nondeterminism leads to mismatches during assertions, causing the tests to fail.

To resolve this issue for the first two tests (testBuild and builder), we introduced a new method called normalizeConnectionString(). This method parses the connection string, extracts the query parameters, sorts them lexicographically, and reconstructs the string with the sorted parameters. By fixing the order of the query parameters, these tests now pass consistently. For the third test (toConf), we addressed the same issue by introducing a new method called normalizeConfigurationString(). This method ensures that normalizeConnectionString() is called on the URL generated within the configuration that the toConf test checks, thereby standardizing the ordering of query parameters and allowing the test to pass reliably.

In addition to the above issue, the toConf test fails occasionally because the conf.nonMappedOptions (unknown options) in org.mariadb.jdbc.Configuration#toConf are unordered. Although the method ensures that other option types (non-default options, default options) are ordered, the unordered nonMappedOptions lead to inconsistent outputs during assertions, causing the test to fail. To fix this, we modified the toConf method to sort the conf.nonMappedOptions, ensuring consistent ordering and eliminating the flakiness in the toConf test.

PS: Instead of sorting the conf.nonMappedOptions in org.mariadb.jdbc.Configuration#toConf, I can also implement a fix wherein the test is made flexible to accept a configuration string that contains the desired options in any order.

Please let me know if any other changes are required. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant