Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve experience for comparing floating point/decimal numbers #74

Open
chadlwilson opened this issue Dec 6, 2021 · 0 comments
Open
Labels
enhancement New feature or request

Comments

@chadlwilson
Copy link
Collaborator

Context / Goal

As noted in #39 it generally doesn't make sense to be using columns with floating point types as an input to tests, since their exact values are likely to be different across databases due to rounding/precision differences, and even the way they are then converted to Java types.

We should probably either

  • completely remove support for these values
  • or warn the user if they are trying to compare column values of these types
  • or allow the user to configure some approach of precision

Expected Outcome

  • Decide and design an approach to deal with decimal types
    • If we can figure out a way for precision to be configured we would need to change the HashingStrategy to take this precision and coerce the value to a long before adding to the hash; with the value rounded to this precision.
    • It probably only makes sense to express this precision in terms of significant figures

Out of Scope

Additional context / implementation notes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant