Skip to content
This repository has been archived by the owner on Oct 1, 2024. It is now read-only.

Testing Float datatypes #10

Open
pattonw opened this issue Mar 8, 2019 · 4 comments
Open

Testing Float datatypes #10

pattonw opened this issue Mar 8, 2019 · 4 comments

Comments

@pattonw
Copy link
Owner

pattonw commented Mar 8, 2019

Improve testing of float datatypes

Problems with datasets of type float

  1. No error is thrown when writing an array of ints to a dataset of type float, I assume python casts ints to floats somewhere.
  2. When casting to floats, small changes may occur such as 0.1 to 0.100000001490116119384765625.
  3. Overflow does not happen as expected because large floats are just written as inf
@clbarnes
Copy link
Collaborator

When casting to floats, small changes may occur such as 0.1 to 0.100000001490116119384765625.

Could this just be floats being floats? It seems like it may be more of a "how different languages format numbers by default" issue than anything else.

@pattonw
Copy link
Owner Author

pattonw commented Mar 12, 2019

Yeah, it is just floats being floats. But I just wasn't sure if n5 is used in cases where floating point precision might be an issue. You should always be able to write and read float64 exactly, but should we throw a warning or error if you try to write ints to a float dataset which works, but is imprecise

@clbarnes
Copy link
Collaborator

That's a good call. Given numpy arrays have strict types which can easily be cast between anyway, I'd throw an error if someone tried to write an int array to a float dataset - make the caller do my_int_array.astype(float) if that's what they want (explicit is better than implicit).

@pattonw
Copy link
Owner Author

pattonw commented Mar 12, 2019

Thats what I was thinking. Its not a huge priority for now, so I'll leave it with expected fail tests. I think this problem should resolve itself when we build in numpy support to the rust side

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants