Skip to content
This repository has been archived by the owner on May 16, 2020. It is now read-only.

data_get precision loss for numbers with many digits #126

Open
acautin opened this issue Aug 5, 2019 · 4 comments
Open

data_get precision loss for numbers with many digits #126

acautin opened this issue Aug 5, 2019 · 4 comments
Assignees
Labels
bug Something isn't working

Comments

@acautin
Copy link
Contributor

acautin commented Aug 5, 2019

Using data_get after executing the following query results in loss of precision:
SELECT 2.700000000000005 FROM dual
Is returned as 2.70000000000001

I am not really convinced we are properly using the full precision of erlang as this is getting rounded even before the erlang precision is a problem.

Also maybe related: oracle/odpi#4

@acautin acautin added the bug Something isn't working label Aug 5, 2019
@acautin
Copy link
Contributor Author

acautin commented Aug 5, 2019

Question to odpi, ref: oracle/odpi#100 (comment)

@stoch
Copy link

stoch commented Aug 6, 2019

@acautin Just a small remark:

Assuming that tools like TOAD are the norm, we have to be careful:

SELECT to_char(2.700000000000005) FROM dual
gives
TO_CHAR(2.700000000000005)

2.700000000000005
1 row selected.

whereas

SELECT 2.700000000000005 FROM dual
gives:
2.700000000000005

   2.70000000

1 row selected.

The inherent precision is conserved but the default rendering in TOAD (probably Oracle) does some default rounding to 8 digits.

SELECT 2.700100021200305 FROM dual
gives
2.700100021200305

   2.70010002

1 row selected.

@c-bik
Copy link
Member

c-bik commented Aug 7, 2019

@acautin is oracle/odpi#100 (comment) good enough to close this issue?

@acautin
Copy link
Contributor Author

acautin commented Aug 7, 2019

@c-bik haven't tried it yet, as soon as I try it out (after finishing with edits) will confirm and close.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants