-
Notifications
You must be signed in to change notification settings - Fork 202
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Undefined Behaviour in volk_32f_invsqrt_32f #686
Comments
This is: Wikipedia fast inverse square root
It is an approximation. It works. It is a hack. Since this is actually a floating point operation, I suggest to ignore this specific error. |
I suspect the signed integer overflow occurs because the input is negative. So this could just be a case where the test input (uniformly distributed floats in the range -1 .. +1) doesn't make sense. |
As long as the output signature is The C reference for sqrt says:
|
UBSAN shows the following Undefined Behaviour in volk_32f_invsqrt_32f:
/home/argilo/git/volk/kernels/volk/volk_32f_invsqrt_32f.h:71:22: runtime error: signed integer overflow: 1597463007 - -569061536 cannot be represented in type 'int'
This is the problematic line:
volk/kernels/volk/volk_32f_invsqrt_32f.h
Line 71 in e853e9b
The text was updated successfully, but these errors were encountered: