You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Some of the warnings CoTeDe is throwing appear to be due to floating point calculations underlying CoTeDe's spike check. Running only CoTeDe_spike.py without parallelization, random profiles generate errors of the type:
/AutoQC/miniconda/lib/python2.7/site-packages/cotede/qctests/spike.py:63: RuntimeWarning: invalid value encountered in greater
flag[np.nonzero(self.features['spike'] > threshold)] = flag_bad
/AutoQC/miniconda/lib/python2.7/site-packages/cotede/qctests/spike.py:64: RuntimeWarning: invalid value encountered in less_equal
flag[np.nonzero(self.features['spike'] <= threshold)] = flag_good
Re-running from scratch will show the same error, but in a different profile (see sample of quota data I was doing this on here: quota_subset.txt)
Digging in a bit, the invalid values are NaNs appearing in the masked array self.features['spike']. If I dump this array without the mask for a profile that produced this error, I get:
The first and last entries in the array are coming out a bit differently each time, sometimes tripping over into NaN and potentially corrupting the result of this test.
Some of the warnings CoTeDe is throwing appear to be due to floating point calculations underlying CoTeDe's spike check. Running only
CoTeDe_spike.py
without parallelization, random profiles generate errors of the type:Re-running from scratch will show the same error, but in a different profile (see sample of quota data I was doing this on here: quota_subset.txt)
Digging in a bit, the invalid values are
NaN
s appearing in the masked arrayself.features['spike']
. If I dump this array without the mask for a profile that produced this error, I get:Then, if I run the exact same thing again, the test doesn't throw a warning on the same profile and instead produces
The first and last entries in the array are coming out a bit differently each time, sometimes tripping over into
NaN
and potentially corrupting the result of this test.@castelao @s-good thoughts?
The text was updated successfully, but these errors were encountered: