Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve spike/outlier detection #22

Open
squaregoldfish opened this issue Nov 14, 2016 · 1 comment
Open

Improve spike/outlier detection #22

squaregoldfish opened this issue Nov 14, 2016 · 1 comment
Assignees

Comments

@squaregoldfish
Copy link
Member

The current High Delta check is a very simple affair. It defines a delta in units and a timespan, and checks the rate of change between any two consecutive measurements to see if it exceeds this rate. This can be improved by the creation of two separate routines:

Long term drift
Define a unit difference and a long time period. Measurements that are that period apart will be checked to see if the change is greater than the specified unit difference. We may need to specify n measurements to be averaged at each end to take account of short term noise.

Spike detection
A new approach to spike detection: Get the deltas between measurements across the entire data set, expressed as a change in units over time. Calculate the standard deviation of these deltas. A spike can be defined as any delta that's outside three standard deviations.

@squaregoldfish
Copy link
Member Author

These would replace the existing HighDelta routine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants