Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not able to run Timeseries Explainer incase of anomaly detection #90

Open
usnishmukherjee opened this issue Jul 19, 2023 · 4 comments
Open

Comments

@usnishmukherjee
Copy link

I am trying to perform Explainability for Timeseries Anomaly detection ( DBSCAN model). I have loaded the model and prediction funtion for DBSCAN is model.fit_predict
Here is the code snippet:
image

And the Final errror : ValueError: Unable to convert array of bytes/strings into decimal numbers with dtype='numeric'
Although my dataframe contains only numeric values.
Here is the view of my data:
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 300 entries, 0 to 299
Data columns (total 1 columns):
Column Non-Null Count Dtype
sample 300 non-null float64
dtypes: float64(1)

@usnishmukherjee
Copy link
Author

I am able to run the shapExplainer for a timeseries data. But not sure how to read the graph. Not enough information is given in the documentation as well.
Please help to understand the graph.

image

What to infer from the graph? What is the anomaly and why that is anomaly? I have passed a batch of data here Like below:
explanations = explainer.explain(Timeseries.from_pd(test_df))

@yangwenz
Copy link
Collaborator

Hi, the score (value_score) shows the feature importance score of each timestamp if SHAP is applied. Similarly, the timestamp_score is the feature importance score of the timestamp. The timestamp_score will be zero if the timestamps are not used during anomaly detection.

@usnishmukherjee
Copy link
Author

Hi @yangwenz .
Thanks for your help. One more thing, how that importance score help me to identify the anomaly ?

@yangwenz
Copy link
Collaborator

Unfortunately, the lib cannot identify the anomaly, it can only explain why a certain data point or a window is detected as anomaly. For example, when you apply some black-box anomaly detector (especially those based on neural networks), you may need to know why a window of timeseries is detected as an anamaly and use this information to analyze the corresponding incident. This is quite common in AIOps.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants