feat: add Grafana dashboards and alerts to monitor Thanos #73
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of the changes
This PR adds multiple Grafana dashboards and Prometheus alerts to monitor Thanos.
The alerts are conditionally deployed depending on the Thanos components deployed. Note that the sidecar rules are always deployed, because the DevOps Stack was designed for Thanos in the sidecar mode and not in the receiver mode. When we change this implementation, we should be aware of that.
Also, for the time being, the dashboards are always deployed, even if certain components are not. Note that certain dashboard files are suffixed with
.ignore
to disable them. I'll let this be done statically until I figure out a way to do it while still using the iteration over the files in thegrafana-dashboard.yaml
template.Breaking change
Tests executed on which distribution(s)