Skip to content

Commit

Permalink
grammar pass on alerts and notifications dir (netdata#17729)
Browse files Browse the repository at this point in the history
  • Loading branch information
Ancairon authored May 22, 2024
1 parent bde40f3 commit 0c8277f
Show file tree
Hide file tree
Showing 4 changed files with 123 additions and 122 deletions.
Original file line number Diff line number Diff line change
@@ -1,45 +1,44 @@
# Creating Alerts with Netdata Alerts Configuration Manager

The Netdata Alerts Configuration Manager enables subscribers to easily set up alerts directly from the Netdata Dashboard. More details on subscriptions can be found [here](https://www.netdata.cloud/pricing/).
The Netdata Alerts Configuration Manager enables subscribers to easily set up Alerts directly from the Netdata Dashboard. More details on subscriptions can be found [here](https://www.netdata.cloud/pricing/).

## Using the Alerts Configuration Manager

1. Navigate to the **Metrics** tab and select the chart you want to configure for alerts.
1. Navigate to the **Metrics** tab and select the chart you want to configure for Alerts.
2. Click the **Alert icon** in the top right corner of the chart.
3. The Alert Configuration Manager will open, showing the default thresholds. Modify these thresholds as needed, the alert definition on the right will update automatically.
3. The Alert Configuration Manager will open, showing the default thresholds. Modify these thresholds as needed; the Alert definition on the right will update automatically.
4. For additional settings, toggle **Show advanced**.
5. After configuring the alert, copy the generated alert definition from the code box. Paste this into an existing or new custom health configuration file located at `<path to netdata install>/etc/netdata/health.d/` on a Parent Agent or a Standalone Child Agent. The guide to edit health configuration files is available [here](/src/health/REFERENCE.md#edit-health-configuration-files).
6. To activate the new alert, run the command `<path to netdata install>/usr/sbin/netdatacli reload-health`.
5. After configuring the Alert, copy the generated Alert definition from the code box. Paste this into an existing or new custom health configuration file located at `<path to netdata install>/etc/netdata/health.d/` on a Parent Agent or a Standalone Child Agent. The guide to edit health configuration files is available [here](/src/health/REFERENCE.md#edit-health-configuration-files).
6. To activate the new Alert, run the command `<path to netdata install>/usr/sbin/netdatacli reload-health`.

## Alerts Configuration Manager Sections

### Alert Detection Method

An alert is triggered whenever a metric crosses a threshold:
An Alert is triggered whenever a metric crosses a threshold:

- **Standard Threshold**: Triggered when a metric crosses a predefined value.
- **Metric Variance**: Triggered based on the variance of the metric.
- **Anomaly Rate**: Triggered based on the anomaly rate of the metric.

### Metrics Lookup, Filtering and Formula Section
### Metrics Lookup, Filtering, and Formula Section

You can read more about the different options in the [Alerts reference documentation](/src/health/REFERENCE.md)
You can read more about the different options in the [Alerts reference documentation](/src/health/REFERENCE.md).

- **Metrics Lookup**: Adjust the database lookup parameters directly in the UI, including method (`avg`, `sum`, `min`, `max`, etc.), computation style, dimensions, duration, and options like `absolute` or `percentage`.
- **Alert Filtering**: The **show advanced** checkbox allows filtering of alert health checks for specific infrastructure components. Options include selecting hosts, nodes, instances, chart labels, and operating systems.
- **Formula / Calculation**: The **show advanced** checkbox allows defining a formula for the metric value, which is then used to set alert thresholds.
- **Alert Filtering**: The **show advanced** checkbox allows filtering of Alert health checks for specific infrastructure components. Options include selecting hosts, nodes, instances, chart labels, and operating systems.
- **Formula / Calculation**: The **show advanced** checkbox allows defining a formula for the metric value, which is then used to set Alert thresholds.

### Alerting Conditions

- **Thresholds**: Set thresholds for warning and critical alert states, specifying whether the alert should trigger above or below these thresholds. Advanced settings allow for custom formulas.
- **Recovery Thresholds**: Set thresholds for downgrading the alert from critical to warning or from warning to clear.
- **Thresholds**: Set thresholds for warning and critical Alert states, specifying whether the Alert should trigger above or below these thresholds. Advanced settings allow for custom formulas.
- **Recovery Thresholds**: Set thresholds for downgrading the Alert from critical to warning or from warning to clear.
- **Check Interval**: Define how frequently the health check should run.
- **Delay Notifications**: Manage notification delays for alert escalations or de-escalations.
- **Delay Notifications**: Manage notification delays for Alert escalations or de-escalations.
- **Agent Specific Options**: Options exclusive to the Netdata Agent, like repeat notification frequencies and notification recipients.
- **Custom Exec Script**: Define custom scripts to execute when an alert triggers.
- **Custom Exec Script**: Define custom scripts to execute when an Alert triggers.

### Alert Name, Description and Summary Section
### Alert Name, Description, and Summary Section

- **Alert Template Name**: Provide a unique name for the alert.
- **Alert Template Description**: Offer a brief explanation of what the alert monitors.
- **Alert Summary**: Create a custom title for alert notifications, which can be integrated with centralized notification systems available [here](https://learn.netdata.cloud/docs/alerts-&-notifications/notifications/centralized-cloud-notifications).
- **Alert Template Name**: Provide a unique name for the Alert.
- **Alert Template Description**: Offer a brief explanation of what the Alert
Original file line number Diff line number Diff line change
@@ -1,24 +1,26 @@
# Centralized Cloud Notifications Reference

Netdata Cloud sends alert notifications for nodes in warning, critical, or unreachable states, ensuring alerts are managed centrally and efficiently.
Netdata Cloud sends Alert notifications for nodes in warning, critical, or unreachable states, ensuring Alerts are managed centrally and efficiently.

## Benefits of Centralized Notifications

- Consolidate health status views across all infrastructure in one place.
- Set up and [manage your alert notifications easily](/docs/alerts-and-notifications/notifications/centralized-cloud-notifications/manage-notification-methods.md).
- Set up and [manage your Alert notifications easily](/docs/alerts-and-notifications/notifications/centralized-cloud-notifications/manage-notification-methods.md).
- Expedite troubleshooting with tools like [Metric Correlations](/docs/metric-correlations.md) and the [Anomaly Advisor](/docs/dashboards-and-charts/anomaly-advisor-tab.md).

To avoid notification overload, **flood protection** is triggered when a node frequently disconnects or sends excessive alerts, highlighting potential issues.
> **Note**
>
> To avoid notification overload, **flood protection** is triggered when a node frequently disconnects or sends excessive Alerts, highlighting potential issues.
Admins must [enable alert notifications](/docs/alerts-and-notifications/notifications/centralized-cloud-notifications/manage-notification-methods.md#manage-space-notification-settings) for their Space(s). All users can then customize their notification preferences through their [account menu](/docs/alerts-and-notifications/notifications/centralized-cloud-notifications/manage-notification-methods.md#manage-user-notification-settings).
Administrators must [enable Alert notifications](/docs/alerts-and-notifications/notifications/centralized-cloud-notifications/manage-notification-methods.md#manage-space-notification-settings) for their Space(s). All users can then customize their notification preferences through their [account menu](/docs/alerts-and-notifications/notifications/centralized-cloud-notifications/manage-notification-methods.md#manage-user-notification-settings).

> **Note**
>
> Centralized alerts in Netdata Cloud are separate from the [Netdata Agent](/docs/alerts-and-notifications/notifications/README.md) notifications. Alerts must be [configured individually](/src/health/REFERENCE.md) on each node.
> Centralized Alerts in Netdata Cloud are separate from the [Netdata Agent](/docs/alerts-and-notifications/notifications/README.md) notifications. Agent Alerts must be [configured individually](/src/health/REFERENCE.md) on each node.
## Alert Notifications

Notifications can be sent via email or through third-party services like PagerDuty or Slack. Admins manage notification settings for the entire Space, while individual users can personalize settings in their profile.
Notifications can be sent via email or through third-party services like PagerDuty or Slack. Administrators can manage notification settings for the entire Space, while individual users can personalize settings in their profile.

### Service Level

Expand All @@ -28,7 +30,7 @@ Notifications are sent to user-specific destinations, such as email, which are m

#### System

These notifications go to general targets like a Slack channel, with admins setting rules for notification targets based on workspace or alert level.
These notifications go to general targets like a Slack channel, with administrators setting rules for notification targets based on workspace or Alert level.

### Service Classification

Expand All @@ -42,11 +44,11 @@ Exclusive to [paid plans](/docs/netdata-cloud/netdata-subscription-plans.md), in

## Alert Notification Silencing Rules

Netdata Cloud offers a silencing rule engine to mute alert notifications based on specific conditions related to nodes or alert types. Learn how to manage these settings [here](/docs/alerts-and-notifications/notifications/centralized-cloud-notifications/manage-alert-notification-silencing-rules.md).
Netdata Cloud offers a silencing rule engine to mute Alert notifications based on specific conditions related to nodes or Alert types. Learn how to manage these settings [here](/docs/alerts-and-notifications/notifications/centralized-cloud-notifications/manage-alert-notification-silencing-rules.md).

## Flood Protection

If a node repeatedly changes state or fires alerts, flood protection limits notifications to prevent overload. You can still access node details through Netdata Cloud or directly via the local Agent dashboard.
If a node repeatedly changes state or raises Alerts, flood protection limits notifications to prevent overload. You can still access node details through Netdata Cloud or directly via the local Agent dashboard.

## Anatomy of an Email Alert Notification

Expand All @@ -55,13 +57,13 @@ Email notifications provide comprehensive details:
- The Space's name
- The node's name
- Alert status: critical, warning, cleared
- Previous alert status
- Time at which the alert triggered
- Chart context that triggered the alert
- Name and information about the triggered alert
- Previous Alert status
- Time at which the Alert triggered
- Chart context that triggered the Alert
- Name and information about the triggered Alert
- Alert value
- Total number of warning and critical alerts on that node
- Threshold for triggering the given alert state
- Total number of warning and critical Alerts on that node
- Threshold for triggering the given Alert state
- Calculation or database lookups that Netdata uses to compute the value
- Source of the alert, including which file you can edit to configure this alert on an individual node
- Source of the Alert, including which file you can edit to configure this Alert on an individual node
- Direct link to the node’s chart in Cloud dashboards.
Loading

0 comments on commit 0c8277f

Please sign in to comment.