Skip to content

Commit

Permalink
docs: Edits in the main Netdata Cloud readme (netdata#19139)
Browse files Browse the repository at this point in the history 
* edits in the readme

* more structured

---------

Co-authored-by: ilyam8 <[email protected]>
  • Loading branch information
@Ancairon @ilyam8
Ancairon and ilyam8 authored Dec 6, 2024
1 parent cb731f3 commit 0584ecc
Show file tree
Hide file tree
Showing 2 changed files with 55 additions and 80 deletions.
133 changes: 54 additions & 79 deletions docs/netdata-cloud/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,17 +1,24 @@
# Netdata Cloud

Netdata Cloud is a service that complements Netdata installations. It is a key part in achieving optimal cost structure for large scale observability.
Netdata Cloud is a powerful service that transforms standalone Netdata Agent installations into a unified, scalable observability solution. It achieves this without centralizing metric storage, ensuring optimal performance and cost-effectiveness even at enterprise scale.

Technically, Netdata Cloud is a thin control plane that allows the Netdata ecosystem to be a virtually unlimited scalable and flexible observability pipeline. With Netdata Cloud, this observability pipeline can span multiple teams, cloud providers, data centers and services, while remaining a uniform and highly integrated infrastructure, providing real-time and high-fidelity insights.
By serving as a lightweight control plane, Netdata Cloud provides:

- Seamless coordination across multiple teams and environments
- Unified visibility across cloud providers and data centers
- Real-time, high-fidelity monitoring at any scale
- Flexible observability pipelines that grow with your infrastructure

```mermaid
flowchart TB
NC("<b>☁️ Netdata Cloud</b>
access from anywhere,
horizontal scalability,
role based access,
custom dashboards,
central notifications")
Horizontal scalability,
Role based access,
Access from anywhere,
Central dispatch of Alert notifications
Custom Dashboards,
Advanced customization,
")
Users[["<b>✨ Unified Dashboards</b>
across the infrastructure,
multi-cloud, hybrid-cloud"]]
Expand All @@ -37,103 +44,71 @@ flowchart TB
NC <-->|secure connection| Agents
```

Netdata Cloud provides the following features, on top of what the Netdata Agents already provide:

1. **Horizontal scalability**: Netdata Cloud allows scaling the observability infrastructure horizontally, by adding more independent Netdata Parents and Children. It can aggregate such, otherwise independent, observability islands into one uniform and integrated infrastructure.

Netdata Cloud is a fundamental component for achieving an optimal cost structure and flexibility, in structuring observability the way that is best suited for each case.

2. **Role Based Access Control (RBAC)**: Netdata Cloud has all the mechanisms for user-management and access control. It allows assigning all users a role, segmenting the infrastructure into rooms, and associating Rooms with roles and users.

3. **Access from anywhere**: Netdata Agents are installed on-prem, and this is where all your data is always stored. Netdata Cloud allows querying all the Netdata Agents (Standalone, Children and Parents) in real-time when dashboards are accessed via Netdata Cloud.

This enables a much simpler access control, eliminating the complexities of setting up VPNs to access observability, and the bandwidth costs for centralizing all metrics to one place.

4. **Central dispatch of alert notifications**: Netdata Cloud allows controlling the dispatch of alert notifications centrally. By default, all Netdata Agents (Standalone, Children and Parents) send their own notifications. This becomes increasingly complex as the infrastructure grows. So, Netdata Cloud steps in to simplify this process and provide central control of all notifications.

Netdata Cloud also enables the use of the **Netdata Mobile App** offering mobile push notifications for all users in commercial plans.

5. **Custom Dashboards**: Netdata Cloud enables the creation, storage and sharing custom dashboards.

Custom dashboards are created directly from the UI, without the need for learning a query language. Netdata Cloud provides all the APIs to the Netdata dashboards to store, browse and retrieve custom dashboards created by all users.

6. **Advanced Customization**: Netdata Cloud provides all the APIs for the dashboard to have different default settings per space, per Room and per user, allowing administrators and users to customize the Netdata dashboards and charts the way they see fit.
Netdata Cloud provides the following features, on top of what the Agents already provide:

## Data Exposed to Netdata Cloud
| Feature | Description |
|:------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| **Horizontal Scalability** | • Scale your observability infrastructure effortlessly<br/>• Add [Parents and Children](/docs/observability-centralization-points/README.md) as needed<br/>• Manage all nodes from a single [Space](/docs/netdata-cloud/organize-your-infrastructure-invite-your-team.md#spaces) |
| [**Role-Based Access Control (RBAC)**](/docs/netdata-cloud/authentication-and-authorization/role-based-access-model.md) | • Fine-grained access management<br/>• Control team member privileges across your Space<br/>• Secure, role-appropriate access to monitoring data |
| **Global Remote Access** | • Access your monitoring from anywhere<br/>• No VPN configuration required<br/>• Secure access to local dashboards while data stays on premises |
| **Centralized Alert Management** | • Unified alert dispatch from a central location<br/>• Cloud-specific alerts and monitoring<br/>• Mobile push notifications via [Netdata Mobile App](/integrations/cloud-notifications/integrations/netdata_mobile_app.md) (paid plans) |
| [**Custom Dashboards**](/docs/dashboards-and-charts/dashboards-tab.md) | • Create and save custom views<br/>• Share dashboards across teams<br/>• Build focused views for specific needs |
| **Personal Customization** | • Individual user visualization preferences<br/>• Tailored dashboard experiences<br/>• Flexible viewing options for different roles |

Netdata Cloud serves as a thin layer on top of Netdata Agents. It does not receive the samples collected, or the logs Netdata Agents maintain.
## Stored metadata

Netdata's design deliberately avoids centralizing raw metrics and logs. This prevents the common constraints of traditional observability solutions: reduced metric resolution, forced data filtering, and higher costs.
Netdata Cloud doesn't store your metrics or logs.

Instead, Netdata Cloud only stores metadata such as:
**What Netdata Cloud Does Store**:

- Node information and labels
- Metric names, labels, and retention periods
- Active collectors and modules
- Active collectors
- Alert configurations and state changes

This information is a small fraction of the total information maintained by Netdata Agents, allowing Netdata Cloud to remain high-resolution, high-fidelity and real-time, while being able to:

- dispatch alerts centrally for all alert transitions.
- know which Netdata Agents to query when users view the dashboards.

Metric samples and logs are transferred via Netdata Cloud to your Web Browser, only when you view them via Netdata Cloud. And even then, Netdata Cloud does not store this information. It only aggregates the responses of multiple Netdata Agents to a single response for your web browser to visualize.

## High-Availability

You can subscribe to Netdata Cloud updates at the [Netdata Cloud Status](https://status.netdata.cloud/) page.

Netdata Cloud is a highly available, auto-scalable solution; however, being a monitoring solution, we need to ensure dashboards are accessible during crisis.

Netdata Agents provide the same dashboard Netdata Cloud provides, with the following limitations:

1. Netdata Agents (Children and Parents) dashboards are limited to their databases, while on Netdata Cloud the dashboard presents the entire infrastructure, from all Netdata Agents connected to it.

2. When you are not logged in or the Agent is not connected to Netdata Cloud, certain features of the Netdata Agent dashboard will not be available.

When you are logged in and the Agent is connected to Netdata Cloud, the dashboard has the same functionality as Netdata Cloud.

To ensure dashboard high availability, Netdata Agent dashboards are available by directly accessing them, even when the connectivity between Children and Parents or Netdata Cloud faces issues. This allows the use of the individual Netdata Agents' dashboards during crisis, at different levels of aggregation.

## Fidelity and Insights
**How Data Flows**:

Netdata Cloud queries Netdata Agents, so it provides exactly the same fidelity and insights Netdata Agents provide. Dashboards have the same resolution, the same number of metrics, exactly the same data.
1. Metadata (listed above) is synchronized between Agents and Cloud
2. Metric data and logs remain stored locally on your Agents
3. When you view dashboards:
- Data is transferred directly from Agents to your browser via Cloud
- Cloud aggregates responses from multiple Agents into a unified view
- No metric or log data is stored in Cloud during this process

## Performance
## Fidelity and Resolution

The Netdata Agent and Netdata Cloud have similar query performance, but there are additional network latencies involved when the dashboards are viewed via Netdata Cloud.
Netdata Cloud delivers the exact same high-quality data that Agents collect. When you view dashboards through the Cloud:

Accessing Netdata Agents on the same LAN has marginal network latency, and their response time is only affected by the queries. However, accessing the same Netdata Agents via Netdata Cloud has a bigger network round-trip time that looks like this:
- No reduction in metric resolution
- No loss of data fidelity
- No compromise in data granularity

1. Your web browser makes a request to Netdata Cloud.
2. Netdata Cloud sends the request to your Netdata Agents. If multiple Netdata Agents are involved, they are queried in parallel.
3. Netdata Cloud receives their responses and aggregates them into a single response.
4. Netdata Cloud replies to your web browser.
The data you see is identical to what you would get by accessing Agents directly.

If you are sitting on the same LAN as the Netdata Agents, the latency will be two times the round-trip network latency between this LAN and Netdata Cloud.
## FAQ

With multiple Netdata Agents, queries are faster than single-point monitoring solutions. Netdata Cloud distributes each query across multiple Agents, where each Agent processes only a portion of the query. This distributed approach uses your infrastructure's combined processing power, delivering superior performance compared to centralized solutions.
<details><summary>details</summary>

## Does Netdata Cloud require Observability Centralization Points?
### Does the Cloud require Observability Centralization Points?

No. Any or all Netdata Agents can be connected to Netdata Cloud.
No. Any or all Agents can be connected directly to the Cloud.

We recommend to create [observability centralization points](/docs/observability-centralization-points/README.md), as required for operational efficiency (ephemeral nodes, teams or services isolation, central control of alerts, production systems performance), security policies (internet isolation), or cost optimization (use existing capacities before allocating new ones).
We recommend creating [Observability Centralization Points](/docs/observability-centralization-points/README.md), as required for operational efficiency (ephemeral nodes, teams or services isolation, central control of alerts, production systems performance), security policies (internet isolation), or cost optimization (use existing capacities before allocating new ones).

We suggest reviewing the [Best Practices for Observability Centralization Points](/docs/observability-centralization-points/best-practices.md).
### When I have Parents, do I need to connect the Children to the Cloud too?

## When I have Netdata Parents, do I need to connect Netdata Children to Netdata Cloud too?
No, it is not necessary, but it provides high availability.

No, it is unnecessary, but it provides high availability.
When Parents are connected to the Cloud, all their Children are available via them.

When Netdata Parents are connected to Netdata Cloud, all their Netdata Children are available via these Parents.
When multiple Parents store data from the same Children (in clusters or multi-level hierarchies), the Cloud queries each unique node once through a single available Parent.

When multiple Parent nodes store data from the same Child nodes (in clusters or multi-level hierarchies), Netdata Cloud queries each unique node once through a single available Parent.
The Cloud prefers:

Netdata Cloud prefers:
- The most distant (from the Child) Parent that is available when doing metrics visualization queries (since usually these Parents have been added for this purpose).

- The most distant (from the Child) Parent is available when doing metrics visualization queries (since usually these Parents have been added for this purpose).
- The closest (to the Child) Parent available for [Top Monitoring](/docs/top-monitoring-netdata-functions.md). The streaming protocol of Parents and Children is able to forward such requests to the leaf child, via the Parents, to respond with live and accurate data.

- The closest (to the Child) Parent available for [Top Monitoring](/docs/top-monitoring-netdata-functions.md) (since top-monitoring provides live data, like the processes running, the list of sockets open, etc.). The streaming protocol of Netdata Parents and Children is able to forward such requests to the right child, via the Parents, to respond with live and accurate data.
Children may be connected to the Cloud for high-availability, in cases where their Parents become unreachable.

Netdata Children may be connected to Netdata Cloud for high-availability, in case the Netdata Parents are unreachable.
</details>
2 changes: 1 addition & 1 deletion docs/netdata-cloud/authentication-and-authorization/role-based-access-model.md
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Role-Based Access model
# Role-Based Access model (RBAC)

Netdata Cloud's role-based-access mechanism allows you to control what functionalities in the app users can access. Each user can be assigned only one role, which fully specifies all the capabilities they are afforded.

Expand Down

0 comments on commit 0584ecc

Please sign in to comment.