diff --git a/advocacy_docs/supported-open-source/pgbackrest/06-use_case_1.mdx b/advocacy_docs/supported-open-source/pgbackrest/06-use_case_1.mdx index 905641e2c6d..ffe8d85dd55 100644 --- a/advocacy_docs/supported-open-source/pgbackrest/06-use_case_1.mdx +++ b/advocacy_docs/supported-open-source/pgbackrest/06-use_case_1.mdx @@ -61,6 +61,10 @@ repo1-s3-key-type=auto repo1-s3-role=authrole ``` +To automatically retrieve web identity credentials, set the `repo1-s3-key-type=web-id` option and define the `AWS_ROLE_ARN` and `AWS_WEB_IDENTITY_TOKEN_FILE` environment variables. + +Set the `repo1-s3-kms-key-id` option to enable server-side encryption using the specified AWS key management service key. + ##### Azure-compatible Object Stores ```ini @@ -210,6 +214,7 @@ See the **Quick start** [backups](03-quick_start#backups) and [restore](03-quick [`repo-s3-key-secret`](https://pgbackrest.org/configuration.html#section-repository/option-repo-s3-key-secret) [`repo-s3-key-type`](https://pgbackrest.org/configuration.html#section-repository/option-repo-s3-key-type) [`repo-s3-key`](https://pgbackrest.org/configuration.html#section-repository/option-repo-s3-key) +[`repo-s3-kms-key-id`](https://pgbackrest.org/configuration.html#section-repository/option-repo-s3-kms-key-id) [`repo-s3-region`](https://pgbackrest.org/configuration.html#section-repository/option-repo-s3-region) [`repo-s3-role`](https://pgbackrest.org/configuration.html#section-repository/option-repo-s3-role) [`repo-type`](https://pgbackrest.org/configuration.html#section-repository/option-repo-type) diff --git a/advocacy_docs/supported-open-source/pgbackrest/09-tls_server.mdx b/advocacy_docs/supported-open-source/pgbackrest/09-tls_server.mdx new file mode 100644 index 00000000000..89181b89d13 --- /dev/null +++ b/advocacy_docs/supported-open-source/pgbackrest/09-tls_server.mdx @@ -0,0 +1,424 @@ +--- +title: 'TLS connections' +description: "TLS Server Support Feature" +--- + +### Description + +The TLS server is an alternative to using SSH for protocol connections to remote hosts. +It is supported since the [2.37](https://github.com/pgbackrest/pgbackrest/releases/tag/release%2F2.37) release. + +In this section, we will examine a configuration used to run pgBackRest from a dedicated repository host like described in the [Use Case 2](07-use_case_2), but using TLS rather than SSH connections. + +We will set up a **demo** cluster (using **EDB Postgres Advanced Server** version 14 on Rocky Linux 8) where the repository host will be called **backup-srv** and the 3 Postgres nodes in _Streaming Replication_: **pg1-srv**, **pg2-srv**, **pg3-srv**. + + +### Create a Dedicated User on the Repository Host + +The `pgbackrest` user is created to own the backups and archives repository. Any user can own the repository, but it is recommended not to use `postgres` or `enterprisedb` to avoid confusion. + +Create the user and the repository location on **backup-srv**: + +```bash +$ sudo groupadd pgbackrest +$ sudo adduser -g pgbackrest -n pgbackrest +$ sudo chown pgbackrest: /var/log/pgbackrest +``` + +The repository should be reachable from the backup server. It can be located on the various supported types described in the [first use case](06-use_case_1#global-section). + +For example, let us create `/backup_space` to store our backups and archives locally: + +```bash +$ sudo mkdir /backup_space +$ sudo chown pgbackrest: /backup_space +``` + + +### Certificates generation + +The TLS server implementation relies on certificates. + +In this example, we will: + * create our own root certificate authority (CA), + * generate keys and certificates for the Postgres nodes and for the backup server. + +```bash +$ mkdir certs && cd certs +$ openssl req -new -x509 -days 365 -nodes -out ca.crt -keyout ca.key -subj "/CN=root-ca" +$ openssl req -new -nodes -out backup-srv.csr -keyout backup-srv.key -subj "/CN=backup-srv" +$ openssl req -new -nodes -out pg1-srv.csr -keyout pg1-srv.key -subj "/CN=pg1-srv" +$ openssl req -new -nodes -out pg2-srv.csr -keyout pg2-srv.key -subj "/CN=pg2-srv" +$ openssl req -new -nodes -out pg3-srv.csr -keyout pg3-srv.key -subj "/CN=pg3-srv" +$ openssl x509 -req -in backup-srv.csr -days 365 -CA ca.crt -CAkey ca.key -CAcreateserial -out backup-srv.crt +$ openssl x509 -req -in pg1-srv.csr -days 365 -CA ca.crt -CAkey ca.key -CAcreateserial -out pg1-srv.crt +$ openssl x509 -req -in pg2-srv.csr -days 365 -CA ca.crt -CAkey ca.key -CAcreateserial -out pg2-srv.crt +$ openssl x509 -req -in pg3-srv.csr -days 365 -CA ca.crt -CAkey ca.key -CAcreateserial -out pg3-srv.crt +$ rm *.csr +``` + +Then, you have to deploy it on each server (`ca.crt` + `server_name.crt` + `server_name.key`). + +Here is an example using a shared folder between the hosts: + +```bash +# on backup-srv +$ sudo -u pgbackrest mkdir ~pgbackrest/certs +$ sudo cp `hostname`.* ~pgbackrest/certs +$ sudo cp ca.crt ~pgbackrest/certs +$ sudo chown -R pgbackrest: ~pgbackrest/certs +$ sudo chmod -R og-rwx ~pgbackrest/certs + +# on pg1-srv, pg2-srv and pg3-srv +$ sudo -u enterprisedb mkdir ~enterprisedb/certs +$ sudo cp `hostname`.* ~enterprisedb/certs +$ sudo cp ca.crt ~enterprisedb/certs +$ sudo chown -R enterprisedb: ~enterprisedb/certs +$ sudo chmod -R og-rwx ~enterprisedb/certs +``` + + +### Configuration + +We will now prepare the configuration for our stanza called `demo`. + +Update the **backup-srv** pgBackRest configuration file: + +```ini +# /etc/pgbackrest.conf +[global] +# repo details +repo1-path=/backup_space +repo1-retention-full=2 + +# general options +process-max=2 +log-level-console=info +log-level-file=debug +start-fast=y +delta=y + +# tls server options +tls-server-address=* +tls-server-cert-file=/home/pgbackrest/certs/backup-srv.crt +tls-server-key-file=/home/pgbackrest/certs/backup-srv.key +tls-server-ca-file=/home/pgbackrest/certs/ca.crt +tls-server-auth=pg1-srv=demo +tls-server-auth=pg2-srv=demo +tls-server-auth=pg3-srv=demo + +[demo] +pg1-host=pg1-srv +pg1-host-user=enterprisedb +pg1-path=/var/lib/edb/as14/data +pg1-host-type=tls +pg1-host-cert-file=certs/backup-srv.crt +pg1-host-key-file=certs/backup-srv.key +pg1-host-ca-file=certs/ca.crt +``` + +Update the **pg1-srv**, **pg2-srv** and **pg3-srv** pgBackRest configuration file: + +```ini +# /etc/pgbackrest.conf +[global] +repo1-host=backup-srv +repo1-host-user=pgbackrest +repo1-host-type=tls +repo1-host-cert-file=/var/lib/edb/certs/pg1-srv.crt +repo1-host-key-file=/var/lib/edb/certs/pg1-srv.key +repo1-host-ca-file=/var/lib/edb/certs/ca.crt + +# general options +process-max=2 +log-level-console=info +log-level-file=debug + +# tls server options +tls-server-address=* +tls-server-cert-file=/var/lib/edb/certs/pg1-srv.crt +tls-server-key-file=/var/lib/edb/certs/pg1-srv.key +tls-server-ca-file=/var/lib/edb/certs/ca.crt +tls-server-auth=backup-srv=demo + +[demo] +pg1-path=/var/lib/edb/as14/data +pg1-user=enterprisedb +pg1-port=5444 +``` + +( Remark: obviously, adjust _pg1-srv_ by _pg2-srv_ on **pg2-srv** and _pg3-srv_ on **pg3-srv**. ) + +Clients are authorized on the server by verifying their certificate and checking the certificate CN (_Common Name_) against a list on the server configured with the `tls-server-auth`. +A CN can be authorized for as many stanzas as needed by repeating the `tls-server-auth` option, or for all stanzas by specifying `tls-server-auth=*`. + +The idea is to have a TLS server running on each host to serve the request coming from the other. +In example, the `backup` command running on the repository host will act as client for the TLS server running on the Postgres node. +The `archive-push` command running on the Postgres node will act as client for the TLS server running on the repository host. + +The [`server`](https://pgbackrest.org/command.html#command-server) command can be used to start the TLS server and will run until terminated by the SIGINT signal (Control+C). + +If not done by the PGDG package already, create a service file on the **backup-srv** server: + +```ini +# /etc/systemd/system/pgbackrest.service +[Unit] +Description=pgBackRest Server +After=network.target +StartLimitIntervalSec=0 + +[Service] +Type=simple +User=pgbackrest +Restart=always +RestartSec=1 +ExecStart=/usr/bin/pgbackrest server +ExecReload=kill -HUP $MAINPID + +[Install] +WantedBy=multi-user.target +``` + +Start the server and check it's running: + +```bash +$ sudo systemctl daemon-reload +$ sudo systemctl enable pgbackrest +$ sudo systemctl start pgbackrest +$ pgbackrest server-ping +``` + +The `server-ping` command serves as an aliveness check only since no authentication is attempted. + +Don't forget to terminate/restart the process when the `tls-*` configuration is changed. + +Now, apply the same steps (service configuration and start) on the on **pg1-srv**, **pg2-srv** and **pg3-srv** using this configuration: + +```ini +# /etc/systemd/system/pgbackrest.service +[Unit] +Description=pgBackRest Server +After=network.target +StartLimitIntervalSec=0 + +[Service] +Type=simple +User=enterprisedb +Restart=always +RestartSec=1 +ExecStart=/usr/bin/pgbackrest server +ExecReload=kill -HUP $MAINPID + +[Install] +WantedBy=multi-user.target +``` + +Now, let us configure Postgres archiving in the `postgresql.conf` file (on **pg1-srv**): + +``` +listen_addresses = '*' +archive_mode = on +archive_command = 'pgbackrest --stanza=demo archive-push %p' +``` + +The Postgres instance must be restarted after making these changes and before performing a backup. + +Let us finally create the stanza and check the configuration on **backup-srv**: + +```bash +$ sudo -iu pgbackrest pgbackrest --stanza=demo stanza-create +... +P00 INFO: stanza-create command end: completed successfully + +$ sudo -iu pgbackrest pgbackrest --stanza=demo check +... +P00 INFO: check command end: completed successfully +``` + + +### Perform a backup + +If everything is working correctly, we can now take our first backup: + +```bash +$ sudo -iu pgbackrest pgbackrest --stanza=demo --type=full backup +P00 INFO: backup command begin 2.38: .. +P00 INFO: execute non-exclusive pg_start_backup(): backup begins after the requested + immediate checkpoint completes +P00 INFO: backup start archive = 000000010000000000000004, lsn = 0/4000060 +P00 INFO: check archive for prior segment 000000010000000000000003 +P00 INFO: execute non-exclusive pg_stop_backup() and wait for all WAL segments to archive +P00 INFO: backup stop archive = 000000010000000000000004, lsn = 0/4000138 +P00 INFO: check archive for segment(s) 000000010000000000000004:000000010000000000000004 +P00 INFO: new backup label = 20220308-143110F +P00 INFO: full backup size = 56.5MB, file total = 1691 +P00 INFO: backup command end: completed successfully +``` + +The `info` command can then be executed from any server where pgBackRest is correctly configured: + +```bash +# From the Postgres node +$ sudo -iu enterprisedb pgbackrest --stanza=demo info +stanza: demo + status: ok + cipher: none + + db (current) + wal archive min/max (14): 000000010000000000000002/000000010000000000000004 + + full backup: 20220308-143110F + timestamp start/stop: 2022-03-08 14:31:10 / 2022-03-08 14:31:20 + wal start/stop: 000000010000000000000004 / 000000010000000000000004 + database size: 56.5MB, database backup size: 56.5MB + repo1: backup set size: 9.1MB, backup size: 9.1MB + +# From the backup server +$ sudo -iu pgbackrest pgbackrest --stanza=demo info +stanza: demo + status: ok + cipher: none + + db (current) + wal archive min/max (14): 000000010000000000000002/000000010000000000000004 + + full backup: 20220308-143110F + timestamp start/stop: 2022-03-08 14:31:10 / 2022-03-08 14:31:20 + wal start/stop: 000000010000000000000004 / 000000010000000000000004 + database size: 56.5MB, database backup size: 56.5MB + repo1: backup set size: 9.1MB, backup size: 9.1MB +``` + + +### Prepare the servers for Streaming Replication + +On **pg1-srv**, create a specific user for the replication: + +```bash +$ sudo -iu enterprisedb psql -d postgres +postgres=# CREATE ROLE replic_user WITH LOGIN REPLICATION PASSWORD 'mypwd'; +``` + +Configure `pg_hba.conf`: + +``` +host replication replic_user pg2-srv scram-sha-256 +host replication replic_user pg3-srv scram-sha-256 +``` + +Reload configuration: + +```bash +$ sudo systemctl reload edb-as-14.service +``` + +Configure `~enterprisedb/.pgpass` on **pg2-srv** and **pg3-srv**: + +```bash +$ sudo -iu enterprisedb +$ echo "*:*:replication:replic_user:mypwd" >> ~enterprisedb/.pgpass +$ chown enterprisedb: ~enterprisedb/.pgpass +$ chmod 0600 ~enterprisedb/.pgpass +``` + + +### Setup the standby servers + +Adjust `/etc/pgbackrest.conf` on **pg2-srv** and **pg3-srv** to add the [recovery-option](https://pgbackrest.org/configuration.html#section-restore/option-recovery-option). +The idea is to automatically configure the _Streaming Replication_ connection string with the restore command. + +```ini +recovery-option=primary_conninfo=host=pg1-srv user=replic_user +``` + +Then, make sure the configuration is correct by executing the `info` command. It should print the same output as above. + +Restore the backup taken from the **pg1-srv** server on **pg2-srv** and **pg3-srv**: + +```bash +$ sudo -iu enterprisedb pgbackrest --stanza=demo --type=standby restore +P00 INFO: restore command begin 2.38: ... +P00 INFO: repo1: restore backup set 20220308-143110F, recovery will start at ... +P00 INFO: write updated /var/lib/edb/as14/data/postgresql.auto.conf +P00 INFO: restore global/pg_control +(performed last to ensure aborted restores cannot be started) +P00 INFO: restore size = 56.5MB, file total = 1691 +P00 INFO: restore command end: completed successfully +``` + +The restore will add extra information to the `postgresql.auto.conf` file: + +```ini +# Recovery settings generated by pgBackRest restore on ... +primary_conninfo = 'host=pg1-srv user=replic_user' +restore_command = 'pgbackrest --stanza=demo archive-get %f "%p"' +``` + +The `--type=standby` option creates the `standby.signal` needed for Postgres to start in standby mode. All we have to do now is to start the Postgres instances: + +```bash +$ sudo systemctl enable edb-as-14.service +$ sudo systemctl start edb-as-14.service +``` + +If the replication setup is correct, you should see those processes on the **pg1-srv** server: + +``` +$ sudo -u enterprisedb ps -o pid,cmd fx + PID CMD + 30594 /usr/edb/as14/bin/edb-postmaster -D /var/lib/edb/as14/data + ... + 31436 \_ postgres: walsender replic_user ... streaming 0/5001FB0 + 31439 \_ postgres: walsender replic_user ... streaming 0/5001FB0 + 30222 /usr/bin/pgbackrest server +``` + +We now have a 3-nodes cluster working with _Streaming Replication_ and archives recovery as safety net. + + +### Take backups from the standby servers + +Add the following settings to the pgBackRest configuration file on **backup-srv**, in the `[demo]` stanza section: + +```ini +pg2-host=pg2-srv +pg2-host-user=enterprisedb +pg2-path=/var/lib/edb/as14/data +pg2-host-type=tls +pg2-host-cert-file=certs/backup-srv.crt +pg2-host-key-file=certs/backup-srv.key +pg2-host-ca-file=certs/ca.crt +pg3-host=pg3-srv +pg3-host-user=enterprisedb +pg3-path=/var/lib/edb/as14/data +pg3-host-type=tls +pg3-host-cert-file=certs/backup-srv.crt +pg3-host-key-file=certs/backup-srv.key +pg3-host-ca-file=certs/ca.crt +backup-standby=y +``` + +Now, perform a backup fetching the data from the first standby server found in the configuration: + +```bash +$ sudo -iu pgbackrest pgbackrest --stanza=demo --type=full backup +P00 INFO: backup command begin 2.38: ... +P00 INFO: execute non-exclusive pg_start_backup(): backup begins after the requested + immediate checkpoint completes +P00 INFO: backup start archive = 000000010000000000000006, lsn = 0/6000028 +P00 INFO: wait for replay on the standby to reach 0/6000028 +P00 INFO: replay on the standby reached 0/6000028 +P00 INFO: check archive for prior segment 000000010000000000000005 +P00 INFO: execute non-exclusive pg_stop_backup() and wait for all WAL segments to archive +P00 INFO: backup stop archive = 000000010000000000000006, lsn = 0/6000138 +P00 INFO: check archive for segment(s) 000000010000000000000006:000000010000000000000006 +P00 INFO: new backup label = 20220308-145104F +P00 INFO: full backup size = 56.5MB, file total = 1691 +P00 INFO: backup command end: completed successfully +``` + + +### Conclusion + +The TLS server provides an alternative to SSH for remote operations. +This new feature will fit perfectly in a containerized world where it can significantly improve performance and user experience. diff --git a/advocacy_docs/supported-open-source/pgbackrest/99-faq.mdx b/advocacy_docs/supported-open-source/pgbackrest/99-faq.mdx index 97359d7402b..a51f03ce3ed 100644 --- a/advocacy_docs/supported-open-source/pgbackrest/99-faq.mdx +++ b/advocacy_docs/supported-open-source/pgbackrest/99-faq.mdx @@ -23,3 +23,11 @@ The databases that are not explicitly included with this option will be restored Yes. Since the 2.37 release, pgBackRest can use TLS with client certificates to enable communication between the hosts. A TLS server must then be configured and started on each host. See [Setup TLS[(https://pgbackrest.org/user-guide-rhel.html#repo-host/config) for more information. + +--- + +- How many CPU cores are needed on a dedicated remote backup server? + +Since compression/decompression is performed on the Postgres host, the repository host does only need to encrypt/decrypt SSH or TLS connections. Usually 1 repository core per 4 Postgres cores is a good starting point. + + diff --git a/advocacy_docs/supported-open-source/pgbackrest/index.mdx b/advocacy_docs/supported-open-source/pgbackrest/index.mdx index 39b85d0e4eb..c3d25cf114e 100644 --- a/advocacy_docs/supported-open-source/pgbackrest/index.mdx +++ b/advocacy_docs/supported-open-source/pgbackrest/index.mdx @@ -17,6 +17,6 @@ pgBackRest is an easy-to-use backup and restore tool that aims to enable your or !!! Note Looking for pgBackRest documentation? Head over to [http://www.pgbackrest.org](http://www.pgbackrest.org) -pgBackRest [v2.37](https://github.com/pgbackrest/pgbackrest/releases/tag/release/2.37) is the current stable release. Release notes are on the [Releases](http://www.pgbackrest.org/release.html) page. +pgBackRest [v2.38](https://github.com/pgbackrest/pgbackrest/releases/tag/release/2.38) is the current stable release. Release notes are on the [Releases](http://www.pgbackrest.org/release.html) page. EDB collaborates with the community on this open-source software. The packages are provided by the **PostgreSQL** community. All of the use cases shown in this document are fully tested and supported by **EDB Postgres Advanced Server**. diff --git a/product_docs/docs/biganimal/release/using_cluster/05_monitoring_and_logging/06_metrics.mdx b/product_docs/docs/biganimal/release/using_cluster/05_monitoring_and_logging/06_metrics.mdx index 43103daa219..db9afe6c487 100644 --- a/product_docs/docs/biganimal/release/using_cluster/05_monitoring_and_logging/06_metrics.mdx +++ b/product_docs/docs/biganimal/release/using_cluster/05_monitoring_and_logging/06_metrics.mdx @@ -33,7 +33,7 @@ specific to BigAnimal: | -------------------- | --------------------------------------------------------------------------------------------------------------- | | PostgresLogs_CL | Logs of the Customer clusters databases (all postgres related logs) | | PostgresAuditLogs_CL | Audit Logs of the Customer clusters databases, if enabled | -| InsightsMetrics | Metrics streams from BigAnimal Prometheus and Azure Monitor. BigAnimal metrics have `namespace == "prometheus"` | +| DpMetrics_CL | Metrics streams from BigAnimal Prometheus. | You can use the KQL Query editor in the Log Workspace view to compose queries over these tables. @@ -53,12 +53,12 @@ that originated the log message. ## Metrics overview BigAnimal collects a wide set of metrics about Postgres instances into the -`InsightsMetrics` log analytics table. Most of these metrics are acquired +`DpMetrics_CL` log analytics table. Most of these metrics are acquired directly from Postgres system tables, views, and functions. The Postgres documentation serves as the main reference for these metrics. You can use KQL to analyze time-series metrics, report latest samples of -metrics, and so on by querying the `InsightsMetrics` table. +metrics, and so on by querying the `DpMetrics_CL` table. Some data from Postgres monitoring system views, tables, and functions are transformed to be easier to consume in Prometheus metrics format. For example, @@ -76,8 +76,8 @@ are mapped to Azure metrics Dimensions vary depending on the individual metric and are documented separately for each group of related metrics. -The forwarded Prometheus metrics use structured JSON fields, particularly for -the `Tags` field. Effective use of them requires use of the +The forwarded Prometheus metrics use structured JSON fields, particularly the `Message` and `Message.labels` fields. +Effective use of them requires the use of the [`todynamic()`](https://docs.microsoft.com/en-us/azure/data-explorer/kusto/query/parsejsonfunction) function in queries. @@ -86,44 +86,42 @@ removed or renamed. Where feasible, an effort will be made not to change the meaning or type of existing metrics without also changing the metric name. At time of writing, all metrics forwarded from Prometheus are in the -`prometheus` namespace. This might change in a future release. +`DpMetrics_CL` table. This might change in a future release. Effective use of the available metrics requires an understanding of Azure time-series data, metrics dimensions, and of the tagging conventions used in the metrics streams. -### Metrics tags +### Metrics labels -All Postgres metrics share a common tagging scheme. Entries generally -have at least the following tags: +All Postgres metrics share a common labelling scheme. Entries generally +have at least the following labels: | Name | Description | | ---------------------------- | ------------------------------------------------------------- | -| address | IP address of the host the metric originated from | +| instance | IP address of the host the metric originated from | | postgresql | BigAnimal postgres cluster identifier, e.g., `p-abcdef012345` | | role | Postgres instance role, "primary" or "replica" | | datname | Postgres database name (where applicable) | -| pod_name | k8s pod name | -| hostName | AKS node host name | -| container.azm.ms/clusterName | AKS cluster name | +| pod | k8s pod name | +| aks_cluster_name | AKS cluster name | -When querying for tags, for best performance apply filters that don't -require inspection of tags (for example, filters by metric name) before tag-based filters. +When querying for labels, for best performance apply filters that don't +require inspection of labels (for example, filters by metric name) before label-based filters. -The `Tags` field of a metrics entry is a JSON-typed field that you can query -for individual values with `todynamic(Tags).keyname` in KQL. Some uses of values -might require explicit casts to another type, for example, `tostring(...)`. +The `labels` field of a metrics entry is a nested field under the JSON-typed `Message` field. +To query the field for individual values you JSON parse the `Message` field and dot +reference (`m.labels`) the labels property. Some uses of values might require explicit casts +to another type, for example, `tostring(...)`. Example usage: ``` -InsightsMetrics -| where Namespace == "prometheus" and Name startswith "cnp_" -| extend t = todynamic(Tags) -| where t.role == "primary" -| project postgres_cluster_id = tostring(t.postgresql), dbname = tostring(t.datname) -| where not (dbname has_any("template0", "template1")) -| distinct postgres_cluster_id, dbname +DpMetrics_CL +| where Message has "cnp_" +| extend m = todynamic(Message) +| where m.labels.role == "primary" +| project postgres_cluster_id = tostring(m.labels.postgresql), metric_name = m.name, metric_value = m.value ``` [comment1]: # "Generated content see upm-substrate repo config monitoring dir" @@ -566,7 +564,7 @@ See also: - [Kubernetes cluster metrics](https://kubernetes.io/docs/concepts/cluster-administration/system-metrics/). -The cloud platform can supply additional streams of metrics +The cloud platform can supply additional streams of metrics directly to your metrics, analytics, and dashboarding endpoint. ### Dive deeper