feat(cluster): support db import, replication, replicationSlots, volu…

…meSnapshots, multiple poolers, expend monitoring options

Signed-off-by: Dmitriy Alekseev <[email protected]>

charts/cluster/README.md.gotmpl

@@ -83,10 +83,10 @@ The chart has three modes of operation. These are configured via the `mode` para
 
 CNPG implements disaster recovery via [Barman](https://pgbarman.org/). The following section configures the barman object
 store where backups will be stored. Barman performs backups of the cluster filesystem base backup and WALs. Both are
-stored in the specified location. The backup provider is configured via the `backups.provider` parameter. The following
-providers are supported:
+stored in the specified location. The backup provider is configured via the `backups.objectStorage.provider` parameter.
+The following providers are supported:
 
-* S3 or S3-compatible stores, like MinIO
+* S3 or S3-compatible stores, like MinIO or Ceph Rados
 * Microsoft Azure Blob Storage
 * Google Cloud Storage
 
@@ -102,8 +102,8 @@ backups:
 ```
 
 Each backup adapter takes it's own set of parameters, listed in the [Configuration options](#Configuration-options) section
-below. Refer to the table for the full list of parameters and place the configuration under the appropriate key: `backup.s3`,
-`backup.azure`, or `backup.google`.
+below. Refer to the table for the full list of parameters and place the configuration under the appropriate key:
+`backups.objectStorage.providerSettings.s3`, `backups.objectStorage.providerSettings.azure` or `backups.objectStorage.providerSettings.google`.
 
 
 Recovery

charts/cluster/docs/Getting Started.md

@@ -1,6 +1,6 @@
 # Getting Started
 
-The CNPG cluster chart follows a convention over configuration approach. This means that the chart will create a reasonable 
+The CNPG cluster chart follows a convention over configuration approach. This means that the chart will create a reasonable
 CNPG setup with sensible defaults. However, you can override these defaults to create a more customized setup. Note that
 you still need to configure backups and monitoring separately. The chart will not install a Prometheus stack for you.
 
@@ -44,14 +44,14 @@ are likely looking for the `standalone` option.
 
 ### Backup configuration
 
-Most importantly you should configure your backup storage. 
+Most importantly you should configure your backup storage.
 
 CNPG implements disaster recovery via [Barman](https://pgbarman.org/). The following section configures the barman object
 store where backups will be stored. Barman performs backups of the cluster filesystem base backup and WALs. Both are
-stored in the specified location. The backup provider is configured via the `backups.provider` parameter. The following
-providers are supported:
+stored in the specified location. The backup provider is configured via the `backups.objectStorage.provider` parameter.
+The following providers are supported:
 
-* S3 or S3-compatible stores, like MinIO
+* S3 or S3-compatible stores, like MinIO or Ceph Rados
 * Microsoft Azure Blob Storage
 * Google Cloud Storage
 
@@ -67,16 +67,16 @@ Additionally you can specify the following parameters:
   ```
 
 Each backup adapter takes it's own set of parameters, listed in the [Configuration options](../README.md#Configuration-options) section
-below. Refer to the table for the full list of parameters and place the configuration under the appropriate key: `backup.s3`,
-`backup.azure`, or `backup.google`.
+below. Refer to the table for the full list of parameters and place the configuration under the appropriate key:
+`backups.objectStorage.providerSettings.s3`, `backups.objectStorage.providerSettings.azure` or `backups.objectStorage.providerSettings.google`.
 
 ### Cluster configuration
 
 There are several important cluster options. Here are the most important ones:
 
 `cluster.instances` - The number of instances in the cluster. Defaults to `1`, but you should set this to `3` for production.
 `cluster.imageName` - This allows you to override the Docker image used for the cluster. The chart will choose a default
-  for you based on the setting you chose for `type`. If you need to run a configuration that is not supported, you can 
+  for you based on the setting you chose for `type`. If you need to run a configuration that is not supported, you can
   create your own Docker image. You can use the [postgres-containers](https://github.com/cloudnative-pg/postgres-containers)
   repository for a starting point.
   You will likely need to set your own repository access credentials via: `cluster.imagePullPolicy` and `cluster.imagePullSecrets`.
@@ -93,7 +93,7 @@ There are several important cluster options. Here are the most important ones:
   cluster:
     postgresql:
       max_connections: "200"
-      shared_buffers: "2GB"  
+      shared_buffers: "2GB"
   ```
 `cluster.initSQL` - Allows you to run custom SQL queries during the cluster initialization. This is useful for creating
 extensions, schemas and databases. Note that these are as a superuser.

charts/cluster/docs/Recovery.md

@@ -7,21 +7,23 @@ CNPG does not support recovery in-place. Instead you need to create a new cluste
 
 You can find more information about the recovery process in the [CNPG documentation](https://cloudnative-pg.io/documentation/current/backup_recovery).
 
-There are 3 types of recovery possible with CNPG:
+There are 4 types of recovery possible with CNPG:
 * Recovery from a backup object in the same Kubernetes namespace.
+* Recovery from a volume snapshot in the same Kubernetes namespace, if supported by the CSI provider.
 * Recovery from a Barman Object Store, that could be located anywhere.
 * Streaming replication from an operating cluster using `pg_basebackup`.
 
 When performing a recovery you are strongly advised to use the same configuration and PostgreSQL version as the original cluster.
+Recovering from older version of PostgreSQL is not posible, but it's possible to use [import](https://cloudnative-pg.io/documentation/current/database_import/) instead.
 
 To begin, create a `values.yaml` that contains the following:
 
 1. Set `mode: recovery` to indicate that you want to perform bootstrap the new cluster from an existing one.
-2. Set the `recovery.method` to the type of recovery you want to perform.
-3. Set either the `recovery.backupName` or the Barman Object Store configuration - i.e. `recovery.provider` and appropriate S3, Azure or GCS configuration. In case of `pg_basebackup` complete the `recovery.pgBaseBackup` section. 
-4. Optionally set the `recovery.pitrTarget.time` in RFC3339 format to perform a point-in-time recovery (not applicable for `pgBaseBackup`).
-5. Retain the identical PostgreSQL version and configuration as the original cluster.
-6. Make sure you don't use the same backup section name as the original cluster. We advise you change the `path` within the storage location if you want to reuse the same storage location/bucket.
+2. Set the `recovery.method` to the type of recovery you want to perform. Supported methods are listed under `recovery.methodSettings`.
+3. Configure `recovery.methodSettings` for selected `recovery.method`.
+4. Optionally set the `recovery.pitrTarget.time` in RFC3339 format to perform a point-in-time recovery (supported with `backup`, `objectStorage` and `volumeSnapshot`).
+5. Retain the identical major PostgreSQL version and same/newer minor version as on the original cluster.
+6. **Important**: make sure you don't use the same backup section name as the original cluster. We advise you change the `path` within the storage location if you want to reuse the same storage location/bucket.
     One pattern is adding a version number at the end of the path, e.g. `/v1` or `/v2` after each recovery procedure.
 
 Example recovery configurations can be found in the [examples](../examples) directory.

charts/cluster/examples/basic.yaml

@@ -1,5 +1,4 @@
 mode: standalone
+
 cluster:
   instances: 1
-backups:
-  enabled: false

charts/cluster/examples/custom-queries.yaml

@@ -1,4 +1,5 @@
 type: postgresql
+
 mode: standalone
 
 cluster:
@@ -8,7 +9,7 @@ cluster:
     customQueries:
       - name: "pg_cache_hit"
         query: |
-          SELECT 
+          SELECT
             current_database() as datname,
             sum(heap_blks_hit) / (sum(heap_blks_hit) + sum(heap_blks_read)) as ratio
           FROM pg_statio_user_tables;
@@ -19,6 +20,3 @@ cluster:
           - ratio:
               usage: GAUGE
               description: "Cache hit ratio"
-
-backups:
-  enabled: false

charts/cluster/examples/import-microservice.yaml

@@ -0,0 +1,23 @@
+mode: import
+
+cluster:
+  instances: 1
+
+import:
+  type: microservice
+  typeSettings:
+    microservice:
+      database: source-db
+
+recovery:
+  method: pgBasebackup
+  methodSettings:
+    pgBasebackup:
+      connectionParameters:
+        host: source-db.foo.com
+        user: postgres
+        sslMode: require
+      auth: password
+      authDetails:
+        password: |-
+          superuser-password

charts/cluster/examples/import-monolith-full.yaml

@@ -0,0 +1,26 @@
+mode: import
+
+cluster:
+  instances: 1
+
+import:
+  type: monolith
+  typeSettings:
+    monolith:
+      databases:
+        - *
+      roles:
+        - *
+
+recovery:
+  method: pgBasebackup
+  methodSettings:
+    pgBasebackup:
+      connectionParameters:
+        host: source-db.foo.com
+        user: postgres
+        sslMode: require
+      auth: password
+      authDetails:
+        password: |-
+          superuser-password

charts/cluster/examples/import-monolith.yaml

@@ -0,0 +1,29 @@
+mode: import
+
+cluster:
+  instances: 1
+
+import:
+  type: monolith
+  typeSettings:
+    monolith:
+      databases:
+        - db1
+        - db2
+        - db3
+      roles:
+        - role1
+        - role2
+
+recovery:
+  method: pgBasebackup
+  methodSettings:
+    pgBasebackup:
+      connectionParameters:
+        host: source-db.foo.com
+        user: postgres
+        sslMode: require
+      auth: password
+      authDetails:
+        password: |-
+          superuser-password

charts/cluster/examples/pgbouncer.yaml

@@ -1,8 +1,12 @@
 mode: standalone
+
 cluster:
   instances: 1
-backups:
-  enabled: false
-pooler:
-  enabled: true
-  instances: 1
+
+poolers:
+  - name: rw
+    type: rw
+    instances: 1
+  - name: ro
+    type: ro
+    instances: 1

charts/cluster/examples/postgis.yaml

@@ -1,6 +1,6 @@
 type: postgis
+
 mode: standalone
+
 cluster:
   instances: 1
-backups:
-  enabled: false

charts/cluster/examples/recovery-backup.yaml

@@ -1,22 +1,26 @@
 mode: recovery
 
-recovery:
-  method: backup
-  backupName: "database-clustermarket-database-daily-backup-1683244800"
-
 cluster:
   instances: 1
 
+recovery:
+  method: backup
+  methodSettings:
+    backup:
+      name: database-clustermarket-database-daily-backup-1683244800
+
 backups:
-  provider: s3
-  s3:
-    region: "eu-west-1"
-    bucket: "db-backups"
-    path: "/v1-restore"
-    accessKey: "AWS_S3_ACCESS_KEY"
-    secretKey: "AWS_S3_SECRET_KEY"
+  objectStorage:
+    provider: s3
+    providerSettings:
+      s3:
+        region: eu-west-1
+        bucket: db-backups
+        path: /v1-restore
+        accessKey: "AWS_S3_ACCESS_KEY"
+        secretKey: "AWS_S3_SECRET_KEY"
   scheduledBackups:
     - name: daily-backup # Daily at midnight
       schedule: "0 0 0 * * *" # Daily at midnight
       backupOwnerReference: self
-  retentionPolicy: "30d"
+  retentionPolicy: 30d

charts/cluster/examples/recovery-object_store-aws.yaml

@@ -0,0 +1,33 @@
+mode: recovery
+
+cluster:
+  instances: 1
+
+recovery:
+  method: objectStorage
+  methodSettings:
+  objectStorage:
+    clusterName: cluster-name-to-recover-from
+    provider: s3
+    s3:
+      region: eu-west-1
+      bucket: db-backups
+      path: /v1-restore
+      accessKey: "AWS_S3_ACCESS_KEY"
+      secretKey: "AWS_S3_SECRET_KEY"
+
+backups:
+  objectStorage:
+    provider: s3
+    providerSettings:
+      s3:
+        region: eu-west-1
+        bucket: db-backups
+        path: /v2-restore
+        accessKey: "AWS_S3_ACCESS_KEY"
+        secretKey: "AWS_S3_SECRET_KEY"
+  scheduledBackups:
+    - name: daily-backup # Daily at midnight
+      schedule: "0 0 0 * * *" # Daily at midnight
+      backupOwnerReference: self
+  retentionPolicy: 30d

charts/cluster/examples/recovery-object_store-minio.yaml

@@ -0,0 +1,40 @@
+mode: recovery
+
+cluster:
+  instances: 1
+
+recovery:
+  method: objectStorage
+  methodSettings:
+    objectStorage:
+      clusterName: standalone-cluster
+      endpointURL: https://minio.minio.svc.cluster.local
+      endpointCA:
+        name: kube-root-ca.crt
+        key: ca.crt
+      provider: s3
+      s3:
+        bucket: mybucket
+        path: /v1
+        accessKey: minio
+        secretKey: minio123
+
+backups:
+  objectStorage:
+    provider: s3
+    providerSettings:
+      endpointURL: https://minio.minio.svc.cluster.local
+      endpointCA:
+        name: kube-root-ca.crt
+        key: ca.crt
+      s3:
+        bucket: mybucket
+        path: /v1
+        accessKey: minio
+        secretKey: minio123
+    wal:
+      encryption: ""
+    data:
+      encryption: ""
+  scheduledBackups: []
+  retentionPolicy: 30d