Skip to content

Commit

Permalink
Merge pull request #2353 from EnterpriseDB/release/2022-02-17
Browse files Browse the repository at this point in the history
Release: 2022-02-17
  • Loading branch information
drothery-edb authored Feb 17, 2022
2 parents 1a56042 + af0c4b1 commit 9d01b92
Show file tree
Hide file tree
Showing 40 changed files with 1,457 additions and 327 deletions.
68 changes: 43 additions & 25 deletions advocacy_docs/kubernetes/cloud_native_postgresql/api_reference.mdx

Large diffs are not rendered by default.

149 changes: 107 additions & 42 deletions advocacy_docs/kubernetes/cloud_native_postgresql/backup_recovery.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,11 @@ The operator can orchestrate a continuous backup infrastructure
that is based on the [Barman](https://pgbarman.org) tool. Instead
of using the classical architecture with a Barman server, which
backs up many PostgreSQL instances, the operator relies on the
`barman-cloud-wal-archive`, `barman-cloud-backup`, `barman-cloud-backup-list`,
and `barman-cloud-backup-delete` tools.
As a result, base backups will be *tarballs*. Both base backups and WAL files
can be compressed and encrypted.
`barman-cloud-wal-archive`, `barman-cloud-check-wal-archive`,
`barman-cloud-backup`, `barman-cloud-backup-list`, and
`barman-cloud-backup-delete` tools. As a result, base backups will
be *tarballs*. Both base backups and WAL files can be compressed
and encrypted.

For this, it is required an image with `barman-cli-cloud` installed.
You can use the image `quay.io/enterprisedb/postgresql` for this scope,
Expand Down Expand Up @@ -57,7 +58,7 @@ The access key used must have permission to upload files in
the bucket. Given that, you must create a k8s secret with the
credentials, and you can do that with the following command:

```shell
```sh
kubectl create secret generic aws-creds \
--from-literal=ACCESS_KEY_ID=<access key here> \
--from-literal=ACCESS_SECRET_KEY=<secret key here>
Expand Down Expand Up @@ -135,7 +136,7 @@ MinIO Gateway as an endpoint, using previously created credentials and service.
MinIO secrets will be used by both the PostgreSQL cluster and the MinIO instance.
Therefore you must create them in the same namespace:

```shell
```sh
kubectl create secret generic minio-creds \
--from-literal=MINIO_ACCESS_KEY=<minio access key here> \
--from-literal=MINIO_SECRET_KEY=<minio secret key here>
Expand Down Expand Up @@ -179,42 +180,42 @@ Here is an example using AWS S3 as Cloud Object Storage:
apiVersion: apps/v1
kind: Deployment
[...]
spec:
containers:
- name: minio
image: minio/minio:RELEASE.2020-06-03T22-13-49Z
args:
- gateway
- s3
env:
# MinIO access key and secret key
- name: MINIO_ACCESS_KEY
valueFrom:
secretKeyRef:
name: minio-creds
key: MINIO_ACCESS_KEY
- name: MINIO_SECRET_KEY
valueFrom:
secretKeyRef:
name: minio-creds
key: MINIO_SECRET_KEY
# AWS credentials
- name: AWS_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
name: aws-creds
key: ACCESS_KEY_ID
- name: AWS_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
name: aws-creds
key: ACCESS_SECRET_KEY
spec:
containers:
- name: minio
image: minio/minio:RELEASE.2020-06-03T22-13-49Z
args:
- gateway
- s3
env:
# MinIO access key and secret key
- name: MINIO_ACCESS_KEY
valueFrom:
secretKeyRef:
name: minio-creds
key: MINIO_ACCESS_KEY
- name: MINIO_SECRET_KEY
valueFrom:
secretKeyRef:
name: minio-creds
key: MINIO_SECRET_KEY
# AWS credentials
- name: AWS_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
name: aws-creds
key: ACCESS_KEY_ID
- name: AWS_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
name: aws-creds
key: ACCESS_SECRET_KEY
# Uncomment the below section if session token is required
# - name: AWS_SESSION_TOKEN
# valueFrom:
# secretKeyRef:
# name: aws-creds
# key: ACCESS_SESSION_TOKEN
# - name: AWS_SESSION_TOKEN
# valueFrom:
# secretKeyRef:
# name: aws-creds
# key: ACCESS_SESSION_TOKEN
ports:
- containerPort: 9000
```
Expand Down Expand Up @@ -574,4 +575,68 @@ spec:
any point in time between `PoR` and the latest successfully archived WAL
file, starting from the first valid backup. Base backups that are older
than the first valid backup will be marked as *obsolete* and permanently
removed after the next backup is completed.
removed after the next backup is completed.

## Compression algorithms

Cloud Native PostgreSQL by default archives backups and WAL files in an
uncompressed fashion. However, it also supports the following compression
algorithms via `barman-cloud-backup` (for backups) and
`barman-cloud-wal-archive` (for WAL files):

- bzip2
- gzip
- snappy

The compression settings for backups and WALs are independent. See the
[DataBackupConfiguration](api_reference.md#DataBackupConfiguration) and
[WALBackupConfiguration](api_reference.md#WalBackupConfiguration) sections in
the API reference.

It is important to note that archival time, restore time, and size change
between the algorithms, so the compression algorithm should be chosen according
to your use case.

The Barman team has performed an evaluation of the performance of the supported
algorithms for Barman Cloud. The following table summarizes a scenario where a
backup is taken on a local MinIO deployment. The Barman GitHub project includes
a [deeper analysis](https://github.com/EnterpriseDB/barman/issues/344#issuecomment-992547396).

| Compression | Backup Time (ms) | Restore Time (ms) | Uncompressed size (MB) | Compressed size (MB) | Approx ratio |
| ----------- | ---------------- | ----------------- | ---------------------- | -------------------- | ------------ |
| None | 10927 | 7553 | 395 | 395 | 1:1 |
| bzip2 | 25404 | 13886 | 395 | 67 | 5.9:1 |
| gzip | 116281 | 3077 | 395 | 91 | 4.3:1 |
| snappy | 8134 | 8341 | 395 | 166 | 2.4:1 |

## Tagging of backup objects

Barman 2.18 introduces support for tagging backup resources when saving them in
object stores via `barman-cloud-backup` and `barman-cloud-wal-archive`. As a
result, if your PostgreSQL container image includes Barman with version 2.18 or
higher, Cloud Native PostgreSQL enables you to specify tags as key-value pairs
for backup objects, namely base backups, WAL files and history files.

You can use two properties in the `.spec.backup.barmanObjectStore` definition:

- `tags`: key-value pair tags to be added to backup objects and archived WAL
file in the backup object store
- `historyTags`: key-value pair tags to be added to archived history files in
the backup object store

The excerpt of a YAML manifest below provides an example of usage of this
feature:

```yaml
apiVersion: postgresql.k8s.enterprisedb.io/v1
kind: Cluster
[...]
spec:
backup:
barmanObjectStore:
[...]
tags:
backupRetentionPolicy: "expire"
historyTags:
backupRetentionPolicy: "keep"
```
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ specific to Kubernetes and PostgreSQL.
| [Operator](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/) | An *operator* is a custom resource that automates those steps that are normally performed by a human operator when managing one or more applications or given services. An operator assists Kubernetes in making sure that the resource's defined state always matches the observed one. |
| [kubectl](https://kubernetes.io/docs/reference/kubectl/overview/) | `kubectl` is the command-line tool used to manage a Kubernetes cluster. |

Cloud Native PostgreSQL requires Kubernetes 1.18 or higher.
Cloud Native PostgreSQL requires Kubernetes 1.19 or higher.

## PostgreSQL terminology

Expand Down
28 changes: 27 additions & 1 deletion advocacy_docs/kubernetes/cloud_native_postgresql/cnp-plugin.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,31 @@ The plugin also works with `oc` in an OpenShift environment.

You can install the plugin in your system with:

```shell
```sh
curl -sSfL \
https://github.com/EnterpriseDB/kubectl-cnp/raw/main/install.sh | \
sudo sh -s -- -b /usr/local/bin
```

### Supported Architectures

Cloud Native PostgreSQL Plugin is currently build for the following operating system and architectures:

- Linux
- amd64
- arm 5/6/7
- arm64
- s390x
- ppc64le
- macOS
- amd64
- arm64
- Windows
- 386
- amd64
- arm 5/6/7
- arm64

## Use

Once the plugin was installed and deployed, you can start using it like this:
Expand Down Expand Up @@ -70,6 +89,13 @@ Working WAL archiving: OK
Last Archived WAL: 00000008000003AE00000079 @ 2021-12-14T10:16:29.340047Z
Last Failed WAL: -

Certificates Status
Certificate Name Expiration Date Days Left Until Expiration
---------------- --------------- --------------------------
cluster-example-ca 2022-05-05 15:02:42 +0000 UTC 87.23
cluster-example-replication 2022-05-05 15:02:42 +0000 UTC 87.23
cluster-example-server 2022-05-05 15:02:42 +0000 UTC 87.23

Streaming Replication status
Name Sent LSN Write LSN Flush LSN Replay LSN Write Lag Flush Lag Replay Lag State Sync State Sync Priority
---- -------- --------- --------- ---------- --------- --------- ---------- ----- ---------- -------------
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ with the following requirements:
- `barman-cloud-backup`
- `barman-cloud-restore`
- `barman-cloud-backup-list`
- `barman-cloud-check-wal-archive`
- PGAudit extension installed (optional - only if PGAudit is required
in the deployed clusters)
- Sensible locale settings
Expand All @@ -34,8 +35,7 @@ Native PostgreSQL overrides it with its instance manager.

EnterpriseDB provides and supports public container images for Cloud Native
PostgreSQL and publishes them on
[Quay.io](https://quay.io/repository/enterprisedb/postgresql) and
[DockerHub](https://hub.docker.com/r/enterprisedb/postgresql).
[Quay.io](https://quay.io/repository/enterprisedb/postgresql).

## Image tag requirements

Expand Down
32 changes: 0 additions & 32 deletions advocacy_docs/kubernetes/cloud_native_postgresql/credits.mdx

This file was deleted.

1 change: 0 additions & 1 deletion advocacy_docs/kubernetes/cloud_native_postgresql/e2e.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ process:
- 1.21
- 1.20
- 1.19
- 1.18

The following PostgreSQL versions are tested:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -107,14 +107,14 @@ Now you will be able to reach the PostgreSQL Cluster from outside your Kubernete

On Minikube you can setup the ingress controller running:

```shell
```sh
minikube addons enable ingress
```

Then, patch the `tcp-service` ConfigMap to redirect to the primary the
connections on port 5432 of the Ingress:

```shell
```sh
kubectl patch configmap tcp-services -n kube-system \
--patch '{"data":{"5432":"default/cluster-example-rw:5432"}}'
```
Expand All @@ -135,12 +135,12 @@ spec:

and apply it to the `nginx-ingress-controller deployment`:

```shell
```sh
kubectl patch deployment nginx-ingress-controller --patch "$(cat patch.yaml)" -n kube-system
```

You can access the primary from your machine running:

```shell
```sh
psql -h $(minikube ip) -p 5432 -U postgres
```
Original file line number Diff line number Diff line change
Expand Up @@ -29,13 +29,13 @@ If you want to prevent the operator from reusing a certain PVC you need to
remove the PVC before deleting the Pod. For this purpose, you can use the
following command:

```shell
```sh
kubectl delete -n [namespace] pvc/[cluster-name]-[serial] pod/[cluster-name]-[serial]
```

For example:

```shell
```sh
$ kubectl delete -n default pvc/cluster-example-1 pod/cluster-example-1
persistentvolumeclaim "cluster-example-1" deleted
pod "cluster-example-1" deleted
Expand Down Expand Up @@ -96,6 +96,10 @@ is set to `off` (default: `on` during maintenance windows, `off` otherwise).
The `PodDisruptionBudget` may prevent the pod from being evicted if there
is at least another pod that is not ready.

!!! Note
Single instance clusters prevent node drain when `reusePVC` is
set to `false`. Refer to the [Kubernetes Upgrade section](kubernetes_upgrade.md).

Self-healing will happen as soon as the *apiserver* is notified.

### Worker node failure
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit 9d01b92

Please sign in to comment.