Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat( cluster ): cluster.imageCatalogRef; ImageCatalog support; TimescaleDB support #306

Merged
merged 24 commits into from
Sep 8, 2024
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
868e74a
Merge remote-tracking branch 'origin/main' into dev/276
itay-grudev May 25, 2024
ac0fbb2
Added support for imageCatalogRef
itay-grudev May 30, 2024
c0ed873
Added support for custom ImageCatalog
itay-grudev May 30, 2024
56be815
Added support for TimescaleDB
itay-grudev May 30, 2024
1bfc4ea
Corrected lint issues
itay-grudev May 30, 2024
f738653
Merge remote-tracking branch 'origin/main' into dev/276
itay-grudev May 30, 2024
ed7994a
Alternative postgresql version approach allowing for minor version in…
itay-grudev May 30, 2024
1419262
Added versions to relevant examples
itay-grudev May 30, 2024
860f767
Merge branch 'main' into dev/276
itay-grudev Jul 3, 2024
9ce2833
lint: removed trailing spaces after merge commit
itay-grudev Jul 3, 2024
c0d135e
Merge branch 'main' into dev/276
itay-grudev Jul 30, 2024
19766bf
Merge branch 'main' into dev/276
itay-grudev Aug 28, 2024
2b983bb
Merge branch 'main' into dev/276
itay-grudev Aug 28, 2024
1b70087
Merge branch 'main' into dev/276
itay-grudev Aug 28, 2024
db8e7c6
Merge branch 'main' into dev/276
itay-grudev Aug 29, 2024
8060676
Merge branch 'main' into dev/276
itay-grudev Sep 3, 2024
d080597
Update charts/cluster/templates/_helpers.tpl
itay-grudev Sep 4, 2024
f892831
Merge branch 'main' into dev/276
itay-grudev Sep 4, 2024
3dffac6
Separated backup paths within the MinIO bucket
itay-grudev Sep 7, 2024
ad73d24
Removed kubectl from a test steps that don't use it
itay-grudev Sep 8, 2024
711baec
Added a TimescaleDB test suite to the chart
itay-grudev Sep 8, 2024
adf9d89
Style fixes
itay-grudev Sep 8, 2024
9d161a4
Test FIx: incorrect timestamp due to operation order
itay-grudev Sep 8, 2024
bbf8520
Fixed typo in test spec
itay-grudev Sep 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 9 additions & 3 deletions charts/cluster/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,7 @@ refer to the [CloudNativePG Documentation](https://cloudnative-pg.io/documentat
| cluster.annotations | object | `{}` | |
| cluster.certificates | object | `{}` | The configuration for the CA and related certificates. See: https://cloudnative-pg.io/documentation/current/cloudnative-pg.v1/#postgresql-cnpg-io-v1-CertificatesConfiguration |
| cluster.enableSuperuserAccess | bool | `true` | When this option is enabled, the operator will use the SuperuserSecret to update the postgres user password. If the secret is not present, the operator will automatically create one. When this option is disabled, the operator will ignore the SuperuserSecret content, delete it when automatically created, and then blank the password of the postgres user by setting it to NULL. |
| cluster.imageCatalogRef | object | `{}` | Reference to `ImageCatalog` of `ClusterImageCatalog`, if specified takes precedence over `cluster.imageName` |
| cluster.imageName | string | `""` | Name of the container image, supporting both tags (<image>:<tag>) and digests for deterministic and repeatable deployments: <image>:<tag>@sha256:<digestValue> |
| cluster.imagePullPolicy | string | `"IfNotPresent"` | Image pull policy. One of Always, Never or IfNotPresent. If not defined, it defaults to IfNotPresent. Cannot be updated. More info: https://kubernetes.io/docs/concepts/containers/images#updating-images |
| cluster.imagePullSecrets | list | `[]` | The list of pull secrets to be used to pull the images. See: https://cloudnative-pg.io/documentation/current/cloudnative-pg.v1/#postgresql-cnpg-io-v1-LocalObjectReference |
Expand All @@ -164,8 +165,8 @@ refer to the [CloudNativePG Documentation](https://cloudnative-pg.io/documentat
| cluster.monitoring.podMonitor.enabled | bool | `true` | Whether to enable the PodMonitor |
| cluster.monitoring.prometheusRule.enabled | bool | `true` | Whether to enable the PrometheusRule automated alerts |
| cluster.monitoring.prometheusRule.excludeRules | list | `[]` | Exclude specified rules |
| cluster.postgresGID | int | `26` | The GID of the postgres user inside the image, defaults to 26 |
| cluster.postgresUID | int | `26` | The UID of the postgres user inside the image, defaults to 26 |
| cluster.postgresGID | int | `-1` | The GID of the postgres user inside the image, defaults to 26 |
| cluster.postgresUID | int | `-1` | The UID of the postgres user inside the image, defaults to 26 |
| cluster.postgresql.parameters | object | `{}` | PostgreSQL configuration options (postgresql.conf) |
| cluster.postgresql.pg_hba | list | `[]` | PostgreSQL Host Based Authentication rules (lines to be appended to the pg_hba.conf file) |
| cluster.primaryUpdateMethod | string | `"switchover"` | Method to follow to upgrade the primary server during a rolling update procedure, after all replicas have been successfully updated. It can be switchover (default) or restart. |
Expand All @@ -180,6 +181,8 @@ refer to the [CloudNativePG Documentation](https://cloudnative-pg.io/documentat
| cluster.walStorage.size | string | `"1Gi"` | |
| cluster.walStorage.storageClass | string | `""` | |
| fullnameOverride | string | `""` | Override the full name of the chart |
| imageCatalog.create | bool | `true` | Whether to provision an image catalog. If imageCatalog.images is empty this option will be ignored. |
| imageCatalog.images | list | `[]` | List of images to be provisioned in an image catalog. |
| mode | string | `"standalone"` | Cluster mode of operation. Available modes: * `standalone` - default mode. Creates new or updates an existing CNPG cluster. * `replica` - Creates a replica cluster from an existing CNPG cluster. # TODO * `recovery` - Same as standalone but creates a cluster from a backup, object store or via pg_basebackup. |
| nameOverride | string | `""` | Override the name of the chart |
| pooler.enabled | bool | `false` | Whether to enable PgBouncer |
Expand Down Expand Up @@ -236,7 +239,10 @@ refer to the [CloudNativePG Documentation](https://cloudnative-pg.io/documentat
| recovery.s3.secretKey | string | `""` | |
| recovery.secret.create | bool | `true` | Whether to create a secret for the backup credentials |
| recovery.secret.name | string | `""` | Name of the backup credentials secret |
| type | string | `"postgresql"` | Type of the CNPG database. Available types: * `postgresql` * `postgis` |
| type | string | `"postgresql"` | Type of the CNPG database. Available types: * `postgresql` * `postgis` * `timescaledb` |
| version.postgis | string | `"3.4"` | If using PostGIS, specify the version |
| version.postgresql | string | `"16"` | PostgreSQL major version to use |
| version.timescaledb | string | `"2.15"` | If using TimescaleDB, specify the version |

## Maintainers

Expand Down
2 changes: 2 additions & 0 deletions charts/cluster/examples/basic.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
mode: standalone
version:
postgresql: "16"
cluster:
instances: 1
backups:
Expand Down
12 changes: 12 additions & 0 deletions charts/cluster/examples/image-catalog-ref.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
type: postgresql
mode: standalone
version:
major: "16"
timescaledb: "2.15"
cluster:
instances: 1
imageCatalogRef:
kind: ImageCatalog
name: my-image-catalog
backups:
enabled: false
14 changes: 14 additions & 0 deletions charts/cluster/examples/image-catalog.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
type: postgresql
mode: standalone
version:
major: "16"
timescaledb: "2.15"
cluster:
instances: 1
backups:
enabled: false
imageCatalog:
create: true
images:
- major: 16
image: my-custom-postgres-image:mytag
5 changes: 4 additions & 1 deletion charts/cluster/examples/postgis.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
type: postgis
mode: standalone
version:
postgresql: "16"
postgis: "3.4"
cluster:
instances: 1
backups:
enabled: false
enabled: false
9 changes: 9 additions & 0 deletions charts/cluster/examples/timescaledb.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
type: timescaledb
mode: standalone
version:
postgresql: "15.7"
timescaledb: "2.15"
cluster:
instances: 1
backups:
enabled: false
11 changes: 9 additions & 2 deletions charts/cluster/templates/NOTES.txt
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ Configuration
{{- $redundancyColor = "ok" -}}
{{- end }}

{{ $scheduledBackups := (first .Values.backups.scheduledBackups).name }}
{{- $scheduledBackups := (first .Values.backups.scheduledBackups).name -}}
{{- range (rest .Values.backups.scheduledBackups) -}}
{{ $scheduledBackups = printf "%s, %s" $scheduledBackups .name }}
{{- end -}}
Expand All @@ -54,12 +54,19 @@ Configuration
{{- end -}}
{{- end -}}

{{- $image := (include "cluster.image" .) | fromYaml -}}
{{- if $image.imageCatalogRef -}}
{{- $image = printf "%s: %s(%s)" $image.imageCatalogRef.kind $image.imageCatalogRef.name (include "cluster.postgresqlMajor" .) -}}
{{- else if $image.imageName -}}
{{- $image = $image.imageName -}}
{{- end }}

╭───────────────────┬──────────────────────────────────────────────────────────╮
│ Configuration │ Value │
┝━━━━━━━━━━━━━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┥
│ Cluster mode │ {{ printf "%-56s" $mode }} │
│ Type │ {{ printf "%-56s" .Values.type }} │
│ Image │ {{ include "cluster.color-info" (printf "%-56s" (include "cluster.imageName" .)) }} │
│ Image │ {{ include "cluster.color-info" (printf "%-56s" $image) }} │
{{- if eq .Values.mode "recovery" }}
│ Source │ {{ printf "%-56s" $source }} │
{{- end }}
Expand Down
73 changes: 69 additions & 4 deletions charts/cluster/templates/_helpers.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,20 @@ app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/part-of: cloudnative-pg
{{- end }}

{{/*
Whether we need to use TimescaleDB defaults
*/}}
{{- define "cluster.useTimescaleDBDefaults" -}}
{{ and (eq .Values.type "timescaledb") .Values.imageCatalog.create (empty .Values.cluster.imageCatalogRef.name) (empty .Values.imageCatalog.images) (empty .Values.cluster.imageName) }}
{{- end -}}

{{/*
Get the PostgreSQL major version from .Values.version.postgresql
*/}}
{{- define "cluster.postgresqlMajor" -}}
{{ index (regexSplit "\\." (toString .Values.version.postgresql) 2) 0 }}
{{- end -}}

{{/*
Cluster Image Name
If a custom imageName is available, use it, otherwise use the defaults based on the .Values.type
Expand All @@ -59,12 +73,63 @@ If a custom imageName is available, use it, otherwise use the defaults based on
{{- if .Values.cluster.imageName -}}
{{- .Values.cluster.imageName -}}
{{- else if eq .Values.type "postgresql" -}}
{{- "ghcr.io/cloudnative-pg/postgresql:15.2" -}}
{{- printf "ghcr.io/cloudnative-pg/postgresql:%s" .Values.version.postgresql -}}
{{- else if eq .Values.type "postgis" -}}
{{- "ghcr.io/cloudnative-pg/postgis:14" -}}
{{- else if eq .Values.type "timescaledb" -}}
{{ fail "You need to provide your own cluster.imageName as an official timescaledb image doesn't exist yet." }}
{{- printf "ghcr.io/cloudnative-pg/postgis:%s-%s" .Values.version.postgresql .Values.version.postgis -}}
{{- else -}}
{{ fail "Invalid cluster type!" }}
{{- end }}
{{- end -}}

{{/*
Cluster Image
If imageCatalogRef defined, use it, otherwice calculate ordinary imageName.
*/}}
{{- define "cluster.image" }}
{{- if .Values.cluster.imageCatalogRef.name }}
imageCatalogRef:
apiGroup: postgresql.cnpg.io
{{- toYaml .Values.cluster.imageCatalogRef | nindent 2 }}
major: {{ include "cluster.postgresqlMajor" . }}
{{- else if and .Values.imageCatalog.create (not (empty .Values.imageCatalog.images )) }}
imageCatalogRef:
apiGroup: postgresql.cnpg.io
kind: ImageCatalog
name: {{ include "cluster.fullname" . }}
major: {{ include "cluster.postgresqlMajor" . }}
{{- else if eq (include "cluster.useTimescaleDBDefaults" .) "true" -}}
imageCatalogRef:
apiGroup: postgresql.cnpg.io
kind: ImageCatalog
name: {{ include "cluster.fullname" . }}-timescaledb-ha
major: {{ include "cluster.postgresqlMajor" . }}
{{- else }}
imageName: {{ include "cluster.imageName" . }}
{{- end }}
{{- end }}

{{/*
Postgres UID
*/}}
{{- define "cluster.postgresUID" -}}
{{- if ge (int .Values.cluster.postgresUID) 0 -}}
{{- .Values.cluster.postgresUID }}
{{- else if and (eq (include "cluster.useTimescaleDBDefaults" .) "true") (eq .Values.type "timescaledb") -}}
{{- 1000 -}}
{{- else -}}
{{- 26 -}}
{{- end -}}
{{- end -}}

{{/*
Postgres GID
*/}}
{{- define "cluster.postgresGID" -}}
{{- if ge (int .Values.cluster.postgresGID) 0 -}}
{{- .Values.cluster.postgresGID }}
{{- else if and (eq (include "cluster.useTimescaleDBDefaults" .) "true") (eq .Values.type "timescaledb") -}}
{{- 1000 -}}
{{- else -}}
{{- 26 -}}
{{- end -}}
{{- end -}}
8 changes: 4 additions & 4 deletions charts/cluster/templates/cluster.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,14 @@ metadata:
{{- end }}
spec:
instances: {{ .Values.cluster.instances }}
imageName: {{ include "cluster.imageName" . }}
{{- include "cluster.image" . | nindent 2 }}
imagePullPolicy: {{ .Values.cluster.imagePullPolicy }}
{{- with .Values.cluster.imagePullSecrets}}
{{- with .Values.cluster.imagePullSecrets }}
imagePullSecrets:
{{- . | toYaml | nindent 4 }}
{{- end }}
postgresUID: {{ .Values.cluster.postgresUID }}
postgresGID: {{ .Values.cluster.postgresGID }}
postgresUID: {{ include "cluster.postgresUID" . }}
postgresGID: {{ include "cluster.postgresGID" . }}
storage:
size: {{ .Values.cluster.storage.size }}
storageClass: {{ .Values.cluster.storage.storageClass }}
Expand Down
18 changes: 18 additions & 0 deletions charts/cluster/templates/image-catalog-timescaledb-ha.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
{{- if eq (include "cluster.useTimescaleDBDefaults" .) "true" -}}
apiVersion: postgresql.cnpg.io/v1
kind: ImageCatalog
metadata:
name: {{ include "cluster.fullname" . }}-timescaledb-ha
spec:
images:
- major: 12
image: timescale/timescaledb-ha:pg15-ts{{ .Values.version.timescaledb }}
- major: 13
image: timescale/timescaledb-ha:pg15-ts{{ .Values.version.timescaledb }}
- major: 14
image: timescale/timescaledb-ha:pg15-ts{{ .Values.version.timescaledb }}
- major: 15
image: timescale/timescaledb-ha:pg15-ts{{ .Values.version.timescaledb }}
- major: 16
image: timescale/timescaledb-ha:pg16-ts{{ .Values.version.timescaledb }}
{{ end }}
12 changes: 12 additions & 0 deletions charts/cluster/templates/image-catalog.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{{ if and .Values.imageCatalog.create (not (empty .Values.imageCatalog.images )) }}
apiVersion: postgresql.cnpg.io/v1
kind: ImageCatalog
metadata:
name: {{ include "cluster.fullname" . }}
spec:
images:
{{- range $image := .Values.imageCatalog.images }}
- image: {{ $image.image }}
major: {{ $image.major }}
{{- end }}
{{- end }}
28 changes: 28 additions & 0 deletions charts/cluster/values.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -187,6 +187,9 @@
"enableSuperuserAccess": {
"type": "boolean"
},
"imageCatalogRef": {
"type": "object"
},
"imageName": {
"type": "string"
},
Expand Down Expand Up @@ -300,6 +303,17 @@
"fullnameOverride": {
"type": "string"
},
"imageCatalog": {
"type": "object",
"properties": {
"create": {
"type": "boolean"
},
"images": {
"type": "array"
}
}
},
"mode": {
"type": "string"
},
Expand Down Expand Up @@ -564,6 +578,20 @@
},
"type": {
"type": "string"
},
"version": {
"type": "object",
"properties": {
"postgis": {
"type": "string"
},
"postgresql": {
"type": "string"
},
"timescaledb": {
"type": "string"
}
}
}
}
}
25 changes: 23 additions & 2 deletions charts/cluster/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,17 @@ fullnameOverride: ""
# -- Type of the CNPG database. Available types:
# * `postgresql`
# * `postgis`
# * `timescaledb`
type: postgresql

version:
# -- PostgreSQL major version to use
postgresql: "16"
# -- If using TimescaleDB, specify the version
timescaledb: "2.15"
# -- If using PostGIS, specify the version
postgis: "3.4"

###
# -- Cluster mode of operation. Available modes:
# * `standalone` - default mode. Creates new or updates an existing CNPG cluster.
Expand Down Expand Up @@ -123,6 +132,11 @@ cluster:
# <image>:<tag>@sha256:<digestValue>
imageName: "" # Default value depends on type (postgresql/postgis/timescaledb)

# -- Reference to `ImageCatalog` of `ClusterImageCatalog`, if specified takes precedence over `cluster.imageName`
imageCatalogRef: {}
# kind: ImageCatalog
# name: postgresql

# -- Image pull policy. One of Always, Never or IfNotPresent. If not defined, it defaults to IfNotPresent. Cannot be updated.
# More info: https://kubernetes.io/docs/concepts/containers/images#updating-images
imagePullPolicy: IfNotPresent
Expand All @@ -141,10 +155,10 @@ cluster:
storageClass: ""

# -- The UID of the postgres user inside the image, defaults to 26
postgresUID: 26
postgresUID: -1

# -- The GID of the postgres user inside the image, defaults to 26
postgresGID: 26
postgresGID: -1

# -- Resources requirements of every generated Pod.
# Please refer to https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/ for more information.
Expand Down Expand Up @@ -328,6 +342,13 @@ backups:
# -- Retention policy for backups
retentionPolicy: "30d"

imageCatalog:
# -- Whether to provision an image catalog. If imageCatalog.images is empty this option will be ignored.
create: true
# -- List of images to be provisioned in an image catalog.
images: []
# - image: ghcr.io/your_repo/your_image:your_tag
# major: 16

pooler:
# -- Whether to enable PgBouncer
Expand Down
Loading