ParadeDB Support (#1)

Signed-off-by: Philippe Noël <[email protected]> Co-authored-by: Philippe Noël <[email protected]> ci: AWS EKS LocalStack Tests (#14) Co-authored-by: Itay Grudev <[email protected]> Co-authored-by: Itay Grudev <[email protected]> Updated documentation workflows and cleanup (#18) Revert "Add postgresql default to tests" This reverts commit a9f340d. Add repository_dispatch Signed-off-by: Philippe Noël <[email protected]> chore: Configure repository for ParadeDB (#15) Co-authored-by: Itay Grudev <[email protected]> chore: Create artifacthub-repo.yml (#20) Signed-off-by: Philippe Noël <[email protected]> Put artifacthub-repo.yml in the right location Bug Fix: Tests (#21) Renamed chart to `paradedb-cluster` (#22) Co-authored-by: Philippe Noël <[email protected]> Rm .DS_Store Remove extra HTML tag Signed-off-by: Philippe Noël <[email protected]> Remove repository_dispatch Signed-off-by: Philippe Noël <[email protected]> chore: Rename paradedb-cluster to paradedb (#28) chore: Try with adding missing -cluster (#33) chore: Final Cleanup (#34) chore: Remove PostGIS and Timescale (#35) Using the default UID/GID 999 from the postgres docker image (#26) Co-authored-by: Philippe Noël <[email protected]> chore: Update README and Schema (#37) Signed-off-by: Philippe Noël <[email protected]> chore(deps): pin sigstore/cosign-installer action to 4959ce0 (cloudnative-pg#403) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Rm pgvectorscale (#39) Signed-off-by: Philippe Noël <[email protected]> feat: Add GitHub Actions Workflow to check for typos (#40) Signed-off-by: Philippe Noël <[email protected]> Co-authored-by: Philippe Noël <[email protected]> chore(deps): update kyverno/action-install-chainsaw action to v0.2.11 (cloudnative-pg#410) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Rename the default database from app to paradedb (#41) Co-authored-by: Philippe Noël <[email protected]> docs: Clarify the docs (#42) Update Depot Revert "Update Depot" This reverts commit cb85b81. feat: Add `pg_cron` and Postgis (#44) Signed-off-by: Philippe Noël <[email protected]> chore: Use the new 0.11.0 release (#47) feat: Enable monitoring via Prometheus and Grafana (rebased) (#48) Rm extra files Remove duplicate block in README
paradedb · Jan 20, 2025 · 506ddc1 · 506ddc1
1 parent 7656400
commit 506ddc1
Show file tree

Hide file tree

Showing 33 changed files with 756 additions and 3 deletions.
diff --git a/.github/workflows/tests-cluster-chainsaw.yaml b/.github/workflows/tests-cluster-chainsaw.yaml
@@ -59,6 +59,29 @@ jobs:
           echo "Chart.yaml:"
           cat Chart.yaml
 
+      # Added by ParadeDB: Authenticate to Docker Hub to avoid rate limits
+      - name: Login to Docker Hub
+        uses: docker/login-action@v3
+        with:
+          username: ${{ secrets.DOCKERHUB_USERNAME }}
+          password: ${{ secrets.DOCKERHUB_ACCESS_TOKEN }}
+
+      # Added by ParadeDB: Always pull the latest version of paradedb/paradedb
+      - name: Set ParadeDB Version to Latest
+        working-directory: charts/paradedb/
+        env:
+          GH_TOKEN: ${{ secrets.GHA_CREATE_RELEASE_PAT }}
+        run: |
+          LATEST_TAG=$(curl -s https://api.github.com/repos/paradedb/paradedb/tags | jq -r '.[0].name')
+          APP_VERSION=${LATEST_TAG#v}
+          sed -i "s/^[[:space:]]*paradedb: .*/  paradedb: \"$APP_VERSION\"/" values.yaml
+          sed -i "s/^version: .*/version: ${{ vars.CHART_VERSION_MAJOR }}.${{ vars.CHART_VERSION_MINOR }}.${{ vars.CHART_VERSION_PATCH }}/" Chart.yaml
+          echo "values.yaml:"
+          cat values.yaml
+          echo "----------------------------------------"
+          echo "Chart.yaml:"
+          cat Chart.yaml
+
       - name: Setup kind
         uses: ./.github/actions/setup-kind
 

diff --git a/charts/paradedb/README.md b/charts/paradedb/README.md
@@ -211,15 +211,15 @@ refer to  the [CloudNativePG Documentation](https://cloudnative-pg.io/documentat
 | cluster.monitoring.customQueries | list | `[]` | Custom Prometheus metrics Will be stored in the ConfigMap |
 | cluster.monitoring.customQueriesSecret | list | `[]` | The list of secrets containing the custom queries |
 | cluster.monitoring.disableDefaultQueries | bool | `false` | Whether the default queries should be injected. Set it to true if you don't want to inject default queries into the cluster. |
-| cluster.monitoring.enabled | bool | `false` | Whether to enable monitoring |
+| cluster.monitoring.enabled | bool | `true` | Whether to enable monitoring |
 | cluster.monitoring.podMonitor.enabled | bool | `true` | Whether to enable the PodMonitor |
 | cluster.monitoring.podMonitor.metricRelabelings | list | `[]` | The list of metric relabelings for the PodMonitor. Applied to samples before ingestion. |
 | cluster.monitoring.podMonitor.relabelings | list | `[]` | The list of relabelings for the PodMonitor. Applied to samples before scraping. |
 | cluster.monitoring.prometheusRule.enabled | bool | `true` | Whether to enable the PrometheusRule automated alerts |
 | cluster.monitoring.prometheusRule.excludeRules | list | `[]` | Exclude specified rules |
 | cluster.postgresGID | int | `-1` | The GID of the postgres user inside the image, defaults to 26 |
 | cluster.postgresUID | int | `-1` | The UID of the postgres user inside the image, defaults to 26 |
-| cluster.postgresql.parameters | object | `{}` | PostgreSQL configuration options (postgresql.conf) |
+| cluster.postgresql.parameters | object | `{"cron.database_name":"postgres"}` | PostgreSQL configuration options (postgresql.conf) |
 | cluster.postgresql.pg_hba | list | `[]` | PostgreSQL Host Based Authentication rules (lines to be appended to the pg_hba.conf file) |
 | cluster.postgresql.pg_ident | list | `[]` | PostgreSQL User Name Maps rules (lines to be appended to the pg_ident.conf file) |
 | cluster.postgresql.shared_preload_libraries | list | `[]` | Lists of shared preload libraries to add to the default ones |

diff --git a/charts/paradedb/test/postgresql-minio-backup-restore/00-minio_cleanup-assert.yaml b/charts/paradedb/test/postgresql-minio-backup-restore/00-minio_cleanup-assert.yaml
@@ -0,0 +1,6 @@
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: data-test
+status:
+  succeeded: 1
diff --git a/charts/paradedb/test/postgresql-minio-backup-restore/00-minio_cleanup.yaml b/charts/paradedb/test/postgresql-minio-backup-restore/00-minio_cleanup.yaml
@@ -0,0 +1,16 @@
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: minio-cleanup
+spec:
+  template:
+    spec:
+      restartPolicy: OnFailure
+      containers:
+      - name: minio-cleanup
+        image: minio/mc
+        command: ['sh', '-c']
+        args:
+         - |
+           mc alias set myminio https://minio.minio.svc.cluster.local minio minio123
+           mc rm --recursive --force myminio/mybucket/paradedb
diff --git a/charts/paradedb/test/postgresql-minio-backup-restore/01-standalone_cluster-assert.yaml b/charts/paradedb/test/postgresql-minio-backup-restore/01-standalone_cluster-assert.yaml
@@ -0,0 +1,6 @@
+apiVersion: postgresql.cnpg.io/v1
+kind: Cluster
+metadata:
+  name: import-paradedb
+status:
+  readyInstances: 2
diff --git a/charts/paradedb/test/postgresql-minio-backup-restore/01-standalone_cluster.yaml b/charts/paradedb/test/postgresql-minio-backup-restore/01-standalone_cluster.yaml
@@ -0,0 +1,29 @@
+type: paradedb
+mode: standalone
+version:
+  major: "17"
+  paradedb: "0.14.0"
+cluster:
+  instances: 2
+  storage:
+    size: 256Mi
+
+backups:
+  enabled: true
+  provider: s3
+  endpointURL: "https://minio.minio.svc.cluster.local"
+  endpointCA:
+    name: kube-root-ca.crt
+    key: ca.crt
+  wal:
+    encryption: ""
+  data:
+    encryption: ""
+  s3:
+    bucket: "mybucket"
+    path: "/paradedb/v1"
+    accessKey: "minio"
+    secretKey: "minio123"
+    region: "local"
+  scheduledBackups: []
+  retentionPolicy: "30d"
diff --git a/charts/paradedb/test/postgresql-minio-backup-restore/02-data_write-assert.yaml b/charts/paradedb/test/postgresql-minio-backup-restore/02-data_write-assert.yaml
@@ -0,0 +1,6 @@
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: data-write
+status:
+  succeeded: 1
diff --git a/charts/paradedb/test/postgresql-minio-backup-restore/02-data_write.yaml b/charts/paradedb/test/postgresql-minio-backup-restore/02-data_write.yaml
@@ -0,0 +1,23 @@
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: data-write
+spec:
+  template:
+    spec:
+      restartPolicy: OnFailure
+      containers:
+      - name: data-write
+        env:
+          - name: DB_URI
+            valueFrom:
+              secretKeyRef:
+                name: standalone-paradedb-superuser
+                key: uri
+        image: alpine:3.19
+        command: ['sh', '-c']
+        args:
+         - |
+           apk --no-cache add postgresql-client
+           DB_URI=$(echo $DB_URI | sed "s|/\*|/|" )
+           psql "$DB_URI" -c "CREATE TABLE mygoodtable (id serial PRIMARY KEY);"
diff --git a/charts/paradedb/test/postgresql-minio-backup-restore/03-backup.yaml b/charts/paradedb/test/postgresql-minio-backup-restore/03-backup.yaml
@@ -0,0 +1,8 @@
+apiVersion: postgresql.cnpg.io/v1
+kind: Backup
+metadata:
+  name: post-init-backup
+spec:
+  method: barmanObjectStore
+  cluster:
+    name: standalone-paradedb
diff --git a/charts/paradedb/test/postgresql-minio-backup-restore/03-backup_completed-assert.yaml b/charts/paradedb/test/postgresql-minio-backup-restore/03-backup_completed-assert.yaml
@@ -0,0 +1,10 @@
+apiVersion: postgresql.cnpg.io/v1
+kind: Backup
+metadata:
+  name: post-init-backup
+spec:
+  cluster:
+    name: standalone-paradedb
+  method: barmanObjectStore
+status:
+  phase: completed
diff --git a/charts/paradedb/test/postgresql-minio-backup-restore/03-backup_running-assert.yaml b/charts/paradedb/test/postgresql-minio-backup-restore/03-backup_running-assert.yaml
@@ -0,0 +1,10 @@
+apiVersion: postgresql.cnpg.io/v1
+kind: Backup
+metadata:
+  name: post-init-backup
+spec:
+  cluster:
+    name: standalone-paradedb
+  method: barmanObjectStore
+status:
+  phase: running
diff --git a/charts/paradedb/test/postgresql-minio-backup-restore/03-checkpoint.yaml b/charts/paradedb/test/postgresql-minio-backup-restore/03-checkpoint.yaml
@@ -0,0 +1,27 @@
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: backup-checkpoint
+spec:
+  template:
+    spec:
+      restartPolicy: OnFailure
+      containers:
+      - name: create-checkpoint
+        env:
+          - name: DB_URI
+            valueFrom:
+              secretKeyRef:
+                name: standalone-paradedb-superuser
+                key: uri
+        image: alpine:3.19
+        command: ['sh', '-c']
+        args:
+         - |
+           apk --no-cache add postgresql-client
+           DB_URI=$(echo $DB_URI | sed "s|/\*|/|" )
+           END_TIME=$(( $(date +%s) + 30 ))
+           while [ $(date +%s) -lt $END_TIME ]; do
+             psql "$DB_URI" -c "SELECT pg_switch_wal();CHECKPOINT;"
+             sleep 5
+           done
diff --git a/charts/paradedb/test/postgresql-minio-backup-restore/04-post_backup_data_write-assert.yaml b/charts/paradedb/test/postgresql-minio-backup-restore/04-post_backup_data_write-assert.yaml
@@ -0,0 +1,6 @@
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: paradedb-enterprise-index-test
+status:
+  succeeded: 1
diff --git a/charts/paradedb/test/postgresql-minio-backup-restore/04-post_backup_data_write.yaml b/charts/paradedb/test/postgresql-minio-backup-restore/04-post_backup_data_write.yaml
@@ -0,0 +1,57 @@
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: configmap-creator-sa
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: Role
+metadata:
+  name: configmap-creator
+rules:
+- apiGroups: [""]
+  resources: ["configmaps"]
+  verbs: ["create"]
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  name: configmap-creator-binding
+subjects:
+- kind: ServiceAccount
+  name: configmap-creator-sa
+roleRef:
+  kind: Role
+  name: configmap-creator
+  apiGroup: rbac.authorization.k8s.io
+---
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: data-write-post-backup
+spec:
+  template:
+    spec:
+      serviceAccountName: configmap-creator-sa
+      restartPolicy: OnFailure
+      containers:
+      - name: data-write
+        env:
+          - name: DB_URI
+            valueFrom:
+              secretKeyRef:
+                name: standalone-paradedb-superuser
+                key: uri
+          - name: NAMESPACE
+            valueFrom:
+              fieldRef:
+                fieldPath: metadata.namespace
+        image: alpine:3.19
+        command: ['sh', '-c']
+        args:
+         - |
+           apk --no-cache add postgresql-client kubectl coreutils
+           DB_URI=$(echo $DB_URI | sed "s|/\*|/|" )
+           DATE_NO_BAD_TABLE=$(date --rfc-3339=ns)
+           sleep 30
+           psql "$DB_URI" -c "CREATE TABLE mybadtable (id serial PRIMARY KEY);"
+           kubectl create configmap date-no-bad-table --from-literal=date="$DATE_NO_BAD_TABLE"
diff --git a/charts/paradedb/test/postgresql-minio-backup-restore/05-recovery_backup_cluster-assert.yaml b/charts/paradedb/test/postgresql-minio-backup-restore/05-recovery_backup_cluster-assert.yaml
@@ -0,0 +1,6 @@
+apiVersion: postgresql.cnpg.io/v1
+kind: Cluster
+metadata:
+  name: pg-basebackup-paradedb
+status:
+  readyInstances: 2
diff --git a/charts/paradedb/test/postgresql-minio-backup-restore/05-recovery_backup_cluster.yaml b/charts/paradedb/test/postgresql-minio-backup-restore/05-recovery_backup_cluster.yaml
@@ -0,0 +1,48 @@
+type: postgresql
+mode: recovery
+
+cluster:
+  instances: 2
+  storage:
+    size: 256Mi
+
+recovery:
+  method: backup
+  backupName: "post-init-backup"
+  provider: s3
+  endpointURL: "https://minio.minio.svc.cluster.local"
+  endpointCA:
+    name: kube-root-ca.crt
+    key: ca.crt
+  wal:
+    encryption: ""
+  data:
+    encryption: ""
+  s3:
+    bucket: "mybucket"
+    path: "/postgresql-minio-backup-restore/v1"
+    accessKey: "minio"
+    secretKey: "minio123"
+    region: "local"
+  scheduledBackups: []
+  retentionPolicy: "30d"
+
+backups:
+  enabled: true
+  provider: s3
+  endpointURL: "https://minio.minio.svc.cluster.local"
+  endpointCA:
+    name: kube-root-ca.crt
+    key: ca.crt
+  wal:
+    encryption: ""
+  data:
+    encryption: ""
+  s3:
+    bucket: "mybucket"
+    path: "/postgresql-minio-backup-restore/v2"
+    accessKey: "minio"
+    secretKey: "minio123"
+    region: "local"
+  scheduledBackups: []
+  retentionPolicy: "30d"
diff --git a/charts/paradedb/test/postgresql-minio-backup-restore/06-data_test-assert.yaml b/charts/paradedb/test/postgresql-minio-backup-restore/06-data_test-assert.yaml
@@ -0,0 +1,6 @@
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: data-test-schemaonly
+status:
+  succeeded: 1
diff --git a/charts/paradedb/test/postgresql-minio-backup-restore/06-data_test.yaml b/charts/paradedb/test/postgresql-minio-backup-restore/06-data_test.yaml
@@ -0,0 +1,24 @@
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: data-test-schemaonly
+spec:
+  template:
+    spec:
+      restartPolicy: OnFailure
+      containers:
+      - name: data-test
+        env:
+          - name: DB_URI
+            valueFrom:
+              secretKeyRef:
+                name: import-schemaonly-paradedb-superuser
+                key: uri
+        image: alpine:3.19
+        command: ['sh', '-c']
+        args:
+         - |
+           apk --no-cache add postgresql-client
+           DB_URI=$(echo $DB_URI | sed "s|/\*|/|" )
+           test "$(psql "${DB_URI}mygooddb" -t -c 'SELECT EXISTS (SELECT FROM information_schema.tables WHERE table_name = $$mygoodtable$$)' --csv -q 2>/dev/null)" = "t"
+           test "$(psql "${DB_URI}mygooddb" -t -c 'SELECT EXISTS (SELECT FROM mygoodtable WHERE id = 314159265)' --csv -q 2>/dev/null)" = "f"
diff --git a/...aradedb/test/postgresql-minio-backup-restore/07-recovery_object_store_cluster-assert.yaml b/...aradedb/test/postgresql-minio-backup-restore/07-recovery_object_store_cluster-assert.yaml
@@ -0,0 +1,6 @@
+apiVersion: postgresql.cnpg.io/v1
+kind: Cluster
+metadata:
+  name: recovery-object-store-paradedb
+status:
+  readyInstances: 2