Skip to content

Commit

Permalink
feat: add chart for pgbench (cloudnative-pg#43)
Browse files Browse the repository at this point in the history
Signed-off-by: Jitendra Wadle <[email protected]>
Signed-off-by: Gabriele Bartolini <[email protected]>
Signed-off-by: Hai He <[email protected]>
Co-authored-by: Gabriele Bartolini <[email protected]>
Co-authored-by: Hai He <[email protected]>
  • Loading branch information
3 people authored Oct 12, 2022
1 parent 7ab800f commit 1892b17
Show file tree
Hide file tree
Showing 12 changed files with 797 additions and 19 deletions.
11 changes: 11 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ schema: ## Generate charts' schema usign helm schema-gen plugin
(echo "Please, run: helm plugin install https://github.com/karuppiah7890/helm-schema-gen.git" && exit 1)
@helm schema-gen charts/cnpg-sandbox/values.yaml > charts/cnpg-sandbox/values.schema.json || \
(@echo "Please, run: helm plugin install https://github.com/karuppiah7890/helm-schema-gen.git" && exit 1)
@helm schema-gen charts/pgbench/values.yaml > charts/pgbench/values.schema.json || \
(@echo "Please, run: helm plugin install https://github.com/karuppiah7890/helm-schema-gen.git" && exit 1)

.PHONY: sandbox-deploy
sandbox-deploy: ## Installs cnpg-sandbox chart
Expand All @@ -32,3 +34,12 @@ sandbox-deploy-dev: ## Installs cnpg-sandbox chart with a development version of
sandbox-uninstall: ## Uninstalls cnpg-sandbox chart if present
@helm uninstall cnpg-sandbox
@kubectl delete cluster cnpg-sandbox

.PHONY: pgbench-deploy
pgbench-deploy: ## Installs pgbench chart
helm dependency update charts/pgbench
helm upgrade --install pgbench --atomic charts/pgbench

.PHONY: pgbench-uninstall
pgbench-uninstall: ## Uninstalls cnpg-pgbench chart if present
@helm uninstall pgbench
160 changes: 141 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,16 +52,6 @@ provided Grafana dashboard.
- [Helm](https://helm.sh/) 3.7
- A supported Kubernetes cluster with enough RBAC permissions to deploy the required resources

## Deployment

Deployment using the latest release:

```console
helm repo add cnpg https://cloudnative-pg.github.io/charts
helm repo update
helm upgrade --install cnpg-sandbox \
cnpg/cnpg-sandbox
```

Then simply follow the instructions that will appear on the terminal once the
installation is completed.
Expand All @@ -84,24 +74,156 @@ make sandbox-uninstall

From the Grafana interface, you can find the dashboard by selecting: `Dashboards` > `Manage` > `CloudNativePg`.

## Benchmarking
# Benchmarking the database with `pgbench`

[pgbench](https://www.postgresql.org/docs/current/pgbench.html) is the default
benchmarking application for PostgreSQL. The chart for `pgbench` is contained
in the `charts/pgbench` directory.

**IMPORTANT:** the `pgbench` chart is considered experimental. It will be replaced
by a command in the `cnpg` plugin in the future.


## Deployment

Deployment using the latest release:

```console
helm repo add cnpg https://cloudnative-pg.github.io/charts
helm repo update
helm upgrade --install pgbench \
--namespace pgbench-ns \
--create-namespace \
charts/pgbench
```

Then simply follow the instructions that will appear on the terminal once the
installation is completed.

### Deployment from local source

To deploy the operator from sources you can run the following command:

```console
make pgbench-deploy
```

You can run a `pgbench` benchmark on:

- a disposable PostgreSQL cluster created by the CNPG operator specifically for
the benchmark
- an existing PostgreSQL cluster, by providing connection information (host,
port, database name, and user)

The `cnpg.existingCluster` option is the one that controls the above behavior.

While running a job on a cluster that lives for the sole duration of the test
is useful, we recommend that you first create your PostgreSQL cluster, possibly
with `cnpg-sandbox` installed, and then run `pgbench` on that cluster as explained
in the "Running `pgbench` on an existing Postgres cluster" section below.

## Running `pgbench` on a disposable CNPG cluster

When `cnpg.existingCluster` is set to `false` (default), the chart will:

1. Create a CNPG cluster based on the user-defined values;
1. Execute a user-defined `pgbench` job on it.

You can use the `kubectl wait` command to wait until the job is complete:

``` sh
kubectl wait --for=condition=complete -n pgbench-ns job/pgbench
```

It is suggested to label nodes and use node selectors to avoid pgbench and
PostgreSQL pods running on the same node. By default, the chart expects
the nodes on which pgbench can run to be labelled with `workload: pgbench`
and the node for CNPG instances to be labelled with `workload: postgres`.

``` sh
kubectl label node/NODE_NAME workload:pgbench
kubectl label node/OTHER_NODE_NAME workload:postgres
```

You can gather the results after the job is completed running:

``` sh
kubectl logs -n pgbench-ns job/pgbench
```

Below is an example of pgbench output:

```console
starting vacuum...end.
transaction type: <builtin: TPC-B (sort of)>
scaling factor: 1
query mode: simple
number of clients: 1
number of threads: 1
duration: 30 s
number of transactions actually processed: 16964
latency average = 1.768 ms
initial connection time = 9.924 ms
tps = 565.639903 (without initial connection time)
```

### Adding a connection pooler

CNPG has native support for the PgBouncer pooler. You can create a database
access layer with PgBouncer by managing the `cnpg.pooler` section of the values
file. By default, PgBouncer will be placed on those nodes with the `workload:
pooler` label.

Look at the `pgbench/values.yaml` for an example, as well as the CNPG
documentation for more information on the PgBouncer implementation.

### Running `pgbench` on an existing Postgres cluster

Suppose you already have your PostgreSQL database setup (not necessarily with CNPG).
You can use `pgbench` to run a `pgbench` test.


``` yaml
cnpg:
existingCluster: true
# Name of the host (or service in K8s) or IP address where Postgres is running
existingHost: mydb
# You need to point `existingCredentials` to a Kubernetes `basic-auth`secret
# containing username and password to connect to the database
existingCredentials: mydb-app
# Name of the database on which to run pgbench
existingDatabase: pgbench

pgbench:
# Node where to run pgbench
nodeSelector:
workload: pgbench
initialize: true
scaleFactor: 1
time: 30
clients: 1
jobs: 1
skipVacuum: false
reportLatencies: false
```
The `cnpg` section above, points to the existing database.

You can use `cnpg-sandbox` in conjuction with
[`cnp-bench`](https://github.com/EnterpriseDB/cnp-bench) to benchmark your
PostgreSQL environment and observe its behaviour in real-time.
The `pgbench` section contains the parameters you can use to run the `pgbench` job.
For example, you can create a job that initializes only the `pgbench` database
for a given scale with different settings of clients, time and jobs.

## Contributing
# Contributing

Please read the [code of conduct](CODE-OF-CONDUCT.md) and the
[guidelines](CONTRIBUTING.md) to contribute to the project.

## Disclaimer
# Disclaimer

`cnpg-sandbox`is open source software and comes "as is". Please carefully
read the [license](LICENSE) before you use this software, in particular
the "Disclaimer of Warranty" and "Limitation of Liability" items.

## Copyright
# Copyright

`cnpg-sandbox` is distributed under Apache License 2.0.
`cnpg` is distributed under Apache License 2.0.
Helm charts for CloudNativePG are distributed under Apache License 2.0.
23 changes: 23 additions & 0 deletions charts/pgbench/.helmignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Patterns to ignore when building packages.
# This supports shell glob matching, relative path matching, and
# negation (prefixed with !). Only one pattern per line.
.DS_Store
# Common VCS dirs
.git/
.gitignore
.bzr/
.bzrignore
.hg/
.hgignore
.svn/
# Common backup files
*.swp
*.bak
*.tmp
*.orig
*~
# Various IDEs
.project
.idea/
*.tmproj
.vscode/
28 changes: 28 additions & 0 deletions charts/pgbench/Chart.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
#
# Copyright The CloudNativePG Contributors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
apiVersion: v2
name: pgbench
description: A Helm chart that starts a CNPG Cluster and executes a
PgBench job on it.
type: application
version: 0.1.0
home: https://cloudnative-pg.io
sources:
- https://github.com/cloudnative-pg/charts

maintainers:
- name: Jitendra
email: [email protected]
50 changes: 50 additions & 0 deletions charts/pgbench/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# pgbench

![Version: 0.1.0](https://img.shields.io/badge/Version-0.1.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square)

A Helm chart that starts a CNPG Cluster and executes a PgBench job on it.

**Homepage:** <https://cloudnative-pg.io>

## Maintainers

| Name | Email | Url |
| ---- | ------ | --- |
| Jitendra | <[email protected]> | |

## Source Code

* <https://github.com/cloudnative-pg/charts>

## Values

| Key | Type | Default | Description |
|-----|------|---------|-------------|
| cnpg.existingCluster | bool | `false` | Whether the benchmark should be run against an existing cluster or a new one has to be created |
| cnpg.existingCredentials | string | `""` | The name of a Secret of type basic-auth containing the existing cluster credentials |
| cnpg.existingDatabase | string | `""` | The port where PostgreSQL is listening on the specified host (default: 5432) |
| cnpg.existingHost | string | `""` | The address of the existing cluster (default: empty) |
| cnpg.existingPort | string | `""` | The name of the existing database (default: empty) |
| cnpg.image | string | `"ghcr.io/cloudnative-pg/postgresql:14.5"` | The PostgreSQL image used by CNPG and PgBench. |
| cnpg.instances | int | `1` | The amount of PostgreSQL instances in the CNPG Cluster. |
| cnpg.monitoring | object | `{"customQueriesConfigMap":[],"customQueriesSecret":[]}` | Configures custom queries for monitoring. The arrays accept a Dictionary made by name: string (resource name), key: string (resource data field containing the queries). Documentation on the accepted values: https://docs.enterprisedb.io/cloud-native-postgresql/latest/monitoring/ |
| cnpg.nodeSelector | object | `{"workload":"postgres"}` | Dictionary of key-value pairs used to define the nodes where the cluster instances can run; used to avoid pgbench and PostgreSQL running on the same node. |
| cnpg.pooler.instances | int | `0` | The number of pooler replicas that receive the connections. If >0 the benchmarks are run with connection pooling |
| cnpg.pooler.nodeSelector.workload | string | `"pooler"` | |
| cnpg.pooler.pgbouncer.parameters | object | `{}` | PgBouncer configuration. |
| cnpg.pooler.pgbouncer.poolMode | string | `"session"` | The pool mode, accepted values: session, transaction |
| cnpg.postgreSQLParameters | object | `{"log_autovacuum_min_duration":"1s","log_checkpoints":"on","log_lock_waits":"on","log_min_duration_statement":"1000","log_statement":"ddl","log_temp_files":"1024","maintenance_work_mem":"128MB","shared_buffers":"512MB"}` | Dictionary of key-value pairs representing PostgreSQL configuration. |
| cnpg.storage.size | string | `"1Gi"` | The size of the PVCs used by CNPG instances. |
| cnpg.storage.storageClass | string | `""` | The storage class used to create PVCs for CNPG instances. |
| pgbench.clients | int | `1` | The number of clients used by pgbench. |
| pgbench.initialize | bool | `true` | Invoke the initialization mode (TPC-B-like test scenario) |
| pgbench.jobs | int | `1` | The number of jobs used by pgbench. |
| pgbench.nodeSelector | object | `{"workload":"pgbench"}` | Dictionary of key-value pairs used to define the nodes where the pgbench pod can run; used to avoid pgbench and PostgreSQL running on the same node. |
| pgbench.reportLatencies | bool | `false` | Report the average per-statement latency (execution time from the perspective of the client) of each command after the benchmark finishes. See below for details |
| pgbench.scaleFactor | int | `1` | Scale factor used to initialize pgbench (if initialize is set to true). |
| pgbench.skipVacuum | bool | `false` | Perform no vacuuming before running the test. |
| pgbench.time | int | `30` | The amount of seconds the pgbench will run for. |
| pgbench.warmTime | int | `0` | If >0, run an initContainer that runs pgbench for the defined amount of time (using the -T option) with the same clients and jobs that will be used for the main pgbench run; can be useful with storage classes that allow I/O bursts where could affect the actual benchmark result. |

----------------------------------------------
Autogenerated from chart metadata using [helm-docs v1.11.0](https://github.com/norwoodj/helm-docs/releases/v1.11.0)
26 changes: 26 additions & 0 deletions charts/pgbench/templates/NOTES.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
{{ if .Values.cnpg.nodeSelector -}}
Make sure at least a node has the following labels to be able to schedule the
{{ include "pgbench.fullname" . }} CNPG cluster:
{{- toYaml .Values.cnpg.nodeSelector | nindent 2}}
{{- end }}

{{ if .Values.pgbench.nodeSelector -}}
Make sure at least a node has the following labels to be able to schedule the
{{ include "pgbench.fullname" . }} job running pgbench:
{{- toYaml .Values.pgbench.nodeSelector | nindent 2}}
{{- end }}

{{ if .Values.cnpg.pooler.instances -}}
The benchmark is being run with connection pooling.
Make sure at least a node has the following labels to be able to schedule the
pooler-{{ include "pgbench.fullname" . }} pooler:
{{- toYaml .Values.cnpg.pooler.nodeSelector | nindent 2}}
{{ end }}

After the {{ include "pgbench.fullname" . }} job completes run

kubectl logs -n {{ .Release.Namespace }} job/{{ include "pgbench.fullname" . }}

to gather the pgbench results. You can then uninstall the chart

helm uninstall {{ .Release.Name }} -n {{ .Release.Namespace }}
Loading

0 comments on commit 1892b17

Please sign in to comment.