Skip to content

Commit

Permalink
Merge pull request #4671 from EnterpriseDB/feature/p4k-sourcing/rewri…
Browse files Browse the repository at this point in the history
…te-private-repo-urls-to-public

Rewrite references to internal repo for P4K to public docs
  • Loading branch information
josh-heyer authored Sep 14, 2023
2 parents 9778081 + 6460d66 commit 7f40609
Show file tree
Hide file tree
Showing 15 changed files with 437 additions and 28 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ kubectl cnp pgbench \

You can see the progress of the job with:

```shell 
```shell
kubectl logs jobs/pgbench-run
```

Expand Down
2 changes: 1 addition & 1 deletion product_docs/docs/postgres_for_kubernetes/1/bootstrap.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ When only the streaming connection is defined, the source can be used for the
source can be used for the `recovery` method. When both are defined, any of the
two bootstrap methods can be chosen.

Furthermore, in case of `pg_basebackup` or full `recovery`point in time), the
Furthermore, in case of `pg_basebackup` or full `recovery` point in time, the
cluster is eligible for replica cluster mode. This means that the cluster is
continuously fed from the source, either via streaming, via WAL shipping
through the PostgreSQL's `restore_command`, or any of the two.
Expand Down
383 changes: 383 additions & 0 deletions product_docs/docs/postgres_for_kubernetes/1/default-monitoring.yaml

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ for a more comprehensive example.
ports, as explained in the official
[docs](https://cloud.google.com/kubernetes-engine/docs/how-to/private-clusters#add_firewall_rules)
and by this
[issue](https://github.com/EnterpriseDB/cloud-native-postgres/issues/1360).
[issue](https://github.com/cloudnative-pg/cloudnative-pg/issues/1360).
You'll need to either change the `targetPort` in the webhook service, to be
one of the allowed ones, or open the webhooks' port (`9443`) on the
firewall.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -941,4 +941,4 @@ A specific `VolumeSnapshotClass` can be requested via the `-c` option:

```shell
kubectl cnp snapshot cluster-example -c longhorn
```
```
15 changes: 8 additions & 7 deletions product_docs/docs/postgres_for_kubernetes/1/monitoring.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,8 @@ All monitoring queries that are performed on PostgreSQL are:
- executed with `application_name` set to `cnp_metrics_exporter`
- executed as user `postgres`

Please refer to the "Default roles" section in PostgreSQL
[documentation](https://www.postgresql.org/docs/current/default-roles.html)
Please refer to the "Predefined Roles" section in PostgreSQL
[documentation](https://www.postgresql.org/docs/current/predefined-roles.html)
for details on the `pg_monitor` role.

Queries, by default, are run against the *main database*, as defined by
Expand Down Expand Up @@ -363,7 +363,7 @@ go_threads 18
### User defined metrics

This feature is currently in *beta* state and the format is inspired by the
[queries.yaml file](https://github.com/prometheus-community/postgres_exporter/blob/master/queries.yaml) <!-- wokeignore:rule=master -->
[queries.yaml file (release 0.12)](https://github.com/prometheus-community/postgres_exporter/blob/v0.12.1/queries.yaml)
of the PostgreSQL Prometheus Exporter.

Custom metrics can be defined by users by referring to the created `Configmap`/`Secret` in a `Cluster` definition
Expand Down Expand Up @@ -442,7 +442,7 @@ data:
```

A list of basic monitoring queries can be found in the
[`default-monitoring.yaml` file](https://github.com/EnterpriseDB/cloud-native-postgres/blob/main/config/manager/default-monitoring.yaml)
[`default-monitoring.yaml` file](default-monitoring.yaml)
that is already installed in your EDB Postgres for Kubernetes deployment (see ["Default set of metrics"](#default-set-of-metrics)).

#### Example of a user defined metric running on multiple databases
Expand Down Expand Up @@ -543,7 +543,7 @@ Here is a short description of all the available fields:

- `<MetricName>`: the name of the Prometheus metric
- `query`: the SQL query to run on the target database to generate the metrics
- `primary`: whether to run the query only on the primary instance <!-- wokeignore:rule=master -->
- `primary`: whether to run the query only on the primary instance
- `master`: same as `primary` (for compatibility with the Prometheus PostgreSQL exporter's syntax - deprecated) <!-- wokeignore:rule=master -->
- `runonserver`: a semantic version range to limit the versions of PostgreSQL the query should run on
(e.g. `">=11.0.0"` or `">=12.0.0 <=15.0.0"`)
Expand Down Expand Up @@ -737,16 +737,17 @@ kubectl delete -f curl.yaml
These resources are provided for illustration and experimentation, and do
not represent any kind of recommendation for your production system

In the [`doc/src/samples/monitoring/`](https://github.com/EnterpriseDB/cloud-native-postgres/tree/main/docs/src/samples/monitoring)
In the [`doc/src/samples/monitoring/`](https://github.com/EnterpriseDB/docs/tree/main/product_docs/docs/postgres_for_kubernetes/1/samples/monitoring)
directory you will find a series of sample files for observability.
Please refer to [Part 4 of the quickstart](quickstart.md#part-4-monitor-clusters-with-prometheus-and-grafana)
section for context:

- `kube-stack-config.yaml`: a configuration file for the kube-stack helm chart
installation. It ensures that Prometheus listens for all PodMonitor resources.
- `postgresql-operator-prometheusrule.yaml`: a `PrometheusRule` with alerts for EDB Postgres for Kubernetes.
- `prometheusrule.yaml`: a `PrometheusRule` with alerts for EDB Postgres for Kubernetes.
NOTE: this does not include inter-operation with notification services. Please refer
to the [Prometheus documentation](https://prometheus.io/docs/alerting/latest/alertmanager/).
- `podmonitor.yaml`: a `PodMonitor` for the EDB Postgres for Kubernetes Operator deployment.
- `grafana-configmap.yaml`: a ConfigMap containing the definition of the sample
EDB Postgres for Kubernetes Dashboard. Note the labels in the definition, which ensure that
the Grafana deployment will find the ConfigMap.
Expand Down
2 changes: 1 addition & 1 deletion product_docs/docs/postgres_for_kubernetes/1/openshift.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -984,7 +984,7 @@ enabled, so you can peek the `cnp_` prefix:
![Prometheus queries](./images/openshift/prometheus-queries.png)

It is easy to define Alerts based on the default metrics as `PrometheusRules`.
You can find some examples of rules in the [cnp-prometheusrule.yaml](../samples/monitoring/cnp-prometheusrule.yaml)
You can find some examples of rules in the [prometheusrule.yaml](../samples/monitoring/prometheusrule.yaml)
file, which you can download.

Before applying the rules, again, some OpenShift setup may be necessary.
Expand Down
22 changes: 10 additions & 12 deletions product_docs/docs/postgres_for_kubernetes/1/operator_conf.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -134,23 +134,21 @@ Following the above example, if the `Cluster` definition contains a `categories`
annotation and any of the `environment`, `workload`, or `app` labels, these will
be inherited by all the resources generated by the deployment.

## PPROF HTTP SERVER
## pprof HTTP Server

The operator can expose a PPROF HTTP server with the following endpoints on localhost:6060:
The operator can expose a PPROF HTTP server with the following endpoints on `localhost:6060`:

```
- `/debug/pprof/`. Responds to a request for "/debug/pprof/" with an HTML page listing the available profiles
- `/debug/pprof/cmdline`. Responds with the running program's command line, with arguments separated by NUL bytes.
- `/debug/pprof/profile`. Responds with the pprof-formatted cpu profile. Profiling lasts for duration specified in seconds GET parameter, or for 30 seconds if not specified.
- `/debug/pprof/symbol`. Looks up the program counters listed in the request, responding with a table mapping program counters to function names.
- `/debug/pprof/trace`. Responds with the execution trace in binary form. Tracing lasts for duration specified in seconds GET parameter, or for 1 second if not specified.
```
- `/debug/pprof/`. Responds to a request for "/debug/pprof/" with an HTML page listing the available profiles
- `/debug/pprof/cmdline`. Responds with the running program's command line, with arguments separated by NULL bytes.
- `/debug/pprof/profile`. Responds with the pprof-formatted cpu profile. Profiling lasts for duration specified in seconds GET parameter, or for 30 seconds if not specified.
- `/debug/pprof/symbol`. Looks up the program counters listed in the request, responding with a table mapping program counters to function names.
- `/debug/pprof/trace`. Responds with the execution trace in binary form. Tracing lasts for duration specified in seconds GET parameter, or for 1 second if not specified.

To enable the operator you need to edit the operator deployment add the flag `--pprof-server=true`.

You can do this by executing these commands:

```
```shell
kubectl edit deployment -n postgresql-operator-system postgresql-operator-controller-manager
```

Expand All @@ -173,12 +171,12 @@ Save the changes, the deployment now will execute a rollout and the new pod will
Once the pod is running you can exec inside the container by doing:
```
```shell
kubectl exec -ti -n postgresql-operator-system <pod name> -- bash
```

Once inside execute:

```
```shell
curl localhost:6060/debug/pprof/
```
6 changes: 3 additions & 3 deletions product_docs/docs/postgres_for_kubernetes/1/quickstart.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -246,7 +246,7 @@ helm repo add prometheus-community \
https://prometheus-community.github.io/helm-charts

helm upgrade --install \
-f https://raw.githubusercontent.com/EnterpriseDB/cloud-native-postgres/main/docs/src/samples/monitoring/kube-stack-config.yaml \
-f https://raw.githubusercontent.com/EnterpriseDB/docs/main/product_docs/docs/postgres_for_kubernetes/1/samples/monitoring/kube-stack-config.yaml \
prometheus-community \
prometheus-community/kube-prometheus-stack
```
Expand Down Expand Up @@ -330,7 +330,7 @@ You can now define some alerts by creating a `prometheusRule`:

```sh
kubectl apply -f \
https://raw.githubusercontent.com/EnterpriseDB/cloud-native-postgres/main/docs/src/samples/monitoring/postgresql-operator-prometheusrule.yaml
https://raw.githubusercontent.com/EnterpriseDB/docs/main/product_docs/docs/postgres_for_kubernetes/1/samples/monitoring/prometheusrule.yaml
```

You should see the default alerts now:
Expand Down Expand Up @@ -361,7 +361,7 @@ We can now install our sample Grafana dashboard:

```sh
kubectl apply -f \
https://raw.githubusercontent.com/EnterpriseDB/cloud-native-postgres/main/docs/src/samples/monitoring/grafana-configmap.yaml
https://raw.githubusercontent.com/EnterpriseDB/docs/main/product_docs/docs/postgres_for_kubernetes/1/samples/monitoring/grafana-configmap.yaml
```

Which will be picked up by the Grafana page in a few seconds. You should now
Expand Down
2 changes: 1 addition & 1 deletion product_docs/docs/postgres_for_kubernetes/1/security.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,7 @@ PostgreSQL `Cluster` resource name.
[recommended way to access the API server from within a Pod](https://kubernetes.io/docs/tasks/run-application/access-api-from-pod/).

For transparency, the permissions associated with the service account are defined in the
[roles.go](https://github.com/EnterpriseDB/cloud-native-postgres/blob/main/pkg/specs/roles.go)
[roles.go](https://github.com/cloudnative-pg/cloudnative-pg/blob/main/pkg/specs/roles.go)
file. For example, to retrieve the permissions of a generic `mypg` cluster in the
`myns` namespace, you can type the following command:

Expand Down
22 changes: 22 additions & 0 deletions scripts/fileProcessor/processors/cnp/replace-github-urls.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
// Replace URLs beginning with the following patterns...
// - https://raw.githubusercontent.com/EnterpriseDB/cloud-native-postgres/main/config/manager/default-monitoring.yaml
// - https://github.com/EnterpriseDB/cloud-native-postgres/tree/main/docs/
// - https://raw.githubusercontent.com/EnterpriseDB/cloud-native-postgres/main/docs/src/
// ...with equivalent URLs referencing postgres for kubernetes in the public EDB Docs repo

const replacements = [
{pattern: /https:\/\/raw\.githubusercontent\.com\/EnterpriseDB\/cloud-native-postgres\/main\/config\/manager\/default-monitoring\.yaml/g, replacement: "default-monitoring.yaml"},
{pattern: /https:\/\/github\.com\/EnterpriseDB\/cloud-native-postgres\/tree\/main\/docs\/src\//g, replacement: "https://github.com/EnterpriseDB/docs/tree/main/product_docs/docs/postgres_for_kubernetes/1/"},
{pattern: /https:\/\/github\.com\/EnterpriseDB\/cloud-native-postgres\/blob\/main\/docs\/src\//g, replacement: "https://github.com/EnterpriseDB/docs/blob/main/product_docs/docs/postgres_for_kubernetes/1/"},
{pattern: /https:\/\/raw\.githubusercontent\.com\/EnterpriseDB\/cloud-native-postgres\/main\/docs\/src\//g, replacement: "https://raw.githubusercontent.com/EnterpriseDB/docs/main/product_docs/docs/postgres_for_kubernetes/1/"},
];

export const process = (filename, content) => {
for (const r of replacements)
content = content.replace(r.pattern, r.replacement);

return {
newFilename: filename,
newContent: content,
};
};
4 changes: 4 additions & 0 deletions scripts/source/process-cnp-docs.sh
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,16 @@ node $DESTINATION_CHECKOUT/scripts/source/files-to-ignore.mjs \

cd $SOURCE_CHECKOUT/docs

# grab key bit of source for use in docs
cp $SOURCE_CHECKOUT/config/manager/default-monitoring.yaml $SOURCE_CHECKOUT/docs/src/

node $DESTINATION_CHECKOUT/scripts/fileProcessor/main.mjs \
-f "src/**/quickstart.md" \
-p cnp/add-quickstart-content

node $DESTINATION_CHECKOUT/scripts/fileProcessor/main.mjs \
-f "src/**/*.md" \
-p "cnp/replace-github-urls" \
-p "cnp/update-yaml-links" \
-p "cnp/add-frontmatters" \
-p "cnp/rename-to-mdx"
Expand Down
1 change: 1 addition & 0 deletions scripts/source/process-pgd4k-docs.sh
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ cd $SOURCE_CHECKOUT/docs

node $DESTINATION_CHECKOUT/scripts/fileProcessor/main.mjs \
-f "src/**/*.md" \
-p "cnp/replace-github-urls" \
-p "cnp/update-yaml-links" \
-p "cnp/add-frontmatters" \
-p "cnp/rename-to-mdx"
Expand Down

1 comment on commit 7f40609

@github-actions
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please sign in to comment.