From 3fe6f2217a101460d3c01bb06eaf6aceb3c18349 Mon Sep 17 00:00:00 2001 From: gvasquezvargas Date: Thu, 5 Sep 2024 17:46:31 +0200 Subject: [PATCH 1/4] TDE: pg to pge tutotial while enabling TDE --- .../docs/tde/15/enabling_tde_epas.mdx | 2 + product_docs/docs/tde/15/index.mdx | 2 +- .../docs/tde/15/pg_upgrade_arguments.mdx | 2 +- .../index.mdx} | 5 +- .../postgres_to_extended.mdx | 160 ++++++++++++++++++ 5 files changed, 167 insertions(+), 4 deletions(-) rename product_docs/docs/tde/15/{upgrade_overview.mdx => upgrade_use_cases/index.mdx} (89%) create mode 100644 product_docs/docs/tde/15/upgrade_use_cases/postgres_to_extended.mdx diff --git a/product_docs/docs/tde/15/enabling_tde_epas.mdx b/product_docs/docs/tde/15/enabling_tde_epas.mdx index 7eef65b3f53..b4e2c993c11 100644 --- a/product_docs/docs/tde/15/enabling_tde_epas.mdx +++ b/product_docs/docs/tde/15/enabling_tde_epas.mdx @@ -28,6 +28,8 @@ and use `pg_upgrade` to transfer data from the existing source cluster to the ne ## Worked example +This example enables Transparent Data Encryption on an EDB Postgres Advanced Server version 16 running on an Ubuntu 22.04 machine. The location of the BIN and CONFIG directories differs depending on your operating system and Postgres versions. + ### Preparing your upgrade Use [pg_dumpall](https://www.postgresql.org/docs/current/app-pg-dumpall.html), [pgBackRest](/supported-open-source/pgbackrest/), or [Barman](/supported-open-source/barman/) to create a backup of your unencrypted source server. diff --git a/product_docs/docs/tde/15/index.mdx b/product_docs/docs/tde/15/index.mdx index cb02e5338b9..8766f4fa226 100644 --- a/product_docs/docs/tde/15/index.mdx +++ b/product_docs/docs/tde/15/index.mdx @@ -13,7 +13,7 @@ navigation: - troubleshooting - backups - "#Upgrading" -- upgrade_overview +- upgrade_use_cases - pg_upgrade_arguments - "#Testing" - single_user diff --git a/product_docs/docs/tde/15/pg_upgrade_arguments.mdx b/product_docs/docs/tde/15/pg_upgrade_arguments.mdx index d1d27ba2e0c..0aa60923fe6 100644 --- a/product_docs/docs/tde/15/pg_upgrade_arguments.mdx +++ b/product_docs/docs/tde/15/pg_upgrade_arguments.mdx @@ -15,7 +15,7 @@ For added certainty, if the old cluster is encrypted and the new cluster was ini See the description of the [initdb --copy-key-from=<file> option](enabling_tde/#using-initdb-tde-options) for information on copying a key from an existing cluster when preparing a new cluster as a target for `pg_upgrade`. -See [Tutorials](upgrade_overview/#tutorials) for `--copy-by-block` usage examples. +See [Tutorials](upgrade_use_cases/#tutorials) for `--copy-by-block` usage examples. ## `--key-unwrap-command=` diff --git a/product_docs/docs/tde/15/upgrade_overview.mdx b/product_docs/docs/tde/15/upgrade_use_cases/index.mdx similarity index 89% rename from product_docs/docs/tde/15/upgrade_overview.mdx rename to product_docs/docs/tde/15/upgrade_use_cases/index.mdx index c13032daad8..4d7ec8bc033 100644 --- a/product_docs/docs/tde/15/upgrade_overview.mdx +++ b/product_docs/docs/tde/15/upgrade_use_cases/index.mdx @@ -3,7 +3,7 @@ title: "TDE pg_upgrade use cases" navTitle: TDE pg_upgrade use cases --- -EDB supports using [pg_upgrade](https://www.postgresql.org/docs/current/pgupgrade.html) with additional [EDB upgrade arguments](pg_upgrade_arguments) to add encryption to unencrypted systems. +EDB supports using [pg_upgrade](https://www.postgresql.org/docs/current/pgupgrade.html) with additional [EDB upgrade arguments](../pg_upgrade_arguments) to add encryption to unencrypted systems. This table provides an overview of supported use cases. | Use case | Source unencrypted server | Target encrypted server | @@ -27,4 +27,5 @@ To enable encryption: ## Tutorials -* [Enable TDE on an existing EDB Postgres Advanced Server database cluster](enabling_tde_epas). \ No newline at end of file +* [Enable TDE on an existing EDB Postgres Advanced Server database cluster](../enabling_tde_epas). +* [Upgrade a PostgreSQL database server to EDB Postgres Extended Server while enabling TDE](postgres_to_extended). \ No newline at end of file diff --git a/product_docs/docs/tde/15/upgrade_use_cases/postgres_to_extended.mdx b/product_docs/docs/tde/15/upgrade_use_cases/postgres_to_extended.mdx new file mode 100644 index 00000000000..1bb10440f49 --- /dev/null +++ b/product_docs/docs/tde/15/upgrade_use_cases/postgres_to_extended.mdx @@ -0,0 +1,160 @@ +--- +title: "Upgrading PostgreSQL to EDB Postgres Extended Server while enabling TDE" +navTitle: Upgrading PostgreSQL to EDB Postgres Extended Server +deepToC: true +--- + +Create a new EDB Postgres Extended Server cluster with TDE enabled and use `pg_upgrade` to transfer data from the existing PostgreSQL cluster to the new encrypted cluster. + +- [Prepare your upgrade](#preparing-your-upgrade) by performing a backup of the existing instance. +- [Create a new database server](#creating-an-encrypted-server) + - Create an empty directory for the new server and ensure `postgres` owns it. + - Set the environment variables to export the `wrap` and `unwrap` commands for encryption. + - Initialize a server with encryption enabled. + - Change the default port, so the new server is available at another port. + - Start the database server. + - Connect to the database server and ensure it is functioning. +- [Upgrade to the encrypted server](#upgrading-to-the-encrypted-server) + - Stop both the source and the new server. + - Use `pg_upgrade` with `--copy-by-block` option to copy data from the source server to the new server. Specify the source and target bin and data directories. + - Start the new encrypted databaser server. + - Connect to the encrypted database server and ensure the data was transfered. +- [Clean up and delete the source server](#cleaning-up-after-upgrade) + - Clean up the database and its statistics. + - Remove the source EDB Postgres Advanced Server cluster with the script provided by `pg_upgrade`. + +## Worked example + +This example upgrades a PostgreSQL 16 instance to EDB Postgres Extended Server 16 while enabling Transparent Data Encryption on an Ubuntu 22.04 machine. The location of the BIN and CONFIG directories differs depending on your operating system and Postgres versions. + +### Preparing your upgrade + +- Install EDB Postgres Extended Server from the [EDB repository](https://www.enterprisedb.com/repos-downloads). Ensure you install a version within the same major version than the source database cluster. Upgrades between different major versions is not supported with `pg_upgrade`. + +- Use [pg_dumpall](https://www.postgresql.org/docs/current/app-pg-dumpall.html), [pgBackRest](/supported-open-source/pgbackrest/), or [Barman](/supported-open-source/barman/) to create a backup of your unencrypted source server. + +### Creating an encrypted server + +1. Create an empty directory for the new server. In this example, the directory name is **TDE**. + + ``` + mkdir /var/lib/edb-pge/16/TDE + ``` + +1. Ensure the `postgres` user owns the directory: + + ``` + sudo chown postgres /var/lib/edb-pge/16/TDE + sudo chgrp /var/lib/edb-pge/16/TDE + ``` + +1. Set environment variables to export the `wrap` and `unwrap` commands: + + ``` + export PGDATAKEYWRAPCMD='openssl enc -e -aes-128-cbc -pass pass:ok -out %p' + export PGDATAKEYUNWRAPCMD='openssl enc -d -aes-128-cbc -pass pass:ok -in %p' + ``` + + !!!note + Alternatively, use the `--key-unwrap-command=` and `--key-wrap-command=` arguments when initializing the encrypted server to include the `wrap` and `unwrap` commands. + + See [Using initdb TDE options](../enabling_tde/#using-initdb-tde-options) for more information on possible configurations. + +1. Initialize the new server with encryption: + + ``` + /usr/lib/edb-pge/16/bin/initdb --data-encryption -D /var/lib/edb-pge/16/TDE + ``` + + This command initializes a CONFIG directory with all configuration files for the encrypted server. + +1. Modify the default port number in the configuration file of the encrypted instance. Uncomment the line with `#port` and change the port number. For example: + + ``` + port 5590 + ``` + +1. Start the encrypted server: + + ``` + /usr/lib/edb-pge/16/bin/pg_ctl -D /var/lib/edb-pge/16/TDE start + ``` + +1. Connect to the server: + + ``` + /usr/lib/edb-pge/16/bin/psql -p 5590 + ``` + + !!!note + If you're using two different Postgres versions, use the psql utility of the encrypted server. Otherwise, the system will attempt to use psql from the previous instance. + +1. To ensure the new server is encrypted, [check for TDE presence](../enabling_tde/#checking-for-tde-presence-using-sql). + +### Upgrading to the encrypted server + +1. Stop both servers: + + ``` + /usr/lib/postgresql/16/bin/pg_ctl -D /var/lib/postgresql/16/non-TDE stop + /usr/lib/edb-pge/16/bin/pg_ctl -D /var/lib/edb-pge/16/TDE stop + ``` + +1. To test for incompatibilities, run the `pg_upgrade` command in check mode. + + With `-b` and `-B`, specify the source and target BIN directories. With `-d` and `-D`, specify the source and target CONFIG directories. + Include the `--copy-by-block` option. + + ``` + /usr/lib/edb-pge/16/bin/pg_upgrade -b /usr/lib/postgresql/16/bin -B /usr/lib/edb-pge/16/bin -d /var/lib/postgresql/16/non-TDE -D /var/lib/edb-pge/16/TDE --copy-by-block --check + ``` + + !!!note + The `--check` mode performs preliminary checks without executing the command. + +1. To copy data from the source server to the target server, run the `pg_upgrade` command in normal mode: + + ``` + /usr/lib/edb-pge/16/bin/pg_upgrade -b /usr/lib/postgresql/16/bin -B /usr/lib/edb-pge/16/bin -d /var/lib/postgresql/16/non-TDE -D /var/lib/edb-pge/16/TDE --copy-by-block + ``` + +1. Restart the encrypted server: + + ``` + /usr/lib/edb-pge/16/bin/pg_ctl -D /var/lib/edb-pge/16/TDE start + ``` + +1. Connect to the encrypted database server: + + ``` + /usr/lib/edb-pge/16/bin/psql -p 5590 + ``` + +1. Perform a spot check to ensure the databases, tables, schemas, and resources you had in the unencrypted server are available in the new server. For example, list all databases: + + ``` + \l + ``` + + Connect to a database, for example `hr`, and search for existing tables: + + ``` + \c hr + SELECT * FROM dept; + ``` + +### Cleaning up after upgrade + +After you verify that `pg_upgrade` encrypted the data successfully, perform a cleanup. + +1. As the `postgres` user, clean up the database and its statistics: + + ``` + /usr/lib/edb-pge/16/bin/vacuumdb --all --analyze-in-stages + ``` + +1. Remove all data files of the unencrypted server with the script generated by `pg_upgrade`: + + ``` + ./delete_old_cluster.sh + ``` From ea9b87cc6f81d7dd347c225b9c730beecfe41d67 Mon Sep 17 00:00:00 2001 From: gvasquezvargas Date: Fri, 6 Sep 2024 08:35:01 +0200 Subject: [PATCH 2/4] rebase and streamlining tutorial --- product_docs/docs/tde/15/enabling_tde_epas.mdx | 2 +- .../docs/tde/15/upgrade_use_cases/postgres_to_extended.mdx | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/product_docs/docs/tde/15/enabling_tde_epas.mdx b/product_docs/docs/tde/15/enabling_tde_epas.mdx index b4e2c993c11..2eb094d15dd 100644 --- a/product_docs/docs/tde/15/enabling_tde_epas.mdx +++ b/product_docs/docs/tde/15/enabling_tde_epas.mdx @@ -36,7 +36,7 @@ Use [pg_dumpall](https://www.postgresql.org/docs/current/app-pg-dumpall.html), [ ### Creating an encrypted server -1. Create an empty directory for the new server: +1. Create an empty directory for the new server. In this example, the directory name is **TDE**. ``` mkdir /var/lib/edb-as/16/TDE diff --git a/product_docs/docs/tde/15/upgrade_use_cases/postgres_to_extended.mdx b/product_docs/docs/tde/15/upgrade_use_cases/postgres_to_extended.mdx index 1bb10440f49..6632034d549 100644 --- a/product_docs/docs/tde/15/upgrade_use_cases/postgres_to_extended.mdx +++ b/product_docs/docs/tde/15/upgrade_use_cases/postgres_to_extended.mdx @@ -35,7 +35,7 @@ This example upgrades a PostgreSQL 16 instance to EDB Postgres Extended Server 1 ### Creating an encrypted server -1. Create an empty directory for the new server. In this example, the directory name is **TDE**. +1. Create an empty directory for the new server. In this example, the directory name is **TDE**. ``` mkdir /var/lib/edb-pge/16/TDE From d0a3bb64440beadd0f49fe99e13d78e54cdfa1b5 Mon Sep 17 00:00:00 2001 From: gvasquezvargas Date: Fri, 6 Sep 2024 09:57:24 +0200 Subject: [PATCH 3/4] Edited preparation steps --- .../docs/tde/15/upgrade_use_cases/postgres_to_extended.mdx | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/product_docs/docs/tde/15/upgrade_use_cases/postgres_to_extended.mdx b/product_docs/docs/tde/15/upgrade_use_cases/postgres_to_extended.mdx index 6632034d549..7fcc0a73473 100644 --- a/product_docs/docs/tde/15/upgrade_use_cases/postgres_to_extended.mdx +++ b/product_docs/docs/tde/15/upgrade_use_cases/postgres_to_extended.mdx @@ -21,7 +21,7 @@ Create a new EDB Postgres Extended Server cluster with TDE enabled and use `pg_u - Connect to the encrypted database server and ensure the data was transfered. - [Clean up and delete the source server](#cleaning-up-after-upgrade) - Clean up the database and its statistics. - - Remove the source EDB Postgres Advanced Server cluster with the script provided by `pg_upgrade`. + - Remove the source PostgreSQL cluster with the script provided by `pg_upgrade`. ## Worked example @@ -29,9 +29,9 @@ This example upgrades a PostgreSQL 16 instance to EDB Postgres Extended Server 1 ### Preparing your upgrade -- Install EDB Postgres Extended Server from the [EDB repository](https://www.enterprisedb.com/repos-downloads). Ensure you install a version within the same major version than the source database cluster. Upgrades between different major versions is not supported with `pg_upgrade`. +- Install EDB Postgres Extended Server from the [EDB repository](https://www.enterprisedb.com/repos-downloads). Ensure the version you install has the same major version than the source server. `pg_upgrade` supports upgrades between minor and patch versions, but not between different major versions. -- Use [pg_dumpall](https://www.postgresql.org/docs/current/app-pg-dumpall.html), [pgBackRest](/supported-open-source/pgbackrest/), or [Barman](/supported-open-source/barman/) to create a backup of your unencrypted source server. +- Use [pg_dumpall](https://www.postgresql.org/docs/current/app-pg-dumpall.html), [pgBackRest](/supported-open-source/pgbackrest/), or [Barman](/supported-open-source/barman/) to create a backup of your unencrypted source server. ### Creating an encrypted server From 3846670d6c813b63470fbe8e2ea2d70dbefd8cf7 Mon Sep 17 00:00:00 2001 From: gvasquezvargas Date: Wed, 25 Sep 2024 13:08:55 +0200 Subject: [PATCH 4/4] Implemented feedback from Tushar --- product_docs/docs/tde/15/enabling_tde_epas.mdx | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/product_docs/docs/tde/15/enabling_tde_epas.mdx b/product_docs/docs/tde/15/enabling_tde_epas.mdx index 2eb094d15dd..199e434121a 100644 --- a/product_docs/docs/tde/15/enabling_tde_epas.mdx +++ b/product_docs/docs/tde/15/enabling_tde_epas.mdx @@ -28,7 +28,9 @@ and use `pg_upgrade` to transfer data from the existing source cluster to the ne ## Worked example -This example enables Transparent Data Encryption on an EDB Postgres Advanced Server version 16 running on an Ubuntu 22.04 machine. The location of the BIN and CONFIG directories differs depending on your operating system and Postgres versions. +This example enables Transparent Data Encryption on an EDB Postgres Advanced Server version 16 running on an Ubuntu 22.04 machine. + +A similar workflow applies to other versions of EDB Postgres Advanced Server and EDB Postgres Extended Server. Note that the location of the BIN and CONFIG directories differs depending on your operating system and the Postgres version. ### Preparing your upgrade