diff --git a/install_template/templates/products/edb-jdbc-connector/index.njk b/install_template/templates/products/edb-jdbc-connector/index.njk index 3e5551f8b2d..8b7166578c1 100644 --- a/install_template/templates/products/edb-jdbc-connector/index.njk +++ b/install_template/templates/products/edb-jdbc-connector/index.njk @@ -23,6 +23,7 @@ legacyRedirectsGenerated: {% block navigation %} {{- super() -}} - windows +- using_maven - configuring_for_java - upgrading {% endblock navigation %} diff --git a/product_docs/docs/biganimal/release/administering_cluster/projects.mdx b/product_docs/docs/biganimal/release/administering_cluster/projects.mdx index b996e91b73e..fb5d528b9fa 100644 --- a/product_docs/docs/biganimal/release/administering_cluster/projects.mdx +++ b/product_docs/docs/biganimal/release/administering_cluster/projects.mdx @@ -89,12 +89,12 @@ To delete a project that you created: 1. From the **Settings** list, select **Security**. 1. Select **Add a key**. 1. On the **Add a key** page, select the **Cloud Service Provider**. -1. Select the **Region** for the key. The interface only displays the regions available in the cloud account you configured. +1. Select the **Region** for the key. Only the regions available in the cloud account you configured are listed. 1. Complete the remaining fields according to your cloud provider. -1. Select **Add Key** to finalize the configuration. +1. To finalize the configuration, select **Add Key**. !!!note Note for GCP keys - If the key you added was created in a different Google Cloud Platform account than the TDE-enabled cluster you want to create, ensure you enable the [Cloud KMS API](https://cloud.google.com/kms/docs/create-encryption-keys#before-you-begin) in the Google console before assigning it to your cluster in BigAnimal. + If the key you added was created in a Google Cloud Platform account different from the TDE-enabled cluster you want to create, ensure you enable the [Cloud KMS API](https://cloud.google.com/kms/docs/create-encryption-keys#before-you-begin) in the Google console before assigning it to your cluster in BigAnimal. Now, use this TDE key to [create a cluster](/biganimal/release/getting_started/creating_a_cluster/#security). diff --git a/product_docs/docs/biganimal/release/getting_started/creating_a_cluster/creating_a_dha_cluster.mdx b/product_docs/docs/biganimal/release/getting_started/creating_a_cluster/creating_a_dha_cluster.mdx index 7adca3144b5..e21e9b904e8 100644 --- a/product_docs/docs/biganimal/release/getting_started/creating_a_cluster/creating_a_dha_cluster.mdx +++ b/product_docs/docs/biganimal/release/getting_started/creating_a_cluster/creating_a_dha_cluster.mdx @@ -140,7 +140,7 @@ The following options aren't available when creating your cluster: 1. In the **Networking** section: - In **Connectivity Type**, specify whether to use private or public networking. Networking is set to **Public** by default. Public means that any client can connect to your cluster's public IP address over the internet. Optionally, you can limit traffic to your public cluster by specifying an IP allowlist, which allows access only to certain blocks of IP addresses. To limit access, tick the **Use allowlists** box and add one or more classless inter-domain routing (CIDR) blocks. CIDR is a method for allocating IP addresses and IP routing to a whole network or subnet. If you have any CIDR block entries, access is limited to those IP addresses. If none are specified, all network traffic is allowed. + In **Connectivity Type**, specify whether to use private or public networking. Networking is set to **Public** by default. Public means that any client can connect to your cluster's public IP address over the internet. Optionally, you can limit traffic to your public cluster by specifying an IP allowlist, which allows access only to certain blocks of IP addresses. To limit access, select **Use allowlists** and add one or more classless inter-domain routing (CIDR) blocks. CIDR is a method for allocating IP addresses and IP routing to a whole network or subnet. If you have any CIDR block entries, access is limited to those IP addresses. If none are specified, all network traffic is allowed. Private networking allows only IP addresses in your private network to connect to your cluster. diff --git a/product_docs/docs/biganimal/release/getting_started/creating_a_cluster/index.mdx b/product_docs/docs/biganimal/release/getting_started/creating_a_cluster/index.mdx index d726de29a77..28ebf644a96 100644 --- a/product_docs/docs/biganimal/release/getting_started/creating_a_cluster/index.mdx +++ b/product_docs/docs/biganimal/release/getting_started/creating_a_cluster/index.mdx @@ -43,7 +43,7 @@ The following options aren't available when creating your cluster: - [Primary/Standby High Availability](../../overview/02_high_availability/primary_standby_highavailability/) creates a cluster with one primary and one or two standby replicas in different availability zones. You can create primary/standby high-availability clusters running PostgreSQL, EDB Postgres Extended Server or EDB Postgres Advanced Server. If you enable read-only workloads, then you might have to raise the IP address resource limits for the cluster. - - [Distributed High Availability](../../overview/02_high_availability/distributed_highavailability/) creates a cluster, powered by EDB Postgres Distributed, with up to two data groups spread across multiple cloud regions to deliver higher performance and faster recovery. You can create distributed high availability clusters running PostgreSQL, EDB Postgres Extended Server or EDB Postgres Advanced Server. See [Creating a distributed high-availability cluster](creating_a_dha_cluster) for instructions. + - [Distributed High Availability](../../overview/02_high_availability/distributed_highavailability/) creates a cluster, powered by EDB Postgres Distributed, with up to two data groups spread across multiple cloud regions to deliver higher performance and faster recovery. You can create distributed high-availability clusters running PostgreSQL, EDB Postgres Extended Server, or EDB Postgres Advanced Server. See [Creating a distributed high-availability cluster](creating_a_dha_cluster) for instructions. See [Supported cluster types](/biganimal/latest/overview/02_high_availability/) for more information about the different cluster types. @@ -143,7 +143,7 @@ The following options aren't available when creating your cluster: 1. In the **Network, Logs, & Telemetry** section: - In **Connectivity Type**, specify whether to use private or public networking. Networking is set to **Public** by default. Public means that any client can connect to your cluster’s public IP address over the internet. Optionally, you can limit traffic to your public cluster by specifying an IP allowlist, which allows access only to certain blocks of IP addresses. To limit access, tick the **Use allowlists** box and add one or more classless inter-domain routing (CIDR) blocks. CIDR is a method for allocating IP addresses and IP routing to a whole network or subnet. If you have any CIDR block entries, access is limited to those IP addresses. If none are specified, all network traffic is allowed. + In **Connectivity Type**, specify whether to use private or public networking. Networking is set to **Public** by default. Public means that any client can connect to your cluster’s public IP address over the internet. Optionally, you can limit traffic to your public cluster by specifying an IP allowlist, which allows access only to certain blocks of IP addresses. To limit access, select **Use allowlists** and add one or more classless inter-domain routing (CIDR) blocks. CIDR is a method for allocating IP addresses and IP routing to a whole network or subnet. If you have any CIDR block entries, access is limited to those IP addresses. If none are specified, all network traffic is allowed. Private networking allows only IP addresses in your private network to connect to your cluster. @@ -233,29 +233,27 @@ Enable **Superuser Access** to grant superuser privileges to the edb_admin role. Enable **Transparent Data Encryption (TDE)** to use your own encryption key. This option is available for EDB Postgres Advanced Server and EDB Postgres Extended Server for version 15 and later. Select an encryption key from your project and region to encrypt the cluster with TDE. To learn more about TDE support, see [Transparent Data Encryption](../../overview/03_security/#your-own-encryption-key---transparent-data-encryption-tde). !!!Note "Important" -- To enable and use TDE for a cluster, the encryption key must be enabled and added at the project level before creating a cluster. +- To enable and use TDE for a cluster, you must first enable the encryption key and add it at the project level before creating a cluster. To add a key, see [Adding a TDE key at project level](../../administering_cluster/projects.mdx/#adding-a-tde-key). - To enable and use TDE for a cluster, you must complete the configuration on the platform of your key management provider after creating a cluster. See [Completing the TDE configuration](#completing-the-TDE-configuration) for more information. !!! #### Completing the TDE configuration -After you create the cluster in the BigAnimal console, the UI will display the **Waiting for access to encryption key** state. To complete the configuration and enable the key sync between BigAnimal and the key management platform you must grant encrypt and decrypt permissions to your key: +After you create the cluster in the BigAnimal console, the cluster is in the **Waiting for access to encryption key** state. To complete the configuration and enable the key sync between BigAnimal and the key management platform, you must grant encrypt and decrypt permissions to your key: 1. In BigAnimal, select the cluster name and access the cluster's page. See the **Action required: grant key permissions to activate the cluster**. -1. Copy the **Principal** identifier (AWS), **service account** (GCP) or **MSI Workload Identity** (Azure) to your clipboard. - -1. Follow the on-screen guide to grant encrypt and decrypt permissions to your key. Here is additional information in case you require further guidance: +1. Follow the on-screen guide to grant encrypt and decrypt permissions to your key. The instructions differ depending on the cloud provider of your key. Some additional guidance:
AWS 1. Copy the **Principal** identifier to your clipboard. - 1. Go to the AWS console, and navigate to the **Key Management Service** + 1. Go to the AWS console, and navigate to the **Key Management Service**. 1. Select **Customer-managed keys**, and **Edit policy** for your key. - 1. Append a new policy statement where the `Principal.AWS` field equals the **Principal** identifier you copied to your clipboard and where the `Principal.Action` field contains **kms:Encrypt** and **kms:Decrypt** permissions. + 1. Append a new policy statement where the `Principal.AWS` field contains the **Principal** identifier you copied earlier and the `Principal.Action` field contains **kms:Encrypt** and **kms:Decrypt** permissions. - This example contains the default AWS policy statement and the BigAnimal policy statement that corresponds to the TDE configuration. + This example contains the default AWS policy statement and the BigAnimal policy statement that corresponds to the TDE configuration: ``` { @@ -292,7 +290,7 @@ After you create the cluster in the BigAnimal console, the UI will display the *
GCP 1. Copy the **service account** to your clipboard. - 1. Go to the Google Cloud console, select **Security**, **VIEW BY PRINCIPALS**, **GRANT ACCESS** for your key. + 1. On the Google Cloud console, select **Security**, **VIEW BY PRINCIPALS**, and **GRANT ACCESS** for your key. 1. Paste the service account into the **New principals** field. 1. Assign the `Cloud KMS CryptoKey Decrypter` and `Cloud KMS CryptoKey Encrypter` roles and save. @@ -301,11 +299,12 @@ After you create the cluster in the BigAnimal console, the UI will display the *
Azure 1. Copy the **MSI Workload Identity** to your clipboard. - 1. Got to the Microsoft Azure console, and navigate to **Key vaults**. - 1. Select the key, go to **Access configuration** and set the **Permission model** to **Vault access policy**. - 1. Select **Access policies**, and **Create**. + 1. On the Microsoft Azure console, navigate to **Key vaults**. + 1. Select the key. Go to **Access configuration** and set the **Permission model** to **Vault access policy**. + 1. Select **Access policies**. + 1. Select **Create**. 1. In **Permissions**, select **Encrypt** and **Decrypt**. - 1. In **Principal**, paste the MSI Workload Identity you copied to your clipboard and finish creating the policy. + 1. In **Principal**, paste the MSI Workload Identity you copied earlier and finish creating the policy.
diff --git a/product_docs/docs/biganimal/release/overview/02_high_availability/distributed_highavailability.mdx b/product_docs/docs/biganimal/release/overview/02_high_availability/distributed_highavailability.mdx index b3a5a84741e..d34d13fe545 100644 --- a/product_docs/docs/biganimal/release/overview/02_high_availability/distributed_highavailability.mdx +++ b/product_docs/docs/biganimal/release/overview/02_high_availability/distributed_highavailability.mdx @@ -59,25 +59,25 @@ Cross-cloud service provider witness nodes are available with AWS, Azure, and Go ## Read-only workloads -When you enable the read-only workloads option during the cluster creation, a read-only connection string is created for the data group. You can use this connection to allow your application or service to route read-only requests through the shadow nodes (non-write leaders) to lighten the load on the write leaders and improve the Distributed High Availability cluster's performance. +When you enable the read-only workloads option during the cluster creation, a read-only connection string is created for the data group. You can use this connection to allow your application or service to route read-only requests through the shadow nodes (non-write leaders) to lighten the load on the write leaders and improve the distributed high-availability cluster's performance. If you have more than one data group, you can choose whether to enable the read-only workloads option on a per-data-group basis. -Since the infrastructure of a Distributed High Availability cluster is almost entirely based on EDB Postgres Distributed, the same [PGD Proxy read-only routing rules](/pgd/latest/routing/readonly/) apply. +Since the infrastructure of a distributed high-availability cluster is almost entirely based on EDB Postgres Distributed, the same [PGD Proxy read-only routing rules](/pgd/latest/routing/readonly/) apply. !!! Important Once you have configured a read-only connection string with your application, read-only workloads are routed to shadow nodes (non-write leaders). The connection is read-only because it accepts read-only queries through the shadow nodes. However, commands that run on read-only connections aren't filtered by BigAnimal, so shadow nodes can still perfom write operations to the contents of database tables. We recommend that you use a Postgres role with minimal read-only privileges for your application or pass `default_transaction_read_only=on` in the connection string. This way, you can avoid write operations on shadow nodes that could cause conflicts between the write leader and the shadow nodes. -**IP addresses** +### IP addresses -If you are configuring read-only workload connections on your own cloud account, you might have to raise the IP address resource limits for the cluster: +If you're configuring read-only workload connections on your own cloud account, you might have to raise the IP address resource limits for the cluster: -- For Azure, the IP address quota is Standard Public IP Address. +- For Azure, the IP address quota is standard public IP address. - For AWS, the IP address quota is Elastic IP. You might also have to increase the **Network Load Balancers per Region** value. -**Advisory locks** +### Advisory locks Advisory locks aren't replicated between Postgres nodes, so advisory locks taken on a shadow node don't conflict with advisory locks taken on the lead. We recommend that applications that rely on advisory locking avoid using read-only workloads for those transactions. diff --git a/product_docs/docs/biganimal/release/overview/03_security/index.mdx b/product_docs/docs/biganimal/release/overview/03_security/index.mdx index 4194e204d65..d90580eb892 100644 --- a/product_docs/docs/biganimal/release/overview/03_security/index.mdx +++ b/product_docs/docs/biganimal/release/overview/03_security/index.mdx @@ -25,14 +25,14 @@ All data in BigAnimal is encrypted in motion and at rest. Network traffic is enc ### Your own encryption key - Transparent Data Encryption (TDE) -Optionally enable Transparent Data Encryption (TDE) at the database level on BigAnimal's cloud account, on AWS, GCP or Azure. TDE encrypts all data files, the write-ahead log (WAL) and temporary files used during query processing and database system operations. +Optionally enable Transparent Data Encryption (TDE) at the database level on BigAnimal's cloud account, AWS, GCP, or Azure. TDE encrypts all data files, the write-ahead log (WAL), and temporary files used during query processing and database system operations. You can't enable nor disable TDE on existing clusters. To enable TDE, first connect the encryption keys to BigAnimal at the project level, and then select those keys while creating a cluster. -EDB supports enabling TDE with your own encryption key on Single Node and Primary/Standby High Availability deployments running EDB Postgres Advanced Server or EDB Postgres Extended Server versions 15 and later. +EDB supports enabling TDE with your own encryption key on single-node and primary/standby high-availability deployments running EDB Postgres Advanced Server or EDB Postgres Extended Server versions 15 and later. Both the key and cluster must be in the same region and hosted by the same underlying cloud provider. -This overview shows the supported cluster-to-key combinations: +This overview shows the supported cluster-to-key combinations. | | AWS cluster (BYOA) | AWS cluster (BAH) | GCP cluster (BYOA) | GCP cluster (BAH) | Azure cluster (BYOA) | Azure cluster (BAH) | |-----------------------------|--------------------|-------------------|--------------------|-------------------|----------------------|---------------------| @@ -41,15 +41,15 @@ This overview shows the supported cluster-to-key combinations: | Azure Key Vault | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | -**BYOA or Bring your own account:** BigAnimal deploys the cluster on your own cloud provider account. +**BYOA or bring-your-own-account:** BigAnimal deploys the cluster on your own cloud provider account. -**BAH or BigAnimal hosted:** BigAnimal deploys the cluster on a cloud provider account owned and managed by EDB. +**BAH or BigAnimal hosted:** BigAnimal deploys the cluster on a cloud provider account owned and managed by EDB. !!!note - The process of encryption and decryption adds additional overhead in terms of CPU and RAM consumption, performance, and for managing keys for faraway replicas. + The process of encryption and decryption adds overhead in terms of CPU and RAM consumption, performance, and for managing keys for faraway replicas. !!! -**To enable TDE**: +To enable TDE: - Before you create a TDE-enabled cluster, you must [add a TDE key](../../administering_cluster/projects##adding-a-tde-key). diff --git a/product_docs/docs/biganimal/release/release_notes/2024_05_may_rel_notes.mdx b/product_docs/docs/biganimal/release/release_notes/2024_05_may_rel_notes.mdx index abf6990f60b..a0016154b9c 100644 --- a/product_docs/docs/biganimal/release/release_notes/2024_05_may_rel_notes.mdx +++ b/product_docs/docs/biganimal/release/release_notes/2024_05_may_rel_notes.mdx @@ -5,7 +5,7 @@ navTitle: May 2024 BigAnimal's May 2024 release includes the following enhancements and bug fixes: -| Type | Description | -|-------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| Type | Description | +|-------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Enhancement | As part of this launch, we rebranded BigAnimal to the EDB Postgres AI Cloud Service. Our cloud service is part of a broader launch of [EDB Postgres AI](https://www.enterprisedb.com/products/edb-postgres-ai). EDB Postgres AI is an intelligent platform for transactional, analytical, and AI workloads managed across hybrid and multi-cloud environments.

Through the EDB Postgres AI Console, you can now discover new capabilities including:

• **Analytics**: Postgres Lakehouse clusters are now available in the EDB Postgres AI Cloud Service, enabling 30X average faster analytical queries versus standard Postgres. This feature is initially limited to AWS-hosted customers only.

• **Hybrid data estate management**: You can now have visibility into your self-managed Postgres databases, RDS Postgres databases, and EDB Postgres AI Cloud Service DB clusters from a “single pane of glass.” Using an installable agent, you can collect metadata from your self-managed DBs and view it in the EDB Postgres AI Console.

With this release, we also added a new **Health Status** tab to the cluster view. This tab provides real-time insight into the topology and heath of the cluster, including the role of each node in the cluster, the status of each replication slot, and a selection of key health metrics. | -| Enhancement | BigAnimal now supports the pg_squeeze extension for clusters running on PostgreSQL, EDB Postgres Extended Server, and EDB Postgres Advanced Server. Learn more about how pg_squeeze can enhance disk space efficiency and improve query performance [here](https://www.enterprisedb.com/docs/pg_extensions/pg_squeeze/). | +| Enhancement | BigAnimal now supports the pg_squeeze extension for clusters running on PostgreSQL, EDB Postgres Extended Server, and EDB Postgres Advanced Server. Learn more about how pg_squeeze can enhance disk space efficiency and improve query performance [here](https://www.enterprisedb.com/docs/pg_extensions/pg_squeeze/). | diff --git a/product_docs/docs/biganimal/release/release_notes/2024_06_jun_rel_notes.mdx b/product_docs/docs/biganimal/release/release_notes/2024_06_jun_rel_notes.mdx new file mode 100644 index 00000000000..5c3bc2d3ea6 --- /dev/null +++ b/product_docs/docs/biganimal/release/release_notes/2024_06_jun_rel_notes.mdx @@ -0,0 +1,16 @@ +--- +title: BigAnimal June 2024 release notes +navTitle: June 2024 +--- + +BigAnimal's June 2024 release includes the following enhancements and bug fixes: + +| Type | Description | +|-------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| Enhancement | Added support for PGD 5.5.1 on Cloud Service. | +| Enhancement | Added [read-only endpoints for shadow nodes on PGD](/biganimal/latest/overview/02_high_availability/distributed_highavailability/#read-only-workloads) on Cloud Service. Service. | +| Enhancement | Added Volume Snapshot Backups for single region. | +| Enhancement | Added support for TDE on Microsoft Azure and Google Cloud. | +| Enhancement | Added PgBouncer and PGD-Proxy co-location with PG nodes. | +| Enhancement | Added support for Query Diagnostics Wait states. | +| Enhancement | EDB Postgres AI CLI v3.7.1 is now available. Learn more about what’s new [here](https://cli.biganimal.com/versions/v3.7.1). | \ No newline at end of file diff --git a/product_docs/docs/biganimal/release/release_notes/index.mdx b/product_docs/docs/biganimal/release/release_notes/index.mdx index ef653adfe6f..d793fbe9f52 100644 --- a/product_docs/docs/biganimal/release/release_notes/index.mdx +++ b/product_docs/docs/biganimal/release/release_notes/index.mdx @@ -2,6 +2,7 @@ title: BigAnimal release notes navTitle: Release notes navigation: +- 2024_06_jun_rel_notes - 2024_05_may_rel_notes - 2024_04_apr_rel_notes - 2024_03_mar_rel_notes @@ -25,6 +26,7 @@ The BigAnimal documentation describes the latest version of BigAnimal, including | 2024 | |--------------------------------------| +| [June 2024](2024_06_jun_rel_notes) | | [May 2024](2024_05_may_rel_notes) | | [April 2024](2024_04_apr_rel_notes) | | [March 2024](2024_03_mar_rel_notes) | diff --git a/product_docs/docs/jdbc_connector/42.5.4.2/installing/index.mdx b/product_docs/docs/jdbc_connector/42.5.4.2/installing/index.mdx index bfba199284c..52878f5dff8 100644 --- a/product_docs/docs/jdbc_connector/42.5.4.2/installing/index.mdx +++ b/product_docs/docs/jdbc_connector/42.5.4.2/installing/index.mdx @@ -27,6 +27,7 @@ navigation: - linux_x86_64 - linux_arm64 - windows + - using_maven - configuring_for_java - upgrading --- diff --git a/product_docs/docs/jdbc_connector/42.5.4.2/installing/using_maven.mdx b/product_docs/docs/jdbc_connector/42.5.4.2/installing/using_maven.mdx new file mode 100644 index 00000000000..38b2c7533d0 --- /dev/null +++ b/product_docs/docs/jdbc_connector/42.5.4.2/installing/using_maven.mdx @@ -0,0 +1,20 @@ +--- +title: "Installing EDB JDBC Connector using Maven" +navTitle: "Using Maven" +--- + +EDB supports installing EDB JDBC Connector using the Maven dependency manager. +EDB-JDBC is published in the [Maven Central Repository](https://central.sonatype.com/artifact/com.enterprisedb/edb-jdbc) with the following groupId and artifactId: + +- groupId: `com.enterprisedb` +- artifactId: `edb-jdbc` + +Add the following dependency for EDB-JDBC in your `pom.xml` file to install and configure the EDB JDBC Connector. Ensure you provide the correct version to install: + +``` + + com.enterprisedb + edb-jdbc + 42.5.4.2 + +``` diff --git a/product_docs/docs/migration_toolkit/55/07_invoking_mtk/08_mtk_command_options.mdx b/product_docs/docs/migration_toolkit/55/07_invoking_mtk/08_mtk_command_options.mdx index fec55811d50..12040e45338 100644 --- a/product_docs/docs/migration_toolkit/55/07_invoking_mtk/08_mtk_command_options.mdx +++ b/product_docs/docs/migration_toolkit/55/07_invoking_mtk/08_mtk_command_options.mdx @@ -1,7 +1,6 @@ --- title: "Migration Toolkit command options" navTitle: "Command options" - redirects: - /migration_toolkit/latest/08_mtk_command_options/ --- @@ -10,7 +9,7 @@ redirects: To control details of the migration, append migration options when you run Migration Toolkit. For example, to migrate all schemas in a database, append the `-allSchemas` option to the command: -`$ ./runMTK.sh -allSchemas` +`./runMTK.sh -allSchemas` !!! Note - The `-allSchemas` parameter is supported only for the Oracle, EDB Postgres Advanced Server, and PostgreSQL source database. It isn't supported for Sybase, MS SQL Server, and MySQL source databases. @@ -42,37 +41,37 @@ If you specify the `-offlineMigration` option in the command line, Migration Too To perform an offline migration of both schema and data, specify the `‑offlineMigration` keyword, followed by the schema name: ```shell -$ ./runMTK.sh -offlineMigration +./runMTK.sh -offlineMigration ``` Each database object definition is saved in a separate file with a name derived from the schema name and object type in your home folder. To specify an alternative file destination, include a directory name after the `‑offlineMigration` option: ```shell -$ ./runMTK.sh -offlineMigration +./runMTK.sh -offlineMigration ``` To perform an offline migration of only schema objects (creating empty tables), specify the `‑schemaOnly` keyword in addition to the `‑offlineMigration` keyword when invoking Migration Toolkit: ```shell -$ ./runMTK.sh -offlineMigration -schemaOnly +./runMTK.sh -offlineMigration -schemaOnly ``` To perform an offline migration of only data, omitting any schema object definitions, specify the `‑dataOnly` keyword and the `‑offlineMigration` keyword when invoking Migration Toolkit: ```shell -$ ./runMTK.sh -offlineMigration -dataOnly +./runMTK.sh -offlineMigration -dataOnly ``` By default, data is written in COPY format. To write the data in a plain SQL format, include the `‑safeMode` keyword: ```shell -$ ./runMTK.sh -offlineMigration -dataOnly -safeMode +./runMTK.sh -offlineMigration -dataOnly -safeMode ``` By default, when you perform an offline migration that contains table data, a separate file is created for each table. To create a single file that contains the data from multiple tables, specify the `‑singleDataFile` keyword: ```shell -$ ./runMTK.sh -offlineMigration -dataOnly -singleDataFile -safeMode +./runMTK.sh -offlineMigration -dataOnly -singleDataFile -safeMode ``` !!! Note @@ -87,25 +86,25 @@ You can use the edb-psql or psql command line to execute the scripts generated d 1. Use the `createdb` command to create the acctg database, into which you'll restore the migrated database objects: ```shell - $ createdb -U enterprisedb acctg + createdb -U enterprisedb acctg ``` 2. Connect to the new database with edb-psql: ```shell - $ edb-psql -U enterprisedb acctg + edb-psql -U enterprisedb acctg ``` 3. Use the `\i` meta-command to invoke the migration script that creates the object definitions: ```shell - $ acctg=# \i ./mtk_hr_ddl.sql + acctg=# \i ./mtk_hr_ddl.sql ``` 4. If the `-offlineMigration` command included the `‑singleDataFile` keyword, the `mtk_hr_data.sql` script will contain the commands required to re-create all of the objects in the new target database. Populate the database with the command: ```shell - $ acctg=# \i ./mtk_hr_data.sql + acctg=# \i ./mtk_hr_data.sql ``` @@ -116,19 +115,19 @@ By default, Migration Toolkit assumes the source database is Oracle and the targ By default, Migration Toolkit imports both the data and the object definition when migrating a schema. Alternatively, you can choose to import either the data or the object definitions. -`-sourcedbtype ` +### `-sourcedbtype ` The `-sourcedbtype` option specifies the source database type. For `source_type`, use one of the following values: `mysql`, `oracle`, `sqlserver`, `sybase`, `postgresql` or `enterprisedb`. `source_type` isn't case sensitive. By default, `source_type` is `oracle`. -`-targetdbtype ` +### `-targetdbtype ` The `-targetdbtype` option specifies the target database type. For `target_type`, use one of the following values: `enterprisedb`, `postgres`, or `postgresql`. `target_type` isn't case sensitive. By default, `target_type` is `enterprisedb`. -`-schemaOnly` +### `-schemaOnly` This option imports the schema definition and creates all selected schema objects in the target database. You can't use this option with the `‑dataOnly` option. -`-dataOnly` +### `-dataOnly` This option copies only the data. When used with the `-tables` option, Migration Toolkit imports data only for the selected tables. You can't use this option with the `-schemaOnly` option. @@ -138,11 +137,11 @@ This option copies only the data. When used with the `-tables` option, Migration By default, Migration Toolkit imports the source schema objects or data into a schema of the same name. If the target schema doesn't exist, Migration Toolkit creates a schema. Alternatively, you can specify a custom schema name by using the `‑targetSchema` option. You can choose to drop the existing schema and create a new schema using the following option: -`-dropSchema [true|false]` +### `-dropSchema [true|false]` With this option set to `true`, Migration Toolkit drops the existing schema and any objects in that schema and creates a new schema. By default, `-dropSchema` is `false`. -`-targetSchema ` +### `-targetSchema ` Use the `-targetSchema` option to specify the name of the migrated schema. If you're migrating multiple schemas, specify a name for each schema in a comma-separated list with no intervening space characters. Without the `-targetSchema` option, the name of the new schema is the same as the name of the source schema. @@ -154,121 +153,181 @@ You can't specify `information-schema`, `dbo`, `sys`, or `pg_catalog` as target Use the following options to select specific schema objects to migrate: -`-allTables` +### `-allTables` Import all tables from the source schema. -`-tables ` +### `-tables ` + +Import the selected tables from the source schema. `table_list` is a comma-separated list of table names with no intervening space characters. + +For example: `-tables emp,dept,acctg` + +When migrating multiple schemas, or all schemas (with the `-allSchemas` option), the table name must be schema-qualified, for example: + +For example: `-allSchemas -tables comp_schema.emp,comp_schema.dept,finance_schema.acctg` -Import the selected tables from the source schema. `table_list` is a comma-separated list of table names with no intervening space characters (for example, `-tables emp,dept,acctg`). +### `-excludeTables ` -`-excludeTables ` +Exclude the selected tables from migration. The `table_list` is a comma-separated list of table names with no intervening space characters. This option applies when all tables from a schema are selected for migration using the `-allTables` option. -Exclude the selected tables from migration. The `table_list` is a comma-separated list of table names with no intervening space characters (for example, `-excludeTables emp,jobhist`). This option applies when all tables from a schema are selected for migration using the `-allTables` option. +For example: `-allTables -excludeTables emp,jobhist` -`-constraints` +When migrating multiple schemas, or all schemas (with the `-allSchemas` option), the excluded table name must be schema-qualified. + +For example: `-allSchemas -allTables -excludeTables comp_schema.emp,finance_schema.jobhist` + +### `-constraints` Import the table constraints. This option is valid only when importing an entire schema or when you specify the `-allTables` or `-tables ` options. -`-ignoreCheckConstFilter` +### `-ignoreCheckConstFilter` By default, Migration Toolkit doesn't implement migration of check constraints and default clauses from a Sybase database. To migrate constraints and default clauses from a Sybase database, include the `‑ignoreCheckConstFilter` parameter when specifying the `-constraints` parameter. -`-skipCKConst` +### `-skipCKConst` Omit the migration of check constraints. This option is useful when migrating check constraints that are based on built-in functions in the source database that aren't supported in the target database. This option is valid only when importing an entire schema or when the `-allTables` or `-tables ` options are specified. -`-skipFKConst` +### `-skipFKConst` Omit migrating foreign-key constraints. This option is valid only when importing an entire schema or when the `-allTables` or `-tables ` options are specified. -`-skipColDefaultClause` +### `-skipColDefaultClause` Omit migrating the column `DEFAULT` clause. -`-indexes` +### `-indexes` Import the table indexes. This option is valid when importing an entire schema or when the `-allTables` or `-tables ` option is specified. -`-triggers` +### `-triggers` Import the table triggers. This option is valid when importing an entire schema or when the `allTables` or `-tables ` option is specified. -`-allViews` +### `-allViews` Import the views from the source schema. This option migrates dynamic and materialized views from the source. Oracle and Postgres materialized views are supported. -`-views ` +### `-views ` + +Import the specified materialized or dynamic views from the source schema. Oracle and Postgres materialized views are supported. `view_list` is a comma-separated list of view names with no intervening space characters. + +For example: `-views all_emp,mgmt_list,acct_list` -Import the specified materialized or dynamic views from the source schema. Oracle and Postgres materialized views are supported. `view_list` is a comma-separated list of view names with no intervening space characters (for example, `-views all_emp,mgmt_list,acct_list`). +When migrating multiple schemas, or all schemas (with the `-allSchemas` option), the view name must be schema-qualified. -`-excludeViews ` +For example: `-allSchemas -views comp_schema.all_emp,comp_schema.mgmt_list,finance_schema.acct_list` -Exclude the selected views from migration. The `view_list` is a comma-separated list of view names with no intervening space characters (for example, `-excludeViews all_emp,acct_list`). This option applies when all views from a schema are selected for migration using the `-allViews` option. +### `-excludeViews ` -`-allSequences` +Exclude the selected views from migration. The `view_list` is a comma-separated list of view names with no intervening space characters. This option applies when all views from a schema are selected for migration using the `-allViews` option. + +For example: `-allViews -excludeViews all_emp,acct_list` + +When migrating multiple schemas, or all schemas (with the `-allSchemas` option), the excluded view name must be schema-qualified: + +For example: `-allSchemas -allViews -excludeViews comp_schema.all_emp,finance_schema.acct_list` + +### `-allSequences` Import all sequences from the source schema. -`-sequences ` +### `-sequences ` Import the selected sequences from the source schema. `` is a comma-separated list of sequence names with no intervening space characters. -`-excludeSequences ` +For example: `-sequences my_sequence,my_sequence_with_increment` + +When migrating multiple schemas, or all schemas (with the `-allSchemas` option), the sequence name must be schema-qualified. + +For example: `-allSchemas -sequences comp_schema.my_sequence,finance_schema.my_sequence_with_increment` + +### `-excludeSequences ` Exclude selected sequences from the migration. The `sequence_list` is a comma-separated list of sequence names with no intervening space characters. This option applies when all sequences from a schema are selected for migration using the `-allSequences` option. -Example: `-allSequences -excludeSequences my_sequence,my_sequence_with_increment` +For example: `-allSequences -excludeSequences my_sequence,my_sequence_with_increment` + +When migrating multiple schemas, or all schemas (with the `-allSchemas` option), the excluded sequence name must be schema-qualified. -`-allProcs` +For example: `-allSchemas -allSequences -excludeSequences comp_schema.my_sequence,finance_schema.my_sequence_with_increment` + +### `-allProcs` Import all stored procedures from the source schema. -`-procs ` +### `-procs ` Import the selected stored procedures from the source schema. `procedures_list` is a comma-separated list of procedure names with no intervening space characters. -`-excludeProcs ` +For example: `-procs show_notice,show_custom_notice` + +When migrating multiple schemas, or all schemas (with the `-allSchema` option), the procedure name must be schema-qualified. + +For example: `-allSchema -procs comp_schema.show_notice,finance_schema.show_custom_notice` + +### `-excludeProcs ` Exclude selected procedures from migration. The `procedures_list` is a comma-separated list of procedure names with no intervening space characters. This option applies when all procedures from a schema are selected for migration using the `-allProcs` option. -Example: `-allProcs -excludeProcs show_notice,show_custom_notice` +For example: `-allProcs -excludeProcs show_notice,show_custom_notice` -`-allFuncs` +When migrating multiple schemas, or all schemas (with the `-allSchemas` option), the excluded procedure name must be schema-qualified. + +For example: `-allSchemas -allProcs -excludeProcs comp_schema.show_notice,finance_schema.show_custom_notice` + +### `-allFuncs` Import all functions from the source schema. -`-funcs ` +### `-funcs ` Import the selected functions from the source schema. `function_list` is a comma-separated list of function names with no intervening space characters. -`-excludeFuncs ` +For example: `-funcs calculate_average_salary,add_two_numbers` + +When migrating multiple schemas, or all schemas (with the `-allSchemas` option), the function name must be schema-qualified. + +For example: `-allSchemas -funcs comp_schema.calculate_average_salary,finance_schema.add_two_numbers` + +### `-excludeFuncs ` Exclude selected functions from migration. The `function_list` is a comma-separated list of function names with no intervening space characters. This option applies when all functions from a schema are selected for migration using the `-allFuncs` option. -Example: `-allFuncs -excludeFuncs calculate_average_salary,add_two_numbers` +For example: `-allFuncs -excludeFuncs calculate_average_salary,add_two_numbers` + +When migrating multiple schemas, or all schemas (with the `-allSchemas` option), the excluded function name must be schema-qualified. -`-checkFunctionBodies [true/false]` +For example: `-allSchemas -allFuncs -excludeFuncs comp_schema.calculate_average_salary,finance_schema.add_two_numbers` + +### `-checkFunctionBodies [true/false]` When `false`, disables validation of the function body during function creation. Disabling this validation avoids errors if the function contains forward references. The default value is `true`. -`-allPackages` +### `-allPackages` Import all packages from the source schema. -`-packages ` +### `-packages ` Import the selected packages from the source schema. `package_list` is a comma-separated list of package names with no intervening space characters. -`-excludePackages ` +For example: `-packages my_package1, mypackage2` + +When migrating multiple schemas, or all schemas (with the `-allSchemas` option), the package name must be schema-qualified. + +For example: `-allSchemas -packages comp_schema.my_package1,finance_schema.mypackage2` + +### `-excludePackages ` Exclude selected packages from migration. The `package_list` is a comma-separated list of package names with no intervening space characters. @@ -276,19 +335,23 @@ This option applies when all packages from a schema are selected for migration u Example: `-allPackages -excludePackages my_package1, mypackage2` -`-allDomains` +When migrating multiple schemas, or all schemas (with the `-allSchemas` option), the excluded package name must be schema-qualified. + +For example: `-allSchemas -allPackages -excludePackages comp_schema.my_package1,finance_schema.mypackage2` + +### `-allDomains` Import all domain, enumeration, and composite types from the source database. This option is valid only when both the source and target are stored on a Postgres host. -`-allQueues` +### `-allQueues` Import all queues from the source schema. These are queues created and managed by the DBMS_AQ and DBMS_AQADM built-in packages. When Oracle is the source database, you must also specify the `-objectTypes` option. When EDB Postgres Advanced Server is the source database, you must also specify the `-allDomains` and `-allTables` options. Oracle and EDB Postgres Advanced Server queues are supported. -`-queues ` +### `-queues ` Import the selected queues from the source schema. `queue_list` is a comma-separated list of queue names with no intervening space characters. These are queues created and managed by the DBMS_AQ and DBMS_AQADM built-in packages. When Oracle is the source database, you must also specify the `-objectTypes`. When EDB Postgres Advanced Server is the source database, you must also specify `-allDomains` and `-allTables`. Oracle and EDB Postgres Advanced Server queues are supported. -`-excludeQueues ` +### `-excludeQueues ` Exclude selected queues from migration. The `queue_list` is a comma-separated list of queue names with no intervening space characters. @@ -296,15 +359,15 @@ This option applies when all queues from a schema are selected for migration usi Example: `-allQueues -excludeQueues EMP_QUEUE1,EMP_QUEUE2` -`-allRules` +### `-allRules` Import all rules from the source database. This option is valid only when both the source and target are stored on a Postgres host. -`-allGroups` +### `-allGroups` Import all groups from the source database. -`-groups ` +### `-groups ` The selected groups from the source database. The `` is a comma-separated list of group names e.g. `-groups acct_emp,mkt_emp`. @@ -314,63 +377,63 @@ The selected groups from the source database. The `` is a comma-sepa Use the migration options to control the details of the migration process. -`-truncLoad` +### `-truncLoad` Truncate the data from the table before importing new data. Use this option with the `-dataOnly` option. -`-enableConstBeforeDataLoad` +### `-enableConstBeforeDataLoad` Include the `-enableConstBeforeDataLoad` option if a nonpartitioned source table is mapped to a partitioned table. This option enables all triggers on the target table, including any triggers that redirect data to individual partitions, before the data migration. `-enableConstBeforeDataLoad` is valid only if you also specify the `-truncLoad` parameter. -`-retryCount []` +### `-retryCount []` If you're performing a multiple-schema migration, objects that fail to migrate during the first migration attempt due to cross-schema dependencies might successfully migrate during a later migration. Use the `-retryCount` option to specify the number of attempts for Migration Toolkit to make to migrate an object that failed during an initial migration attempt. Specify a value greater than 0. The default value is 2. -`-safeMode` +### `-safeMode` If you include the `-safeMode` option, Migration Toolkit commits each row as migrated. If the migration fails to transfer all records, rows inserted prior to the point of failure remain in the target database. -`-fastCopy` +### `-fastCopy` Including the `-fastCopy` option specifies for Migration Toolkit to bypass WAL logging to perform the COPY operation in an optimized way. It is disabled by default. If you choose to use the `-fastCopy` option, you might not be able to recover the migrated data in the target database if the migration is interrupted. -`-replaceNullChar ` +### `-replaceNullChar ` The Migration Toolkit supports importing a column with a value of NULL. However, the Migration Toolkit doesn't support importing NULL character values (embedded binary zeros 0x00) with the JDBC connection protocol. If you're importing data that includes the NULL character, use the `-replaceNullChar` option to replace the NULL character with a single, non-NULL replacement character. Don't enclose the replacement character in quotes or apostrophes. Once the data is migrated, use a SQL statement to replace the character specified by `-replaceNullChar` with binary zeros. -`-analyze` +### `-analyze` Include the `-analyze` option to invoke the Postgres `ANALYZE` operation against a target database. The optimizer consults the statistics collected by the `ANALYZE` operation, using the information to construct efficient query plans. -`-vacuumAnalyze` +### `-vacuumAnalyze` Include the `-vacuumAnalyze` option to invoke both the `VACUUM` and `ANALYZE` operations against a target database. The optimizer consults the statistics collected by the `ANALYZE` operation, using the information to construct efficient query plans. The `VACUUM` operation reclaims any storage space occupied by dead tuples in the target database. -`-copyDelimiter` +### `-copyDelimiter` Specify a single character to use as a delimiter in the COPY command when loading table data. The default value is `'\t'` (tab). -`-batchSize` +### `-batchSize` Specify the batch size of bulk inserts. Valid values are 1 to 1000. The default batch size is 1000. Reduce the value of `-batchSize` if Out of Memory exceptions occur. -`-cpBatchSize` +### `-cpBatchSize` Specify the batch size in MB to use in the COPY command. Any value greater than 0 is valid. The default batch size is 8MB. -`-lobBatchSize` +### `-lobBatchSize` Specify the number of rows to load in a batch for LOB data types. The data migration for a table containing a large object type (LOB) column, such as `BYTEA`, `BLOB`, or `CLOB`, is performed one row at a time by default. This is to avoid an out-of-heap-space error in case an individual LOB column holds hundreds of megabytes of data. In case the LOB column average data size is at a lower end, you can customize the LOB batch size by specifying the number of rows in each batch with any value greater than 0. -`-fetchSize` +### `-fetchSize` Use the `-fetchSize` option to specify the number of rows fetched in a result set. If the designated `-fetchSize` is too large, you might encounter Out of Memory exceptions. Include the `-fetchSize` option to avoid this pitfall when migrating large tables. The default fetch size is specific to the JDBC driver implementation and varies by database. MySQL users note: By default, the MySQL JDBC driver fetches all of the rows in a table into the Migration Toolkit in a single network round trip. This behavior can easily exceed available memory for large tables. If you encounter an out-of-heap-space error, specify `-fetchSize 1` as a command line argument to force Migration Toolkit to load the table data one row at a time. -`-filterProp ` +### `-filterProp ` `` specifies the name of a file that contains constraints in key=value pairs. Each record read from the database is evaluated against the constraints. Those that satisfy the constraints are migrated. @@ -382,7 +445,7 @@ The right side specifies a condition that must be true for each row migrated. For example, this code migrates only those countries with a `COUNTRY_ID` value that isn't equal to `AR`: -`COUNTRIES=COUNTRY_ID<>'AR'` +### `COUNTRIES=COUNTRY_ID<>'AR'` This constraint applies to the COUNTRIES table. @@ -447,7 +510,7 @@ Migration Toolkit resumes the migration based on the mode that it was using to m You can specify several connection retry options: -`-connRetryCount []` +### `-connRetryCount []` Use the `-connRetryCount` option to specify the number of retry attempts to perform if the target database connection fails. The `[]` value must be a number between 0 and 50. The default is 3 retry attempts. @@ -457,10 +520,10 @@ Since the retry applies to the migration of data, it isn't compatible with the ` Example: ``` -$ ./runMTK.sh -connRetryCount 2 -dataOnly -tables dept,emp,jobhist public +./runMTK.sh -connRetryCount 2 -dataOnly -tables dept,emp,jobhist public ``` -`-connRetryInterval []` +### `-connRetryInterval []` Use the `-connRetryInterval` option to specify the seconds to wait before each subsequent reconnection attempt if the target database connection fails. @@ -471,10 +534,10 @@ Since the retry applies to the migration of data, it isn't compatible with the ` Example: ``` -$ ./runMTK.sh -connRetryCount 2 -connRetryInterval 50 -dataOnly -tables dept,emp,jobhist public +./runMTK.sh -connRetryCount 2 -connRetryInterval 50 -dataOnly -tables dept,emp,jobhist public ``` -`-abortConnOnFailure` [true/false]` +### `-abortConnOnFailure` [true/false]` Specify whether to abort the migration if all the reconnection attempts failed. @@ -486,10 +549,10 @@ Since the retry applies to the migration of data, it isn't compatible with the ` Example: ``` -$ ./runMTK.sh -connRetryCount 2 -connRetryInterval 50 -abortConnOnFailure false -dataOnly -tables dept,emp,jobhist public +./runMTK.sh -connRetryCount 2 -connRetryInterval 50 -abortConnOnFailure false -dataOnly -tables dept,emp,jobhist public ``` -`-pgIdleTxSessionTimeOut []` +### `-pgIdleTxSessionTimeOut []` Specify the PostgreSQL or EDB Postgres Advanced Server `idle_in_transaction_session_timeout`, which defines the time after which the database terminates the session when a migration transaction is in idle state. @@ -519,7 +582,7 @@ Parallel data migration at the schema level uses multiple threads to migrate sev The following options control the way MTK migrates data in parallel. -`-loaderCount []` +### `-loaderCount []` Use the `-loaderCount` option to specify the number of parallel threads available in the pool to perform data load in parallel. This option is particularly useful if the host system that's running Migration Toolkit has high-end CPU and RAM resources. While `value` can be any nonzero, positive number, we recommend that value not exceed the number of CPU cores. For example, a dual-core CPU has an optimal value of `2`. The default is `1`. @@ -529,13 +592,13 @@ The number of table-level threads can introduce overhead on the source database When multiple threads are used, depending on the actual number of threads and table data size, you might need to adjust the memory heap size for the Migration Toolkit. -`-parallelLoadRowLimit []` +### `-parallelLoadRowLimit []` Use the `-parallelLoadRowLimit` option to specify the minimum number of rows required in a table to perform data load in parallel at the table level. The `` parameter must be greater than 0. The default is 100,000. For example, if the value of `-parallelLoadRowLimit` is 10,000, only tables with the number of rows 10,000 and greater are loaded in parallel at the table level. The other tables migrate in sequential mode or at the schema level. -`-tableLoaderLimit []` +### `-tableLoaderLimit []` Use the `-tableLoaderLimit` option to specify the maximum number of threads to assign from the thread pool to load a single table data in parallel chunks at the table level. The `` parameter must not be greater than the value of `-loaderCount`. The default value is equal to the value of the `-loaderCount` option. This option is implicitly changed based on the nondefault custom value of `-loaderCount`. @@ -557,7 +620,7 @@ The algorithm works in three steps: In this scenario, migrate four tables—`tab1, tab2, tab3, tab4`—with parallel data loading: ```text -# ./runMTK.sh -sourcedbtype enterprisedb -targetdbtype enterprisedb -loaderCount 4 -parallelLoadRowLimit 10000 -tableLoaderLimit 2 -tables tab1,tab2,tab3,tab4 public +./runMTK.sh -sourcedbtype enterprisedb -targetdbtype enterprisedb -loaderCount 4 -parallelLoadRowLimit 10000 -tableLoaderLimit 2 -tables tab1,tab2,tab3,tab4 public ``` Table `tab1` has 50,000 rows, table `tab2` has 9,500 rows, table `tab3` has 50,001 rows and table `tab4` has 9,999 rows. The list of tables for step 1 is `tab1, tab2, tab3`, and `tab4`. @@ -613,7 +676,7 @@ This way, four threads are used to migrate four tables in parallel at the mixed- If you change command options to set `-tableLoaderLimit` to `1`, you can have pure schema-level parallelism: ```text -# ./runMTK.sh -sourcedbtype enterprisedb -targetdbtype enterprisedb -loaderCount 4 -parallelLoadRowLimit 10000 -tableLoaderLimit 1 -tables tab1,tab2,tab3,tab4 public +./runMTK.sh -sourcedbtype enterprisedb -targetdbtype enterprisedb -loaderCount 4 -parallelLoadRowLimit 10000 -tableLoaderLimit 1 -tables tab1,tab2,tab3,tab4 public ``` Each table is processed simultaneously in one thread: @@ -635,19 +698,19 @@ Choose the number of threads depending on the CPU and RAM resources available on The following options apply only when the source database is Oracle. -`-objectTypes` +### `-objectTypes` Import the user-defined object types from the schema list specified at the end of the `runMTK.sh` command. -`-allUsers` +### `-allUsers` Import all users and roles from the source database. The `‑allUsers` option is supported only when migrating from an Oracle database to an EDB Postgres Advanced Server database. -`-users ` +### `-users ` Import the selected users or roles from the source Oracle database. `` is a comma-separated list of user/role names with no intervening space characters (e.g., `-users MTK, SAMPLE, acctg`). The `-users` option is supported only when migrating from an Oracle database to an EDB Postgres Advanced Server database. -`-allProfiles` +### `-allProfiles` Import all custom (that is, user-created) profiles from the source database. Other Oracle noncustom profiles such as `DEFAULT` and `MONITORING_PROFILE` aren't imported. @@ -672,7 +735,7 @@ For the imported profiles, only the following password parameters associated wit !!! Note The `‑allProfiles` option is supported only when migrating from an Oracle database to an EDB Postgres Advanced Server database. -`-profiles ` +### `-profiles ` Import the selected, custom (that is, user-created) profiles from the source Oracle database. `profile_list` is a comma-separated list of profile names with no intervening space characters (e.g., `-profiles ADMIN_PROFILE,USER_PROFILE`). Oracle noncustom profiles such as `DEFAULT` and `MONITORING_PROFILE` aren't imported. @@ -681,30 +744,30 @@ As with the `-allProfiles` option, only the password parameters are imported. Th !!! Note The `-profiles` option is supported only when migrating from an Oracle database to an EDB Postgres Advanced Server database. -`-importPartitionAsTable ` +### `-importPartitionAsTable ` Include the `-importPartitionAsTable` parameter to import the contents of a partitioned table that resides on an Oracle host into a single nonpartitioned table. `table_list` is a comma-separated list of table names with no intervening space characters (for example, `-importPartitionAsTable emp,dept,acctg`). -`-copyViaDBLinkOra` +### `-copyViaDBLinkOra` The `dblink_ora` module provides EDB Postgres Advanced Server-to-Oracle connectivity at the SQL level. `dblink_ora` is bundled and installed as part of the EDB Postgres Advanced Server database installation. `dblink_ora` uses the `COPY API` method to transfer data between databases. This method is considerably faster than the `JDBC COPY` method. This example uses the `dblink_ora` `COPY API` to migrate all tables from the `HR` schema: ```shell -$./runMTK.sh -copyViaDBLinkOra -allTables HR +./runMTK.sh -copyViaDBLinkOra -allTables HR ``` The target EDB Postgres Advanced Server database must have `dblink_ora` installed and configured. See [dblink_ora](/epas/latest/working_with_oracle_data/06_dblink_ora/). -`-allDBLinks [link_Name_1=password_1,link_Name_2=password_2,...]` +### `-allDBLinks [link_Name_1=password_1,link_Name_2=password_2,...]` Choose this option to migrate Oracle database links. The password information for each link connection in the source database is encrypted so, unless specified, the dummy password edb is substituted. To migrate all database links using edb as the dummy password for the connected user: ```shell -$./runMTK.sh -allDBLinks HR +./runMTK.sh -allDBLinks HR ``` You can alternatively specify the password for each of the database links through a comma-separated list of name=value pairs with no intervening space characters. Specify the link name on the left side of the pair and the password value on the right side. @@ -712,20 +775,20 @@ You can alternatively specify the password for each of the database links throug To migrate all database links with the actual passwords specified on the command line: ```shell -$./runMTK.sh -allDBLinks LINK_NAME1=abc,LINK_NAME2=xyz HR +./runMTK.sh -allDBLinks LINK_NAME1=abc,LINK_NAME2=xyz HR ``` Migration Toolkit migrates only the database link types that are currently supported by EnterpriseDB. These types include fixed user links of public and private type. -`-allSynonyms` +### `-allSynonyms` Include the `-allSynonyms` option to migrate all public and private synonyms from an Oracle database to an EDB Postgres Advanced Server database. If a synonym with the same name already exists in the target database, the existing synonym is replaced with the migrated version. -`-allPublicSynonyms` +### `-allPublicSynonyms` Include the `-allPublicSynonyms` option to migrate all public synonyms from an Oracle database to an EDB Postgres Advanced Server database. If a synonym with the same name already exists in the target database, the existing synonym is replaced with the migrated version. -`-excludeSynonyms ` +### `-excludeSynonyms ` Exclude selected synonyms from migration. The `synonym_list` is a comma-separated list of synonym names with no intervening space characters. @@ -733,11 +796,11 @@ This option applies when all synonyms from a schema are selected for migration u Example: `-allSynonyms -excludeSynonyms SYNEMP1,SYNEMP2` -`-allPrivateSynonyms` +### `-allPrivateSynonyms` Include the `-allPrivateSynonyms` option to migrate all private synonyms from an Oracle database to an EDB Postgres Advanced Server database. If a synonym with the same name already exists in the target database, the existing synonym is replaced with the migrated version. -`-useOraCase` +### `-useOraCase` Include the `-useOraCase` option to preserve the Oracle default, uppercase naming convention for all database objects when migrating from an Oracle database to an EDB Postgres Advanced Server database. @@ -826,7 +889,7 @@ DEPTNO | DNAME | LOC (4 rows) ``` -`-skipUserSchemaCreation` +### `-skipUserSchemaCreation` When an Oracle user is migrated, a role (that is, a user name) is created in the target database server for the Oracle user if the role doesn't already exist. The role name is created in lowercase letters. When a new role is created, a schema with the same name is also created in lowercase letters. @@ -840,11 +903,11 @@ Thus, if the `-useOraCase` option is specified without the `-skipUserSchemaCreat Use these migration options to view Migration Toolkit help and version information. You can also use these options to control Migration Toolkit feedback and logging options. -`-help` +### `-help` Display the application command-line usage information. -`-logDir ` +### `-logDir ` Include this option to specify where to write the log files. `` is the location for saving log files. By default, on Linux log files are written to: @@ -854,15 +917,15 @@ On Windows, the log files are saved to: `%HOMEDRIVE%%HOMEPATH%\.enterprisedb\migration-toolkit\logs` -`-logFileCount ` +### `-logFileCount ` Include this option to specify the number of files used in log file rotation. Specify a value of `0` to disable log file rotation and create a single log file. This file is truncated when it reaches the value specified using the `logFileSize` option. `` must be greater than or equal to 0. The default is 20. -`-logFileSize ` +### `-logFileSize ` Include this option to specify the maximum file size limit in MB before rotating to a new log file. `file_size` must be greater than 0. The default is 50. -`-logBadSQL` +### `-logBadSQL` Include this option to save the schema definition (DDL script) of any failed objects to a file. The file is saved under the same path used for the error logs and is named in the format: @@ -870,11 +933,11 @@ Include this option to save the schema definition (DDL script) of any failed obj Where `schema_name` is the name of the schema and `timestamp` is the timestamp of the Migration Toolkit run. -`-verbose [on|off]` +### `-verbose [on|off]` Display application log messages on standard output. By default, verbose is on. -`-version` +### `-version` Display the Migration Toolkit version. @@ -899,7 +962,7 @@ TARGET_DB_PASSWORD=password The following command invokes Migration Toolkit: ```shell -$ ./runMTK.sh EDB +./runMTK.sh EDB __OUTPUT__ Running EnterpriseDB Migration Toolkit (Build 48.0.0) ... Source database connectivity info... diff --git a/product_docs/docs/pgd/5/consistency/conflicts.mdx b/product_docs/docs/pgd/5/consistency/conflicts.mdx deleted file mode 100644 index 335ffdaf285..00000000000 --- a/product_docs/docs/pgd/5/consistency/conflicts.mdx +++ /dev/null @@ -1,774 +0,0 @@ ---- -title: Conflicts -redirects: - - /pgd/latest/bdr/conflicts/ ---- - -EDB Postgres Distributed is an active/active or multi-master DBMS. If used asynchronously, writes to the same or related rows from multiple different nodes can result in data conflicts when using standard data types. - -Conflicts aren't errors. In most cases, they are events that PGD can detect and resolve as they occur. Resolving them depends on the nature of the application and the meaning of the data, so it's important for -PGD to provide the application a range of choices for how to resolve conflicts. - -By default, conflicts are resolved at the row level. When changes from two nodes conflict, PGD picks either the local or remote tuple and the discards the other. For example, the commit timestamps might be compared for the two conflicting changes and the newer one kept. This approach ensures that all nodes converge to the same result and establishes commit-order-like semantics on the whole cluster. - -Conflict handling is configurable, as described in [Conflict resolution](#conflict-resolution). PGD can detect conflicts and handle them differently for each table using conflict triggers, described in [Stream triggers](../striggers). - -Column-level conflict detection and resolution is available with PGD, described in [CLCD](column-level-conflicts). - -By default, all conflicts are logged to `bdr.conflict_history`. If conflicts are possible, then table owners must monitor for them and analyze how to avoid them or make plans to handle them regularly as an application task. The [LiveCompare](/livecompare/latest) tool is also available to scan regularly for divergence. - -Some clustering systems use distributed lock mechanisms to prevent concurrent access to data. These can perform reasonably when servers are very close to each other but can't support geographically distributed applications where very low latency is critical for acceptable performance. - -Distributed locking is essentially a pessimistic approach. PGD advocates an optimistic approach, which is to avoid conflicts where possible but allow some types of conflicts to occur and resolve them when they arise. - -## How conflicts happen - -Inter-node conflicts arise as a result of sequences of events that can't happen if all the involved transactions happen concurrently on the same node. Because the nodes exchange changes only after the transactions commit, each transaction is individually valid on the node it committed on. It isn't -valid if applied on another node that did other conflicting work at the same time. - -Since PGD replication essentially replays the transaction on the other nodes, the replay operation can fail if there's a conflict between a transaction being applied and a transaction that was committed on the receiving node. - -Most conflicts can't happen when all transactions run on a single node because Postgres has inter-transaction communication mechanisms to prevent it. Examples of these mechanisms are `UNIQUE` indexes, `SEQUENCE` operations, row and relation locking, and `SERIALIZABLE` dependency tracking. All of these mechanisms are ways to communicate between ongoing transactions to prevent undesirable concurrency -issues. - -PGD doesn't have a distributed transaction manager or lock manager. That's part of why it performs well with latency and network partitions. As a result, transactions on different nodes execute entirely independently from each other when using the default, which is lazy replication. Less independence between nodes can avoid conflicts altogether, which is why PGD also offers Eager Replication for when this is important. - -## Types of conflict - -### PRIMARY KEY or UNIQUE conflicts - -The most common conflicts are row conflicts, where two operations affect a row with the same key in ways they can't on a single node. PGD can detect most of those and applies the `update_if_newer` conflict resolver. - -Row conflicts include: - -- `INSERT` versus `INSERT` -- `UPDATE` versus `UPDATE` -- `UPDATE` versus `DELETE` -- `INSERT` versus `UPDATE` -- `INSERT` versus `DELETE` -- `DELETE` versus `DELETE` - -The view `bdr.node_conflict_resolvers` provides information on how conflict resolution is currently configured for all known conflict types. - -#### INSERT/INSERT conflicts - -The most common conflict, `INSERT`/`INSERT`, arises where `INSERT` operations on two different nodes create a tuple with the same `PRIMARY KEY` values (or if no `PRIMARY KEY` exists, the same values for a single `UNIQUE` constraint). - -PGD handles this situation by retaining the most recently inserted tuple of the two according to the originating node's timestamps. (A user-defined conflict handler can override this behavior.) - -This conflict generates the `insert_exists` conflict type, which is by default resolved by choosing the newer row, based on commit time, and keeping only that one (`update_if_newer` resolver). You can configure other resolvers. See [Conflict resolution](#conflict-resolution) for details. - -To resolve this conflict type, you can also use column-level conflict resolution and user-defined conflict triggers. - -You can effectively eliminate this type of conflict by using [global sequences](../sequences). - -#### INSERT operations that violate multiple UNIQUE constraints - -An `INSERT`/`INSERT` conflict can violate more than one `UNIQUE` constraint, of which one might be the `PRIMARY KEY`. If a new row violates more than one `UNIQUE` constraint and that results in a conflict against more than one other row, then applying the replication change produces a `multiple_unique_conflicts` conflict. - -In case of such a conflict, you must remove some rows for replication to continue. Depending on the resolver setting for `multiple_unique_conflicts`, the apply process either exits with error, skips the incoming row, or deletes some of the rows. The deletion tries to preserve the row with the correct `PRIMARY KEY` and delete the others. - -!!! Warning - In case of multiple rows conflicting this way, if the result of conflict resolution is to proceed with the insert operation, some of the data is always deleted. - -You can also define a different behavior using a conflict trigger. - -#### UPDATE/UPDATE conflicts - -Where two concurrent `UPDATE` operations on different nodes change the same tuple but not its `PRIMARY KEY`, an `UPDATE`/`UPDATE` conflict can occur on replay. - -These can generate different conflict kinds based on the configuration and situation. If the table is configured with [row version conflict detection](#row-version-conflict-detection), then the original (key) row is compared with the local row. If they're different, the `update_differing` conflict is generated. When using [origin conflict detection](#origin-conflict-detection), the origin of the row is checked. (The origin is the node that the current local row came from.) If that changed, the `update_origin_change` conflict is generated. In all other cases, the `UPDATE` is normally applied without generating a conflict. - -Both of these conflicts are resolved the same way as `insert_exists`, described in [INSERT/INSERT conflicts](#insertinsert-conflicts). - -#### UPDATE conflicts on the PRIMARY KEY - -PGD can't currently perform conflict resolution where the `PRIMARY KEY` is changed by an `UPDATE` operation. You can update the primary key, but you must ensure that no conflict with existing values is possible. - -Conflicts on the update of the primary key are [divergent conflicts](#divergent-conflicts) and require manual intervention. - -Updating a primary key is possible in Postgres, but there are issues in both Postgres and PGD. - -A simple schema provides an example that explains: - -```sql -CREATE TABLE pktest (pk integer primary key, val integer); -INSERT INTO pktest VALUES (1,1); -``` - -Updating the Primary Key column is possible, so this SQL succeeds: - -```sql -UPDATE pktest SET pk=2 WHERE pk=1; -``` - -However, suppose the table has multiple rows: - -```sql -INSERT INTO pktest VALUES (3,3); -``` - -Some UPDATE operations succeed: - -```sql -UPDATE pktest SET pk=4 WHERE pk=3; - -SELECT * FROM pktest; - pk | val -----+----- - 2 | 1 - 4 | 3 -(2 rows) -``` - -Other UPDATE operations fail with constraint errors: - -```sql -UPDATE pktest SET pk=4 WHERE pk=2; -ERROR: duplicate key value violates unique constraint "pktest_pkey" -DETAIL: Key (pk)=(4) already exists -``` - -So for Postgres applications that update primary keys, be careful to avoid runtime errors, even without PGD. - -With PGD, the situation becomes more complex if UPDATE operations are allowed from multiple locations at same time. - -Executing these two changes concurrently works: - -```sql -node1: UPDATE pktest SET pk=pk+1 WHERE pk = 2; -node2: UPDATE pktest SET pk=pk+1 WHERE pk = 4; - -SELECT * FROM pktest; - pk | val -----+----- - 3 | 1 - 5 | 3 -(2 rows) -``` - -Executing these next two changes concurrently causes a divergent error, since both changes are accepted. But applying the changes on the other node results in `update_missing` conflicts. - -```sql -node1: UPDATE pktest SET pk=1 WHERE pk = 3; -node2: UPDATE pktest SET pk=2 WHERE pk = 3; -``` - -This scenario leaves the data different on each node: - -```sql -node1: -SELECT * FROM pktest; - pk | val -----+----- - 1 | 1 - 5 | 3 -(2 rows) - -node2: -SELECT * FROM pktest; - pk | val -----+----- - 2 | 1 - 5 | 3 -(2 rows) -``` - -You can identify and resolve this situation using [LiveCompare](/livecompare/latest). - -Concurrent conflicts present problems. Executing these two changes concurrently isn't easy to resolve: - -```sql -node1: UPDATE pktest SET pk=6, val=8 WHERE pk = 5; -node2: UPDATE pktest SET pk=6, val=9 WHERE pk = 5; -``` - -Both changes are applied locally, causing a divergence between the nodes. But the apply on the target fails on both nodes with a duplicate key-value violation error. This error causes the replication to halt and requires manual resolution. - -You can avoid this duplicate key violation error, and replication doesn't break, if you set the conflict_type `update_pkey_exists` to `skip`, `update`, or `update_if_newer`. This approach can still lead to divergence depending on the nature of the update. - -You can avoid divergence in cases where the same old key is being updated by the same new key concurrently by setting `update_pkey_exists` to `update_if_newer`. However, in certain situations, -divergence occurs even with `update_if_newer`, namely when two different rows both are updated concurrently to the same new primary key. - -As a result, we recommend strongly against allowing primary key UPDATE operations in your applications, especially with PGD. If parts of your application change primary keys, then to avoid concurrent -changes, make those changes using Eager Replication. - -!!! Warning - In case the conflict resolution of `update_pkey_exists` conflict results in update, one of the rows is always deleted. - -#### UPDATE operations that violate multiple UNIQUE constraints - -Like [INSERT operations that violate multiple UNIQUE constraints](#insert-operations-that-violate-multiple-unique-constraints), when an incoming `UPDATE` violates more than one `UNIQUE` index (or the `PRIMARY KEY`), PGD raises a `multiple_unique_conflicts` conflict. - -PGD supports deferred unique constraints. If a transaction can commit on the source, then it applies cleanly on target, unless it sees conflicts. However, you can't use a deferred primary key as a REPLICA IDENTITY, so the use cases are already limited by that and the warning about using multiple unique constraints. - -#### UPDATE/DELETE conflicts - -One node can update a row that another node deletes at ths same time. In this case an `UPDATE`/`DELETE` conflict can occur on replay. - -If the deleted row is still detectable (the deleted row wasn't removed by `VACUUM`), the `update_recently_deleted` conflict is generated. By default, the `UPDATE` is skipped, but you can configure the resolution for this. See [Conflict resolution](#conflict-resolution) for details. - -The database can clean up the deleted row by the time the `UPDATE` is received in case the local node is lagging behind in replication. In this case, PGD can't differentiate between `UPDATE`/`DELETE` conflicts and [INSERT/UPDATE conflicts](#insertupdate-conflicts). It generates the `update_missing` conflict. - -Another type of conflicting `DELETE` and `UPDATE` is a `DELETE` that comes after the row was updated locally. In this situation, A `delete_recently_updated` conflict is generated. The default resolution for a `delete_recently_updated` conflict is to `skip` the deletion. However, you can configure the resolution or a conflict trigger can be configured to handle it. - -#### INSERT/UPDATE conflicts - -When using the default asynchronous mode of operation, a node might receive an `UPDATE` of a row before the original `INSERT` was received. This can happen only when three or more nodes are active (see [Conflicts with three or more nodes](#conflicts-with-three-or-more-nodes)). - -When this happens, the `update_missing` conflict is generated. The default conflict resolver is `insert_or_skip`, though you can use `insert_or_error` or `skip` instead. Resolvers that do insert-or-action first try to `INSERT` a new row based on data from the `UPDATE` when possible (when the whole row was received). For reconstructing the row to be possible, the table either needs to have `REPLICA IDENTITY FULL` or the row must not contain any toasted data. - -See [TOAST support details](#toast-support-details) for more info about toasted data. - -#### INSERT/DELETE conflicts - -Similar to the `INSERT`/`UPDATE` conflict, the node might also receive a `DELETE` operation on a row for which it didn't yet receive an `INSERT`. This is again possible only with three or more nodes set up (see [Conflicts with three or more nodes](#conflicts-with-three-or-more-nodes)). - -PGD can't currently detect this conflict type. The `INSERT` operation doesn't generate any conflict type, and the `INSERT` is applied. - -The `DELETE` operation always generates a `delete_missing` conflict, which is by default resolved by skipping the operation. - -#### DELETE/DELETE conflicts - -A `DELETE`/`DELETE` conflict arises when two different nodes concurrently delete the same tuple. - -This scenario always generates a `delete_missing` conflict, which is by default resolved by skipping the operation. - -This conflict is harmless since both `DELETE` operations have the same effect. You can safely ignroe one of them. - -#### Conflicts with three or more nodes - -If one node inserts a row that's then replayed to a second node and updated there, a third node can receive the `UPDATE` from the second node before it receives the `INSERT` from the first node. This scenario is an `INSERT`/`UPDATE` conflict. - -These conflicts are handled by discarding the `UPDATE`, which can lead to different data on different nodes. These are [divergent conflicts](#divergent-conflicts). - -This conflict type can happen only with three or more masters. At least two masters must be actively writing. - -Also, the replication lag from node 1 to node 3 must be high enough to allow the following sequence of actions: - -1. node 2 receives INSERT from node 1 -2. node 2 performs UPDATE -3. node 3 receives UPDATE from node 2 -4. node 3 receives INSERT from node 1 - -Using `insert_or_error` (or in some cases the `insert_or_skip` conflict resolver for the `update_missing` conflict type) is a viable mitigation strategy for these conflicts. However, enabling this option opens the door for `INSERT`/`DELETE` conflicts: - -1. node 1 performs UPDATE -2. node 2 performs DELETE -3. node 3 receives DELETE from node 2 -4. node 3 receives UPDATE from node 1, turning it into an INSERT - -If these are problems, we recommend tuning freezing settings for a table or database so that they're correctly detected as `update_recently_deleted`. - -Another alternative is to use [Eager Replication](eager) to prevent these conflicts. - -`INSERT`/`DELETE` conflicts can also occur with three or more nodes. Such a conflict is identical to `INSERT`/`UPDATE` except with the `UPDATE` replaced by a `DELETE`. This can result in a `delete_missing` -conflict. - -PGD could choose to make each `INSERT` into a check-for-recently deleted, as occurs with an `update_missing` conflict. However, the cost of doing this penalizes the majority of users, so at this time it instead logs `delete_missing`. - -Future releases will automatically resolve `INSERT`/`DELETE` anomalies by way of rechecks using [LiveCompare](/livecompare/latest/) when `delete_missing` conflicts occur. Applications can perform these manually by checking the `bdr.conflict_history_summary` view. - -These conflicts can occur in two main problem use cases: - -- `INSERT` followed rapidly by a `DELETE`, as can be used in queuing applications -- Any case where the primary key identifier of a table is reused - -Neither of these cases is common. We recommend not replicating the affected tables if these problem use cases occur. - -PGD has problems with the latter case because PGD relies on the uniqueness of identifiers to make replication work correctly. - -Applications that insert, delete, and then later reuse the same unique identifiers can cause difficulties. This is known as the [ABA problem](https://en.wikipedia.org/wiki/ABA_problem). PGD has no way of knowing whether the rows are the current row, the last row, or much older rows. - -Unique identifier reuse is also a business problem, since it is prevents unique identification over time, which prevents auditing, traceability, and sensible data quality. Applications don't need to reuse unique identifiers. - -Any identifier reuse that occurs in the time interval it takes for changes to pass across the system causes difficulties. Although that time might be short in normal operation, down nodes can extend that -interval to hours or days. - -We recommend that applications don't reuse unique identifiers. If they do, take steps to avoid reuse in less than a year. - -This problem doesn't occur in applications that use sequences or UUIDs. - -### Foreign key constraint conflicts - -Conflicts between a remote transaction being applied and existing local data can also occur for `FOREIGN KEY` (FK) constraints. - -PGD applies changes with `session_replication_role = 'replica'`, so foreign keys aren't rechecked when applying changes. In an active/active environment, this situation can result in FK violations if deletes occur to the referenced table at the same time as inserts into the referencing table. This scenario is similar to an `INSERT`/`DELETE` conflict. - -In single-master Postgres, any `INSERT`/`UPDATE` that refers to a value in the referenced table must wait for `DELETE` operations to finish before they can gain a row-level lock. If a `DELETE` removes a referenced value, then the `INSERT`/`UPDATE` fails the FK check. - -In multi-master PGD. there are no inter-node row-level locks. An `INSERT` on the referencing table doesn't wait behind a `DELETE` on the referenced table, so both actions can occur concurrently. Thus an `INSERT`/`UPDATE` on one node on the referencing table can use a value at the same time as a `DELETE` -on the referenced table on another node. The result, then, is a value in the referencing table that's no longer present in the referenced table. - -In practice, this situation occurs if the `DELETE` operations occurs on referenced tables in separate transactions from `DELETE` operations on referencing tables, which isn't a common operation. - -In a parent-child relationship such as Orders -> OrderItems, it isn't typical to do this. It's more likely to mark an OrderItem as canceled than to remove it completely. For reference/lookup data, it's unusual to completely remove entries at the same time as using those same values for new fact data. - -While dangling FKs are possible, the risk of this in general is very low. Thus PGD doesn't impose a generic solution to cover this case. Once you understand the situation in which this occurs, two solutions are possible. - -The first solution is to restrict the use of FKs to closely related entities that are generally modified from only one node at a time, are infrequently modified, or where the modification's concurrency is -application mediated. This approach avoids any FK violations at the application level. - -The second solution is to add triggers to protect against this case using the PGD-provided functions `bdr.ri_fkey_trigger()` and `bdr.ri_fkey_on_del_trigger()`. When called as `BEFORE` triggers, these functions use `FOREIGN KEY` information to avoid FK anomalies by setting referencing columns to NULL, much as if you had a SET NULL constraint. This approach rechecks all FKs in one trigger, so you need to add only one trigger per table to prevent FK violation. - -As an example, suppose you have two tables: Fact and RefData. Fact has an FK that references RefData. Fact is the referencing table, and RefData is the referenced table. You need to add one trigger to each table. - -Add a trigger that sets columns to NULL in Fact if the referenced row in RefData was already deleted: - -```sql -CREATE TRIGGER bdr_replica_fk_iu_trg - BEFORE INSERT OR UPDATE ON fact - FOR EACH ROW - EXECUTE PROCEDURE bdr.ri_fkey_trigger(); - -ALTER TABLE fact - ENABLE REPLICA TRIGGER bdr_replica_fk_iu_trg; -``` - -Add a trigger that sets columns to NULL in Fact at the time a DELETE occurs on the RefData table: - -```sql -CREATE TRIGGER bdr_replica_fk_d_trg - BEFORE DELETE ON refdata - FOR EACH ROW - EXECUTE PROCEDURE bdr.ri_fkey_on_del_trigger(); - -ALTER TABLE refdata - ENABLE REPLICA TRIGGER bdr_replica_fk_d_trg; -``` - -Adding both triggers avoids dangling foreign keys. - -### TRUNCATE conflicts - -`TRUNCATE` behaves similarly to a `DELETE` of all rows but performs this action by physically removing the table data rather than row-by-row deletion. As a result, row-level conflict handling isn't available, so `TRUNCATE` commands don't generate conflicts with other DML actions, even when there's a clear conflict. - -As a result, the ordering of replay can cause divergent changes if another DML is executed concurrently on other nodes to the `TRUNCATE`. - -You can take one of the following actions: - -- Ensure `TRUNCATE` isn't executed alongside other concurrent DML. Rely on [LiveCompare](/livecompare/latest) to highlight any such inconsistency. - -- Replace `TRUNCATE` with a `DELETE` statement with no `WHERE` clause. This approach is likely to have poor performance on larger tables. - -- Set `bdr.truncate_locking = 'on'` to set the `TRUNCATE` command’s locking behavior. This setting determines whether `TRUNCATE` obeys the `bdr.ddl_locking` setting. This isn't the default behavior for `TRUNCATE` since it requires all nodes to be up. This configuration might not be possible or wanted in all cases. - -### Exclusion constraint conflicts - -PGD doesn't support exclusion constraints and prevents their creation. - -If an existing standalone database is converted to a PGD database, then drop all exclusion constraints manually. - -In a distributed asynchronous system, you can't ensure that no set of rows that violate the constraint exists because all transactions on different nodes are fully isolated. Exclusion constraints lead to replay deadlocks where replay can't progress from any node to any other node because of exclusion constraint violations. - -If you force PGD to create an exclusion constraint, or you don't drop existing ones when converting a standalone database to PGD, expect replication to break. To get it to progress again, remove or alter the -local tuples that an incoming remote tuple conflicts with so that the remote transaction can be applied. - -### Data conflicts for roles and tablespace differences - -Conflicts can also arise where nodes have global (Postgres-system-wide) data, like roles, that differ. This conflict can result in operations—mainly `DDL`—that can run successfully and commit on one node but then fail to apply to other nodes. - -For example, node1 might have a user named fred, and that user wasn't created on node2. If fred on node1 creates a table, the table is replicated with its owner set to fred. When the DDL command is applied to -node2, the DDL fails because there's no user named fred. This failure generates an error in the Postgres logs. - -Administrator intervention is required to resolve this conflict by creating the user fred in the database where PGD is running. You can set `bdr.role_replication = on` to resolve this in future. - -### Lock conflicts and deadlock aborts - -Because PGD writer processes operate much like normal user sessions, they're subject to the usual rules around row and table locking. This can sometimes lead to PGD writer processes waiting on locks held by user transactions or even by each other. - -Relevant locking includes: - -- Explicit table-level locking (`LOCK TABLE ...`) by user sessions -- Explicit row-level locking (`SELECT ... FOR UPDATE/FOR SHARE`) by user sessions -- Implicit locking because of row `UPDATE`, `INSERT`, or `DELETE` operations, either from local activity or from replication from other nodes - -A PGD writer process can deadlock with a user transaction, where the user transaction is waiting on a lock held by the writer process and vice versa. Two writer processes can also deadlock with each other. Postgres's deadlock detector steps in and terminates one of the problem transactions. If the PGD writer process is terminated, it retries and generally succeeds. - -All these issues are transient and generally require no administrator action. If a writer process is stuck for a long time behind a lock on an idle user session, the administrator can terminate the user session to get replication flowing again. However, this is no different from a user holding a long lock that impacts another user session. - -Use of the [log_lock_waits](https://www.postgresql.org/docs/current/runtime-config-logging.html#GUC-LOG-LOCK-WAITS) facility in Postgres can help identify locking related replay stalls. - -### Divergent conflicts - -Divergent conflicts arise when data that should be the same on different nodes differs unexpectedly. Divergent conflicts shouldn't occur, but not all such conflicts can be reliably prevented at the time of writing. - -Changing the `PRIMARY KEY` of a row can lead to a divergent conflict if another node changes the key of the same row before all nodes have replayed the change. Avoid changing primary keys, or change them only on one designated node. - -Divergent conflicts involving row data generally require administrator action to manually adjust the data on one of the nodes to be consistent with the other one. Such conflicts don't arise so long as you use PGD as documented and avoid settings or functions marked as unsafe. - -The administrator must manually resolve such conflicts. You might need to use the advanced options such as `bdr.ddl_replication` and `bdr.ddl_locking` depending on the nature of the conflict. However, careless use of these options can make things much worse and create a conflict that generic instructions can't address. - -### TOAST support details - -Postgres uses out-of-line storage for larger columns called [TOAST](https://www.postgresql.org/docs/current/storage-toast.html). - -The TOAST values handling in logical decoding (which PGD is built on top of) and logical replication is different from inline data stored as part of the main row in the table. - -The TOAST value is logged into the transaction log (WAL) only if the value changed. This can cause problems, especially when handling UPDATE conflicts, because an `UPDATE` statement that didn't change a value of a toasted column produces a row without that column. As mentioned in [INSERT/UPDATE conflicts](#insertupdate-conflicts), PGD reports an error if an `update_missing` conflict is resolved using `insert_or_error` and there are missing TOAST columns. - -However, more subtle issues than this one occur in case of concurrent workloads with asynchronous replication. (Eager transactions aren't affected.) Imagine, for example, the following workload on an EDB Postgres Distributed cluster with three nodes called A, B, and C: - -1. On node A: txn A1 does an UPDATE SET col1 = 'toast data...' and commits first. -2. On node B: txn B1 does UPDATE SET other_column = 'anything else'; and commits after A1. -3. On node C: the connection to node A lags behind. -4. On node C: txn B1 is applied first, it misses the TOASTed column in col1, but gets applied without conflict. -5. On node C: txn A1 conflicts (on update_origin_change) and is skipped. -6. Node C misses the toasted data from A1 forever. - -This scenario isn't usually a problem when using PGD. (It is when using either built-in logical replication or plain pglogical for multi-master.) PGD adds its own logging of TOAST columns when it detects a local `UPDATE` to a row that recently replicated a TOAST column modification and the local -`UPDATE` isn't modifying the TOAST. Thus PGD prevents any inconsistency for toasted data across different nodes. This situation causes increased WAL logging when updates occur on multiple nodes, that is, when origin changes for a tuple. Additional WAL overhead is zero if all updates are made from a single node, -as is normally the case with PGD AlwaysOn architecture. - -!!! Note - Running `VACUUM FULL` or `CLUSTER` on just the TOAST table without doing same on the main table removes metadata needed for the extra logging to work. This means that, for a short period after such a statement, the protection against these concurrency issues isn't present. - -!!! Warning - The additional WAL logging of TOAST is done using the `BEFORE UPDATE` trigger on standard Postgres. This trigger must be sorted alphabetically last based on trigger name among all `BEFORE UPDATE` triggers on the table. It's prefixed with `zzzz_bdr_` to make this easier, but make sure - you don't create any trigger with a name that sorts after it. Otherwise you won't have the protection against the concurrency issues. - -For the `insert_or_error` conflict resolution, the use of `REPLICA IDENTITY FULL` is still required. - -None of these problems associated with toasted columns affect tables with `REPLICA IDENTITY FULL`. This setting always logs a toasted value as part of the key since the whole row is considered to be part of the key. PGD can reconstruct the new row, filling the missing data from the key row. As a result, using `REPLICA IDENTITY FULL` can increase WAL size significantly. - -## Avoiding or tolerating conflicts - -In most cases, you can design the application to avoid or tolerate conflicts. - -Conflicts can happen only if things are happening at the same time on multiple nodes. The simplest way to avoid conflicts is to only ever write to one node or to only ever write to a specific row in a specific way from one specific node at a time. - -This avoidance happens naturally in many applications. For example, many consumer applications allow only the owning user to change data, such as changing the default billing address on an account. Such data changes seldom have update conflicts. - -You might make a change just before a node goes down, so the change seems to be lost. You might then make the same change again, leading to two updates on different nodes. When the down node comes back up, it tries to send the older change to other nodes. It's rejected because the last update of the data is kept. - -For `INSERT`/`INSERT` conflicts, use [global sequences](../sequences) to prevent this type of conflict. - -For applications that assign relationships between objects, such as a room-booking application, applying `update_if_newer` might not give an acceptable business outcome. That is, it isn't useful to confirm to two people separately that they have booked the same room. The simplest resolution is to use Eager Replication to ensure that only one booking succeeds. More complex ways might be possible depending on the application. For example, you can assign 100 seats to each node and allow those to be booked by a writer on that node. But if none are available locally, use a distributed locking scheme or Eager Replication after most seats are reserved. - -Another technique for ensuring certain types of updates occur only from one specific node is to route different types of transactions through different nodes. For example: - -- Receiving parcels on one node but delivering parcels using another node -- A service application where orders are input on one node and work is prepared on a second node and then served back to customers on another - -Frequently, the best course is to allow conflicts to occur and design the application to work with PGD's conflict resolution mechanisms to cope with the conflict. - -## Conflict detection - -PGD provides these mechanisms for conflict detection: - -- [Origin conflict detection](#origin-conflict-detection) (default) -- [Row version conflict detection](#row-version-conflict-detection) -- [Column-level conflict detection](column-level-conflicts) - -### Origin conflict detection - -Origin conflict detection uses and relies on commit timestamps as recorded on the node the transaction originates from. This requires clocks to be in sync to work correctly or to be within a tolerance of the fastest message between two nodes. If this isn't the case, conflict resolution tends to favor the node that's further ahead. You can manage clock skew between nodes using the parameters `bdr.maximum_clock_skew` and `bdr.maximum_clock_skew_action`. - -Row origins are available only if `track_commit_timestamp = on`. - -Conflicts are first detected based on whether the replication origin changed, so conflict triggers are called in situations that might not turn out to be conflicts. Hence, this mechanism isn't precise, since it can generate false-positive conflicts. - -Origin info is available only up to the point where a row is frozen. Updates arriving for a row after it was frozen don't raise a conflict so are applied in all cases. This is the normal case when adding a new node by `bdr_init_physical`, so raising conflicts causes many false-positive results in that case. - -A node that was offline that reconnects and begins sending data changes can cause divergent -errors if the newly arrived updates are older than the frozen rows that they update. Inserts and deletes aren't affected by this situation. - -We suggest that you don't leave down nodes for extended outages, as discussed in [Node restart and down node recovery](../node_management). - -On EDB Postgres Extended Server and EDB Postgres Advanced Server, PGD holds back the freezing of rows while a node is down. This mechanism handles this situation gracefully so you don't need to change parameter settings. - -On other variants of Postgres, you might need to manage this situation with some care. - -Freezing normally occurs when a row being vacuumed is older than `vacuum_freeze_min_age` xids from the current xid, which means that you need to configure suitably high values for these parameters: - -- `vacuum_freeze_min_age` -- `vacuum_freeze_table_age` -- `autovacuum_freeze_max_age` - -Choose values based on the transaction rate, giving a grace period of downtime before removing any conflict data from the database node. For example, when `vacuum_freeze_min_age` is set to 500 million, a node performing 1000 TPS can be down for just over 5.5 days before conflict data is removed. The CommitTS data structure takes on-disk space of 5 GB with that setting, so lower transaction rate systems can benefit from lower settings. - -Initially recommended settings are: - -```sql -# 1 billion = 10GB -autovacuum_freeze_max_age = 1000000000 - -vacuum_freeze_min_age = 500000000 - -# 90% of autovacuum_freeze_max_age -vacuum_freeze_table_age = 900000000 -``` - -Note that: - -- You can set `autovacuum_freeze_max_age` only at node start. -- You can set `vacuum_freeze_min_age`, so using a low value freezes rows early and can result in conflicts being ignored. You can also set `autovacuum_freeze_min_age` and `toast.autovacuum_freeze_min_age` for individual tables. -- Running the `CLUSTER` or `VACUUM FREEZE` commands also freezes rows early and can result in conflicts being ignored. - -### Row version conflict detection - -PGD provides the option to use row versioning and make conflict detection independent of the nodes' system clock. - -Row version conflict detection requires that you enable two things. If any of these steps aren't performed correctly then [origin conflict detection](#origin-conflict-detection) is used. - -- Enable `REPLICA IDENTITY FULL` on all tables that use row version conflict detection. - -- Enable row version tracking on the table by using `bdr.alter_table_conflict_detection`. This function adds a column with a name you specify and an `UPDATE` trigger that manages the new column value. The column is created as `INTEGER` type. - -Although the counter is incremented only on `UPDATE`, this technique allows conflict detection for both `UPDATE` and `DELETE`. - -This approach resembles Lamport timestamps and fully prevents the ABA problem for conflict detection. - -!!! Note - The row-level conflict resolution is still handled based on the [conflict resolution](#conflict-resolution) configuration even with row versioning. The way the row version is generated is useful only for detecting conflicts. Don't rely on it as authoritative information about which - version of row is newer. - -To determine the current conflict resolution strategy used for a specific table, refer to the column `conflict_detection` of the view `bdr.tables`. - -### `bdr.alter_table_conflict_detection` - -Allows the table owner to change how conflict detection works for a given table. - -#### Synopsis - -```sql -bdr.alter_table_conflict_detection(relation regclass, - method text, - column_name name DEFAULT NULL) -``` - -#### Parameters - -- `relation` — Name of the relation for which to set the new conflict detection method. -- `method` — The conflict detection method to use. -- `column_name` — The column to use for storing the column detection data. This can be skipped, in which case the column name is chosen based on the conflict detection method. The `row_origin` method doesn't require an extra column for metadata storage. - -The recognized methods for conflict detection are: - -- `row_origin` — Origin of the previous change made on the tuple (see [Origin conflict detection](#origin-conflict-detection)). This is the only method supported that doesn't require an extra column in the table. -- `row_version` — Row version column (see [Row version conflict detection](#row-version-conflict-detection)). -- `column_commit_timestamp` — Per-column commit timestamps (described in [CLCD](column-level-conflicts)). -- `column_modify_timestamp` — Per-column modification timestamp (described in [CLCD](column-level-conflicts)). - -#### Notes - -For more information about the difference between `column_commit_timestamp` and `column_modify_timestamp` conflict detection methods, see [Current versus commit timestamp](column-level-conflicts#current-versus-commit-timestamp). - -This function uses the same replication mechanism as `DDL` statements. This means the replication is affected by the [ddl filters](../repsets#ddl-replication-filtering) configuration. - -The function takes a `DML` global lock on the relation for which column-level conflict resolution is being enabled. - -This function is transactional. You can roll back the effects with the `ROLLBACK` of the transaction, and the changes are visible to the current transaction. - -Only the owner of the `relation` can execute the `bdr.alter_table_conflict_detection` function unless `bdr.backwards_compatibility` is set to 30618 or less. - -!!! Warning - When changing the conflict detection method from one that uses an extra column to store metadata, that column is dropped. - -!!! Warning - This function disables CAMO and gives a warning, as long as warnings aren't disabled with `bdr.camo_enable_client_warnings`. - -### List of conflict types - -PGD recognizes the following conflict types, which can be used as the `conflict_type` parameter: - -- `insert_exists` — An incoming insert conflicts with an existing row by way of a primary key or a unique key/index. -- `update_differing` — An incoming update's key row differs from a local row. This can happen only when using [row version conflict detection](#row-version-conflict-detection). -- `update_origin_change` — An incoming update is modifying a row that was last changed by a different node. -- `update_missing` — An incoming update is trying to modify a row that doesn't exist. -- `update_recently_deleted` — An incoming update is trying to modify a row that was recently deleted. -- `update_pkey_exists` — An incoming update has modified the `PRIMARY KEY` to a value that already exists on the node that's applying the change. -- `multiple_unique_conflicts` — The incoming row conflicts with multiple UNIQUE constraints/indexes in the target table. -- `delete_recently_updated` — An incoming delete with an older commit timestamp than the most recent update of the row on the current node or when using [row version conflict detection](#row-version-conflict-detection). -- `delete_missing` — An incoming delete is trying to remove a row that doesn't exist. -- `target_column_missing` — The target table is missing one or more columns present in the incoming row. -- `source_column_missing` — The incoming row is missing one or more columns that are present in the target table. -- `target_table_missing` — The target table is missing. -- `apply_error_ddl` — An error was thrown by Postgres when applying a replicated DDL command. - -## Conflict resolution - -Most conflicts can be resolved automatically. PGD defaults to a last-update-wins mechanism or, more accurately, the `update_if_newer` conflict resolver. This mechanism retains the most recently inserted or changed row of the two conflicting ones based on the same commit timestamps used for conflict detection. The behavior in certain corner-case scenarios depends on the settings used for `bdr.create_node_group` and -alternatively for `bdr.alter_node_group`. - -PGD lets you override the default behavior of conflict resolution by using the following function. - -### bdr.alter_node_set_conflict_resolver - -This function sets the behavior of conflict resolution on a given node. - -#### Synopsis - -```sql -bdr.alter_node_set_conflict_resolver(node_name text, - conflict_type text, - conflict_resolver text) -``` - -#### Parameters - -- `node_name` — Name of the node that's being changed. -- `conflict_type` — Conflict type for which to apply the setting (see [List of conflict types](#list-of-conflict-types)). -- `conflict_resolver` — Resolver to use for the given conflict type (see [List of conflict resolvers](#list-of-conflict-resolvers)). - -#### Notes - -Currently you can change only the local node. The function call isn't replicated. If you want to change settings on multiple nodes, you must run the function on each of them. - -The configuration change made by this function overrides any default behavior of conflict resolutions specified by `bdr.create_node_group` or `bdr.alter_node_group`. - -This function is transactional. You can roll back the changes, and they are visible to the current transaction. - -### List of conflict resolvers - -Several conflict resolvers are available in PGD, with differing coverages of the conflict types they can handle: - -- `error` — Throws error and stops replication. Can be used for any conflict type. -- `skip` — Skips processing the remote change and continues replication with the next change. Can be used for `insert_exists`, `update_differing`, `update_origin_change`, `update_missing`, `update_recently_deleted`, `update_pkey_exists`, `delete_recently_updated`, `delete_missing`, `target_table_missing`, `target_column_missing`, and `source_column_missing` conflict types. -- `skip_if_recently_dropped` — Skips the remote change if it's for a table that doesn't exist downstream because it was recently (within one day) dropped on the downstream. Throw an error otherwise. Can be used for the `target_table_missing` conflict type. `skip_if_recently_dropped` conflict -resolver can pose challenges if a table with the same name is re-created shortly after it's dropped. In that case, one of the nodes might see the DMLs on the re-created table before it sees the DDL to re-create the table. It then incorrectly skips the remote data, assuming that the table is recently dropped, and causes data loss. We hence recommend that you don't reuse the object names immediately after they're dropped along with this conflict resolver. -- `skip_transaction` — Skips the whole transaction that generated the conflict. Can be used for `apply_error_ddl` conflict. -- `update_if_newer` — Updates if the remote row was committed later (as determined by the wall clock of the originating node) than the conflicting local row. If the timestamps are same, the node id is used as a tie-breaker to ensure that same row is picked on all nodes (higher nodeid wins). Can be used for `insert_exists`, `update_differing`, `update_origin_change`, and `update_pkey_exists` conflict types. -- `update` — Always performs the replicated action. Can be used for `insert_exists` (turns the `INSERT` into `UPDATE`), `update_differing`, `update_origin_change`, `update_pkey_exists`, and `delete_recently_updated` (performs the delete). -- `insert_or_skip` — Tries to build a new row from available information sent by the origin and INSERT it. If there isn't enough information available to build a full row, skips the change. Can be used for `update_missing` and `update_recently_deleted` conflict types. -- `insert_or_error` — Tries to build new row from available information sent by origin and insert it. If there isn't enough information available to build full row, throws an error and stops the replication. Can be used for `update_missing` and `update_recently_deleted` conflict types. -- `ignore` — Ignores any missing target column and continues processing. Can be used for the `target_column_missing` conflict type. -- `ignore_if_null` — Ignores a missing target column if the extra column in the remote row contains a NULL value. Otherwise, throws an error and stops replication. Can be used for the `target_column_missing` conflict type. -- `use_default_value` — Fills the missing column value with the default (including NULL if that's the column default) and continues processing. Any error while processing the default or violation of constraints (that is, NULL default on NOT NULL column) stops replication. Can be used for the `source_column_missing` conflict type. - -The `insert_exists`, `update_differing`, `update_origin_change`, `update_missing`, `multiple_unique_conflicts`, `update_recently_deleted`, `update_pkey_exists`, `delete_recently_updated`, and `delete_missing` conflict types can also be resolved by user-defined logic using -[Conflict triggers](../striggers). - -This matrix helps you individuate the conflict types the conflict resolvers can handle. - -| | insert_exists | update_differing | update_origin_change | update_missing | update_recently_deleted | update_pkey_exists | delete_recently_updated | delete_missing | target_column_missing | source_column_missing | target_table_missing | multiple_unique_conflicts | -| :----------------------- | ------------- | ---------------- | -------------------- | -------------- | ----------------------- | ------------------ | ----------------------- | -------------- | --------------------- | --------------------- | -------------------- | ------------------------- | -| error | X | X | X | X | X | X | X | X | X | X | X | X | -| skip | X | X | X | X | X | X | X | X | X | X | X | X | -| skip_if_recently_dropped | | | | | | | | | | | X | | -| update_if_newer | X | X | X | | | X | | | | | | | -| update | X | X | X | | | X | X | | | | | X | -| insert_or_skip | | | | X | X | | | | | | | | -| insert_or_error | | | | X | X | | | | | | | | -| ignore | | | | | | | | | X | | | | -| ignore_if_null | | | | | | | | | X | | | | -| use_default_value | | | | | | | | | | X | | | -| conflict_trigger | X | X | X | X | X | X | X | X | | | | X | - -### Default conflict resolvers - -| Conflict type | Resolver | -| ------------------------- | ------------------------ | -| insert_exists | update_if_newer | -| update_differing | update_if_newer | -| update_origin_change | update_if_newer | -| update_missing | insert_or_skip | -| update_recently_deleted | skip | -| update_pkey_exists | update_if_newer | -| multiple_unique_conflicts | error | -| delete_recently_updated | skip | -| delete_missing | skip | -| target_column_missing | ignore_if_null | -| source_column_missing | use_default_value | -| target_table_missing (see note) | skip_if_recently_dropped | -| apply_error_ddl | error | - -
- -!!! note target_table_missing -This conflict type isn't detected on community Postgresql. If the target table is missing, it causes an error and halts replication. -EDB Postgres servers detect and handle missing target tables and can invoke the resolver. -!!! - -### List of conflict resolutions - -The conflict resolution represents the kind of resolution chosen by the conflict resolver and corresponds to the specific action that was taken to resolve the conflict. - -The following conflict resolutions are currently supported for the `conflict_resolution` parameter: - -- `apply_remote` — The remote (incoming) row was applied. -- `skip` — Processing of the row was skipped (no change was made locally). -- `merge` — A new row was created, merging information from remote and local row. -- `user` — User code (a conflict trigger) produced the row that was written to the target table. - -## Conflict logging - -To ease diagnosing and handling multi-master conflicts, PGD, by default, logs every conflict into the `bdr.conflict_history` table. You can change this behavior with more granularity with the following functions. - -### bdr.alter_node_set_log_config - -Set the conflict logging configuration for a node. - -#### Synopsis - -```sql -bdr.alter_node_set_log_config(node_name text, - log_to_file bool DEFAULT true, - log_to_table bool DEFAULT true, - conflict_type text[] DEFAULT NULL, - conflict_resolution text[] DEFAULT NULL) -``` - -#### Parameters - -- `node_name` — Name of the node that's being changed. -- `log_to_file` — Whether to log to the node log file. -- `log_to_table` — Whether to log to the `bdr.conflict_history` table. -- `conflict_type` — Conflict types to log. NULL (the default) means all. -- `conflict_resolution` — Conflict resolutions to log. NULL (the default) means all. - -#### Notes - -You can change only the local node. The function call isn't replicated. If you want to change settings on multiple nodes, you must run the function on each of them. - -This function is transactional. You can roll back the changes, and they're visible to the current transaction. - -#### Listing conflict logging configurations - -The view `bdr.node_log_config` shows all the logging configurations. It lists the name of the logging configuration, where it logs, and the conflict type and resolution it logs. - -#### Logging conflicts to a table - -If `log_to_table` is set to true, conflicts are logged to a table. The target table for conflict logging is `bdr.conflict_history`. - -This table is range partitioned on the column `local_time`. The table is managed by autopartition. By default, a new partition is created for every day, and conflicts of the last one month are maintained. After that, the old partitions are dropped automatically. Autopartition creates between 7 and 14 partitions in advance. bdr_superuser can change these defaults. - -Since conflicts generated for all tables managed by PGD are logged to this table, it's important to ensure that only legitimate users can read the conflicted data. PGD does this by defining ROW LEVEL SECURITY policies on the `bdr.conflict_history` table. Only owners of the tables are allowed to read conflicts on the respective tables. If the underlying tables have RLS policies defined, enabled, and enforced, then even owners can't read the conflicts. RLS policies created with the FORCE option also apply to owners of the table. In that case, some or all rows in the underlying table might not be readable even to the owner. So PGD also enforces a stricter policy on the conflict log table. - -The default role `bdr_read_all_conflicts` can be granted to users who need to see all conflict details logged to the `bdr.conflict_history` table without also granting them `bdr_superuser` role. - -The default role `bdr_read_all_stats` has access to a catalog view called `bdr.conflict_history_summary`, which doesn't contain user data, allowing monitoring of any conflicts logged. - -### Conflict reporting - -You can summarize conflicts logged to tables in reports. Reports allow application owners to identify, understand, and resolve conflicts and introduce application changes to prevent them. - -```sql -SELECT nspname, relname -, date_trunc('day', local_time) :: date AS date -, count(*) -FROM bdr.conflict_history -WHERE local_time > date_trunc('day', current_timestamp) -GROUP BY 1,2,3 -ORDER BY 1,2; - - nspname | relname | date | count ----------+---------+------------+------- - my_app | test | 2019-04-05 | 1 -(1 row) -``` - -## Data verification with LiveCompare - -LiveCompare is a utility program designed to compare any two databases to verify that they are identical. - -LiveCompare is included as part of the PGD stack and can be aimed at any pair of PGD nodes. By default, it compares all replicated tables and reports differences. LiveCompare also works with non-PGD data sources such as Postgres and Oracle. - -You can also use LiveCompare to continuously monitor incoming rows. You can stop and start it without losing context information, so you can run it at convenient times. - -LiveCompare allows concurrent checking of multiple tables. You can configure it to allow checking of a few tables or just a section of rows in a table. Checks are performed by first comparing whole -row hashes. If different, LiveCompare then compares whole rows. LiveCompare avoids overheads by comparing rows in useful-sized batches. - -If differences are found, they can be rechecked over time, allowing for the delays of eventual consistency. - -See the [LiveCompare](/livecompare/latest/) documentation for further details. diff --git a/product_docs/docs/pgd/5/consistency/conflicts/00_conflicts_overview.mdx b/product_docs/docs/pgd/5/consistency/conflicts/00_conflicts_overview.mdx new file mode 100644 index 00000000000..da19837bec6 --- /dev/null +++ b/product_docs/docs/pgd/5/consistency/conflicts/00_conflicts_overview.mdx @@ -0,0 +1,55 @@ +--- +title: Overview +Description: Conflicts section overview. +deepToC: true +--- + +EDB Postgres Distributed is an active/active or multi-master DBMS. If used asynchronously, writes to the same or related rows from multiple different nodes can result in data conflicts when using standard data types. + +Conflicts aren't errors. In most cases, they are events that PGD can detect and resolve as they occur. Resolving them depends on the nature of the application and the meaning of the data, so it's important for +PGD to provide the application with a range of choices for how to resolve conflicts. + +By default, conflicts are resolved at the row level. When changes from two nodes conflict, PGD picks either the local or remote tuple and the discards the other. For example, the commit timestamps might be compared for the two conflicting changes and the newer one kept. This approach ensures that all nodes converge to the same result and establishes commit-order-like semantics on the whole cluster. + +Conflict handling is configurable, as described in [Conflict resolution](04_conflict_resolution). PGD can detect conflicts and handle them differently for each table using conflict triggers, described in [Stream triggers](../../striggers). + +Column-level conflict detection and resolution is available with PGD, as described in [CLCD](../column-level-conflicts). + +By default, all conflicts are logged to [`bdr.conflict_history`](/pgd/latest/reference/catalogs-visible/#bdrconflict_history). If conflicts are possible, then table owners must monitor for them and analyze how to avoid them or make plans to handle them regularly as an application task. The [LiveCompare](/livecompare/latest) tool is also available to scan regularly for divergence. + +Some clustering systems use distributed lock mechanisms to prevent concurrent access to data. These can perform reasonably when servers are very close to each other but can't support geographically distributed applications where very low latency is critical for acceptable performance. + +Distributed locking is essentially a pessimistic approach. PGD advocates an optimistic approach, which is to avoid conflicts where possible but allow some types of conflicts to occur and resolve them when they arise. + +## How conflicts happen + +Inter-node conflicts arise as a result of sequences of events that can't happen if all the involved transactions happen concurrently on the same node. Because the nodes exchange changes only after the transactions commit, each transaction is individually valid on the node it committed on. It isn't +valid if applied on another node that did other conflicting work at the same time. + +Since PGD replication essentially replays the transaction on the other nodes, the replay operation can fail if there's a conflict between a transaction being applied and a transaction that was committed on the receiving node. + +Most conflicts can't happen when all transactions run on a single node because Postgres has inter-transaction communication mechanisms to prevent it. Examples of these mechanisms are `UNIQUE` indexes, `SEQUENCE` operations, row and relation locking, and `SERIALIZABLE` dependency tracking. All of these mechanisms are ways to communicate between ongoing transactions to prevent undesirable concurrency +issues. + +PGD doesn't have a distributed transaction manager or lock manager. That's part of why it performs well with latency and network partitions. As a result, transactions on different nodes execute entirely independently from each other when using the default, which is lazy replication. Less independence between nodes can avoid conflicts altogether, which is why PGD also offers Eager Replication for when this is important. + +## Avoiding or tolerating conflicts + +In most cases, you can design the application to avoid or tolerate conflicts. + +Conflicts can happen only if things are happening at the same time on multiple nodes. The simplest way to avoid conflicts is to only ever write to one node or to only ever write to a specific row in a specific way from one specific node at a time. + +This avoidance happens naturally in many applications. For example, many consumer applications allow only the owning user to change data, such as changing the default billing address on an account. Such data changes seldom have update conflicts. + +You might make a change just before a node goes down, so the change seems to be lost. You might then make the same change again, leading to two updates on different nodes. When the down node comes back up, it tries to send the older change to other nodes. It's rejected because the last update of the data is kept. + +For `INSERT`/`INSERT` conflicts, use [global sequences](../../sequences/#pgd-global-sequences) to prevent this type of conflict. + +For applications that assign relationships between objects, such as a room-booking application, applying `update_if_newer` might not give an acceptable business outcome. That is, it isn't useful to confirm to two people separately that they have booked the same room. The simplest resolution is to use Eager Replication to ensure that only one booking succeeds. More complex ways might be possible depending on the application. For example, you can assign 100 seats to each node and allow those to be booked by a writer on that node. But if none are available locally, use a distributed locking scheme or Eager Replication after most seats are reserved. + +Another technique for ensuring certain types of updates occur only from one specific node is to route different types of transactions through different nodes. For example: + +- Receiving parcels on one node but delivering parcels using another node +- A service application where orders are input on one node and work is prepared on a second node and then served back to customers on another + +Frequently, the best course is to allow conflicts to occur and design the application to work with PGD's conflict resolution mechanisms to cope with the conflict. \ No newline at end of file diff --git a/product_docs/docs/pgd/5/consistency/conflicts/02_types_of_conflict.mdx b/product_docs/docs/pgd/5/consistency/conflicts/02_types_of_conflict.mdx new file mode 100644 index 00000000000..a8fb57f1d84 --- /dev/null +++ b/product_docs/docs/pgd/5/consistency/conflicts/02_types_of_conflict.mdx @@ -0,0 +1,403 @@ +--- +title: Types of Conflict +Description: Details on different kinds of database conflicts. +deepToC: true +--- + +## PRIMARY KEY or UNIQUE conflicts + +The most common conflicts are row conflicts, where two operations affect a row with the same key in ways they can't on a single node. PGD can detect most of those and applies the `update_if_newer` conflict resolver. + +Row conflicts include: + +- `INSERT` versus `INSERT` +- `UPDATE` versus `UPDATE` +- `UPDATE` versus `DELETE` +- `INSERT` versus `UPDATE` +- `INSERT` versus `DELETE` +- `DELETE` versus `DELETE` + +The view `bdr.node_conflict_resolvers` provides information on how conflict resolution is currently configured for all known conflict types. + +### INSERT/INSERT conflicts + +The most common conflict, `INSERT`/`INSERT`, arises where `INSERT` operations on two different nodes create a tuple with the same `PRIMARY KEY` values (or if no `PRIMARY KEY` exists, the same values for a single `UNIQUE` constraint). + +PGD handles this situation by retaining the most recently inserted tuple of the two according to the originating node's timestamps. (A user-defined conflict handler can override this behavior.) + +This conflict generates the `insert_exists` conflict type, which is by default resolved by choosing the newer row, based on commit time, and keeping only that one (`update_if_newer` resolver). You can configure other resolvers. See [Conflict resolution](04_conflict_resolution) for details. + +To resolve this conflict type, you can also use column-level conflict resolution and user-defined conflict triggers. + +You can effectively eliminate this type of conflict by using [global sequences](../../sequences/#pgd-global-sequences). + +### INSERT operations that violate multiple UNIQUE constraints + +An `INSERT`/`INSERT` conflict can violate more than one `UNIQUE` constraint, of which one might be the `PRIMARY KEY`. If a new row violates more than one `UNIQUE` constraint and that results in a conflict against more than one other row, then applying the replication change produces a `multiple_unique_conflicts` conflict. + +In case of such a conflict, you must remove some rows for replication to continue. Depending on the resolver setting for `multiple_unique_conflicts`, the apply process either exits with error, skips the incoming row, or deletes some of the rows. The deletion tries to preserve the row with the correct `PRIMARY KEY` and delete the others. + +!!! Warning + In case of multiple rows conflicting this way, if the result of conflict resolution is to proceed with the insert operation, some of the data is always deleted. + +You can also define a different behavior using a conflict trigger. + +### UPDATE/UPDATE conflicts + +Where two concurrent `UPDATE` operations on different nodes change the same tuple but not its `PRIMARY KEY`, an `UPDATE`/`UPDATE` conflict can occur on replay. + +These can generate different conflict kinds based on the configuration and situation. If the table is configured with [row version conflict detection](03_conflict_detection/#row-version-conflict-detection), then the original (key) row is compared with the local row. If they're different, the `update_differing` conflict is generated. When using [origin conflict detection](03_conflict_detection/#origin-conflict-detection), the origin of the row is checked. (The origin is the node that the current local row came from.) If that changed, the `update_origin_change` conflict is generated. In all other cases, the `UPDATE` is normally applied without generating a conflict. + +Both of these conflicts are resolved the same way as `insert_exists`, described in [INSERT/INSERT conflicts](#insertinsert-conflicts). + +### UPDATE conflicts on the PRIMARY KEY + +PGD can't currently perform conflict resolution where the `PRIMARY KEY` is changed by an `UPDATE` operation. You can update the primary key, but you must ensure that no conflict with existing values is possible. + +Conflicts on the update of the primary key are [divergent conflicts](#divergent-conflicts) and require manual intervention. + +Updating a primary key is possible in Postgres, but there are issues in both Postgres and PGD. + +A simple schema provides an example that explains: + +```sql +CREATE TABLE pktest (pk integer primary key, val integer); +INSERT INTO pktest VALUES (1,1); +``` + +Updating the Primary Key column is possible, so this SQL succeeds: + +```sql +UPDATE pktest SET pk=2 WHERE pk=1; +``` + +However, suppose the table has multiple rows: + +```sql +INSERT INTO pktest VALUES (3,3); +``` + +Some UPDATE operations succeed: + +```sql +UPDATE pktest SET pk=4 WHERE pk=3; + +SELECT * FROM pktest; + pk | val +----+----- + 2 | 1 + 4 | 3 +(2 rows) +``` + +Other UPDATE operations fail with constraint errors: + +```sql +UPDATE pktest SET pk=4 WHERE pk=2; +ERROR: duplicate key value violates unique constraint "pktest_pkey" +DETAIL: Key (pk)=(4) already exists +``` + +So for Postgres applications that update primary keys, be careful to avoid runtime errors, even without PGD. + +With PGD, the situation becomes more complex if UPDATE operations are allowed from multiple locations at same time. + +Executing these two changes concurrently works: + +```sql +node1: UPDATE pktest SET pk=pk+1 WHERE pk = 2; +node2: UPDATE pktest SET pk=pk+1 WHERE pk = 4; + +SELECT * FROM pktest; + pk | val +----+----- + 3 | 1 + 5 | 3 +(2 rows) +``` + +Executing these next two changes concurrently causes a divergent error, since both changes are accepted. But applying the changes on the other node results in `update_missing` conflicts. + +```sql +node1: UPDATE pktest SET pk=1 WHERE pk = 3; +node2: UPDATE pktest SET pk=2 WHERE pk = 3; +``` + +This scenario leaves the data different on each node: + +```sql +node1: +SELECT * FROM pktest; + pk | val +----+----- + 1 | 1 + 5 | 3 +(2 rows) + +node2: +SELECT * FROM pktest; + pk | val +----+----- + 2 | 1 + 5 | 3 +(2 rows) +``` + +You can identify and resolve this situation using [LiveCompare](/livecompare/latest). + +Concurrent conflicts present problems. Executing these two changes concurrently isn't easy to resolve: + +```sql +node1: UPDATE pktest SET pk=6, val=8 WHERE pk = 5; +node2: UPDATE pktest SET pk=6, val=9 WHERE pk = 5; +``` + +Both changes are applied locally, causing a divergence between the nodes. But the apply on the target fails on both nodes with a duplicate key-value violation error. This error causes the replication to halt and requires manual resolution. + +You can avoid this duplicate key violation error, and replication doesn't break, if you set the conflict_type `update_pkey_exists` to `skip`, `update`, or `update_if_newer`. This approach can still lead to divergence depending on the nature of the update. + +You can avoid divergence in cases where the same old key is being updated by the same new key concurrently by setting `update_pkey_exists` to `update_if_newer`. However, in certain situations, +divergence occurs even with `update_if_newer`, namely when two different rows both are updated concurrently to the same new primary key. + +As a result, we recommend strongly against allowing primary key UPDATE operations in your applications, especially with PGD. If parts of your application change primary keys, then to avoid concurrent +changes, make those changes using Eager Replication. + +!!! Warning + In case the conflict resolution of `update_pkey_exists` conflict results in update, one of the rows is always deleted. + +### UPDATE operations that violate multiple UNIQUE constraints + +Like [INSERT operations that violate multiple UNIQUE constraints](#insert-operations-that-violate-multiple-unique-constraints), when an incoming `UPDATE` violates more than one `UNIQUE` index (or the `PRIMARY KEY`), PGD raises a `multiple_unique_conflicts` conflict. + +PGD supports deferred unique constraints. If a transaction can commit on the source, then it applies cleanly on target, unless it sees conflicts. However, you can't use a deferred primary key as a REPLICA IDENTITY, so the use cases are already limited by that and the warning about using multiple unique constraints. + +### UPDATE/DELETE conflicts + +One node can update a row that another node deletes at ths same time. In this case an `UPDATE`/`DELETE` conflict can occur on replay. + +If the deleted row is still detectable (the deleted row wasn't removed by `VACUUM`), the `update_recently_deleted` conflict is generated. By default, the `UPDATE` is skipped, but you can configure the resolution for this. See [Conflict resolution](04_conflict_resolution) for details. + +The database can clean up the deleted row by the time the `UPDATE` is received in case the local node is lagging behind in replication. In this case, PGD can't differentiate between `UPDATE`/`DELETE` conflicts and [INSERT/UPDATE conflicts](#insertupdate-conflicts). It generates the `update_missing` conflict. + +Another type of conflicting `DELETE` and `UPDATE` is a `DELETE` that comes after the row was updated locally. In this situation, the outcome depends on the type of conflict detection used. When using the +default, [origin conflict detection](03_conflict_detection/#origin-conflict-detection), no conflict is detected, leading to the `DELETE` being applied and the row removed. If you enable [row version conflict detection](03_conflict_detection/#row-version-conflict-detection), a `delete_recently_updated` conflict is generated. The default resolution for a `delete_recently_updated` conflict is to `skip` the deletion. However, you can configure the resolution or a conflict trigger can be configured to handle it. + +### INSERT/UPDATE conflicts + +When using the default asynchronous mode of operation, a node might receive an `UPDATE` of a row before the original `INSERT` was received. This can happen only when three or more nodes are active (see [Conflicts with three or more nodes](#conflicts-with-three-or-more-nodes)). + +When this happens, the `update_missing` conflict is generated. The default conflict resolver is `insert_or_skip`, though you can use `insert_or_error` or `skip` instead. Resolvers that do insert-or-action first try to `INSERT` a new row based on data from the `UPDATE` when possible (when the whole row was received). For reconstructing the row to be possible, the table either needs to have +`REPLICA IDENTITY FULL` or the row must not contain any toasted data. + +See [TOAST support details](#toast-support-details) for more info about toasted data. + +### INSERT/DELETE conflicts + +Similar to the `INSERT`/`UPDATE` conflict, the node might also receive a `DELETE` operation on a row for which it didn't yet receive an `INSERT`. This is again possible only with three or more nodes set up (see [Conflicts with three or more nodes](#conflicts-with-three-or-more-nodes)). + +PGD can't currently detect this conflict type. The `INSERT` operation doesn't generate any conflict type, and the `INSERT` is applied. + +The `DELETE` operation always generates a `delete_missing` conflict, which is by default resolved by skipping the operation. + +### DELETE/DELETE conflicts + +A `DELETE`/`DELETE` conflict arises when two different nodes concurrently delete the same tuple. + +This scenario always generates a `delete_missing` conflict, which is by default resolved by skipping the operation. + +This conflict is harmless since both `DELETE` operations have the same effect. You can safely ignroe one of them. + +### Conflicts with three or more nodes + +If one node inserts a row that's then replayed to a second node and updated there, a third node can receive the `UPDATE` from the second node before it receives the `INSERT` from the first node. This scenario is an `INSERT`/`UPDATE` conflict. + +These conflicts are handled by discarding the `UPDATE`, which can lead to different data on different nodes. These are [divergent conflicts](#divergent-conflicts). + +This conflict type can happen only with three or more masters. At least two masters must be actively writing. + +Also, the replication lag from node 1 to node 3 must be high enough to allow the following sequence of actions: + +1. node 2 receives INSERT from node 1 +2. node 2 performs UPDATE +3. node 3 receives UPDATE from node 2 +4. node 3 receives INSERT from node 1 + +Using `insert_or_error` (or in some cases the `insert_or_skip` conflict resolver for the `update_missing` conflict type) is a viable mitigation strategy for these conflicts. However, enabling this option opens the door for `INSERT`/`DELETE` conflicts: + +1. node 1 performs UPDATE +2. node 2 performs DELETE +3. node 3 receives DELETE from node 2 +4. node 3 receives UPDATE from node 1, turning it into an INSERT + +If these are problems, we recommend tuning freezing settings for a table or database so that they're correctly detected as `update_recently_deleted`. + +Another alternative is to use [Eager Replication](../eager) to prevent these conflicts. + +`INSERT`/`DELETE` conflicts can also occur with three or more nodes. Such a conflict is identical to `INSERT`/`UPDATE` except with the `UPDATE` replaced by a `DELETE`. This can result in a `delete_missing` +conflict. + +PGD could choose to make each `INSERT` into a check-for-recently deleted, as occurs with an `update_missing` conflict. However, the cost of doing this penalizes the majority of users, so at this time it instead logs `delete_missing`. + +Future releases will automatically resolve `INSERT`/`DELETE` anomalies by way of rechecks using [LiveCompare](/livecompare/latest/) when `delete_missing` conflicts occur. Applications can perform these manually by checking the `bdr.conflict_history_summary` view. + +These conflicts can occur in two main problem use cases: + +- `INSERT` followed rapidly by a `DELETE`, as can be used in queuing applications +- Any case where the primary key identifier of a table is reused + +Neither of these cases is common. We recommend not replicating the affected tables if these problem use cases occur. + +PGD has problems with the latter case because PGD relies on the uniqueness of identifiers to make replication work correctly. + +Applications that insert, delete, and then later reuse the same unique identifiers can cause difficulties. This is known as the [ABA problem](https://en.wikipedia.org/wiki/ABA_problem). PGD has no way of knowing whether the rows are the current row, the last row, or much older rows. + +Unique identifier reuse is also a business problem, since it is prevents unique identification over time, which prevents auditing, traceability, and sensible data quality. Applications don't need to reuse unique identifiers. + +Any identifier reuse that occurs in the time interval it takes for changes to pass across the system causes difficulties. Although that time might be short in normal operation, down nodes can extend that +interval to hours or days. + +We recommend that applications don't reuse unique identifiers. If they do, take steps to avoid reuse in less than a year. + +This problem doesn't occur in applications that use sequences or UUIDs. + +## Foreign key constraint conflicts + +Conflicts between a remote transaction being applied and existing local data can also occur for `FOREIGN KEY` (FK) constraints. + +PGD applies changes with `session_replication_role = 'replica'`, so foreign keys aren't rechecked when applying changes. In an active/active environment, this situation can result in FK violations if deletes occur to the referenced table at the same time as inserts into the referencing table. This scenario is similar to an `INSERT`/`DELETE` conflict. + +In single-master Postgres, any `INSERT`/`UPDATE` that refers to a value in the referenced table must wait for `DELETE` operations to finish before they can gain a row-level lock. If a `DELETE` removes a referenced value, then the `INSERT`/`UPDATE` fails the FK check. + +In multi-master PGD. there are no inter-node row-level locks. An `INSERT` on the referencing table doesn't wait behind a `DELETE` on the referenced table, so both actions can occur concurrently. Thus an `INSERT`/`UPDATE` on one node on the referencing table can use a value at the same time as a `DELETE` +on the referenced table on another node. The result, then, is a value in the referencing table that's no longer present in the referenced table. + +In practice, this situation occurs if the `DELETE` operations occurs on referenced tables in separate transactions from `DELETE` operations on referencing tables, which isn't a common operation. + +In a parent-child relationship such as Orders -> OrderItems, it isn't typical to do this. It's more likely to mark an OrderItem as canceled than to remove it completely. For reference/lookup data, it's unusual to completely remove entries at the same time as using those same values for new fact data. + +While dangling FKs are possible, the risk of this in general is very low. Thus PGD doesn't impose a generic solution to cover this case. Once you understand the situation in which this occurs, two solutions are possible. + +The first solution is to restrict the use of FKs to closely related entities that are generally modified from only one node at a time, are infrequently modified, or where the modification's concurrency is +application mediated. This approach avoids any FK violations at the application level. + +The second solution is to add triggers to protect against this case using the PGD-provided functions `bdr.ri_fkey_trigger()` and `bdr.ri_fkey_on_del_trigger()`. When called as `BEFORE` triggers, these functions use `FOREIGN KEY` information to avoid FK anomalies by setting referencing columns to NULL, much as if you had a SET NULL constraint. This approach rechecks all FKs in one trigger, so you need to add only one trigger per table to prevent FK violation. + +As an example, suppose you have two tables: Fact and RefData. Fact has an FK that references RefData. Fact is the referencing table, and RefData is the referenced table. You need to add one trigger to each table. + +Add a trigger that sets columns to NULL in Fact if the referenced row in RefData was already deleted: + +```sql +CREATE TRIGGER bdr_replica_fk_iu_trg + BEFORE INSERT OR UPDATE ON fact + FOR EACH ROW + EXECUTE PROCEDURE bdr.ri_fkey_trigger(); + +ALTER TABLE fact + ENABLE REPLICA TRIGGER bdr_replica_fk_iu_trg; +``` + +Add a trigger that sets columns to NULL in Fact at the time a DELETE occurs on the RefData table: + +```sql +CREATE TRIGGER bdr_replica_fk_d_trg + BEFORE DELETE ON refdata + FOR EACH ROW + EXECUTE PROCEDURE bdr.ri_fkey_on_del_trigger(); + +ALTER TABLE refdata + ENABLE REPLICA TRIGGER bdr_replica_fk_d_trg; +``` + +Adding both triggers avoids dangling foreign keys. + +## TRUNCATE conflicts + +`TRUNCATE` behaves similarly to a `DELETE` of all rows but performs this action by physically removing the table data rather than row-by-row deletion. As a result, row-level conflict handling isn't available, so `TRUNCATE` commands don't generate conflicts with other DML actions, even when there's a clear conflict. + +As a result, the ordering of replay can cause divergent changes if another DML is executed concurrently on other nodes to the `TRUNCATE`. + +You can take one of the following actions: + +- Ensure `TRUNCATE` isn't executed alongside other concurrent DML. Rely on [LiveCompare](/livecompare/latest) to highlight any such inconsistency. + +- Replace `TRUNCATE` with a `DELETE` statement with no `WHERE` clause. This approach is likely to have poor performance on larger tables. + +- Set `bdr.truncate_locking = 'on'` to set the `TRUNCATE` command’s locking behavior. This setting determines whether `TRUNCATE` obeys the `bdr.ddl_locking` setting. This isn't the default behavior for `TRUNCATE` since it requires all nodes to be up. This configuration might not be possible or wanted in all cases. + +## Exclusion constraint conflicts + +PGD doesn't support exclusion constraints and prevents their creation. + +If an existing standalone database is converted to a PGD database, then drop all exclusion constraints manually. + +In a distributed asynchronous system, you can't ensure that no set of rows that violate the constraint exists because all transactions on different nodes are fully isolated. Exclusion constraints lead to replay deadlocks where replay can't progress from any node to any other node because of exclusion constraint violations. + +If you force PGD to create an exclusion constraint, or you don't drop existing ones when converting a standalone database to PGD, expect replication to break. To get it to progress again, remove or alter the +local tuples that an incoming remote tuple conflicts with so that the remote transaction can be applied. + +## Data conflicts for roles and tablespace differences + +Conflicts can also arise where nodes have global (Postgres-system-wide) data, like roles, that differ. This conflict can result in operations—mainly `DDL`—that can run successfully and commit on one node but then fail to apply to other nodes. + +For example, node1 might have a user named fred, and that user wasn't created on node2. If fred on node1 creates a table, the table is replicated with its owner set to fred. When the DDL command is applied to +node2, the DDL fails because there's no user named fred. This failure generates an error in the Postgres logs. + +Administrator intervention is required to resolve this conflict by creating the user fred in the database where PGD is running. You can set `bdr.role_replication = on` to resolve this in future. + +## Lock conflicts and deadlock aborts + +Because PGD writer processes operate much like normal user sessions, they're subject to the usual rules around row and table locking. This can sometimes lead to PGD writer processes waiting on locks held by user transactions or even by each other. + +Relevant locking includes: + +- Explicit table-level locking (`LOCK TABLE ...`) by user sessions +- Explicit row-level locking (`SELECT ... FOR UPDATE/FOR SHARE`) by user sessions +- Implicit locking because of row `UPDATE`, `INSERT`, or `DELETE` operations, either from local activity or from replication from other nodes + +A PGD writer process can deadlock with a user transaction, where the user transaction is waiting on a lock held by the writer process and vice versa. Two writer processes can also deadlock with each other. Postgres's deadlock detector steps in and terminates one of the problem transactions. If the PGD writer process is terminated, it retries and generally succeeds. + +All these issues are transient and generally require no administrator action. If a writer process is stuck for a long time behind a lock on an idle user session, the administrator can terminate the user session to get replication flowing again. However, this is no different from a user holding a long lock that impacts another user session. + +Use of the [log_lock_waits](https://www.postgresql.org/docs/current/runtime-config-logging.html#GUC-LOG-LOCK-WAITS) facility in Postgres can help identify locking related replay stalls. + +## Divergent conflicts + +Divergent conflicts arise when data that should be the same on different nodes differs unexpectedly. Divergent conflicts shouldn't occur, but not all such conflicts can be reliably prevented at the time of writing. + +Changing the `PRIMARY KEY` of a row can lead to a divergent conflict if another node changes the key of the same row before all nodes have replayed the change. Avoid changing primary keys, or change them only on one designated node. + +Divergent conflicts involving row data generally require administrator action to manually adjust the data on one of the nodes to be consistent with the other one. Such conflicts don't arise so long as you use PGD as documented and avoid settings or functions marked as unsafe. + +The administrator must manually resolve such conflicts. You might need to use the advanced options such as `bdr.ddl_replication` and `bdr.ddl_locking` depending on the nature of the conflict. However, careless use of these options can make things much worse and create a conflict that generic instructions can't address. + +## TOAST support details + +Postgres uses out-of-line storage for larger columns called [TOAST](https://www.postgresql.org/docs/current/storage-toast.html). + +The TOAST values handling in logical decoding (which PGD is built on top of) and logical replication is different from inline data stored as part of the main row in the table. + +The TOAST value is logged into the transaction log (WAL) only if the value changed. This can cause problems, especially when handling UPDATE conflicts, because an `UPDATE` statement that didn't change a value of a toasted column produces a row without that column. As mentioned in [INSERT/UPDATE conflicts](#insertupdate-conflicts), PGD reports an error if an `update_missing` conflict is resolved using `insert_or_error` and there are missing TOAST columns. + +However, more subtle issues than this one occur in case of concurrent workloads with asynchronous replication. (Eager transactions aren't affected.) Imagine, for example, the following workload on an EDB Postgres Distributed cluster with three nodes called A, B, and C: + +1. On node A: txn A1 does an UPDATE SET col1 = 'toast data...' and commits first. +2. On node B: txn B1 does UPDATE SET other_column = 'anything else'; and commits after A1. +3. On node C: the connection to node A lags behind. +4. On node C: txn B1 is applied first, it misses the TOASTed column in col1, but gets applied without conflict. +5. On node C: txn A1 conflicts (on update_origin_change) and is skipped. +6. Node C misses the toasted data from A1 forever. + +This scenario isn't usually a problem when using PGD. (It is when using either built-in logical replication or plain pglogical for multi-master.) PGD adds its own logging of TOAST columns when it detects a local `UPDATE` to a row that recently replicated a TOAST column modification and the local +`UPDATE` isn't modifying the TOAST. Thus PGD prevents any inconsistency for toasted data across different nodes. This situation causes increased WAL logging when updates occur on multiple nodes, that is, when origin changes for a tuple. Additional WAL overhead is zero if all updates are made from a single node, +as is normally the case with PGD AlwaysOn architecture. + +!!! Note + Running `VACUUM FULL` or `CLUSTER` on just the TOAST table without doing same on the main table removes metadata needed for the extra logging to work. This means that, for a short period after such a statement, the protection against these concurrency issues isn't present. + +!!! Warning + The additional WAL logging of TOAST is done using the `BEFORE UPDATE` trigger on standard Postgres. This trigger must be sorted alphabetically last based on trigger name among all `BEFORE UPDATE` triggers on the table. It's prefixed with `zzzz_bdr_` to make this easier, but make sure + you don't create any trigger with a name that sorts after it. Otherwise you won't have the protection against the concurrency issues. + +For the `insert_or_error` conflict resolution, the use of `REPLICA IDENTITY FULL` is still required. + +None of these problems associated with toasted columns affect tables with `REPLICA IDENTITY FULL`. This setting always logs a toasted value as part of the key since the whole row is considered to be part of the key. PGD can reconstruct the new row, filling the missing data from the key row. As a result, using `REPLICA IDENTITY FULL` can increase WAL size significantly. \ No newline at end of file diff --git a/product_docs/docs/pgd/5/consistency/conflicts/03_conflict_detection.mdx b/product_docs/docs/pgd/5/consistency/conflicts/03_conflict_detection.mdx new file mode 100644 index 00000000000..fda6f616dfe --- /dev/null +++ b/product_docs/docs/pgd/5/consistency/conflicts/03_conflict_detection.mdx @@ -0,0 +1,80 @@ +--- +title: Conflict detection +Description: How PGD detects database conflicts. +deepToC: true +--- + +PGD provides these mechanisms for conflict detection: + +- [Origin conflict detection](#origin-conflict-detection) (default) +- [Row version conflict detection](#row-version-conflict-detection) +- [Column-level conflict detection](../column-level-conflicts) + +## Origin conflict detection + +Origin conflict detection uses and relies on commit timestamps as recorded on the node the transaction originates from. This requires clocks to be in sync to work correctly or to be within a tolerance of the fastest message between two nodes. If this isn't the case, conflict resolution tends to favor the node that's further ahead. You can manage clock skew between nodes using the parameters `bdr.maximum_clock_skew` and `bdr.maximum_clock_skew_action`. + +Row origins are available only if `track_commit_timestamp = on`. + +Conflicts are first detected based on whether the replication origin changed, so conflict triggers are called in situations that might not turn out to be conflicts. Hence, this mechanism isn't precise, since it can generate false-positive conflicts. + +Origin info is available only up to the point where a row is frozen. Updates arriving for a row after it was frozen don't raise a conflict so are applied in all cases. This is the normal case when adding a new node by `bdr_init_physical`, so raising conflicts causes many false-positive results in that case. + +A node that was offline that reconnects and begins sending data changes can cause divergent +errors if the newly arrived updates are older than the frozen rows that they update. Inserts and deletes aren't affected by this situation. + +We suggest that you don't leave down nodes for extended outages, as discussed in [Node restart and down node recovery](../../node_management/node_recovery/). + +On EDB Postgres Extended Server and EDB Postgres Advanced Server, PGD holds back the freezing of rows while a node is down. This mechanism handles this situation gracefully so you don't need to change parameter settings. + +On other variants of Postgres, you might need to manage this situation with some care. + +Freezing normally occurs when a row being vacuumed is older than `vacuum_freeze_min_age` xids from the current xid, which means that you need to configure suitably high values for these parameters: + +- `vacuum_freeze_min_age` +- `vacuum_freeze_table_age` +- `autovacuum_freeze_max_age` + +Choose values based on the transaction rate, giving a grace period of downtime before removing any conflict data from the database node. For example, when `vacuum_freeze_min_age` is set to 500 million, a node performing 1000 TPS can be down for just over 5.5 days before conflict data is removed. The CommitTS data structure takes on-disk space of 5 GB with that setting, so lower transaction rate systems can benefit from lower settings. + +Initially, recommended settings are: + +```sql +# 1 billion = 10GB +autovacuum_freeze_max_age = 1000000000 + +vacuum_freeze_min_age = 500000000 + +# 90% of autovacuum_freeze_max_age +vacuum_freeze_table_age = 900000000 +``` + +Note that: + +- You can set `autovacuum_freeze_max_age` only at node start. +- You can set `vacuum_freeze_min_age`, so using a low value freezes rows early and can result in conflicts being ignored. You can also set `autovacuum_freeze_min_age` and `toast.autovacuum_freeze_min_age` for individual tables. +- Running the `CLUSTER` or `VACUUM FREEZE` commands also freezes rows early and can result in conflicts being ignored. + +## Row version conflict detection + +PGD provides the option to use row versioning and make conflict detection independent of the nodes' system clock. + +Row version conflict detection requires that you enable three things. If any of these steps aren't performed correctly then [origin conflict detection](#origin-conflict-detection) is used. + +- Enable `check_full_tuple` or the PGD node group. + +- Enable `REPLICA IDENTITY FULL` on all tables that use row version conflict detection. + +- Enable row version tracking on the table by using `bdr.alter_table_conflict_detection`. This function adds a column with a name you specify and an `UPDATE` trigger that manages the new column value. The column is created as `INTEGER` type. + +Although the counter is incremented only on `UPDATE`, this technique allows conflict detection for both `UPDATE` and `DELETE`. + +This approach resembles Lamport timestamps and fully prevents the ABA problem for conflict detection. + +!!! Note + The row-level conflict resolution is still handled based on the [conflict resolution](04_conflict_resolution) configuration even with row versioning. The way the row version is generated is useful only for detecting conflicts. Don't rely on it as authoritative information about which + version of row is newer. + +To determine the current conflict detection strategy used for a specific table, refer to the column `conflict_detection` of the view `bdr.tables`. + +To change the current conflict detection strategy, use [bdr.alter_table_conflict_detection](../../reference/conflict_functions/#bdralter_table_conflict_detection). \ No newline at end of file diff --git a/product_docs/docs/pgd/5/consistency/conflicts/04_conflict_resolution.mdx b/product_docs/docs/pgd/5/consistency/conflicts/04_conflict_resolution.mdx new file mode 100644 index 00000000000..f7406c36d60 --- /dev/null +++ b/product_docs/docs/pgd/5/consistency/conflicts/04_conflict_resolution.mdx @@ -0,0 +1,9 @@ +--- +title: Conflict resolution +Description: How PGD resolves database conflicts. +deepToC: true +--- + +Most conflicts can be resolved automatically. PGD defaults to a last-update-wins mechanism or, more accurately, the `update_if_newer` conflict resolver. This mechanism retains the most recently inserted or changed row of the two conflicting ones based on the same commit timestamps used for conflict detection. The behavior in certain corner-case scenarios depends on the settings used for `bdr.create_node_group` and alternatively for `bdr.alter_node_group`. + +PGD lets you override the default behavior of conflict resolution by using [bdr.alter_node_set_conflict_resolver](../../reference/conflict_functions/#bdralter_node_set_conflict_resolver). \ No newline at end of file diff --git a/product_docs/docs/pgd/5/consistency/conflicts/05_conflict_logging.mdx b/product_docs/docs/pgd/5/consistency/conflicts/05_conflict_logging.mdx new file mode 100644 index 00000000000..bb0ff2a6414 --- /dev/null +++ b/product_docs/docs/pgd/5/consistency/conflicts/05_conflict_logging.mdx @@ -0,0 +1,26 @@ +--- +title: Conflict logging +Description: How PGD logs database conflicts. +deepToC: true +--- + +To ease diagnosing and handling multi-master conflicts, PGD, by default, logs every conflict into the `bdr.conflict_history` table. You can change this behavior with more granularity using [bdr.alter_node_set_log_config](../../reference/conflict_functions/#bdralter_node_set_log_config). + +## Conflict reporting + +You can summarize conflicts logged to tables in reports. Reports allow application owners to identify, understand, and resolve conflicts and introduce application changes to prevent them. + +```sql +SELECT nspname, relname +, date_trunc('day', local_time) :: date AS date +, count(*) +FROM bdr.conflict_history +WHERE local_time > date_trunc('day', current_timestamp) +GROUP BY 1,2,3 +ORDER BY 1,2; + + nspname | relname | date | count +---------+---------+------------+------- + my_app | test | 2019-04-05 | 1 +(1 row) +``` \ No newline at end of file diff --git a/product_docs/docs/pgd/5/consistency/conflicts/06_live_compare.mdx b/product_docs/docs/pgd/5/consistency/conflicts/06_live_compare.mdx new file mode 100644 index 00000000000..ffdbfdf0ac1 --- /dev/null +++ b/product_docs/docs/pgd/5/consistency/conflicts/06_live_compare.mdx @@ -0,0 +1,17 @@ +--- +title: Data verification with LiveCompare +Description: LiveCompare for data verification between PGD nodes. +--- + +LiveCompare is a utility program designed to compare any two databases to verify that they are identical. + +LiveCompare is included as part of the PGD stack and can be aimed at any pair of PGD nodes. By default, it compares all replicated tables and reports differences. LiveCompare also works with non-PGD data sources such as Postgres and Oracle. + +You can also use LiveCompare to continuously monitor incoming rows. You can stop and start it without losing context information, so you can run it at convenient times. + +LiveCompare allows concurrent checking of multiple tables. You can configure it to allow checking of a few tables or just a section of rows in a table. Checks are performed by first comparing whole +row hashes. If different, LiveCompare then compares whole rows. LiveCompare avoids overheads by comparing rows in useful-sized batches. + +If differences are found, they can be rechecked over time, allowing for the delays of eventual consistency. + +See the [LiveCompare](/livecompare/latest/) documentation for further details. diff --git a/product_docs/docs/pgd/5/consistency/conflicts/index.mdx b/product_docs/docs/pgd/5/consistency/conflicts/index.mdx new file mode 100644 index 00000000000..85a6b5ec93e --- /dev/null +++ b/product_docs/docs/pgd/5/consistency/conflicts/index.mdx @@ -0,0 +1,20 @@ +--- +title: Conflicts +redirects: + - /pgd/latest/bdr/conflicts/ +--- +EDB Postgres Distributed is an active/active or multi-master DBMS. If used asynchronously, writes to the same or related rows from multiple different nodes can result in data conflicts when using standard data types. + +Conflicts aren't errors. In most cases, they are events that PGD can detect and resolve as they occur. This section introduces the PGD functionality that allows you to manage that detection and resolution. + +- [Overview](00_conflicts_overview) introduces the idea of conflicts in PGD and explains how they can happen. + +- [Types of conflicts](02_types_of_conflict) lists and discusses the various sorts of conflicts you might run across in PGD. + +- [Conflict detection](03_conflict_detection) introduces the mechanisms PGD provides for conflict detection. + +- [Conflict resolution](04_conflict_resolution) explains how PGD resolves conflicts and how you can change the default behavior. + +- [Conflict logging](05_conflict_logging) points out where PGD keeps conflict logs and explains how you can perform conflict reporting. + +- [Data verification with LiveCompare](06_live_compare) explains how LiveCompare can help keep data consistent by pointing out conflicts as they arise. \ No newline at end of file diff --git a/product_docs/docs/pgd/5/quickstart/connecting_applications.mdx b/product_docs/docs/pgd/5/quickstart/connecting_applications.mdx index b7d18c0cd8b..2e511193114 100644 --- a/product_docs/docs/pgd/5/quickstart/connecting_applications.mdx +++ b/product_docs/docs/pgd/5/quickstart/connecting_applications.mdx @@ -25,7 +25,7 @@ This command returns a string that's the password for the enterprisedb user. If ## Creating a .pgpass file -You can avoid entering passwords for `psql` and other Postgres clients by creating [a `.pgpass` file](https://www.postgresql.org/docs/current/libpq-pgpass.html) in your home directory. It contains password details that applications can look up when connecting. After getting the password (see [Getting credentials](#getting-credentials)), you can open the `.pgpass` file using your preferred editor. +You can avoid entering passwords for psql and other Postgres clients by creating [a `.pgpass` file](https://www.postgresql.org/docs/current/libpq-pgpass.html) in your home directory. It contains password details that applications can look up when connecting. After getting the password (see [Getting credentials](#getting-credentials)), you can open the `.pgpass` file using your preferred editor. In the file, enter: @@ -127,7 +127,7 @@ You can read more about this command in [Add rules to your security group](https ## Getting an appropriate Postgres client -Unless you installed Postgres on your local system, you probably need to install a client application, such as `psql`, to connect to your database cluster. +Unless you installed Postgres on your local system, you probably need to install a client application, such as psql, to connect to your database cluster. On Ubuntu, for example, you can run: @@ -135,11 +135,11 @@ On Ubuntu, for example, you can run: sudo apt install postgresql-client ``` -This command installs `psql`, along with some other tools but without installing the Postgres database locally. +This command installs psql, along with some other tools but without installing the Postgres database locally. ## Connecting the client to the cluster -After you install `psql` or a similar client, you can connect to the cluster. Run: +After you install psql or a similar client, you can connect to the cluster. Run: ```shell psql -h -p 6432 -U enterprisedb bdrdb @@ -168,44 +168,40 @@ Type "help" for help. bdrdb=# ``` -## Creating a Connection URL +## Creating a connection URL -Many applications use a [Connection URL](https://www.postgresql.org/docs/current/libpq-connect.html#id-1.7.3.8.3.6) to connect to the database. To create a Connection URL, you need to assemble a string in the format: +Many applications use a [connection URL](https://www.postgresql.org/docs/current/libpq-connect.html#id-1.7.3.8.3.6) to connect to the database. To create a connection URL, you need to assemble a string in the format: ``` postgresql://@:6432,:6432,:6432/bdrdb ``` -This format of string can be used with the `psql` command, so if our database nodes were on Ip addresses 192.168.9.10, 192.168.10.10 and 192.168.10.11, we could use: +This format of the string can be used with the `psql` command, so if your database nodes are on IP addresses 192.168.9.10, 192.168.10.10, and 192.168.10.11, you can use: ``` psql postgresql://enterprisedb@192.168.9.10:6432,192.168.10.10:6432,192.168.11.10:6432/bdrdb ``` -You can also embed the password in the created URL. If we are using the enterprisedb user and the password for the enterprisedb user is `notasecret` then the URL -would look like: +You can also embed the password in the created URL. If you're using the enterprisedb user, and the password for the enterprisedb user is `notasecret`, then the URL +looks like: ``` psql postgresql://enterprisedb:notasecret@192.168.9.10:6432,192.168.10.10:6432,192.168.11.10:6432/bdrdb ``` -Actual passwords are more complex than that and may contain special characters. You will need to urlencode the password to ensure that it doesn't trip up the shell, the command or the driver you are using. +Actual passwords are more complex than that and can contain special characters. You need to urlencode the password to ensure that it doesn't trip up the shell, the command, or the driver you're using. !!! Warning Passwords should not be embedded While we have shown you how to embed a password, we recommend that you do not do this. The password is easily extracted from the URL and can easily be saved in insecure locations. Consider other ways of passing the password. -### Making a Java Connection URL +### Making a Java connection URL -Finally, the URL we have created is fine for many Postgres applications and clients, but those based on Java require one change to allow Java to invoke the appropriate driver. Precede the URL with `jdbc:` to make a Java compatible URL. +Finally, the URL you created is fine for many Postgres applications and clients, but those based on Java require one change to allow Java to invoke the appropriate driver. Precede the URL with `jdbc:` to make a Java compatible URL: ``` jdbc:postgresql://enterprisedb@192.168.9.10:6432,192.168.10.10:6432,192.168.11.10:6432/bdrdb ``` -## Moving On - -You are now equipped to connect your applications to your cluster, with all the connection credentials and URLs you need. - - - +## Moving on +You're now equipped to connect your applications to your cluster, with all the connection credentials and URLs you need. diff --git a/product_docs/docs/pgd/5/quickstart/further_explore_conflicts.mdx b/product_docs/docs/pgd/5/quickstart/further_explore_conflicts.mdx index 5c56c99a851..2a864590cb0 100644 --- a/product_docs/docs/pgd/5/quickstart/further_explore_conflicts.mdx +++ b/product_docs/docs/pgd/5/quickstart/further_explore_conflicts.mdx @@ -9,7 +9,7 @@ In a multi-master architecture like PGD, conflicts happen. PGD is built to handl A conflict can occur when one database node has an update from an application to a row and another node has a different update to the same row. This type of conflict is called a *row-level conflict*. Conflicts aren't errors. Resolving them effectively is core to how Postgres Distributed maintains consistency. -The best way to handle conflicts is not to have them in the first place! Use PGD's Always-on architecture with proxies to ensure that your applications write to the same server in the cluster. +The best way to handle conflicts is to prevent them! Use PGD's Always-on architecture with proxies to ensure that your applications write to the same server in the cluster. When conflicts occur, though, it's useful to know how PGD resolves them, how you can control that resolution, and how you can find out that they're happening. Row insertion and row updates are two actions that can cause conflicts. @@ -116,7 +116,7 @@ start transaction; insert into test_conflict values (1, 'from kaboom'); ``` -Next, move to the top-right pane using **Control-b →** or **Control-b q 1**. This pane is the kaftan node. Here, you'll also start a transaction and insert into the same row with different data. +Next, move to the top-right pane using **Control-b →** or **Control-b q 1**. This pane is the kaftan node. Here, you'll also start a transaction and insert into the same row with different data: ``` start transaction; @@ -154,8 +154,8 @@ update test_conflict set value_1 = 'update from kaftan' where id = 1; You now have two transactions open on different servers, with an update operation already performed successfully. You need to commit both transactions at this point: -* Use **Control-b cursor-left** or **Control-b q 0**, and then enter `commit;`. -* Use **Control-b cursor-right** or **Control-b q 1**, and then enter `commit;`. +* Use **Control-b ←** or **Control-b q 0**, and then enter `commit;`. +* Use **Control-b →** or **Control-b q 1**, and then enter `commit;`. Again you'll see both commits working. And, again, in the bottom-right pane, you can see the update conflict being detected. @@ -163,10 +163,10 @@ Again you'll see both commits working. And, again, in the bottom-right pane, you An additional row in the conflict history shows an `update_origin_change` conflict occurred and that the resolution was `apply_remote`. This resolution means that the remote change was applied, updating the record. This conflict is called an UPDATE/UPDATE conflict and is explained in more detail in [UPDATE/UPDATE conflicts](../consistency/conflicts/#updateupdate-conflicts). -!!!Tip Exiting Tmux -You can quickly exit Tmux and all the associated sessions. First terminate any running processes, as they will otherwise continue running after the session is killed. Press **Control-b** and then enter `:kill-session`. This approach is simpler than quitting each pane's session one at a time using **Control-D** or `exit`. +!!!Tip Exiting tmux +You can quickly exit tmux and all the associated sessions. First terminate any running processes, as they will otherwise continue running after the session is killed. Press **Control-b** and then enter `:kill-session`. This approach is simpler than quitting each pane's session one at a time using **Control-D** or `exit`. !!! ## Other conflicts -You are now equipped to explore all the possible conflict scenarios and resolutions that can occur. For full details of how conflicts are managed, see [Conflicts](../consistency/conflicts/) in the documentation. While ideally you should avoid conflicts, it's important to know that, when they do happen, they're recorded and managed by Postgres Distributed's integrated and configurable conflict resolver. +You're now equipped to explore all the possible conflict scenarios and resolutions that can occur. For full details of how conflicts are managed, see [Conflicts](../consistency/conflicts/). While ideally you should avoid conflicts, it's important to know that, when they do happen, they're recorded and managed by Postgres Distributed's integrated and configurable conflict resolver. diff --git a/product_docs/docs/pgd/5/quickstart/further_explore_failover.mdx b/product_docs/docs/pgd/5/quickstart/further_explore_failover.mdx index 1e2c93ee793..212175d5c37 100644 --- a/product_docs/docs/pgd/5/quickstart/further_explore_failover.mdx +++ b/product_docs/docs/pgd/5/quickstart/further_explore_failover.mdx @@ -11,9 +11,9 @@ In this exercise, you'll create an application that sends data to the database r ## Your quick started configuration -This exploration assumes that you created your PGD cluster using the [quick start for Docker](quick_start_docker), the [quick start for AWS](quick_start_aws) or the [quick start for Linux hosts](quick_start_linux). +This exploration assumes that you created your PGD cluster using the [quick start for Docker](quick_start_docker), the [quick start for AWS](quick_start_aws), or the [quick start for Linux hosts](quick_start_linux). -At the end of each quick starts, you'll have a cluster with four nodes and these roles: +At the end of each quick start, you'll have a cluster with four nodes and these roles: | Host name | Host role | | --------- | ----------------- | @@ -25,24 +25,24 @@ At the end of each quick starts, you'll have a cluster with four nodes and these You'll use these hostnames throughout this exercise. !!! Note A best practice recommendation -This example is based on the quick started configuration and, for speed -and simplicity, it uses the barman backup server in place of creating a bastion -server. It also uses the barman login to the Postgres cluster. +This example is based on the quick start configuration. For speed +and simplicity, it uses the Barman backup server in place of creating a bastion +server. It also uses the Barman login to the Postgres cluster. -In a production environment, it is recommended that you create a separate -bastion server to run the failover experiment from and to create an appropriate -Postgres user to log into the cluster. +In a production environment, we recommend that you create a separate +bastion server to run the failover experiment from and that you create an appropriate +Postgres user to log in to the cluster. !!! ## Installing xpanes !!! Note Xpanes optional -We recommend the `xpanes` utility for this exercise. It allows you to easily switch between multiple terminal sessions. If you prefer to use multiple terminals, tmux or another terminal multiplexer, you can do so. Just make sure you can easily switch between multiple terminal sessions. +We recommend the xpanes utility for this exercise. It allows you to easily switch between multiple terminal sessions. If you prefer to use multiple terminals, tmux, or another terminal multiplexer, you can do so. Just make sure you can easily switch between multiple terminal sessions. !!! -You'll be using `xpanes`, a utility that allows you to quickly create multiple terminal sessions that you can easily switch between. It isn't installed by default, so you'll have to install it. For this exercise, you'll be launching xpanes from the system where you ran tpaexec to configure your quick-start cluster. +You'll use xpanes, a utility that allows you to quickly create multiple terminal sessions that you can easily switch between. It isn't installed by default, so you have to install it. For this exercise, you launch xpanes from the system where you ran tpaexec to configure your quick-start cluster. -If the system is running Ubuntu, run this: +If the system is running Ubuntu, run: ``` sudo apt install software-properties-common @@ -62,7 +62,7 @@ cd democluster xpanes -d -c "ssh -F ssh_config {}" "kaboom" "kaolin" "kaftan" "kapok" ``` -After running these commands, there are four panes. The four panes are connected to kaboom, kaolin, kaftan, and kapok and logged in as the root user on each. You need this privilege so you can easily stop and start services later in the exercise. +After running these commands, there are four panes. The four panes are connected to kaboom, kaolin, kaftan, and kapok and you're logged in as the root user on each. You need this privilege so you can easily stop and start services later in the exercise. Press **Control-b** followed by **q** to briefly display the numeric values for each pane. @@ -80,14 +80,14 @@ psql -h kaboom -p 6432 bdrdb This code connects to the proxy on the kaboom host, which also runs a Postgres instance as part of the cluster. -The next step is to create the table your application will write to: +The next step is to create the table for your application to write to: ``` drop table if exists ping cascade; CREATE TABLE ping (id SERIAL PRIMARY KEY, node TEXT, timestamp TEXT) ; ``` -This code first drop the `ping` table. Then it recreates the `ping` table with an id primary key and two text fields for a node and timestamp. The table should now be ready. To check, use **Control-b ← Control-b ↑** or **Control-b q 0** to move to the top left pane, which puts you on the kaboom server. In this pane, become the enterprisedb user so you can easily connect to the database: +This code first drops the `ping` table. Then it re-creates the `ping` table with an id primary key and two text fields for a node and timestamp. The table should now be ready. To verify that it is, use **Control-b ← Control-b ↑** or **Control-b q 0** to move to the top left pane, which puts you on the kaboom server. In this pane, become the enterprisedb user so you can easily connect to the database: ```shell sudo -iu enterprisedb @@ -192,7 +192,7 @@ while true; done ``` -In a constant loop, you call the psql command, telling it to connect to any of the three proxies as hosts, giving the proxy port and selecting the bdrdb database. You also pass a command that inserts two values into the ping table. One of the values comes from `bdr.local_node_summary`, which contains the name of the node you're actually connected to. The other value is the current time. +In a constant loop, you call the `psql` command, telling it to connect to any of the three proxies as hosts, giving the proxy port and selecting the bdrdb database. You also pass a command that inserts two values into the ping table. One of the values comes from `bdr.local_node_summary`, which contains the name of the node you're actually connected to. The other value is the current time. Once the loop is running, new entries appear in the table. You'll see them in the top-left pane where you set up the monitor. @@ -202,7 +202,7 @@ You can now start testing failover. For this part of the process, switch to the host kaftan, which is in the lower-left corner. Use **Control-b ←** or **Control-b q 2** to switch focus to it. -To gain appropriate privileges to run `pgd`, at the PGD command line interface, run: +To gain appropriate privileges to run pgd, at the PGD command line interface, run: ```shell sudo -iu enterprisedb @@ -251,7 +251,7 @@ switchover is complete If you look in the top-left pane, you'll see the inserts from the script switching and being written to the node you just switched to. !!!Info Observe the id number -Notice that the id number being generated is from a completely different range of values, too. That's because the system transparently made the sequence generating the ID a global sequence. Read more about global sequences and how they work in [Sequences](../sequences/). +Notice that the id number being generated is from a completely different range of values, too. That's because the system transparently made the sequence generating the ID a global sequence. For more about global sequences and how they work, see [Sequences](../sequences/). !!! You might also notice an error in the lower-right pane, as an inflight update is canceled by the switch. The script then continues writing. @@ -300,7 +300,7 @@ The kaolin node is down, and updates are going to a different write leader. ## Monitoring lag -While kaolin is down, the logical replication at the heart of PGD is tracking how far out of sync kaolin is with the cluster. To see that for yourself, run: +While kaolin is down, the logical replication at the heart of PGD is tracking how far out of sync kaolin is with the cluster. To see the details, run: ```shell psql bdrdb -c "select * from bdr.node_replication_rates;" @@ -368,7 +368,7 @@ The output looks like this: (2 rows) ``` -As you can see, there's no replay lag now, as kaolin has completely caught up automatically. +As you can see, there's no replay lag now, as kaolin has completely caught up. With kaolin fully back in service, you can leave everything as it is. There's no need to change the server that's write leader. The failover mechanism is always ready to bring another server up to write leader when needed. @@ -412,13 +412,13 @@ Bring the proxy service on kaftan back by running: systemctl start pgd-proxy.service ``` -!!!Tip Exiting Tmux -You can quickly exit Tmux and all the associated sessions. First terminate any running processes, as they otherwise continue running after the session is killed. Press **Control-B** and then enter `:kill-session`. This approach is simpler than quitting each pane's session one at a time using **Control-D** or `exit`. +!!!Tip Exiting tmux +You can quickly exit tmux and all the associated sessions. First terminate any running processes, as they otherwise continue running after the session is killed. Press **Control-B** and then enter `:kill-session`. This approach is simpler than quitting each pane's session one at a time using **Control-D** or `exit`. !!! ## Other scenarios -This example uses the quick-start configuration of three data nodes and one backup node. You can configure a cluster to have two data nodes and a witness node, which is less resilient to a node failing. Or you can configure five data nodes, which is much more resilient to a node failing. With this configuration, you can explore how failover works for your applications. For clusters with multiple locations, the same basic rules apply: taking a server down elects a new write leader that proxies now point to. +This example uses the quick start configuration of three data nodes and one backup node. You can configure a cluster to have two data nodes and a witness node, which is less resilient to a node failing. Or you can configure five data nodes, which is much more resilient to a node failing. With this configuration, you can explore how failover works for your applications. For clusters with multiple locations, the same basic rules apply: taking a server down elects a new write leader that proxies now point to. ## Further reading diff --git a/product_docs/docs/pgd/5/quickstart/further_explore_replication.mdx b/product_docs/docs/pgd/5/quickstart/further_explore_replication.mdx index 074c8d02566..17d628798f1 100644 --- a/product_docs/docs/pgd/5/quickstart/further_explore_replication.mdx +++ b/product_docs/docs/pgd/5/quickstart/further_explore_replication.mdx @@ -11,44 +11,47 @@ With the cluster up and running, it's useful to run some basic checks to see how The following example shows one quick way to do this, but you must ensure that any testing you perform is appropriate for your use case. ### Preparation - * Ensure your cluster is ready to perform replication. If you haven't installed a cluster yet, use one of the [quick start](.) guides to get going: - * Log in to the database on the first host. - * Run `select bdr.wait_slot_confirm_lsn(NULL, NULL);`. - * When the query returns, the cluster is ready. + +Ensure your cluster is ready to perform replication. If you haven't installed a cluster yet, use one of the [quick start](.) guides to get going. +1. Log in to the database on the first host. +1. Run `select bdr.wait_slot_confirm_lsn(NULL, NULL);`. + +When the query returns, the cluster is ready. ### Create data - The simplest way to test that the cluster is replicating is to log in to a node, create a table, populate it, and see the data you populated appear on a second node. On the first node: - * Create a table: +The simplest way to test that the cluster is replicating is to log in to a node, create a table, populate it, and see the data you populated appear on a second node. On the first node: +1. Create a table: ```sql CREATE TABLE quicktest ( id SERIAL PRIMARY KEY, value INT ); ``` - * Populate the table: +1. Populate the table: ```sql INSERT INTO quicktest (value) SELECT random()*10000 FROM generate_series(1,10000); ``` - * Monitor replication performance: +1. Monitor replication performance: ```sql select * from bdr.node_replication_rates; ``` - * Get a sum of the value column (for checking): +1. Get a sum of the value column (for checking): ```sql select COUNT(*),SUM(value) from quicktest; ``` ### Check data - * To confirm the data was successfully replicated, log in to a second node. - * Get a sum of the value column (for checking): + +1. To confirm the data was successfully replicated, log in to a second node. + 1. Get a sum of the value column (for checking): ```sql select COUNT(*),SUM(value) from quicktest; ``` - * Compare with the result from the first node. - * Log in to a third node. - * Get a sum of the value column (for checking): + 1. Compare with the result from the first node. +1. Log in to a third node. + 1. Get a sum of the value column (for checking): ```sql select COUNT(*),SUM(value) from quicktest; ``` - * Compare with the result from the first and second nodes. + 1. Compare with the result from the first and second nodes. ## Worked example @@ -75,7 +78,7 @@ __OUTPUT__ (1 row) ``` -This query waits if the cluster is busy initializing and returns when the cluster is ready. +If the cluster is busy initializing, this query waits and returns when the cluster is ready. ### Create data @@ -91,7 +94,7 @@ CREATE TABLE #### On kaboom, populate the table -This command generates a table of 10000 rows of random values. +This command generates a table of 10000 rows of random values: ``` INSERT INTO quicktest (value) SELECT random()*10000 FROM generate_series(1,10000); @@ -101,7 +104,7 @@ INSERT 0 10000 #### On kaboom, monitor performance -As soon as possible, run the following command. It will shows statistics about how quickly that data was replicated to the other two nodes. +As soon as possible, run the following command. It shows statistics about how quickly that data was replicated to the other two nodes. ```sql select * from bdr.node_replication_rates; @@ -113,7 +116,7 @@ __OUTPUT__ (2 rows) ``` -The `replay_lag` values are 0, showing no lag, and the LSN values are in sync, meaning the data is already replicated. +The `replay_lag` values are 0, showing no lag. The LSN values are in sync, meaning the data is already replicated. #### On kaboom get a checksum @@ -173,7 +176,7 @@ You can repeat the process with the third node (kaolin), or generate new data on #### Log in to the third node (kaolin) -The third and last node is kaolin. In another window or session, log in to kaolin and then into kaolin's Postgres server: +The third and last node is kaolin. In another window or session, log in to kaolin and then to kaolin's Postgres server: ``` cd democluster @@ -200,9 +203,7 @@ __OUTPUT__ (1 row) ``` -#### Compare the results. - -Compare the result from the first and second nodes (kaboom and kaftan) with the result from kaolin. The values will be identical on all three nodes. +#### Compare the results -## Next steps +Compare the result from the first and second nodes (kaboom and kaftan) with the result from kaolin. The values are identical on all three nodes. diff --git a/product_docs/docs/pgd/5/quickstart/index.mdx b/product_docs/docs/pgd/5/quickstart/index.mdx index 28d2ebd0359..00c4d288975 100644 --- a/product_docs/docs/pgd/5/quickstart/index.mdx +++ b/product_docs/docs/pgd/5/quickstart/index.mdx @@ -22,9 +22,9 @@ EDB Postgres Distributed (PGD) is a multi-master replicating implementation of P ### Other deployment options -* If you prefer to have a fully managed EDB Postgres Distributed experience, PGD is now available as an option on BigAnimal, EDB's cloud platform for Postgres. Read more about [BigAnimal distributed high-availability clusters](/biganimal/latest/overview/02_high_availability/distributed_highavailability/). +* If you prefer to have a fully managed EDB Postgres Distributed experience, PGD is now available as an option on BigAnimal, EDB's cloud platform for Postgres. See [BigAnimal distributed high-availability clusters](/biganimal/latest/overview/02_high_availability/distributed_highavailability/). -* If you prefer to deploy PGD on Kubernetes, you can use the EDB PGD Operator for Kubernetes. Read more about [EDB PGD Operator for Kubernetes](/postgres_distributed_for_kubernetes/latest/quickstart). +* If you prefer to deploy PGD on Kubernetes, you can use the EDB PGD Operator for Kubernetes. See [EDB PGD Operator for Kubernetes](/postgres_distributed_for_kubernetes/latest/quickstart). ### What's in this quick start @@ -60,31 +60,30 @@ You will then use TPA to provision and deploy the required configuration and sof Three quick starts are currently available: * Docker — Provisions, deploys, and hosts the cluster on Docker containers on a single machine. -* Linux hosts — Deploys and hosts the cluster on Linux servers that you have already provisioned with an operating system and SSH connectivity. These can be actual physical servers or virtual machines, deployed on-premises or in the cloud. +* Linux hosts — Deploys and hosts the cluster on Linux servers that you already provisioned with an operating system and SSH connectivity. These can be actual physical servers or virtual machines, deployed on-premises or in the cloud. * AWS — Provisions, deploys, and hosts the cluster on AWS. ### Docker quick start The Docker quick start is ideal for those looking to initially explore PGD and its capabilities. This configuration of PGD isn't suitable for production use but can be valuable for testing the functionality and behavior of PGD clusters. You might also find it useful when familiarizing yourself with PGD commands and APIs to prepare for deploying on cloud, VM, or Linux hosts. -* [Begin the Docker quick start](quick_start_docker) +* [Begin the Docker quick start](quick_start_docker). ### Linux host quick start -The Linux hosts quick start is suited for those looking to install PGD on their own hosts, where they have complete control of the hardware and software, or in a private cloud. The overall configuration is similar to the Docker configuration but is more persistent over system restarts and closer to a single-region production deployment of PGD. +The Linux hosts quick start is suited if you intend to install PGD on your own hosts, where you have complete control of the hardware and software, or in a private cloud. The overall configuration is similar to the Docker configuration but is more persistent over system restarts and closer to a single-region production deployment of PGD. -* [Begin the Linux host quick start](quick_start_linux) +* [Begin the Linux host quick start](quick_start_linux). ### AWS quick start The AWS quick start is more extensive and deploys the PGD cluster onto EC2 nodes on Amazon's cloud. The cluster's overall configuration is similar to the Docker quick start. However, instead of using Docker containers, it uses t3.micro instances of Amazon EC2 to provide the compute power. The AWS deployment is more persistent and not subject to the limitations of the Docker quick start deployment. However, it requires more initial setup to configure the AWS CLI. -* [Begin the AWS quick start](quick_start_aws) +* [Begin the AWS quick start](quick_start_aws). ## Further explorations with your cluster -* [Connect applications to your PGD cluster](connecting_applications/) -* [Find out how a PGD cluster stands up to downtime of data nodes or proxies](further_explore_failover/) -* [Learn about how EDB Postgres Distributed manages conflicting updates](further_explore_conflicts/) -* [Moving beyond the quick starts](next_steps/) - +* [Connect applications to your PGD cluster](connecting_applications/). +* [Find out how a PGD cluster stands up to downtime of data nodes or proxies](further_explore_failover/). +* [Learn about how EDB Postgres Distributed manages conflicting updates](further_explore_conflicts/). +* [Move beyond the quick starts](next_steps/). diff --git a/product_docs/docs/pgd/5/quickstart/next_steps.mdx b/product_docs/docs/pgd/5/quickstart/next_steps.mdx index eae96251803..ceea92679da 100644 --- a/product_docs/docs/pgd/5/quickstart/next_steps.mdx +++ b/product_docs/docs/pgd/5/quickstart/next_steps.mdx @@ -9,33 +9,32 @@ description: > ### Architecture -In this quick start, we created a single region cluster of high availability Postgres databases. This is the, Always-on Single Location architecture, one of a range of available PGD architectures. Other architectures include Always-on Multi-Location, with clusters in multiple data centers working together, and variations of both with witness nodes enhancing resilience. Read more in [architectural options](../planning/architectures/). +This quick start created a single region cluster of high-availability Postgres databases. This is the Always-on Single Location architecture, one of a range of available PGD architectures. Other architectures include Always-on Multi-Location, with clusters in multiple data centers working together, and variations of both with witness nodes enhancing resilience. See [architectural options](../planning/architectures/). ### Postgres versions -In this quick start, we deployed EDB Postgres Advanced Server (EPAS) to the database nodes. PGD is able to deploy a three different kinds of Postgres distributions, EPAS, EDB Postgres Extended Server and open-source PostgreSQL. The selection of database affects PGD, offering [different capabilities](../planning/choosing_server/) dependant on server. +This quick start deployed EDB Postgres Advanced Server to the database nodes. PGD can deploy three different kinds of Postgres distributions, EDB Postgres Advanced Server, EDB Postgres Extended Server, and open-source PostgreSQL. The selection of database affects PGD, offering [different capabilities](../planning/choosing_server/), depending on server. -* Open-source PostgreSQL does not support CAMO -* EDB Postgres Extended Server supports CAMO, but does not offer Oracle compatibility -* EDB Postgres Advanced Server supports CAMO and offers optional Oracle compatibility +* Open-source PostgreSQL doesn't support CAMO. +* EDB Postgres Extended Server supports CAMO but doesn't offer Oracle compatibility. +* EDB Postgres Advanced Server supports CAMO and offers optional Oracle compatibility. -### Read On +### Further reading -* Learn PGD's [terminology](../terminology/) - from Asynchronous Replication to Write Scalability -* Find out how [applications work](../appusage/) with PGD and how common Postgres features like [sequences](../sequences/) are globally distributed -* Discover how PGD supports [rolling upgrades](../upgrades/) of your clusters -* Take control of [routing](../routing/) and use SQL to control the PGD Proxies -* Engage with the [PGD CLI](../cli/) to manage and monitor your cluster +* Learn PGD's [terminology](../terminology/), from asynchronous replication to write scalability. +* Find out how [applications work](../appusage/) with PGD and how common Postgres features like [sequences](../sequences/) are globally distributed. +* Discover how PGD supports [rolling upgrades](../upgrades/) of your clusters. +* Take control of [routing](../routing/) and use SQL to control the PGD proxies. +* Engage with the [PGD CLI](../cli/) to manage and monitor your cluster. ## Deprovisioning the cluster -When you're done testing the cluster, you'll want to deprovision it. +When you're done testing the cluster, deprovision it. ```shell tpaexec deprovision democluster ``` * With a Docker deployment, deprovisioning tears down the Docker containers, network, and other local configuration. -* With an AWS deployment, it will remove the EC2 instances, VPC configuration and other associated resources. Note that it will leave the S3 bucket it create; this will have to be manually removed. - +* With an AWS deployment, deprovisioning removes the EC2 instances, VPC configuration, and other associated resources. Note that it leaves the S3 bucket it created. You must manually remove it. diff --git a/product_docs/docs/pgd/5/quickstart/quick_start_aws.mdx b/product_docs/docs/pgd/5/quickstart/quick_start_aws.mdx index e6a2aa70631..c120a37cfe9 100644 --- a/product_docs/docs/pgd/5/quickstart/quick_start_aws.mdx +++ b/product_docs/docs/pgd/5/quickstart/quick_start_aws.mdx @@ -16,7 +16,7 @@ This quick start sets up EDB Postgres Distributed with an Always-on Single Locat We created TPA to make installing and managing various Postgres configurations easily repeatable. TPA orchestrates creating and deploying Postgres. In this quick start, you install TPA first. If you already have TPA installed, you can skip those steps. You can use TPA to deploy various configurations of Postgres clusters. -PGD is a multi-master replicating implementation of Postgres designed for high performance and availability. The installation of PGD is orchestrated by TPA. You will use TPA to generate a configuration file for a PGD demonstration cluster. This cluster uses Amazon EC2 instances configures your cluster with three data nodes, cohosting three [PGD Proxy](../routing/proxy/) servers, along with a [Barman](../backup#physical-backup) +PGD is a multi-master replicating implementation of Postgres designed for high performance and availability. The installation of PGD is orchestrated by TPA. You'll use TPA to generate a configuration file for a PGD demonstration cluster. This cluster uses Amazon EC2 instances configures your cluster with three data nodes, cohosting three [PGD Proxy](../routing/proxy/) servers, along with a [Barman](../backup#physical-backup) node for backup. You can then use TPA to provision and deploy the required configuration and software to each node. ## Preparation @@ -31,9 +31,9 @@ To install both TPA and PGD, you need an EDB account. [Sign up for a free EDB account](https://www.enterprisedb.com/accounts/register) if you don't already have one. Signing up gives you a trial subscription to EDB's software repositories. -After you are registered, go to the [EDB Repos 2.0](https://www.enterprisedb.com/repos-downloads) page, where you can obtain your repo token. +After you're registered, go to the [EDB Repos 2.0](https://www.enterprisedb.com/repos-downloads) page, where you can obtain your repo token. -On your first visit to this page, select **Request Access** to generate your repo token. Copy the token using the **Copy Token** icon, and store it safely. +On your first visit to this page, select **Request Access** to generate your repo token. Copy the token using the **Copy Token** icon, and store it safely. ### Install curl @@ -113,12 +113,12 @@ TPA is now installed. ### AWS Credentials TPA uses your AWS credentials to perform the deployment onto AWS. Unless you -have a corporate-managed account, you will need to [get your credentials from -AWS](https://docs.aws.amazon.com/singlesignon/latest/userguide/howtogetcredentials.html). Corporate-managed accounts will have their own process for obtaining credentials. +have a corporate-managed account, you need to [get your credentials from +AWS](https://docs.aws.amazon.com/singlesignon/latest/userguide/howtogetcredentials.html). Corporate-managed accounts have their own process for obtaining credentials. -Your credentials will consist of an AWS Access Key ID and a Secret Access Key. You will also need to select an AWS default region for your work. +Your credentials consist of an AWS Access Key ID and a Secret Access Key. You also need to select an AWS default region for your work. -Set the environment variables `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and `AWS_DEFAULT_REGION` to the values of your AWS credentials. You can add these to your `.bashrc` or similar shell profile to ensure they're always set. +Set the environment variables `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and `AWS_DEFAULT_REGION` to the values of your AWS credentials. To ensure they're always set, you can add these to your `.bashrc` or similar shell profile. ```shell $ export AWS_ACCESS_KEY_ID=THISISJUSTANEXAMPLE @@ -126,7 +126,7 @@ $ export AWS_SECRET_ACCESS_KEY=d0ntU5E/Th1SAs1ts/jUs7anEXAMPLEKEY $ export AWS_DEFAULT_REGION=us-west-2 ``` -Your account will need the necessary permissions to create and manage the resources that TPA uses. The [TPA AWS platform](/tpa/latest/platform-aws/) page details the permissions that you'll need. Consult your AWS administrator if you need help with this. +Your account needs the necessary permissions to create and manage the resources that TPA uses. [TPA AWS platform](/tpa/latest/platform-aws/) details the permissions that you need. Consult your AWS administrator if you need help with this. ## Installing PGD using TPA @@ -147,7 +147,7 @@ tpaexec configure democluster \ --hostnames-unsorted ``` -You specify the PGD-Always-ON architecture (`--architecture PGD-Always-ON`), which sets up the configuration for [PGD 5's Always-on architectures](../planning/architectures/). As part of the default architecture, +You specify the PGD-Always-ON architecture (`--architecture PGD-Always-ON`), which sets up the configuration for [PGD's Always-on architectures](../planning/architectures/). As part of the default architecture, this configures your cluster with three data nodes, cohosting three [PGD Proxy](../routing/proxy/) servers, along with a [Barman](../backup#physical-backup) node for backup. Specify that you're using AWS (`--platform aws`) and eu-west-1 as the region (`--region eu-west-1`). @@ -166,7 +166,7 @@ By default, TPA configures Debian as the default OS for all nodes on AWS. Specify that the data nodes will be running [EDB Postgres Advanced Server v16](/epas/latest/) (`--edb-postgres-advanced 16`) with Oracle compatibility (`--redwood`). -You set the notional location of the nodes to `dc1` using `--location-names`. You then set `--pgd-proxy-routing` to `local` so that proxy routing can route traffic to all nodes within each location. +You set the notional location of the nodes to `dc1` using `--location-names`. You then set `--pgd-proxy-routing` to `local` so that proxy routing can route traffic to all nodes in each location. By default, TPA commits configuration changes to a Git repository. For this example, you don't need to do that, so you pass the `--no-git` flag. @@ -184,9 +184,9 @@ less democluster/config.yml tpaexec configure --architecture PGD-Always-ON --help ``` - More details on PGD-Always-ON configuration options in [Deploying with TPA](../deploy-config/deploy-tpa/deploying/) - - [PGD-Always-ON](/tpa/latest/architecture-PGD-Always-ON/) in the Trusted Postgres Architect documentation - - [`tpaexec configure`](/tpa/latest/tpaexec-configure/) in the Trusted Postgres Architect documentation - - [AWS platform](/tpa/latest/platform-aws/) in the Trusted Postgres Architect documentation + - [PGD-Always-ON](/tpa/latest/architecture-PGD-Always-ON/) in the TPA documentation + - [`tpaexec configure`](/tpa/latest/tpaexec-configure/) in the TPA documentation + - [AWS platform](/tpa/latest/platform-aws/) in the TPA documentation ## Provisioning the cluster: @@ -221,7 +221,7 @@ TPA applies the configuration, installing the needed packages and setting up the ## Connecting to the cluster -You're now ready to log into one of the nodes of the cluster with SSH and then connect to the database. Part of the configuration process set up SSH logins for all the nodes, complete with keys. To use the SSH configuration, you need to be in the `democluster` directory created by the `tpaexec configure` command earlier: +You're now ready to log in to one of the nodes of the cluster with SSH and then connect to the database. Part of the configuration process is to set up SSH logins for all the nodes, complete with keys. To use the SSH configuration, you need to be in the `democluster` directory created by the `tpaexec configure` command earlier: ```shell cd democluster @@ -318,9 +318,8 @@ bdrdb=# ## Explore your cluster -* [Connect your database](connecting_applications) to applications -* [Explore replication](further_explore_replication) with hands-on exercises -* [Explore failover](further_explore_failover) with hands-on exercises -* [Understand conflicts](further_explore_conflicts) by creating and monitoring them -* [Next steps](next_steps) in working with your cluster - +* [Connect your database](connecting_applications) to applications. +* [Explore replication](further_explore_replication) with hands-on exercises. +* [Explore failover](further_explore_failover) with hands-on exercises. +* [Understand conflicts](further_explore_conflicts) by creating and monitoring them. +* Take the [next steps](next_steps) for working with your cluster. diff --git a/product_docs/docs/pgd/5/quickstart/quick_start_cloud.mdx b/product_docs/docs/pgd/5/quickstart/quick_start_cloud.mdx index 218517bab07..2e01eee8bf3 100644 --- a/product_docs/docs/pgd/5/quickstart/quick_start_cloud.mdx +++ b/product_docs/docs/pgd/5/quickstart/quick_start_cloud.mdx @@ -10,13 +10,12 @@ hideToC: True ## Deploying on Azure and Google clouds -For most cloud platforms such as Azure and Google Cloud Platform, you create Linux hosts on the cloud platform you are using. You can then use the [Deploying on Linux hosts](quick_start_linux) quick start to deploy PGD to those Linux hosts. +For most cloud platforms, such as Azure and Google Cloud Platform, you create Linux hosts on the cloud platform you're using. You can then use the [Deploying on Linux hosts](quick_start_linux) quick start to deploy PGD to those Linux hosts. * Azure users can follow [a Microsoft guide](https://learn.microsoft.com/en-us/azure/virtual-machines/linux/quick-create-portal?tabs=ubuntu) on how to provision Azure VMs loaded with Linux. * Google Cloud Platform users can follow [a Google guide](https://cloud.google.com/compute/docs/create-linux-vm-instance) on how to provision GCP VMs with Linux loaded. -Now continue with [Deploying on Linux hosts](quick_start_linux) - -* AWS users should go to the [Deploying on AWS](quick_start_aws) quick start where TPA will -automatically provision the hosts needed to create your cluster. +Then continue with [Deploying on Linux hosts](quick_start_linux). +For AWS users, see the [Deploying on AWS](quick_start_aws) quick start. Using this quick start, TPA +provisions the hosts needed to create your cluster. diff --git a/product_docs/docs/pgd/5/quickstart/quick_start_docker.mdx b/product_docs/docs/pgd/5/quickstart/quick_start_docker.mdx index 0ec807e46c8..61e9c0b885e 100644 --- a/product_docs/docs/pgd/5/quickstart/quick_start_docker.mdx +++ b/product_docs/docs/pgd/5/quickstart/quick_start_docker.mdx @@ -26,7 +26,7 @@ This set of steps is specifically for Ubuntu 22.04 LTS on Intel/AMD processors. ## Prerequisites -To complete this example, you need a system with enough RAM and free storage. You also need `curl` and Docker installed. +To complete this example, you need a system with enough RAM and free storage. You also need curl and Docker installed. ### RAM requirements @@ -34,11 +34,11 @@ You need a minimum of 4GB of RAM on the system. You need this much RAM because y ### Free disk space -You need at least 5GB of free storage, accessible by Docker, to deploy the cluster described by this example. A bit more is probably wise. +You need at least 5GB of free storage, accessible by Docker, to deploy the cluster described by this example. We recommend that you have a bit more than that. ### The curl utility -You will download and run scripts during this quick start using the `curl` utilty which might not be installed by default. To ensure that `curl` is installed, run: +You will download and run scripts during this quick start using the curl utility, which might not be installed by default. To ensure that curl is installed, run: ```shell sudo apt update @@ -67,11 +67,11 @@ sudo apt install docker.io ### EDB account -To install both TPA and PGD, you'll need an EDB account. +To install both TPA and PGD, you need an EDB account. [Sign up for a free EDB account](https://www.enterprisedb.com/accounts/register) if you don't already have one. Signing up gives you a trial subscription to EDB's software repositories. -After you are registered, go to the [EDB Repos 2.0](https://www.enterprisedb.com/repos-downloads) page, where you can obtain your repo token. +After you're registered, go to the [EDB Repos 2.0](https://www.enterprisedb.com/repos-downloads) page, where you can obtain your repo token. On your first visit to this page, select **Request Access** to generate your repo token. Copy the token using the **Copy Token** icon, and store it safely. @@ -154,7 +154,7 @@ tpaexec configure democluster \ ``` You specify the PGD-Always-ON architecture (`--architecture PGD-Always-ON`), which -sets up the configuration for [PGD 5's Always-on +sets up the configuration for [PGD's Always-on architectures](../planning/architectures/). As part of the default architecture, it configures your cluster with three data nodes, cohosting three [PGD Proxy](../routing/proxy/) servers, along with a [Barman](../backup#physical-backup) @@ -170,7 +170,7 @@ Linux as the default image for all nodes. Specify that the data nodes will be running [EDB Postgres Advanced Server v16](/epas/latest/) (`--edb-postgres-advanced 16`) with Oracle compatibility (`--redwood`). -You set the notional location of the nodes to `dc1` using `--location-names`. You then set `--pgd-proxy-routing` to `local` so that proxy routing can route traffic to all nodes within each location. +You set the notional location of the nodes to `dc1` using `--location-names`. You then set `--pgd-proxy-routing` to `local` so that proxy routing can route traffic to all nodes in each location. By default, TPA commits configuration changes to a Git repository. For this example, you don't need to do that, so pass the `--no-git` flag. @@ -190,15 +190,15 @@ less democluster/config.yml tpaexec configure --architecture PGD-Always-ON --help ``` - More details on PGD-Always-ON configuration options in [Deploying with TPA](../deploy-config/deploy-tpa/) - - [PGD-Always-ON](/tpa/latest/architecture-PGD-Always-ON/) in the Trusted Postgres Architect documentation - - [`tpaexec configure`](/tpa/latest/tpaexec-configure/) in the Trusted Postgres Architect documentation - - [Docker platform](/tpa/latest/platform-docker/) in the Trusted Postgres Architect documentation + - [PGD-Always-ON](/tpa/latest/architecture-PGD-Always-ON/) in the TPA documentation + - [`tpaexec configure`](/tpa/latest/tpaexec-configure/) in the TPA documentation + - [Docker platform](/tpa/latest/platform-docker/) in the TPA documentation ### Deploying the cluster You can now [deploy](/tpa/latest/tpaexec-deploy/) the distributed cluster. -For Docker deployments, this will both provision the required Docker containers and deploy the software to those containers: +For Docker deployments, deploying both provisions the required Docker containers and deploys the software to those containers: ```shell tpaexec deploy democluster @@ -211,7 +211,7 @@ TPA applies the configuration, installing the needed packages and setting up the ## Connecting to the cluster -You're now ready to log into one of the nodes of the cluster with SSH and then connect to the database. Part of the configuration process set up SSH logins for all the nodes, complete with keys. To use the SSH configuration, you need to be in the `democluster` directory created by the `tpaexec configure` command earlier: +You're now ready to log in to one of the nodes of the cluster with SSH and then connect to the database. Part of the configuration process is to set up SSH logins for all the nodes, complete with keys. To use the SSH configuration, you need to be in the `democluster` directory created by the `tpaexec configure` command earlier: ```shell cd democluster @@ -309,9 +309,8 @@ bdrdb=# ## Explore your cluster -* [Connect to your database](connecting_applications) to applications -* [Explore replication](further_explore_replication) with hands-on exercises -* [Explore failover](further_explore_failover) with hands-on exercises -* [Understand conflicts](further_explore_conflicts) by creating and monitoring them -* [Next steps](next_steps) in working with your cluster - +* [Connect to your database](connecting_applications) to applications. +* [Explore replication](further_explore_replication) with hands-on exercises. +* [Explore failover](further_explore_failover) with hands-on exercises. +* [Understand conflicts](further_explore_conflicts) by creating and monitoring them. +* Take the [next steps](next_steps) for working with your cluster. diff --git a/product_docs/docs/pgd/5/quickstart/quick_start_linux.mdx b/product_docs/docs/pgd/5/quickstart/quick_start_linux.mdx index 8fb9d2e5538..363c0c2f81e 100644 --- a/product_docs/docs/pgd/5/quickstart/quick_start_linux.mdx +++ b/product_docs/docs/pgd/5/quickstart/quick_start_linux.mdx @@ -13,7 +13,7 @@ We created TPA to make installing and managing various Postgres configurations e PGD is a multi-master replicating implementation of Postgres designed for high performance and availability. The installation of PGD is orchestrated by TPA. You will use TPA to generate a configuration file for a PGD demonstration cluster. -The TPA Linux host option allows users of any cloud or VM platform to use TPA to configure EDB Postgres Distributed. All you need from TPA is for the target system to be configured with a Linux operating system and accessible using SSH. Unlike the other TPA platforms (Docker and AWS), the Linux host configuration doesn't provision the target machines. It's up to you to provision them wherever you decide to deploy. +The TPA Linux host option allows users of any cloud or VM platform to use TPA to configure EDB Postgres Distributed. All you need from TPA is for the target system to be configured with a Linux operating system and accessible using SSH. Unlike the other TPA platforms (Docker and AWS), the Linux host configuration doesn't provision the target machines. You need to provision them wherever you decide to deploy. This cluster uses Linux server instances to host the cluster's nodes. The nodes include three replicating database nodes, three cohosted connection proxies, and one backup node. TPA can then provision, prepare, and deploy the required EDB Postgres Distributed software and configuration to each node. @@ -25,7 +25,7 @@ This set of steps is specifically for users running Ubuntu 22.04 LTS on Intel/AM ### Configure your Linux hosts -You will need to provision four hosts for this quick start. Each host should have a [supported Linux operating system](/tpa/latest/reference/distributions/) installed. To eliminate prompts for password, each host also needs to be SSH-accessible using certificate key pairs. +You need to provision four hosts for this quick start. Each host must have a [supported Linux operating system](/tpa/latest/reference/distributions/) installed. To eliminate prompts for password, each host also needs to be SSH accessible using certificate key pairs. !!! Note On machine provisioning Azure users can follow [a Microsoft guide](https://learn.microsoft.com/en-us/azure/virtual-machines/linux/quick-create-portal?tabs=ubuntu) on how to provision Azure VMs loaded with Linux. Google Cloud Platform users can follow [a Google guide](https://cloud.google.com/compute/docs/create-linux-vm-instance) on how to provision GCP VMs with Linux loaded. You can use any virtual machine technology to host a Linux instance, too. Refer to your virtualization platform's documentation for instructions on how to create instances with Linux loaded on them. @@ -54,11 +54,11 @@ Each machine requires a user account to use for installation. For simplicity, us ### EDB account -You'll need an EDB account to install both TPA and PGD. +You need an EDB account to install both TPA and PGD. [Sign up for a free EDB account](https://www.enterprisedb.com/accounts/register) if you don't already have one. Signing up gives you a trial subscription to EDB's software repositories. -After you are registered, go to the [EDB Repos 2.0](https://www.enterprisedb.com/repos-downloads) page, where you can obtain your repo token. +After you're registered, go to the [EDB Repos 2.0](https://www.enterprisedb.com/repos-downloads) page, where you can obtain your repo token. On your first visit to this page, select **Request Access** to generate your repo token. Copy the token using the **Copy Token** icon, and store it safely. @@ -132,13 +132,13 @@ tpaexec configure democluster \ --hostnames-unsorted ``` -You specify the PGD-Always-ON architecture (`--architecture PGD-Always-ON`), which sets up the configuration for [PGD 5's Always-on architectures](../planning/architectures/). As part of the default architecture, it configures your cluster with three data nodes, cohosting three [PGD Proxy](../routing/proxy/) servers and a [Barman](/backup/#physical-backup) node for backup. +You specify the PGD-Always-ON architecture (`--architecture PGD-Always-ON`), which sets up the configuration for [PGD's Always-on architectures](../planning/architectures/). As part of the default architecture, it configures your cluster with three data nodes, cohosting three [PGD Proxy](../routing/proxy/) servers and a [Barman](/backup/#physical-backup) node for backup. -For Linux hosts, specify that you're targeting a "bare" platform (`--platform bare`). TPA will determine the Linux version running on each host during deployment. See [the EDB Postgres Distributed compatibility table](https://www.enterprisedb.com/resources/platform-compatibility) for details about the supported operating systems. +For Linux hosts, specify that you're targeting a "bare" platform (`--platform bare`). TPA determines the Linux version running on each host during deployment. See [the EDB Postgres Distributed compatibility table](https://www.enterprisedb.com/resources/platform-compatibility) for details about the supported operating systems. -Specify that the data nodes will be running [EDB Postgres Advanced Server v16](https://www.enterprisedb.com/docs/epas/latest/) (`--edb-postgres-advanced 16`) with Oracle compatibility (`--redwood`). +Specify that the data nodes will be running [EDB Postgres Advanced Server v16](/epas/latest/) (`--edb-postgres-advanced 16`) with Oracle compatibility (`--redwood`). -You set the notional location of the nodes to `dc1` using `--location-names`. You then set `--pgd-proxy-routing` to `local` so that proxy routing can route traffic to all nodes within each location. +You set the notional location of the nodes to `dc1` using `--location-names`. You then set `--pgd-proxy-routing` to `local` so that proxy routing can route traffic to all nodes in each location. By default, TPA commits configuration changes to a Git repository. For this example, you don't need to do that, so pass the `--no-git` flag. @@ -228,13 +228,13 @@ You can now run: tpaexec provision democluster ``` -This command prepares for deploying the cluster. (On other platforms, such as Docker and AWS, this command also creates the required hosts. When using Linux hosts, your hosts should already be configured.) +This command prepares for deploying the cluster. (On other platforms, such as Docker and AWS, this command also creates the required hosts. When using Linux hosts, your hosts must already be configured.) !!! SeeAlso "Further reading" - [`tpaexec provision`](/tpa/latest/tpaexec-provision/) in the Trusted Postgres Architect documentation -One part of this process for Linux hosts is creating key-pairs for the hosts for SSH operations later. With those key-pairs created, you will need to copy the public part of the key-pair to the hosts. You can do this with `ssh-copy-id`, giving the democluster identity (`-i`) and the login to each host. For this example, these are the commands: +One part of this process for Linux hosts is creating key-pairs for the hosts for SSH operations later. With those key-pairs created, you need to copy the public part of the key-pair to the hosts. You can do this with `ssh-copy-id`, giving the democluster identity (`-i`) and the login to each host. For this example, these are the commands: ```shell ssh-copy-id -i democluster/id_democluster rocky@172.19.16.27 @@ -268,7 +268,7 @@ TPA applies the configuration, installing the needed packages and setting up the ## Connecting to the cluster -You're now ready to log into one of the nodes of the cluster with SSH and then connect to the database. Part of the configuration process set up SSH logins for all the nodes, complete with keys. To use the SSH configuration, you need to be in the `democluster` directory created by the `tpaexec configure` command earlier: +You're now ready to log in to one of the nodes of the cluster with SSH and then connect to the database. Part of the configuration process set up SSH logins for all the nodes, complete with keys. To use the SSH configuration, you need to be in the `democluster` directory created by the `tpaexec configure` command earlier: ```shell cd democluster @@ -363,11 +363,10 @@ Type "help" for help. bdrdb=# ``` - ## Explore your cluster -* [Connect to your database](connecting_applications) to applications -* [Explore replication](further_explore_replication) with hands-on exercises -* [Explore failover](further_explore_failover) with hands-on exercises -* [Understand conflicts](further_explore_conflicts) by creating and monitoring them -* [Next steps](next_steps) in working with your cluster +* [Connect to your database](connecting_applications) to applications. +* [Explore replication](further_explore_replication) with hands-on exercises. +* [Explore failover](further_explore_failover) with hands-on exercises. +* [Understand conflicts](further_explore_conflicts) by creating and monitoring them. +* Take the [next steps](next_steps) for working with your cluster. diff --git a/product_docs/docs/pgd/5/reference/conflict_functions.mdx b/product_docs/docs/pgd/5/reference/conflict_functions.mdx new file mode 100644 index 00000000000..7d434446382 --- /dev/null +++ b/product_docs/docs/pgd/5/reference/conflict_functions.mdx @@ -0,0 +1,117 @@ +--- +title: Conflict functions +indexdepth: 2 +--- + +## `bdr.alter_table_conflict_detection` + +Allows the table owner to change how conflict detection works for a given table. + +### Synopsis + +```sql +bdr.alter_table_conflict_detection(relation regclass, + method text, + column_name name DEFAULT NULL) +``` + +### Parameters + +- `relation` — Name of the relation for which to set the new conflict detection method. +- `method` — The conflict detection method to use. +- `column_name` — The column to use for storing the column detection data. This can be skipped, in which case the column name is chosen based on the conflict detection method. The `row_origin` method doesn't require an extra column for metadata storage. + +The recognized methods for conflict detection are: + +- `row_origin` — Origin of the previous change made on the tuple (see [Origin conflict detection](../consistency/conflicts/03_conflict_detection/#origin-conflict-detection)). This is the only method supported that doesn't require an extra column in the table. +- `row_version` — Row version column (see [Row version conflict detection](../consistency/conflicts/03_conflict_detection/#row-version-conflict-detection)). +- `column_commit_timestamp` — Per-column commit timestamps (described in [CLCD](../consistency/column-level-conflicts)). +- `column_modify_timestamp` — Per-column modification timestamp (described in [CLCD](../consistency/column-level-conflicts)). + +### Notes + +For more information about the difference between `column_commit_timestamp` and `column_modify_timestamp` conflict detection methods, see [Current versus commit timestamp](../consistency/column-level-conflicts/#current-versus-commit-timestamp). + +This function uses the same replication mechanism as `DDL` statements. This means the replication is affected by the [ddl filters](../repsets/#ddl-replication-filtering) configuration. + +The function takes a `DML` global lock on the relation for which column-level conflict resolution is being enabled. + +This function is transactional. You can roll back the effects with the `ROLLBACK` of the transaction, and the changes are visible to the current transaction. + +Only the owner of the `relation` can execute the `bdr.alter_table_conflict_detection` function unless `bdr.backwards_compatibility` is set to 30618 or less. + +!!! Warning + When changing the conflict detection method from one that uses an extra column to store metadata, that column is dropped. + +!!! Warning + This function disables CAMO and gives a warning, as long as warnings aren't disabled with `bdr.camo_enable_client_warnings`. + +## `bdr.alter_node_set_conflict_resolver` + +This function sets the behavior of conflict resolution on a given node. + +### Synopsis + +```sql +bdr.alter_node_set_conflict_resolver(node_name text, + conflict_type text, + conflict_resolver text) +``` + +### Parameters + +- `node_name` — Name of the node that's being changed. +- `conflict_type` — Conflict type for which to apply the setting (see [List of conflict types](#list-of-conflict-types)). +- `conflict_resolver` — Resolver to use for the given conflict type (see [List of conflict resolvers](#list-of-conflict-resolvers)). + +### Notes + +Currently you can change only the local node. The function call isn't replicated. If you want to change settings on multiple nodes, you must run the function on each of them. + +The configuration change made by this function overrides any default behavior of conflict resolutions specified by `bdr.create_node_group` or `bdr.alter_node_group`. + +This function is transactional. You can roll back the changes, and they are visible to the current transaction. + +## `bdr.alter_node_set_log_config` + +Set the conflict logging configuration for a node. + +### Synopsis + +```sql +bdr.alter_node_set_log_config(node_name text, + log_to_file bool DEFAULT true, + log_to_table bool DEFAULT true, + conflict_type text[] DEFAULT NULL, + conflict_resolution text[] DEFAULT NULL) +``` + +### Parameters + +- `node_name` — Name of the node that's being changed. +- `log_to_file` — Whether to log to the node log file. +- `log_to_table` — Whether to log to the `bdr.conflict_history` table. +- `conflict_type` — Conflict types to log. NULL (the default) means all. +- `conflict_resolution` — Conflict resolutions to log. NULL (the default) means all. + +### Notes + +You can change only the local node. The function call isn't replicated. If you want to change settings on multiple nodes, you must run the function on each of them. + +This function is transactional. You can roll back the changes, and they're visible to the current transaction. + +### Listing conflict logging configurations + +The view `bdr.node_log_config` shows all the logging configurations. It lists the name of the logging configuration, where it logs, and the conflict type and resolution it logs. + +### Logging conflicts to a table + +If `log_to_table` is set to true, conflicts are logged to a table. The target table for conflict logging is `bdr.conflict_history`. + +This table is range partitioned on the column `local_time`. The table is managed by autopartition. By default, a new partition is created for every day, and conflicts of the last one month are maintained. After that, the old partitions are dropped automatically. Autopartition creates between 7 and 14 partitions in advance. bdr_superuser can change these defaults. + +Since conflicts generated for all tables managed by PGD are logged to this table, it's important to ensure that only legitimate users can read the conflicted data. PGD does this by defining ROW LEVEL SECURITY policies on the `bdr.conflict_history` table. Only owners of the tables are allowed to read conflicts on the respective tables. If the underlying tables have RLS policies defined, enabled, and enforced, then even owners can't read the conflicts. RLS policies created with the FORCE option also apply to owners of the table. In that case, some or all rows in the underlying table might not be readable even to the owner. So PGD also enforces a stricter policy on the conflict log table. + +The default role `bdr_read_all_conflicts` can be granted to users who need to see all conflict details logged to the `bdr.conflict_history` table without also granting them `bdr_superuser` role. + +The default role `bdr_read_all_stats` has access to a catalog view called `bdr.conflict_history_summary`, which doesn't contain user data, allowing monitoring of any conflicts logged. diff --git a/product_docs/docs/pgd/5/reference/conflicts.mdx b/product_docs/docs/pgd/5/reference/conflicts.mdx new file mode 100644 index 00000000000..ecd776903ba --- /dev/null +++ b/product_docs/docs/pgd/5/reference/conflicts.mdx @@ -0,0 +1,107 @@ +--- +title: Conflicts +indexdepth: 2 +--- + +# Conflict detection + +## List of conflict types + +PGD recognizes the following conflict types, which can be used as the `conflict_type` parameter: + +- `insert_exists` — An incoming insert conflicts with an existing row by way of a primary key or a unique key/index. +- `update_differing` — An incoming update's key row differs from a local row. This can happen only when using [row version conflict detection](../consistency/conflicts/03_conflict_detection/#row-version-conflict-detection). +- `update_origin_change` — An incoming update is modifying a row that was last changed by a different node. +- `update_missing` — An incoming update is trying to modify a row that doesn't exist. +- `update_recently_deleted` — An incoming update is trying to modify a row that was recently deleted. +- `update_pkey_exists` — An incoming update has modified the `PRIMARY KEY` to a value that already exists on the node that's applying the change. +- `multiple_unique_conflicts` — The incoming row conflicts with multiple UNIQUE constraints/indexes in the target table. +- `delete_recently_updated` — An incoming delete with an older commit timestamp than the most recent update of the row on the current node or when using [row version conflict detection](../consistency/conflicts/03_conflict_detection/#row-version-conflict-detection). +- `delete_missing` — An incoming delete is trying to remove a row that doesn't exist. +- `target_column_missing` — The target table is missing one or more columns present in the incoming row. +- `source_column_missing` — The incoming row is missing one or more columns that are present in the target table. +- `target_table_missing` — The target table is missing. +- `apply_error_ddl` — An error was thrown by Postgres when applying a replicated DDL command. + +# Conflict resolution + +Most conflicts can be resolved automatically. PGD defaults to a last-update-wins mechanism or, more accurately, the `update_if_newer` conflict resolver. This mechanism retains the most recently inserted or changed row of the two conflicting ones based on the same commit timestamps used for conflict detection. The behavior in certain corner-case scenarios depends on the settings used for `bdr.create_node_group` and alternatively for `bdr.alter_node_group`. + +PGD lets you override the default behavior of conflict resolution by using the following function. + +## List of conflict resolvers + +Several conflict resolvers are available in PGD, with differing coverages of the conflict types they can handle: + +- `error` — Throws error and stops replication. Can be used for any conflict type. +- `skip` — Skips processing the remote change and continues replication with the next change. Can be used for `insert_exists`, `update_differing`, `update_origin_change`, `update_missing`, `update_recently_deleted`, `update_pkey_exists`, `delete_recently_updated`, `delete_missing`, `target_table_missing`, `target_column_missing`, and `source_column_missing` conflict types. +- `skip_if_recently_dropped` — Skips the remote change if it's for a table that doesn't exist downstream because it was recently (within one day) dropped on the downstream. Throw an error otherwise. Can be used for the `target_table_missing` conflict type. `skip_if_recently_dropped` conflict +resolver can pose challenges if a table with the same name is re-created shortly after it's dropped. In that case, one of the nodes might see the DMLs on the re-created table before it sees the DDL to re-create the table. It then incorrectly skips the remote data, assuming that the table is recently dropped, and causes data loss. We hence recommend that you don't reuse the object names immediately after they're dropped along with this conflict resolver. +- `skip_transaction` — Skips the whole transaction that generated the conflict. Can be used for `apply_error_ddl` conflict. +- `update_if_newer` — Updates if the remote row was committed later (as determined by the wall clock of the originating node) than the conflicting local row. If the timestamps are same, the node id is used as a tie-breaker to ensure that same row is picked on all nodes (higher nodeid wins). Can be used for `insert_exists`, `update_differing`, `update_origin_change`, and `update_pkey_exists` conflict types. +- `update` — Always performs the replicated action. Can be used for `insert_exists` (turns the `INSERT` into `UPDATE`), `update_differing`, `update_origin_change`, `update_pkey_exists`, and `delete_recently_updated` (performs the delete). +- `insert_or_skip` — Tries to build a new row from available information sent by the origin and INSERT it. If there isn't enough information available to build a full row, skips the change. Can be used for `update_missing` and `update_recently_deleted` conflict types. +- `insert_or_error` — Tries to build new row from available information sent by origin and insert it. If there isn't enough information available to build full row, throws an error and stops the replication. Can be used for `update_missing` and `update_recently_deleted` conflict types. +- `ignore` — Ignores any missing target column and continues processing. Can be used for the `target_column_missing` conflict type. +- `ignore_if_null` — Ignores a missing target column if the extra column in the remote row contains a NULL value. Otherwise, throws an error and stops replication. Can be used for the `target_column_missing` conflict type. +- `use_default_value` — Fills the missing column value with the default (including NULL if that's the column default) and continues processing. Any error while processing the default or violation of constraints (that is, NULL default on NOT NULL column) stops replication. Can be used for the `source_column_missing` conflict type. + +The `insert_exists`, `update_differing`, `update_origin_change`, `update_missing`, `multiple_unique_conflicts`, `update_recently_deleted`, `update_pkey_exists`, `delete_recently_updated`, and `delete_missing` conflict types can also be resolved by user-defined logic using +[Conflict triggers](../striggers). + +This matrix helps you individuate the conflict types the conflict resolvers can handle. + +| | insert_exists | update_differing | update_origin_change | update_missing | update_recently_deleted | update_pkey_exists | delete_recently_updated | delete_missing | target_column_missing | source_column_missing | target_table_missing | multiple_unique_conflicts | +| :----------------------- | ------------- | ---------------- | -------------------- | -------------- | ----------------------- | ------------------ | ----------------------- | -------------- | --------------------- | --------------------- | -------------------- | ------------------------- | +| error | X | X | X | X | X | X | X | X | X | X | X | X | +| skip | X | X | X | X | X | X | X | X | X | X | X | X | +| skip_if_recently_dropped | | | | | | | | | | | X | | +| update_if_newer | X | X | X | | | X | | | | | | | +| update | X | X | X | | | X | X | | | | | X | +| insert_or_skip | | | | X | X | | | | | | | | +| insert_or_error | | | | X | X | | | | | | | | +| ignore | | | | | | | | | X | | | | +| ignore_if_null | | | | | | | | | X | | | | +| use_default_value | | | | | | | | | | X | | | +| conflict_trigger | X | X | X | X | X | X | X | X | | | | X | + +## Default conflict resolvers + +| Conflict type | Resolver | +| ------------------------- | ------------------------ | +| insert_exists | update_if_newer | +| update_differing | update_if_newer | +| update_origin_change | update_if_newer | +| update_missing | insert_or_skip | +| update_recently_deleted | skip | +| update_pkey_exists | update_if_newer | +| multiple_unique_conflicts | error | +| delete_recently_updated | skip | +| delete_missing | skip | +| target_column_missing | ignore_if_null | +| source_column_missing | use_default_value | +| target_table_missing (see note) | skip_if_recently_dropped | +| apply_error_ddl | error | + +
+ +!!! note target_table_missing +This conflict type isn't detected on community Postgresql. If the target table is missing, it causes an error and halts replication. +EDB Postgres servers detect and handle missing target tables and can invoke the resolver. +!!! + +## List of conflict resolutions + +The conflict resolution represents the kind of resolution chosen by the conflict resolver and corresponds to the specific action that was taken to resolve the conflict. + +The following conflict resolutions are currently supported for the `conflict_resolution` parameter: + +- `apply_remote` — The remote (incoming) row was applied. +- `skip` — Processing of the row was skipped (no change was made locally). +- `merge` — A new row was created, merging information from remote and local row. +- `user` — User code (a conflict trigger) produced the row that was written to the target table. + +# Conflict logging + +To ease diagnosing and handling multi-master conflicts, PGD, by default, logs every conflict into the `bdr.conflict_history` table. You can change this behavior with more granularity using [bdr.alter_node_set_log_config](conflict_functions/#bdralter_node_set_log_config). + diff --git a/product_docs/docs/pgd/5/reference/index.json b/product_docs/docs/pgd/5/reference/index.json index 0a844545bb9..e1f4640e238 100644 --- a/product_docs/docs/pgd/5/reference/index.json +++ b/product_docs/docs/pgd/5/reference/index.json @@ -318,5 +318,8 @@ "bdrshow_writers": "/pgd/latest/reference/functions-internal#bdrshow_writers", "bdrtaskmgr_set_leader": "/pgd/latest/reference/functions-internal#bdrtaskmgr_set_leader", "bdrtaskmgr_get_last_completed_workitem": "/pgd/latest/reference/functions-internal#bdrtaskmgr_get_last_completed_workitem", - "bdrtaskmgr_work_queue_check_status": "/pgd/latest/reference/functions-internal#bdrtaskmgr_work_queue_check_status" + "bdrtaskmgr_work_queue_check_status": "/pgd/latest/reference/functions-internal#bdrtaskmgr_work_queue_check_status", + "bdralter_table_conflict_detection": "/pgd/latest/reference/conflict_functions#bdralter_table_conflict_detection", + "bdralter_node_set_conflict_resolver": "/pgd/latest/reference/conflict_functions#bdralter_node_set_conflict_resolver", + "bdralter_node_set_log_config": "/pgd/latest/reference/conflict_functions#bdralter_node_set_log_config" } \ No newline at end of file diff --git a/product_docs/docs/pgd/5/reference/index.mdx b/product_docs/docs/pgd/5/reference/index.mdx index 8ee2d810193..6ce731e3436 100644 --- a/product_docs/docs/pgd/5/reference/index.mdx +++ b/product_docs/docs/pgd/5/reference/index.mdx @@ -434,3 +434,9 @@ The reference section is a definitive listing of all functions, views, and comma * [`bdr.taskmgr_set_leader`](functions-internal#bdrtaskmgr_set_leader) * [`bdr.taskmgr_get_last_completed_workitem`](functions-internal#bdrtaskmgr_get_last_completed_workitem) * [`bdr.taskmgr_work_queue_check_status`](functions-internal#bdrtaskmgr_work_queue_check_status) + + +## [Conflict functions](conflict_functions) + * [`bdr.alter_table_conflict_detection`](conflict_functions#bdralter_table_conflict_detection) + * [`bdr.alter_node_set_conflict_resolver`](conflict_functions#bdralter_node_set_conflict_resolver) + * [`bdr.alter_node_set_log_config`](conflict_functions#bdralter_node_set_log_config) diff --git a/product_docs/docs/tde/15/key_stores.mdx b/product_docs/docs/tde/15/key_stores.mdx index 12be55f08e7..07ecf11c202 100644 --- a/product_docs/docs/tde/15/key_stores.mdx +++ b/product_docs/docs/tde/15/key_stores.mdx @@ -17,7 +17,7 @@ If you don't want key wrapping, for example for testing, then you must set the w Postgres leaves this configuration up to the user, which allows tailoring the setup to local requirements and integrating with existing key management software or similar. To configure the data key protection, you must specify a pair of external commands that take care of the wrapping (encrypting) and unwrapping (decryption). -## Using a passphrase +## Using a passphrase You can protect the data key with a passphrase using the openssl command line utility. The following is an example that sets up this protection: @@ -25,10 +25,10 @@ You can protect the data key with a passphrase using the openssl command line ut initdb -D datadir -y --key-wrap-command='openssl enc -e -aes-128-cbc -pbkdf2 -out "%p"' --key-unwrap-command='openssl enc -d -aes-128-cbc -pbkdf2 -in "%p"' ``` -This example wraps the randomly generated data key (done internally by initdb) by encrypting it using the AES-128-CBC (AESKW) algorithm. The encryption uses a key derived from a passphrase using the PBKDF2 key derivation function and a randomly generated salt. The terminal prompts for the passphrase. (See the openssl-enc manual page for details about these options. Available options vary across versions.) The placeholder `%p` is replaced with the name of the file to store the wrapped key. +This example wraps the randomly generated data key (done internally by initdb) by encrypting it with the AES-128-CBC (AESKW) algorithm. The encryption uses a key derived from a passphrase with the PBKDF2 key derivation function and a randomly generated salt. The terminal prompts for the passphrase. (See the openssl-enc manual page for details about these options. Available options vary across versions.) The initdb utility replaces `%p` with the name of the file that stores the wrapped key. The unwrap command performs the opposite operation. initdb doesn't need the unwrap operation. However, it stores it in the `postgresql.conf` file of the initialized cluster, which uses it when it starts up. - + The key wrap command receives the plaintext key on standard input and needs to put the wrapped key at the file system location specified by the `%p` placeholder. The key unwrap command needs to read the wrapped key from the file system location specified by the `%p` placeholder and write the unwrapped key to the standard output. @@ -43,7 +43,7 @@ export PGDATAKEYWRAPCMD PGDATAKEYUNWRAPCMD ``` Key unwrap commands that prompt for passwords on the terminal don't work when the server is started by pg_ctl or through service managers such as systemd. The server is detached from the terminal in those environments. If you want an interactive password prompt on server start, you need a more elaborate configuration that fetches the password using some indirect mechanism. - + For example, for systemd, you can use `systemd-ask-password`: ``` @@ -52,12 +52,12 @@ PGDATAKEYUNWRAPCMD="bash -c 'openssl enc -d -aes-128-cbc -pbkdf2 -in %p -pass fi ``` You also need an entry like in `/etc/sudoers`: - + ``` postgres ALL = NOPASSWD: /usr/bin/systemd-ask-password ``` -## Using a key store +## Using a key store You can use the key store in an external key management system to manage the data encryption key. The tested and supported key stores are: - Amazon AWS Key Management Service (KMS) @@ -70,7 +70,7 @@ You can use the key store in an external key management system to manage the dat ### AWS Key Management Service example -Create a key with AWS Key Management Service: +Create a key with AWS Key Management Service: ```shell aws kms create-key @@ -86,52 +86,38 @@ PGDATAKEYUNWRAPCMD='aws kms decrypt --key-id alias/pg-tde-master-1 --ciphertext- !!! Note Shell commands with pipes, as in this example, are problematic because the exit status of the pipe is that of the last command. A failure of the first, more interesting command isn't reported properly. Postgres handles this somewhat by recognizing whether the wrap or unwrap command wrote nothing. However, it's better to make this more robust. For example, use the `pipefail` option available in some shells or the `mispipe` command available on some operating systems. Put more complicated commands into an external shell script or other program instead of defining them inline. -Alternatively, you can use the [crypt utility](https://github.com/VirtusLab/crypt) to wrap and unwrap the data encryption key: - -```shell -PGDATAKEYWRAPCMD='crypt encrypt aws --out %p --region us-east-1 --kms alias/pg-tde-master-1' -PGDATAKEYUNWRAPCMD='crypt decrypt aws --in %p --region us-east-1' -``` - ### Azure Key Vault example -Create a key with Azure Key Vault: +Create a key with Azure Key Vault: ```shell az keyvault key create --vault-name pg-tde --name pg-tde-master-1 ``` -Use the `az keyvault` command with the `pg-tde-master-1` key to wrap and unwrap the data encryption key: +Use the `az keyvault key` command with the `pg-tde-master-1` key to wrap and unwrap the data encryption key: ```shell -PGDATAKEYWRAPCMD='crypt encrypt azure --vaultURL https://pg-tde.vault.azure.net --name pg-tde-master-1 --version fa2bf368449e432085318c5bf666754c --out %p' -PGDATAKEYUNWRAPCMD='crypt decrypt azure --vaultURL https://pg-tde.vault.azure.net --name pg-tde-master-1 --version fa2bf368449e432085318c5bf666754c --in %p' +PGDATAKEYWRAPCMD='az keyvault key encrypt --name pg-tde-master-1 --vault-name pg-tde --algorithm A256GCM --value @- --data-type plaintext --only-show-errors --output json | jq -r .result > "%p"' +PGDATAKEYUNWRAPCMD='az keyvault key decrypt --name pg-tde-master-1 --vault-name pg-tde --algorithm A256GCM --value @"%p" --data-type plaintext --only-show-errors --output json | jq -r .result' ``` - -This example uses [crypt](https://github.com/VirtusLab/crypt). You can't use the Azure CLI directly for this purpose because it lacks some functionality. +!!! Note + Shell commands with pipes, as in this example, are problematic because the exit status of the pipe is that of the last command. A failure of the first, more interesting command isn't reported properly. Postgres handles this somewhat by recognizing whether the wrap or unwrap command wrote nothing. However, it's better to make this more robust. For example, use the `pipefail` option available in some shells or the `mispipe` command available on some operating systems. Put more complicated commands into an external shell script or other program instead of defining them inline. ### Google Cloud KMS example -Create a key with Google Cloud KMS: +Create a key with Google Cloud KMS: ```shell gcloud kms keys create pg-tde-master-1 --location=global --keyring=pg-tde --purpose=encryption ``` -Use the `az keyvault` command with the `pg-tde-master-1` key to wrap and unwrap the data encryption key: +Use the `gcloud kms` command with the `pg-tde-master-1` key to wrap and unwrap the data encryption key: ```shell PGDATAKEYWRAPCMD='gcloud kms encrypt --plaintext-file=- --ciphertext-file=%p --location=global --keyring=pg-tde --key=pg-tde-master-1' PGDATAKEYUNWRAPCMD='gcloud kms decrypt --plaintext-file=- --ciphertext-file=%p --location=global --keyring=pg-tde --key=pg-tde-master-1' ``` -Alternatively, you can use the [crypt utility](https://github.com/VirtusLab/crypt) to wrap and unwrap the data encryption key: - -```shell -PGDATAKEYWRAPCMD='crypt encrypt gcp --out=%p --location=global --keyring=pg-tde --key=pg-tde-master-1 --project your-project-123456' -PGDATAKEYUNWRAPCMD='crypt decrypt gcp --in=%p --location=global --keyring=pg-tde --key=pg-tde-master-1 --project your-project-123456' -``` - ### HashiCorp Vault Transit Secrets Engine example ```shell