From 50de6ad018f8beb3220d1ef4fdc6e2dbfc13fd31 Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Tue, 25 Jun 2024 14:48:27 -0400 Subject: [PATCH 01/24] some updates --- .../conflicts/01_types_of_conflict.mdx | 403 ++++++++++++++++++ .../conflicts/02_avoid_tolerate_conflicts.mdx | 24 ++ .../conflicts/03_conflict_detection.mdx | 139 ++++++ .../conflicts/04_conflict_resolution.mdx | 109 +++++ .../conflicts/05_conflict_logging.mdx | 70 +++ .../consistency/conflicts/06_live_compare.mdx | 17 + .../pgd/5/consistency/conflicts/index.mdx | 34 ++ 7 files changed, 796 insertions(+) create mode 100644 product_docs/docs/pgd/5/consistency/conflicts/01_types_of_conflict.mdx create mode 100644 product_docs/docs/pgd/5/consistency/conflicts/02_avoid_tolerate_conflicts.mdx create mode 100644 product_docs/docs/pgd/5/consistency/conflicts/03_conflict_detection.mdx create mode 100644 product_docs/docs/pgd/5/consistency/conflicts/04_conflict_resolution.mdx create mode 100644 product_docs/docs/pgd/5/consistency/conflicts/05_conflict_logging.mdx create mode 100644 product_docs/docs/pgd/5/consistency/conflicts/06_live_compare.mdx create mode 100644 product_docs/docs/pgd/5/consistency/conflicts/index.mdx diff --git a/product_docs/docs/pgd/5/consistency/conflicts/01_types_of_conflict.mdx b/product_docs/docs/pgd/5/consistency/conflicts/01_types_of_conflict.mdx new file mode 100644 index 00000000000..b91ec9f7b88 --- /dev/null +++ b/product_docs/docs/pgd/5/consistency/conflicts/01_types_of_conflict.mdx @@ -0,0 +1,403 @@ +--- +title: Types of Conflict +Description: Explore the different types of possible conflicts. +deepToC: true +--- + +## PRIMARY KEY or UNIQUE conflicts + +The most common conflicts are row conflicts, where two operations affect a row with the same key in ways they can't on a single node. PGD can detect most of those and applies the `update_if_newer` conflict resolver. + +Row conflicts include: + +- `INSERT` versus `INSERT` +- `UPDATE` versus `UPDATE` +- `UPDATE` versus `DELETE` +- `INSERT` versus `UPDATE` +- `INSERT` versus `DELETE` +- `DELETE` versus `DELETE` + +The view `bdr.node_conflict_resolvers` provides information on how conflict resolution is currently configured for all known conflict types. + +### INSERT/INSERT conflicts + +The most common conflict, `INSERT`/`INSERT`, arises where `INSERT` operations on two different nodes create a tuple with the same `PRIMARY KEY` values (or if no `PRIMARY KEY` exists, the same values for a single `UNIQUE` constraint). + +PGD handles this situation by retaining the most recently inserted tuple of the two according to the originating node's timestamps. (A user-defined conflict handler can override this behavior.) + +This conflict generates the `insert_exists` conflict type, which is by default resolved by choosing the newer row, based on commit time, and keeping only that one (`update_if_newer` resolver). You can configure other resolvers. See [Conflict resolution](#conflict-resolution) for details. + +To resolve this conflict type, you can also use column-level conflict resolution and user-defined conflict triggers. + +You can effectively eliminate this type of conflict by using [global sequences](../sequences). + +### INSERT operations that violate multiple UNIQUE constraints + +An `INSERT`/`INSERT` conflict can violate more than one `UNIQUE` constraint, of which one might be the `PRIMARY KEY`. If a new row violates more than one `UNIQUE` constraint and that results in a conflict against more than one other row, then applying the replication change produces a `multiple_unique_conflicts` conflict. + +In case of such a conflict, you must remove some rows for replication to continue. Depending on the resolver setting for `multiple_unique_conflicts`, the apply process either exits with error, skips the incoming row, or deletes some of the rows. The deletion tries to preserve the row with the correct `PRIMARY KEY` and delete the others. + +!!! Warning + In case of multiple rows conflicting this way, if the result of conflict resolution is to proceed with the insert operation, some of the data is always deleted. + +You can also define a different behavior using a conflict trigger. + +### UPDATE/UPDATE conflicts + +Where two concurrent `UPDATE` operations on different nodes change the same tuple but not its `PRIMARY KEY`, an `UPDATE`/`UPDATE` conflict can occur on replay. + +These can generate different conflict kinds based on the configuration and situation. If the table is configured with [row version conflict detection](#row-version-conflict-detection), then the original (key) row is compared with the local row. If they're different, the `update_differing` conflict is generated. When using [origin conflict detection](#origin-conflict-detection), the origin of the row is checked. (The origin is the node that the current local row came from.) If that changed, the `update_origin_change` conflict is generated. In all other cases, the `UPDATE` is normally applied without generating a conflict. + +Both of these conflicts are resolved the same way as `insert_exists`, described in [INSERT/INSERT conflicts](#insertinsert-conflicts). + +### UPDATE conflicts on the PRIMARY KEY + +PGD can't currently perform conflict resolution where the `PRIMARY KEY` is changed by an `UPDATE` operation. You can update the primary key, but you must ensure that no conflict with existing values is possible. + +Conflicts on the update of the primary key are [divergent conflicts](#divergent-conflicts) and require manual intervention. + +Updating a primary key is possible in Postgres, but there are issues in both Postgres and PGD. + +A simple schema provides an example that explains: + +```sql +CREATE TABLE pktest (pk integer primary key, val integer); +INSERT INTO pktest VALUES (1,1); +``` + +Updating the Primary Key column is possible, so this SQL succeeds: + +```sql +UPDATE pktest SET pk=2 WHERE pk=1; +``` + +However, suppose the table has multiple rows: + +```sql +INSERT INTO pktest VALUES (3,3); +``` + +Some UPDATE operations succeed: + +```sql +UPDATE pktest SET pk=4 WHERE pk=3; + +SELECT * FROM pktest; + pk | val +----+----- + 2 | 1 + 4 | 3 +(2 rows) +``` + +Other UPDATE operations fail with constraint errors: + +```sql +UPDATE pktest SET pk=4 WHERE pk=2; +ERROR: duplicate key value violates unique constraint "pktest_pkey" +DETAIL: Key (pk)=(4) already exists +``` + +So for Postgres applications that update primary keys, be careful to avoid runtime errors, even without PGD. + +With PGD, the situation becomes more complex if UPDATE operations are allowed from multiple locations at same time. + +Executing these two changes concurrently works: + +```sql +node1: UPDATE pktest SET pk=pk+1 WHERE pk = 2; +node2: UPDATE pktest SET pk=pk+1 WHERE pk = 4; + +SELECT * FROM pktest; + pk | val +----+----- + 3 | 1 + 5 | 3 +(2 rows) +``` + +Executing these next two changes concurrently causes a divergent error, since both changes are accepted. But applying the changes on the other node results in `update_missing` conflicts. + +```sql +node1: UPDATE pktest SET pk=1 WHERE pk = 3; +node2: UPDATE pktest SET pk=2 WHERE pk = 3; +``` + +This scenario leaves the data different on each node: + +```sql +node1: +SELECT * FROM pktest; + pk | val +----+----- + 1 | 1 + 5 | 3 +(2 rows) + +node2: +SELECT * FROM pktest; + pk | val +----+----- + 2 | 1 + 5 | 3 +(2 rows) +``` + +You can identify and resolve this situation using [LiveCompare](/livecompare/latest). + +Concurrent conflicts present problems. Executing these two changes concurrently isn't easy to resolve: + +```sql +node1: UPDATE pktest SET pk=6, val=8 WHERE pk = 5; +node2: UPDATE pktest SET pk=6, val=9 WHERE pk = 5; +``` + +Both changes are applied locally, causing a divergence between the nodes. But the apply on the target fails on both nodes with a duplicate key-value violation error. This error causes the replication to halt and requires manual resolution. + +You can avoid this duplicate key violation error, and replication doesn't break, if you set the conflict_type `update_pkey_exists` to `skip`, `update`, or `update_if_newer`. This approach can still lead to divergence depending on the nature of the update. + +You can avoid divergence in cases where the same old key is being updated by the same new key concurrently by setting `update_pkey_exists` to `update_if_newer`. However, in certain situations, +divergence occurs even with `update_if_newer`, namely when two different rows both are updated concurrently to the same new primary key. + +As a result, we recommend strongly against allowing primary key UPDATE operations in your applications, especially with PGD. If parts of your application change primary keys, then to avoid concurrent +changes, make those changes using Eager Replication. + +!!! Warning + In case the conflict resolution of `update_pkey_exists` conflict results in update, one of the rows is always deleted. + +### UPDATE operations that violate multiple UNIQUE constraints + +Like [INSERT operations that violate multiple UNIQUE constraints](#insert-operations-that-violate-multiple-unique-constraints), when an incoming `UPDATE` violates more than one `UNIQUE` index (or the `PRIMARY KEY`), PGD raises a `multiple_unique_conflicts` conflict. + +PGD supports deferred unique constraints. If a transaction can commit on the source, then it applies cleanly on target, unless it sees conflicts. However, you can't use a deferred primary key as a REPLICA IDENTITY, so the use cases are already limited by that and the warning about using multiple unique constraints. + +### UPDATE/DELETE conflicts + +One node can update a row that another node deletes at ths same time. In this case an `UPDATE`/`DELETE` conflict can occur on replay. + +If the deleted row is still detectable (the deleted row wasn't removed by `VACUUM`), the `update_recently_deleted` conflict is generated. By default, the `UPDATE` is skipped, but you can configure the resolution for this. See [Conflict resolution](#conflict-resolution) for details. + +The database can clean up the deleted row by the time the `UPDATE` is received in case the local node is lagging behind in replication. In this case, PGD can't differentiate between `UPDATE`/`DELETE` conflicts and [INSERT/UPDATE conflicts](#insertupdate-conflicts). It generates the `update_missing` conflict. + +Another type of conflicting `DELETE` and `UPDATE` is a `DELETE` that comes after the row was updated locally. In this situation, the outcome depends on the type of conflict detection used. When using the +default, [origin conflict detection](#origin-conflict-detection), no conflict is detected, leading to the `DELETE` being applied and the row removed. If you enable [row version conflict detection](#row-version-conflict-detection), a `delete_recently_updated` conflict is generated. The default resolution for a `delete_recently_updated` conflict is to `skip` the deletion. However, you can configure the resolution or a conflict trigger can be configured to handle it. + +### INSERT/UPDATE conflicts + +When using the default asynchronous mode of operation, a node might receive an `UPDATE` of a row before the original `INSERT` was received. This can happen only when three or more nodes are active (see [Conflicts with three or more nodes](#conflicts-with-three-or-more-nodes)). + +When this happens, the `update_missing` conflict is generated. The default conflict resolver is `insert_or_skip`, though you can use `insert_or_error` or `skip` instead. Resolvers that do insert-or-action first try to `INSERT` a new row based on data from the `UPDATE` when possible (when the whole row was received). For reconstructing the row to be possible, the table either needs to have +`REPLICA IDENTITY FULL` or the row must not contain any toasted data. + +See [TOAST support details](#toast-support-details) for more info about toasted data. + +### INSERT/DELETE conflicts + +Similar to the `INSERT`/`UPDATE` conflict, the node might also receive a `DELETE` operation on a row for which it didn't yet receive an `INSERT`. This is again possible only with three or more nodes set up (see [Conflicts with three or more nodes](#conflicts-with-three-or-more-nodes)). + +PGD can't currently detect this conflict type. The `INSERT` operation doesn't generate any conflict type, and the `INSERT` is applied. + +The `DELETE` operation always generates a `delete_missing` conflict, which is by default resolved by skipping the operation. + +### DELETE/DELETE conflicts + +A `DELETE`/`DELETE` conflict arises when two different nodes concurrently delete the same tuple. + +This scenario always generates a `delete_missing` conflict, which is by default resolved by skipping the operation. + +This conflict is harmless since both `DELETE` operations have the same effect. You can safely ignroe one of them. + +### Conflicts with three or more nodes + +If one node inserts a row that's then replayed to a second node and updated there, a third node can receive the `UPDATE` from the second node before it receives the `INSERT` from the first node. This scenario is an `INSERT`/`UPDATE` conflict. + +These conflicts are handled by discarding the `UPDATE`, which can lead to different data on different nodes. These are [divergent conflicts](#divergent-conflicts). + +This conflict type can happen only with three or more masters. At least two masters must be actively writing. + +Also, the replication lag from node 1 to node 3 must be high enough to allow the following sequence of actions: + +1. node 2 receives INSERT from node 1 +2. node 2 performs UPDATE +3. node 3 receives UPDATE from node 2 +4. node 3 receives INSERT from node 1 + +Using `insert_or_error` (or in some cases the `insert_or_skip` conflict resolver for the `update_missing` conflict type) is a viable mitigation strategy for these conflicts. However, enabling this option opens the door for `INSERT`/`DELETE` conflicts: + +1. node 1 performs UPDATE +2. node 2 performs DELETE +3. node 3 receives DELETE from node 2 +4. node 3 receives UPDATE from node 1, turning it into an INSERT + +If these are problems, we recommend tuning freezing settings for a table or database so that they're correctly detected as `update_recently_deleted`. + +Another alternative is to use [Eager Replication](eager) to prevent these conflicts. + +`INSERT`/`DELETE` conflicts can also occur with three or more nodes. Such a conflict is identical to `INSERT`/`UPDATE` except with the `UPDATE` replaced by a `DELETE`. This can result in a `delete_missing` +conflict. + +PGD could choose to make each `INSERT` into a check-for-recently deleted, as occurs with an `update_missing` conflict. However, the cost of doing this penalizes the majority of users, so at this time it instead logs `delete_missing`. + +Future releases will automatically resolve `INSERT`/`DELETE` anomalies by way of rechecks using [LiveCompare](/livecompare/latest/) when `delete_missing` conflicts occur. Applications can perform these manually by checking the `bdr.conflict_history_summary` view. + +These conflicts can occur in two main problem use cases: + +- `INSERT` followed rapidly by a `DELETE`, as can be used in queuing applications +- Any case where the primary key identifier of a table is reused + +Neither of these cases is common. We recommend not replicating the affected tables if these problem use cases occur. + +PGD has problems with the latter case because PGD relies on the uniqueness of identifiers to make replication work correctly. + +Applications that insert, delete, and then later reuse the same unique identifiers can cause difficulties. This is known as the [ABA problem](https://en.wikipedia.org/wiki/ABA_problem). PGD has no way of knowing whether the rows are the current row, the last row, or much older rows. + +Unique identifier reuse is also a business problem, since it is prevents unique identification over time, which prevents auditing, traceability, and sensible data quality. Applications don't need to reuse unique identifiers. + +Any identifier reuse that occurs in the time interval it takes for changes to pass across the system causes difficulties. Although that time might be short in normal operation, down nodes can extend that +interval to hours or days. + +We recommend that applications don't reuse unique identifiers. If they do, take steps to avoid reuse in less than a year. + +This problem doesn't occur in applications that use sequences or UUIDs. + +## Foreign key constraint conflicts + +Conflicts between a remote transaction being applied and existing local data can also occur for `FOREIGN KEY` (FK) constraints. + +PGD applies changes with `session_replication_role = 'replica'`, so foreign keys aren't rechecked when applying changes. In an active/active environment, this situation can result in FK violations if deletes occur to the referenced table at the same time as inserts into the referencing table. This scenario is similar to an `INSERT`/`DELETE` conflict. + +In single-master Postgres, any `INSERT`/`UPDATE` that refers to a value in the referenced table must wait for `DELETE` operations to finish before they can gain a row-level lock. If a `DELETE` removes a referenced value, then the `INSERT`/`UPDATE` fails the FK check. + +In multi-master PGD. there are no inter-node row-level locks. An `INSERT` on the referencing table doesn't wait behind a `DELETE` on the referenced table, so both actions can occur concurrently. Thus an `INSERT`/`UPDATE` on one node on the referencing table can use a value at the same time as a `DELETE` +on the referenced table on another node. The result, then, is a value in the referencing table that's no longer present in the referenced table. + +In practice, this situation occurs if the `DELETE` operations occurs on referenced tables in separate transactions from `DELETE` operations on referencing tables, which isn't a common operation. + +In a parent-child relationship such as Orders -> OrderItems, it isn't typical to do this. It's more likely to mark an OrderItem as canceled than to remove it completely. For reference/lookup data, it's unusual to completely remove entries at the same time as using those same values for new fact data. + +While dangling FKs are possible, the risk of this in general is very low. Thus PGD doesn't impose a generic solution to cover this case. Once you understand the situation in which this occurs, two solutions are possible. + +The first solution is to restrict the use of FKs to closely related entities that are generally modified from only one node at a time, are infrequently modified, or where the modification's concurrency is +application mediated. This approach avoids any FK violations at the application level. + +The second solution is to add triggers to protect against this case using the PGD-provided functions `bdr.ri_fkey_trigger()` and `bdr.ri_fkey_on_del_trigger()`. When called as `BEFORE` triggers, these functions use `FOREIGN KEY` information to avoid FK anomalies by setting referencing columns to NULL, much as if you had a SET NULL constraint. This approach rechecks all FKs in one trigger, so you need to add only one trigger per table to prevent FK violation. + +As an example, suppose you have two tables: Fact and RefData. Fact has an FK that references RefData. Fact is the referencing table, and RefData is the referenced table. You need to add one trigger to each table. + +Add a trigger that sets columns to NULL in Fact if the referenced row in RefData was already deleted: + +```sql +CREATE TRIGGER bdr_replica_fk_iu_trg + BEFORE INSERT OR UPDATE ON fact + FOR EACH ROW + EXECUTE PROCEDURE bdr.ri_fkey_trigger(); + +ALTER TABLE fact + ENABLE REPLICA TRIGGER bdr_replica_fk_iu_trg; +``` + +Add a trigger that sets columns to NULL in Fact at the time a DELETE occurs on the RefData table: + +```sql +CREATE TRIGGER bdr_replica_fk_d_trg + BEFORE DELETE ON refdata + FOR EACH ROW + EXECUTE PROCEDURE bdr.ri_fkey_on_del_trigger(); + +ALTER TABLE refdata + ENABLE REPLICA TRIGGER bdr_replica_fk_d_trg; +``` + +Adding both triggers avoids dangling foreign keys. + +## TRUNCATE conflicts + +`TRUNCATE` behaves similarly to a `DELETE` of all rows but performs this action by physically removing the table data rather than row-by-row deletion. As a result, row-level conflict handling isn't available, so `TRUNCATE` commands don't generate conflicts with other DML actions, even when there's a clear conflict. + +As a result, the ordering of replay can cause divergent changes if another DML is executed concurrently on other nodes to the `TRUNCATE`. + +You can take one of the following actions: + +- Ensure `TRUNCATE` isn't executed alongside other concurrent DML. Rely on [LiveCompare](/livecompare/latest) to highlight any such inconsistency. + +- Replace `TRUNCATE` with a `DELETE` statement with no `WHERE` clause. This approach is likely to have poor performance on larger tables. + +- Set `bdr.truncate_locking = 'on'` to set the `TRUNCATE` command’s locking behavior. This setting determines whether `TRUNCATE` obeys the `bdr.ddl_locking` setting. This isn't the default behavior for `TRUNCATE` since it requires all nodes to be up. This configuration might not be possible or wanted in all cases. + +## Exclusion constraint conflicts + +PGD doesn't support exclusion constraints and prevents their creation. + +If an existing standalone database is converted to a PGD database, then drop all exclusion constraints manually. + +In a distributed asynchronous system, you can't ensure that no set of rows that violate the constraint exists because all transactions on different nodes are fully isolated. Exclusion constraints lead to replay deadlocks where replay can't progress from any node to any other node because of exclusion constraint violations. + +If you force PGD to create an exclusion constraint, or you don't drop existing ones when converting a standalone database to PGD, expect replication to break. To get it to progress again, remove or alter the +local tuples that an incoming remote tuple conflicts with so that the remote transaction can be applied. + +## Data conflicts for roles and tablespace differences + +Conflicts can also arise where nodes have global (Postgres-system-wide) data, like roles, that differ. This conflict can result in operations—mainly `DDL`—that can run successfully and commit on one node but then fail to apply to other nodes. + +For example, node1 might have a user named fred, and that user wasn't created on node2. If fred on node1 creates a table, the table is replicated with its owner set to fred. When the DDL command is applied to +node2, the DDL fails because there's no user named fred. This failure generates an error in the Postgres logs. + +Administrator intervention is required to resolve this conflict by creating the user fred in the database where PGD is running. You can set `bdr.role_replication = on` to resolve this in future. + +## Lock conflicts and deadlock aborts + +Because PGD writer processes operate much like normal user sessions, they're subject to the usual rules around row and table locking. This can sometimes lead to PGD writer processes waiting on locks held by user transactions or even by each other. + +Relevant locking includes: + +- Explicit table-level locking (`LOCK TABLE ...`) by user sessions +- Explicit row-level locking (`SELECT ... FOR UPDATE/FOR SHARE`) by user sessions +- Implicit locking because of row `UPDATE`, `INSERT`, or `DELETE` operations, either from local activity or from replication from other nodes + +A PGD writer process can deadlock with a user transaction, where the user transaction is waiting on a lock held by the writer process and vice versa. Two writer processes can also deadlock with each other. Postgres's deadlock detector steps in and terminates one of the problem transactions. If the PGD writer process is terminated, it retries and generally succeeds. + +All these issues are transient and generally require no administrator action. If a writer process is stuck for a long time behind a lock on an idle user session, the administrator can terminate the user session to get replication flowing again. However, this is no different from a user holding a long lock that impacts another user session. + +Use of the [log_lock_waits](https://www.postgresql.org/docs/current/runtime-config-logging.html#GUC-LOG-LOCK-WAITS) facility in Postgres can help identify locking related replay stalls. + +## Divergent conflicts + +Divergent conflicts arise when data that should be the same on different nodes differs unexpectedly. Divergent conflicts shouldn't occur, but not all such conflicts can be reliably prevented at the time of writing. + +Changing the `PRIMARY KEY` of a row can lead to a divergent conflict if another node changes the key of the same row before all nodes have replayed the change. Avoid changing primary keys, or change them only on one designated node. + +Divergent conflicts involving row data generally require administrator action to manually adjust the data on one of the nodes to be consistent with the other one. Such conflicts don't arise so long as you use PGD as documented and avoid settings or functions marked as unsafe. + +The administrator must manually resolve such conflicts. You might need to use the advanced options such as `bdr.ddl_replication` and `bdr.ddl_locking` depending on the nature of the conflict. However, careless use of these options can make things much worse and create a conflict that generic instructions can't address. + +## TOAST support details + +Postgres uses out-of-line storage for larger columns called [TOAST](https://www.postgresql.org/docs/current/storage-toast.html). + +The TOAST values handling in logical decoding (which PGD is built on top of) and logical replication is different from inline data stored as part of the main row in the table. + +The TOAST value is logged into the transaction log (WAL) only if the value changed. This can cause problems, especially when handling UPDATE conflicts, because an `UPDATE` statement that didn't change a value of a toasted column produces a row without that column. As mentioned in [INSERT/UPDATE conflicts](#insertupdate-conflicts), PGD reports an error if an `update_missing` conflict is resolved using `insert_or_error` and there are missing TOAST columns. + +However, more subtle issues than this one occur in case of concurrent workloads with asynchronous replication. (Eager transactions aren't affected.) Imagine, for example, the following workload on an EDB Postgres Distributed cluster with three nodes called A, B, and C: + +1. On node A: txn A1 does an UPDATE SET col1 = 'toast data...' and commits first. +2. On node B: txn B1 does UPDATE SET other_column = 'anything else'; and commits after A1. +3. On node C: the connection to node A lags behind. +4. On node C: txn B1 is applied first, it misses the TOASTed column in col1, but gets applied without conflict. +5. On node C: txn A1 conflicts (on update_origin_change) and is skipped. +6. Node C misses the toasted data from A1 forever. + +This scenario isn't usually a problem when using PGD. (It is when using either built-in logical replication or plain pglogical for multi-master.) PGD adds its own logging of TOAST columns when it detects a local `UPDATE` to a row that recently replicated a TOAST column modification and the local +`UPDATE` isn't modifying the TOAST. Thus PGD prevents any inconsistency for toasted data across different nodes. This situation causes increased WAL logging when updates occur on multiple nodes, that is, when origin changes for a tuple. Additional WAL overhead is zero if all updates are made from a single node, +as is normally the case with PGD AlwaysOn architecture. + +!!! Note + Running `VACUUM FULL` or `CLUSTER` on just the TOAST table without doing same on the main table removes metadata needed for the extra logging to work. This means that, for a short period after such a statement, the protection against these concurrency issues isn't present. + +!!! Warning + The additional WAL logging of TOAST is done using the `BEFORE UPDATE` trigger on standard Postgres. This trigger must be sorted alphabetically last based on trigger name among all `BEFORE UPDATE` triggers on the table. It's prefixed with `zzzz_bdr_` to make this easier, but make sure + you don't create any trigger with a name that sorts after it. Otherwise you won't have the protection against the concurrency issues. + +For the `insert_or_error` conflict resolution, the use of `REPLICA IDENTITY FULL` is still required. + +None of these problems associated with toasted columns affect tables with `REPLICA IDENTITY FULL`. This setting always logs a toasted value as part of the key since the whole row is considered to be part of the key. PGD can reconstruct the new row, filling the missing data from the key row. As a result, using `REPLICA IDENTITY FULL` can increase WAL size significantly. \ No newline at end of file diff --git a/product_docs/docs/pgd/5/consistency/conflicts/02_avoid_tolerate_conflicts.mdx b/product_docs/docs/pgd/5/consistency/conflicts/02_avoid_tolerate_conflicts.mdx new file mode 100644 index 00000000000..23222d35734 --- /dev/null +++ b/product_docs/docs/pgd/5/consistency/conflicts/02_avoid_tolerate_conflicts.mdx @@ -0,0 +1,24 @@ +--- +title: Avoiding or tolerating conflicts +Description: Explore the different types of possible conflicts. +deepToC: true +--- + +In most cases, you can design the application to avoid or tolerate conflicts. + +Conflicts can happen only if things are happening at the same time on multiple nodes. The simplest way to avoid conflicts is to only ever write to one node or to only ever write to a specific row in a specific way from one specific node at a time. + +This avoidance happens naturally in many applications. For example, many consumer applications allow only the owning user to change data, such as changing the default billing address on an account. Such data changes seldom have update conflicts. + +You might make a change just before a node goes down, so the change seems to be lost. You might then make the same change again, leading to two updates on different nodes. When the down node comes back up, it tries to send the older change to other nodes. It's rejected because the last update of the data is kept. + +For `INSERT`/`INSERT` conflicts, use [global sequences](../sequences) to prevent this type of conflict. + +For applications that assign relationships between objects, such as a room-booking application, applying `update_if_newer` might not give an acceptable business outcome. That is, it isn't useful to confirm to two people separately that they have booked the same room. The simplest resolution is to use Eager Replication to ensure that only one booking succeeds. More complex ways might be possible depending on the application. For example, you can assign 100 seats to each node and allow those to be booked by a writer on that node. But if none are available locally, use a distributed locking scheme or Eager Replication after most seats are reserved. + +Another technique for ensuring certain types of updates occur only from one specific node is to route different types of transactions through different nodes. For example: + +- Receiving parcels on one node but delivering parcels using another node +- A service application where orders are input on one node and work is prepared on a second node and then served back to customers on another + +Frequently, the best course is to allow conflicts to occur and design the application to work with PGD's conflict resolution mechanisms to cope with the conflict. \ No newline at end of file diff --git a/product_docs/docs/pgd/5/consistency/conflicts/03_conflict_detection.mdx b/product_docs/docs/pgd/5/consistency/conflicts/03_conflict_detection.mdx new file mode 100644 index 00000000000..b23035ad474 --- /dev/null +++ b/product_docs/docs/pgd/5/consistency/conflicts/03_conflict_detection.mdx @@ -0,0 +1,139 @@ +--- +title: Conflict detection +Description: Learn how PGD detects conflicts. +deepToC: true +--- + +PGD provides these mechanisms for conflict detection: + +- [Origin conflict detection](#origin-conflict-detection) (default) +- [Row version conflict detection](#row-version-conflict-detection) +- [Column-level conflict detection](column-level-conflicts) + +## Origin conflict detection + +Origin conflict detection uses and relies on commit timestamps as recorded on the node the transaction originates from. This requires clocks to be in sync to work correctly or to be within a tolerance of the fastest message between two nodes. If this isn't the case, conflict resolution tends to favor the node that's further ahead. You can manage clock skew between nodes using the parameters `bdr.maximum_clock_skew` and `bdr.maximum_clock_skew_action`. + +Row origins are available only if `track_commit_timestamp = on`. + +Conflicts are first detected based on whether the replication origin changed, so conflict triggers are called in situations that might not turn out to be conflicts. Hence, this mechanism isn't precise, since it can generate false-positive conflicts. + +Origin info is available only up to the point where a row is frozen. Updates arriving for a row after it was frozen don't raise a conflict so are applied in all cases. This is the normal case when adding a new node by `bdr_init_physical`, so raising conflicts causes many false-positive results in that case. + +A node that was offline that reconnects and begins sending data changes can cause divergent +errors if the newly arrived updates are older than the frozen rows that they update. Inserts and deletes aren't affected by this situation. + +We suggest that you don't leave down nodes for extended outages, as discussed in [Node restart and down node recovery](../node_management). + +On EDB Postgres Extended Server and EDB Postgres Advanced Server, PGD holds back the freezing of rows while a node is down. This mechanism handles this situation gracefully so you don't need to change parameter settings. + +On other variants of Postgres, you might need to manage this situation with some care. + +Freezing normally occurs when a row being vacuumed is older than `vacuum_freeze_min_age` xids from the current xid, which means that you need to configure suitably high values for these parameters: + +- `vacuum_freeze_min_age` +- `vacuum_freeze_table_age` +- `autovacuum_freeze_max_age` + +Choose values based on the transaction rate, giving a grace period of downtime before removing any conflict data from the database node. For example, when `vacuum_freeze_min_age` is set to 500 million, a node performing 1000 TPS can be down for just over 5.5 days before conflict data is removed. The CommitTS data structure takes on-disk space of 5 GB with that setting, so lower transaction rate systems can benefit from lower settings. + +Initially recommended settings are: + +```sql +# 1 billion = 10GB +autovacuum_freeze_max_age = 1000000000 + +vacuum_freeze_min_age = 500000000 + +# 90% of autovacuum_freeze_max_age +vacuum_freeze_table_age = 900000000 +``` + +Note that: + +- You can set `autovacuum_freeze_max_age` only at node start. +- You can set `vacuum_freeze_min_age`, so using a low value freezes rows early and can result in conflicts being ignored. You can also set `autovacuum_freeze_min_age` and `toast.autovacuum_freeze_min_age` for individual tables. +- Running the `CLUSTER` or `VACUUM FREEZE` commands also freezes rows early and can result in conflicts being ignored. + +## Row version conflict detection + +PGD provides the option to use row versioning and make conflict detection independent of the nodes' system clock. + +Row version conflict detection requires that you enable three things. If any of these steps aren't performed correctly then [origin conflict detection](#origin-conflict-detection) is used. + +- Enable `check_full_tuple` or the PGD node group. + +- Enable `REPLICA IDENTITY FULL` on all tables that use row version conflict detection. + +- Enable row version tracking on the table by using `bdr.alter_table_conflict_detection`. This function adds a column with a name you specify and an `UPDATE` trigger that manages the new column value. The column is created as `INTEGER` type. + +Although the counter is incremented only on `UPDATE`, this technique allows conflict detection for both `UPDATE` and `DELETE`. + +This approach resembles Lamport timestamps and fully prevents the ABA problem for conflict detection. + +!!! Note + The row-level conflict resolution is still handled based on the [conflict resolution](#conflict-resolution) configuration even with row versioning. The way the row version is generated is useful only for detecting conflicts. Don't rely on it as authoritative information about which + version of row is newer. + +To determine the current conflict resolution strategy used for a specific table, refer to the column `conflict_detection` of the view `bdr.tables`. + +## bdr.alter_table_conflict_detection + +Allows the table owner to change how conflict detection works for a given table. + +### Synopsis + +```sql +bdr.alter_table_conflict_detection(relation regclass, + method text, + column_name name DEFAULT NULL) +``` + +### Parameters + +- `relation` — Name of the relation for which to set the new conflict detection method. +- `method` — The conflict detection method to use. +- `column_name` — The column to use for storing the column detection data. This can be skipped, in which case the column name is chosen based on the conflict detection method. The `row_origin` method doesn't require an extra column for metadata storage. + +The recognized methods for conflict detection are: + +- `row_origin` — Origin of the previous change made on the tuple (see [Origin conflict detection](#origin-conflict-detection)). This is the only method supported that doesn't require an extra column in the table. +- `row_version` — Row version column (see [Row version conflict detection](#row-version-conflict-detection)). +- `column_commit_timestamp` — Per-column commit timestamps (described in [CLCD](column-level-conflicts)). +- `column_modify_timestamp` — Per-column modification timestamp (described in [CLCD](column-level-conflicts)). + +### Notes + +For more information about the difference between `column_commit_timestamp` and `column_modify_timestamp` conflict detection methods, see [Current versus commit timestamp](column-level-conflicts#current-versus-commit-timestamp). + +This function uses the same replication mechanism as `DDL` statements. This means the replication is affected by the [ddl filters](../repsets#ddl-replication-filtering) configuration. + +The function takes a `DML` global lock on the relation for which column-level conflict resolution is being enabled. + +This function is transactional. You can roll back the effects with the `ROLLBACK` of the transaction, and the changes are visible to the current transaction. + +Only the owner of the `relation` can execute the `bdr.alter_table_conflict_detection` function unless `bdr.backwards_compatibility` is set to 30618 or less. + +!!! Warning + When changing the conflict detection method from one that uses an extra column to store metadata, that column is dropped. + +!!! Warning + This function disables CAMO and gives a warning, as long as warnings aren't disabled with `bdr.camo_enable_client_warnings`. + +## List of conflict types + +PGD recognizes the following conflict types, which can be used as the `conflict_type` parameter: + +- `insert_exists` — An incoming insert conflicts with an existing row by way of a primary key or a unique key/index. +- `update_differing` — An incoming update's key row differs from a local row. This can happen only when using [row version conflict detection](#row-version-conflict-detection). +- `update_origin_change` — An incoming update is modifying a row that was last changed by a different node. +- `update_missing` — An incoming update is trying to modify a row that doesn't exist. +- `update_recently_deleted` — An incoming update is trying to modify a row that was recently deleted. +- `update_pkey_exists` — An incoming update has modified the `PRIMARY KEY` to a value that already exists on the node that's applying the change. +- `multiple_unique_conflicts` — The incoming row conflicts with multiple UNIQUE constraints/indexes in the target table. +- `delete_recently_updated` — An incoming delete with an older commit timestamp than the most recent update of the row on the current node or when using [row version conflict detection](#row-version-conflict-detection). +- `delete_missing` — An incoming delete is trying to remove a row that doesn't exist. +- `target_column_missing` — The target table is missing one or more columns present in the incoming row. +- `source_column_missing` — The incoming row is missing one or more columns that are present in the target table. +- `target_table_missing` — The target table is missing. +- `apply_error_ddl` — An error was thrown by Postgres when applying a replicated DDL command. \ No newline at end of file diff --git a/product_docs/docs/pgd/5/consistency/conflicts/04_conflict_resolution.mdx b/product_docs/docs/pgd/5/consistency/conflicts/04_conflict_resolution.mdx new file mode 100644 index 00000000000..bbbb1f9622d --- /dev/null +++ b/product_docs/docs/pgd/5/consistency/conflicts/04_conflict_resolution.mdx @@ -0,0 +1,109 @@ +--- +title: Conflict resolution +Description: Learn how PGD resolves conflicts. +deepToC: true +--- + +Most conflicts can be resolved automatically. PGD defaults to a last-update-wins mechanism or, more accurately, the `update_if_newer` conflict resolver. This mechanism retains the most recently inserted or changed row of the two conflicting ones based on the same commit timestamps used for conflict detection. The behavior in certain corner-case scenarios depends on the settings used for `bdr.create_node_group` and +alternatively for `bdr.alter_node_group`. + +PGD lets you override the default behavior of conflict resolution by using the following function. + +## bdr.alter_node_set_conflict_resolver + +This function sets the behavior of conflict resolution on a given node. + +### Synopsis + +```sql +bdr.alter_node_set_conflict_resolver(node_name text, + conflict_type text, + conflict_resolver text) +``` + +### Parameters + +- `node_name` — Name of the node that's being changed. +- `conflict_type` — Conflict type for which to apply the setting (see [List of conflict types](#list-of-conflict-types)). +- `conflict_resolver` — Resolver to use for the given conflict type (see [List of conflict resolvers](#list-of-conflict-resolvers)). + +### Notes + +Currently you can change only the local node. The function call isn't replicated. If you want to change settings on multiple nodes, you must run the function on each of them. + +The configuration change made by this function overrides any default behavior of conflict resolutions specified by `bdr.create_node_group` or `bdr.alter_node_group`. + +This function is transactional. You can roll back the changes, and they are visible to the current transaction. + +## List of conflict resolvers + +Several conflict resolvers are available in PGD, with differing coverages of the conflict types they can handle: + +- `error` — Throws error and stops replication. Can be used for any conflict type. +- `skip` — Skips processing the remote change and continues replication with the next change. Can be used for `insert_exists`, `update_differing`, `update_origin_change`, `update_missing`, `update_recently_deleted`, `update_pkey_exists`, `delete_recently_updated`, `delete_missing`, `target_table_missing`, `target_column_missing`, and `source_column_missing` conflict types. +- `skip_if_recently_dropped` — Skips the remote change if it's for a table that doesn't exist downstream because it was recently (within one day) dropped on the downstream. Throw an error otherwise. Can be used for the `target_table_missing` conflict type. `skip_if_recently_dropped` conflict +resolver can pose challenges if a table with the same name is re-created shortly after it's dropped. In that case, one of the nodes might see the DMLs on the re-created table before it sees the DDL to re-create the table. It then incorrectly skips the remote data, assuming that the table is recently dropped, and causes data loss. We hence recommend that you don't reuse the object names immediately after they're dropped along with this conflict resolver. +- `skip_transaction` — Skips the whole transaction that generated the conflict. Can be used for `apply_error_ddl` conflict. +- `update_if_newer` — Updates if the remote row was committed later (as determined by the wall clock of the originating node) than the conflicting local row. If the timestamps are same, the node id is used as a tie-breaker to ensure that same row is picked on all nodes (higher nodeid wins). Can be used for `insert_exists`, `update_differing`, `update_origin_change`, and `update_pkey_exists` conflict types. +- `update` — Always performs the replicated action. Can be used for `insert_exists` (turns the `INSERT` into `UPDATE`), `update_differing`, `update_origin_change`, `update_pkey_exists`, and `delete_recently_updated` (performs the delete). +- `insert_or_skip` — Tries to build a new row from available information sent by the origin and INSERT it. If there isn't enough information available to build a full row, skips the change. Can be used for `update_missing` and `update_recently_deleted` conflict types. +- `insert_or_error` — Tries to build new row from available information sent by origin and insert it. If there isn't enough information available to build full row, throws an error and stops the replication. Can be used for `update_missing` and `update_recently_deleted` conflict types. +- `ignore` — Ignores any missing target column and continues processing. Can be used for the `target_column_missing` conflict type. +- `ignore_if_null` — Ignores a missing target column if the extra column in the remote row contains a NULL value. Otherwise, throws an error and stops replication. Can be used for the `target_column_missing` conflict type. +- `use_default_value` — Fills the missing column value with the default (including NULL if that's the column default) and continues processing. Any error while processing the default or violation of constraints (that is, NULL default on NOT NULL column) stops replication. Can be used for the `source_column_missing` conflict type. + +The `insert_exists`, `update_differing`, `update_origin_change`, `update_missing`, `multiple_unique_conflicts`, `update_recently_deleted`, `update_pkey_exists`, `delete_recently_updated`, and `delete_missing` conflict types can also be resolved by user-defined logic using +[Conflict triggers](../striggers). + +This matrix helps you individuate the conflict types the conflict resolvers can handle. + +| | insert_exists | update_differing | update_origin_change | update_missing | update_recently_deleted | update_pkey_exists | delete_recently_updated | delete_missing | target_column_missing | source_column_missing | target_table_missing | multiple_unique_conflicts | +| :----------------------- | ------------- | ---------------- | -------------------- | -------------- | ----------------------- | ------------------ | ----------------------- | -------------- | --------------------- | --------------------- | -------------------- | ------------------------- | +| error | X | X | X | X | X | X | X | X | X | X | X | X | +| skip | X | X | X | X | X | X | X | X | X | X | X | X | +| skip_if_recently_dropped | | | | | | | | | | | X | | +| update_if_newer | X | X | X | | | X | | | | | | | +| update | X | X | X | | | X | X | | | | | X | +| insert_or_skip | | | | X | X | | | | | | | | +| insert_or_error | | | | X | X | | | | | | | | +| ignore | | | | | | | | | X | | | | +| ignore_if_null | | | | | | | | | X | | | | +| use_default_value | | | | | | | | | | X | | | +| conflict_trigger | X | X | X | X | X | X | X | X | | | | X | + +## Default conflict resolvers + +| Conflict type | Resolver | +| ------------------------- | ------------------------ | +| insert_exists | update_if_newer | +| update_differing | update_if_newer | +| update_origin_change | update_if_newer | +| update_missing | insert_or_skip | +| update_recently_deleted | skip | +| update_pkey_exists | update_if_newer | +| multiple_unique_conflicts | error | +| delete_recently_updated | skip | +| delete_missing | skip | +| target_column_missing | ignore_if_null | +| source_column_missing | use_default_value | +| target_table_missing (see note) | skip_if_recently_dropped | +| apply_error_ddl | error | + +
+ +!!! note target_table_missing +This conflict type isn't detected on community Postgresql. If the target table is missing, it causes an error and halts replication. +EDB Postgres servers detect and handle missing target tables and can invoke the resolver. +!!! + +## List of conflict resolutions + +The conflict resolution represents the kind of resolution chosen by the conflict resolver and corresponds to the specific action that was taken to resolve the conflict. + +The following conflict resolutions are currently supported for the `conflict_resolution` parameter: + +- `apply_remote` — The remote (incoming) row was applied. +- `skip` — Processing of the row was skipped (no change was made locally). +- `merge` — A new row was created, merging information from remote and local row. +- `user` — User code (a conflict trigger) produced the row that was written to the target table. + diff --git a/product_docs/docs/pgd/5/consistency/conflicts/05_conflict_logging.mdx b/product_docs/docs/pgd/5/consistency/conflicts/05_conflict_logging.mdx new file mode 100644 index 00000000000..01cdf93df1e --- /dev/null +++ b/product_docs/docs/pgd/5/consistency/conflicts/05_conflict_logging.mdx @@ -0,0 +1,70 @@ +--- +title: Conflict logging +Description: Learn how PGD logs conflicts. +deepToC: true +--- + +To ease diagnosing and handling multi-master conflicts, PGD, by default, logs every conflict into the `bdr.conflict_history` table. You can change this behavior with more granularity with the following functions. + +## bdr.alter_node_set_log_config + +Set the conflict logging configuration for a node. + +### Synopsis + +```sql +bdr.alter_node_set_log_config(node_name text, + log_to_file bool DEFAULT true, + log_to_table bool DEFAULT true, + conflict_type text[] DEFAULT NULL, + conflict_resolution text[] DEFAULT NULL) +``` + +### Parameters + +- `node_name` — Name of the node that's being changed. +- `log_to_file` — Whether to log to the node log file. +- `log_to_table` — Whether to log to the `bdr.conflict_history` table. +- `conflict_type` — Conflict types to log. NULL (the default) means all. +- `conflict_resolution` — Conflict resolutions to log. NULL (the default) means all. + +### Notes + +You can change only the local node. The function call isn't replicated. If you want to change settings on multiple nodes, you must run the function on each of them. + +This function is transactional. You can roll back the changes, and they're visible to the current transaction. + +### Listing conflict logging configurations + +The view `bdr.node_log_config` shows all the logging configurations. It lists the name of the logging configuration, where it logs, and the conflict type and resolution it logs. + +### Logging conflicts to a table + +If `log_to_table` is set to true, conflicts are logged to a table. The target table for conflict logging is `bdr.conflict_history`. + +This table is range partitioned on the column `local_time`. The table is managed by autopartition. By default, a new partition is created for every day, and conflicts of the last one month are maintained. After that, the old partitions are dropped automatically. Autopartition creates between 7 and 14 partitions in advance. bdr_superuser can change these defaults. + +Since conflicts generated for all tables managed by PGD are logged to this table, it's important to ensure that only legitimate users can read the conflicted data. PGD does this by defining ROW LEVEL SECURITY policies on the `bdr.conflict_history` table. Only owners of the tables are allowed to read conflicts on the respective tables. If the underlying tables have RLS policies defined, enabled, and enforced, then even owners can't read the conflicts. RLS policies created with the FORCE option also apply to owners of the table. In that case, some or all rows in the underlying table might not be readable even to the owner. So PGD also enforces a stricter policy on the conflict log table. + +The default role `bdr_read_all_conflicts` can be granted to users who need to see all conflict details logged to the `bdr.conflict_history` table without also granting them `bdr_superuser` role. + +The default role `bdr_read_all_stats` has access to a catalog view called `bdr.conflict_history_summary`, which doesn't contain user data, allowing monitoring of any conflicts logged. + +## Conflict reporting + +You can summarize conflicts logged to tables in reports. Reports allow application owners to identify, understand, and resolve conflicts and introduce application changes to prevent them. + +```sql +SELECT nspname, relname +, date_trunc('day', local_time) :: date AS date +, count(*) +FROM bdr.conflict_history +WHERE local_time > date_trunc('day', current_timestamp) +GROUP BY 1,2,3 +ORDER BY 1,2; + + nspname | relname | date | count +---------+---------+------------+------- + my_app | test | 2019-04-05 | 1 +(1 row) +``` \ No newline at end of file diff --git a/product_docs/docs/pgd/5/consistency/conflicts/06_live_compare.mdx b/product_docs/docs/pgd/5/consistency/conflicts/06_live_compare.mdx new file mode 100644 index 00000000000..b961a745f97 --- /dev/null +++ b/product_docs/docs/pgd/5/consistency/conflicts/06_live_compare.mdx @@ -0,0 +1,17 @@ +--- +title: Data verification with LiveCompare +Description: Explains the uses of LiveCompare for data verification. +--- + +LiveCompare is a utility program designed to compare any two databases to verify that they are identical. + +LiveCompare is included as part of the PGD stack and can be aimed at any pair of PGD nodes. By default, it compares all replicated tables and reports differences. LiveCompare also works with non-PGD data sources such as Postgres and Oracle. + +You can also use LiveCompare to continuously monitor incoming rows. You can stop and start it without losing context information, so you can run it at convenient times. + +LiveCompare allows concurrent checking of multiple tables. You can configure it to allow checking of a few tables or just a section of rows in a table. Checks are performed by first comparing whole +row hashes. If different, LiveCompare then compares whole rows. LiveCompare avoids overheads by comparing rows in useful-sized batches. + +If differences are found, they can be rechecked over time, allowing for the delays of eventual consistency. + +See the [LiveCompare](/livecompare/latest/) documentation for further details. diff --git a/product_docs/docs/pgd/5/consistency/conflicts/index.mdx b/product_docs/docs/pgd/5/consistency/conflicts/index.mdx new file mode 100644 index 00000000000..96ee2966780 --- /dev/null +++ b/product_docs/docs/pgd/5/consistency/conflicts/index.mdx @@ -0,0 +1,34 @@ +--- +title: Conflicts +redirects: + - /pgd/latest/bdr/conflicts/ +--- + +EDB Postgres Distributed is an active/active or multi-master DBMS. If used asynchronously, writes to the same or related rows from multiple different nodes can result in data conflicts when using standard data types. + +Conflicts aren't errors. In most cases, they are events that PGD can detect and resolve as they occur. Resolving them depends on the nature of the application and the meaning of the data, so it's important for +PGD to provide the application a range of choices for how to resolve conflicts. + +By default, conflicts are resolved at the row level. When changes from two nodes conflict, PGD picks either the local or remote tuple and the discards the other. For example, the commit timestamps might be compared for the two conflicting changes and the newer one kept. This approach ensures that all nodes converge to the same result and establishes commit-order-like semantics on the whole cluster. + +Conflict handling is configurable, as described in [Conflict resolution](#conflict-resolution). PGD can detect conflicts and handle them differently for each table using conflict triggers, described in [Stream triggers](../striggers). + +Column-level conflict detection and resolution is available with PGD, described in [CLCD](column-level-conflicts). + +By default, all conflicts are logged to `bdr.conflict_history`. If conflicts are possible, then table owners must monitor for them and analyze how to avoid them or make plans to handle them regularly as an application task. The [LiveCompare](/livecompare/latest) tool is also available to scan regularly for divergence. + +Some clustering systems use distributed lock mechanisms to prevent concurrent access to data. These can perform reasonably when servers are very close to each other but can't support geographically distributed applications where very low latency is critical for acceptable performance. + +Distributed locking is essentially a pessimistic approach. PGD advocates an optimistic approach, which is to avoid conflicts where possible but allow some types of conflicts to occur and resolve them when they arise. + +## How conflicts happen + +Inter-node conflicts arise as a result of sequences of events that can't happen if all the involved transactions happen concurrently on the same node. Because the nodes exchange changes only after the transactions commit, each transaction is individually valid on the node it committed on. It isn't +valid if applied on another node that did other conflicting work at the same time. + +Since PGD replication essentially replays the transaction on the other nodes, the replay operation can fail if there's a conflict between a transaction being applied and a transaction that was committed on the receiving node. + +Most conflicts can't happen when all transactions run on a single node because Postgres has inter-transaction communication mechanisms to prevent it. Examples of these mechanisms are `UNIQUE` indexes, `SEQUENCE` operations, row and relation locking, and `SERIALIZABLE` dependency tracking. All of these mechanisms are ways to communicate between ongoing transactions to prevent undesirable concurrency +issues. + +PGD doesn't have a distributed transaction manager or lock manager. That's part of why it performs well with latency and network partitions. As a result, transactions on different nodes execute entirely independently from each other when using the default, which is lazy replication. Less independence between nodes can avoid conflicts altogether, which is why PGD also offers Eager Replication for when this is important. \ No newline at end of file From 7f48e0f709b05cb12ca91fac753578a0feef1ac0 Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Tue, 25 Jun 2024 14:58:42 -0400 Subject: [PATCH 02/24] Fixed descriptions. --- .../docs/pgd/5/consistency/conflicts/01_types_of_conflict.mdx | 2 +- .../pgd/5/consistency/conflicts/02_avoid_tolerate_conflicts.mdx | 2 +- .../docs/pgd/5/consistency/conflicts/03_conflict_detection.mdx | 2 +- .../docs/pgd/5/consistency/conflicts/04_conflict_resolution.mdx | 2 +- .../docs/pgd/5/consistency/conflicts/05_conflict_logging.mdx | 2 +- .../docs/pgd/5/consistency/conflicts/06_live_compare.mdx | 2 +- 6 files changed, 6 insertions(+), 6 deletions(-) diff --git a/product_docs/docs/pgd/5/consistency/conflicts/01_types_of_conflict.mdx b/product_docs/docs/pgd/5/consistency/conflicts/01_types_of_conflict.mdx index b91ec9f7b88..66bb791ed45 100644 --- a/product_docs/docs/pgd/5/consistency/conflicts/01_types_of_conflict.mdx +++ b/product_docs/docs/pgd/5/consistency/conflicts/01_types_of_conflict.mdx @@ -1,6 +1,6 @@ --- title: Types of Conflict -Description: Explore the different types of possible conflicts. +Description: Details on different kinds of database conflicts. deepToC: true --- diff --git a/product_docs/docs/pgd/5/consistency/conflicts/02_avoid_tolerate_conflicts.mdx b/product_docs/docs/pgd/5/consistency/conflicts/02_avoid_tolerate_conflicts.mdx index 23222d35734..e75ab220747 100644 --- a/product_docs/docs/pgd/5/consistency/conflicts/02_avoid_tolerate_conflicts.mdx +++ b/product_docs/docs/pgd/5/consistency/conflicts/02_avoid_tolerate_conflicts.mdx @@ -1,6 +1,6 @@ --- title: Avoiding or tolerating conflicts -Description: Explore the different types of possible conflicts. +Description: How to avoid database conflicts. deepToC: true --- diff --git a/product_docs/docs/pgd/5/consistency/conflicts/03_conflict_detection.mdx b/product_docs/docs/pgd/5/consistency/conflicts/03_conflict_detection.mdx index b23035ad474..b7198adf61d 100644 --- a/product_docs/docs/pgd/5/consistency/conflicts/03_conflict_detection.mdx +++ b/product_docs/docs/pgd/5/consistency/conflicts/03_conflict_detection.mdx @@ -1,6 +1,6 @@ --- title: Conflict detection -Description: Learn how PGD detects conflicts. +Description: How PGD detects database conflicts. deepToC: true --- diff --git a/product_docs/docs/pgd/5/consistency/conflicts/04_conflict_resolution.mdx b/product_docs/docs/pgd/5/consistency/conflicts/04_conflict_resolution.mdx index bbbb1f9622d..ec866eaabf8 100644 --- a/product_docs/docs/pgd/5/consistency/conflicts/04_conflict_resolution.mdx +++ b/product_docs/docs/pgd/5/consistency/conflicts/04_conflict_resolution.mdx @@ -1,6 +1,6 @@ --- title: Conflict resolution -Description: Learn how PGD resolves conflicts. +Description: How PGD resolves database conflicts. deepToC: true --- diff --git a/product_docs/docs/pgd/5/consistency/conflicts/05_conflict_logging.mdx b/product_docs/docs/pgd/5/consistency/conflicts/05_conflict_logging.mdx index 01cdf93df1e..d9755312dcb 100644 --- a/product_docs/docs/pgd/5/consistency/conflicts/05_conflict_logging.mdx +++ b/product_docs/docs/pgd/5/consistency/conflicts/05_conflict_logging.mdx @@ -1,6 +1,6 @@ --- title: Conflict logging -Description: Learn how PGD logs conflicts. +Description: How PGD logs database conflicts. deepToC: true --- diff --git a/product_docs/docs/pgd/5/consistency/conflicts/06_live_compare.mdx b/product_docs/docs/pgd/5/consistency/conflicts/06_live_compare.mdx index b961a745f97..ffdbfdf0ac1 100644 --- a/product_docs/docs/pgd/5/consistency/conflicts/06_live_compare.mdx +++ b/product_docs/docs/pgd/5/consistency/conflicts/06_live_compare.mdx @@ -1,6 +1,6 @@ --- title: Data verification with LiveCompare -Description: Explains the uses of LiveCompare for data verification. +Description: LiveCompare for data verification between PGD nodes. --- LiveCompare is a utility program designed to compare any two databases to verify that they are identical. From 030a9241f60552c27102237d128f12e29830b70e Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Tue, 25 Jun 2024 16:25:25 -0400 Subject: [PATCH 03/24] Add bit to reference for testing. --- .../docs/pgd/5/consistency/conflicts.mdx | 774 ------------------ .../docs/pgd/5/reference/conflicts.mdx | 16 + product_docs/docs/pgd/5/reference/index.json | 3 +- product_docs/docs/pgd/5/reference/index.mdx | 4 + 4 files changed, 22 insertions(+), 775 deletions(-) delete mode 100644 product_docs/docs/pgd/5/consistency/conflicts.mdx create mode 100644 product_docs/docs/pgd/5/reference/conflicts.mdx diff --git a/product_docs/docs/pgd/5/consistency/conflicts.mdx b/product_docs/docs/pgd/5/consistency/conflicts.mdx deleted file mode 100644 index 335ffdaf285..00000000000 --- a/product_docs/docs/pgd/5/consistency/conflicts.mdx +++ /dev/null @@ -1,774 +0,0 @@ ---- -title: Conflicts -redirects: - - /pgd/latest/bdr/conflicts/ ---- - -EDB Postgres Distributed is an active/active or multi-master DBMS. If used asynchronously, writes to the same or related rows from multiple different nodes can result in data conflicts when using standard data types. - -Conflicts aren't errors. In most cases, they are events that PGD can detect and resolve as they occur. Resolving them depends on the nature of the application and the meaning of the data, so it's important for -PGD to provide the application a range of choices for how to resolve conflicts. - -By default, conflicts are resolved at the row level. When changes from two nodes conflict, PGD picks either the local or remote tuple and the discards the other. For example, the commit timestamps might be compared for the two conflicting changes and the newer one kept. This approach ensures that all nodes converge to the same result and establishes commit-order-like semantics on the whole cluster. - -Conflict handling is configurable, as described in [Conflict resolution](#conflict-resolution). PGD can detect conflicts and handle them differently for each table using conflict triggers, described in [Stream triggers](../striggers). - -Column-level conflict detection and resolution is available with PGD, described in [CLCD](column-level-conflicts). - -By default, all conflicts are logged to `bdr.conflict_history`. If conflicts are possible, then table owners must monitor for them and analyze how to avoid them or make plans to handle them regularly as an application task. The [LiveCompare](/livecompare/latest) tool is also available to scan regularly for divergence. - -Some clustering systems use distributed lock mechanisms to prevent concurrent access to data. These can perform reasonably when servers are very close to each other but can't support geographically distributed applications where very low latency is critical for acceptable performance. - -Distributed locking is essentially a pessimistic approach. PGD advocates an optimistic approach, which is to avoid conflicts where possible but allow some types of conflicts to occur and resolve them when they arise. - -## How conflicts happen - -Inter-node conflicts arise as a result of sequences of events that can't happen if all the involved transactions happen concurrently on the same node. Because the nodes exchange changes only after the transactions commit, each transaction is individually valid on the node it committed on. It isn't -valid if applied on another node that did other conflicting work at the same time. - -Since PGD replication essentially replays the transaction on the other nodes, the replay operation can fail if there's a conflict between a transaction being applied and a transaction that was committed on the receiving node. - -Most conflicts can't happen when all transactions run on a single node because Postgres has inter-transaction communication mechanisms to prevent it. Examples of these mechanisms are `UNIQUE` indexes, `SEQUENCE` operations, row and relation locking, and `SERIALIZABLE` dependency tracking. All of these mechanisms are ways to communicate between ongoing transactions to prevent undesirable concurrency -issues. - -PGD doesn't have a distributed transaction manager or lock manager. That's part of why it performs well with latency and network partitions. As a result, transactions on different nodes execute entirely independently from each other when using the default, which is lazy replication. Less independence between nodes can avoid conflicts altogether, which is why PGD also offers Eager Replication for when this is important. - -## Types of conflict - -### PRIMARY KEY or UNIQUE conflicts - -The most common conflicts are row conflicts, where two operations affect a row with the same key in ways they can't on a single node. PGD can detect most of those and applies the `update_if_newer` conflict resolver. - -Row conflicts include: - -- `INSERT` versus `INSERT` -- `UPDATE` versus `UPDATE` -- `UPDATE` versus `DELETE` -- `INSERT` versus `UPDATE` -- `INSERT` versus `DELETE` -- `DELETE` versus `DELETE` - -The view `bdr.node_conflict_resolvers` provides information on how conflict resolution is currently configured for all known conflict types. - -#### INSERT/INSERT conflicts - -The most common conflict, `INSERT`/`INSERT`, arises where `INSERT` operations on two different nodes create a tuple with the same `PRIMARY KEY` values (or if no `PRIMARY KEY` exists, the same values for a single `UNIQUE` constraint). - -PGD handles this situation by retaining the most recently inserted tuple of the two according to the originating node's timestamps. (A user-defined conflict handler can override this behavior.) - -This conflict generates the `insert_exists` conflict type, which is by default resolved by choosing the newer row, based on commit time, and keeping only that one (`update_if_newer` resolver). You can configure other resolvers. See [Conflict resolution](#conflict-resolution) for details. - -To resolve this conflict type, you can also use column-level conflict resolution and user-defined conflict triggers. - -You can effectively eliminate this type of conflict by using [global sequences](../sequences). - -#### INSERT operations that violate multiple UNIQUE constraints - -An `INSERT`/`INSERT` conflict can violate more than one `UNIQUE` constraint, of which one might be the `PRIMARY KEY`. If a new row violates more than one `UNIQUE` constraint and that results in a conflict against more than one other row, then applying the replication change produces a `multiple_unique_conflicts` conflict. - -In case of such a conflict, you must remove some rows for replication to continue. Depending on the resolver setting for `multiple_unique_conflicts`, the apply process either exits with error, skips the incoming row, or deletes some of the rows. The deletion tries to preserve the row with the correct `PRIMARY KEY` and delete the others. - -!!! Warning - In case of multiple rows conflicting this way, if the result of conflict resolution is to proceed with the insert operation, some of the data is always deleted. - -You can also define a different behavior using a conflict trigger. - -#### UPDATE/UPDATE conflicts - -Where two concurrent `UPDATE` operations on different nodes change the same tuple but not its `PRIMARY KEY`, an `UPDATE`/`UPDATE` conflict can occur on replay. - -These can generate different conflict kinds based on the configuration and situation. If the table is configured with [row version conflict detection](#row-version-conflict-detection), then the original (key) row is compared with the local row. If they're different, the `update_differing` conflict is generated. When using [origin conflict detection](#origin-conflict-detection), the origin of the row is checked. (The origin is the node that the current local row came from.) If that changed, the `update_origin_change` conflict is generated. In all other cases, the `UPDATE` is normally applied without generating a conflict. - -Both of these conflicts are resolved the same way as `insert_exists`, described in [INSERT/INSERT conflicts](#insertinsert-conflicts). - -#### UPDATE conflicts on the PRIMARY KEY - -PGD can't currently perform conflict resolution where the `PRIMARY KEY` is changed by an `UPDATE` operation. You can update the primary key, but you must ensure that no conflict with existing values is possible. - -Conflicts on the update of the primary key are [divergent conflicts](#divergent-conflicts) and require manual intervention. - -Updating a primary key is possible in Postgres, but there are issues in both Postgres and PGD. - -A simple schema provides an example that explains: - -```sql -CREATE TABLE pktest (pk integer primary key, val integer); -INSERT INTO pktest VALUES (1,1); -``` - -Updating the Primary Key column is possible, so this SQL succeeds: - -```sql -UPDATE pktest SET pk=2 WHERE pk=1; -``` - -However, suppose the table has multiple rows: - -```sql -INSERT INTO pktest VALUES (3,3); -``` - -Some UPDATE operations succeed: - -```sql -UPDATE pktest SET pk=4 WHERE pk=3; - -SELECT * FROM pktest; - pk | val -----+----- - 2 | 1 - 4 | 3 -(2 rows) -``` - -Other UPDATE operations fail with constraint errors: - -```sql -UPDATE pktest SET pk=4 WHERE pk=2; -ERROR: duplicate key value violates unique constraint "pktest_pkey" -DETAIL: Key (pk)=(4) already exists -``` - -So for Postgres applications that update primary keys, be careful to avoid runtime errors, even without PGD. - -With PGD, the situation becomes more complex if UPDATE operations are allowed from multiple locations at same time. - -Executing these two changes concurrently works: - -```sql -node1: UPDATE pktest SET pk=pk+1 WHERE pk = 2; -node2: UPDATE pktest SET pk=pk+1 WHERE pk = 4; - -SELECT * FROM pktest; - pk | val -----+----- - 3 | 1 - 5 | 3 -(2 rows) -``` - -Executing these next two changes concurrently causes a divergent error, since both changes are accepted. But applying the changes on the other node results in `update_missing` conflicts. - -```sql -node1: UPDATE pktest SET pk=1 WHERE pk = 3; -node2: UPDATE pktest SET pk=2 WHERE pk = 3; -``` - -This scenario leaves the data different on each node: - -```sql -node1: -SELECT * FROM pktest; - pk | val -----+----- - 1 | 1 - 5 | 3 -(2 rows) - -node2: -SELECT * FROM pktest; - pk | val -----+----- - 2 | 1 - 5 | 3 -(2 rows) -``` - -You can identify and resolve this situation using [LiveCompare](/livecompare/latest). - -Concurrent conflicts present problems. Executing these two changes concurrently isn't easy to resolve: - -```sql -node1: UPDATE pktest SET pk=6, val=8 WHERE pk = 5; -node2: UPDATE pktest SET pk=6, val=9 WHERE pk = 5; -``` - -Both changes are applied locally, causing a divergence between the nodes. But the apply on the target fails on both nodes with a duplicate key-value violation error. This error causes the replication to halt and requires manual resolution. - -You can avoid this duplicate key violation error, and replication doesn't break, if you set the conflict_type `update_pkey_exists` to `skip`, `update`, or `update_if_newer`. This approach can still lead to divergence depending on the nature of the update. - -You can avoid divergence in cases where the same old key is being updated by the same new key concurrently by setting `update_pkey_exists` to `update_if_newer`. However, in certain situations, -divergence occurs even with `update_if_newer`, namely when two different rows both are updated concurrently to the same new primary key. - -As a result, we recommend strongly against allowing primary key UPDATE operations in your applications, especially with PGD. If parts of your application change primary keys, then to avoid concurrent -changes, make those changes using Eager Replication. - -!!! Warning - In case the conflict resolution of `update_pkey_exists` conflict results in update, one of the rows is always deleted. - -#### UPDATE operations that violate multiple UNIQUE constraints - -Like [INSERT operations that violate multiple UNIQUE constraints](#insert-operations-that-violate-multiple-unique-constraints), when an incoming `UPDATE` violates more than one `UNIQUE` index (or the `PRIMARY KEY`), PGD raises a `multiple_unique_conflicts` conflict. - -PGD supports deferred unique constraints. If a transaction can commit on the source, then it applies cleanly on target, unless it sees conflicts. However, you can't use a deferred primary key as a REPLICA IDENTITY, so the use cases are already limited by that and the warning about using multiple unique constraints. - -#### UPDATE/DELETE conflicts - -One node can update a row that another node deletes at ths same time. In this case an `UPDATE`/`DELETE` conflict can occur on replay. - -If the deleted row is still detectable (the deleted row wasn't removed by `VACUUM`), the `update_recently_deleted` conflict is generated. By default, the `UPDATE` is skipped, but you can configure the resolution for this. See [Conflict resolution](#conflict-resolution) for details. - -The database can clean up the deleted row by the time the `UPDATE` is received in case the local node is lagging behind in replication. In this case, PGD can't differentiate between `UPDATE`/`DELETE` conflicts and [INSERT/UPDATE conflicts](#insertupdate-conflicts). It generates the `update_missing` conflict. - -Another type of conflicting `DELETE` and `UPDATE` is a `DELETE` that comes after the row was updated locally. In this situation, A `delete_recently_updated` conflict is generated. The default resolution for a `delete_recently_updated` conflict is to `skip` the deletion. However, you can configure the resolution or a conflict trigger can be configured to handle it. - -#### INSERT/UPDATE conflicts - -When using the default asynchronous mode of operation, a node might receive an `UPDATE` of a row before the original `INSERT` was received. This can happen only when three or more nodes are active (see [Conflicts with three or more nodes](#conflicts-with-three-or-more-nodes)). - -When this happens, the `update_missing` conflict is generated. The default conflict resolver is `insert_or_skip`, though you can use `insert_or_error` or `skip` instead. Resolvers that do insert-or-action first try to `INSERT` a new row based on data from the `UPDATE` when possible (when the whole row was received). For reconstructing the row to be possible, the table either needs to have `REPLICA IDENTITY FULL` or the row must not contain any toasted data. - -See [TOAST support details](#toast-support-details) for more info about toasted data. - -#### INSERT/DELETE conflicts - -Similar to the `INSERT`/`UPDATE` conflict, the node might also receive a `DELETE` operation on a row for which it didn't yet receive an `INSERT`. This is again possible only with three or more nodes set up (see [Conflicts with three or more nodes](#conflicts-with-three-or-more-nodes)). - -PGD can't currently detect this conflict type. The `INSERT` operation doesn't generate any conflict type, and the `INSERT` is applied. - -The `DELETE` operation always generates a `delete_missing` conflict, which is by default resolved by skipping the operation. - -#### DELETE/DELETE conflicts - -A `DELETE`/`DELETE` conflict arises when two different nodes concurrently delete the same tuple. - -This scenario always generates a `delete_missing` conflict, which is by default resolved by skipping the operation. - -This conflict is harmless since both `DELETE` operations have the same effect. You can safely ignroe one of them. - -#### Conflicts with three or more nodes - -If one node inserts a row that's then replayed to a second node and updated there, a third node can receive the `UPDATE` from the second node before it receives the `INSERT` from the first node. This scenario is an `INSERT`/`UPDATE` conflict. - -These conflicts are handled by discarding the `UPDATE`, which can lead to different data on different nodes. These are [divergent conflicts](#divergent-conflicts). - -This conflict type can happen only with three or more masters. At least two masters must be actively writing. - -Also, the replication lag from node 1 to node 3 must be high enough to allow the following sequence of actions: - -1. node 2 receives INSERT from node 1 -2. node 2 performs UPDATE -3. node 3 receives UPDATE from node 2 -4. node 3 receives INSERT from node 1 - -Using `insert_or_error` (or in some cases the `insert_or_skip` conflict resolver for the `update_missing` conflict type) is a viable mitigation strategy for these conflicts. However, enabling this option opens the door for `INSERT`/`DELETE` conflicts: - -1. node 1 performs UPDATE -2. node 2 performs DELETE -3. node 3 receives DELETE from node 2 -4. node 3 receives UPDATE from node 1, turning it into an INSERT - -If these are problems, we recommend tuning freezing settings for a table or database so that they're correctly detected as `update_recently_deleted`. - -Another alternative is to use [Eager Replication](eager) to prevent these conflicts. - -`INSERT`/`DELETE` conflicts can also occur with three or more nodes. Such a conflict is identical to `INSERT`/`UPDATE` except with the `UPDATE` replaced by a `DELETE`. This can result in a `delete_missing` -conflict. - -PGD could choose to make each `INSERT` into a check-for-recently deleted, as occurs with an `update_missing` conflict. However, the cost of doing this penalizes the majority of users, so at this time it instead logs `delete_missing`. - -Future releases will automatically resolve `INSERT`/`DELETE` anomalies by way of rechecks using [LiveCompare](/livecompare/latest/) when `delete_missing` conflicts occur. Applications can perform these manually by checking the `bdr.conflict_history_summary` view. - -These conflicts can occur in two main problem use cases: - -- `INSERT` followed rapidly by a `DELETE`, as can be used in queuing applications -- Any case where the primary key identifier of a table is reused - -Neither of these cases is common. We recommend not replicating the affected tables if these problem use cases occur. - -PGD has problems with the latter case because PGD relies on the uniqueness of identifiers to make replication work correctly. - -Applications that insert, delete, and then later reuse the same unique identifiers can cause difficulties. This is known as the [ABA problem](https://en.wikipedia.org/wiki/ABA_problem). PGD has no way of knowing whether the rows are the current row, the last row, or much older rows. - -Unique identifier reuse is also a business problem, since it is prevents unique identification over time, which prevents auditing, traceability, and sensible data quality. Applications don't need to reuse unique identifiers. - -Any identifier reuse that occurs in the time interval it takes for changes to pass across the system causes difficulties. Although that time might be short in normal operation, down nodes can extend that -interval to hours or days. - -We recommend that applications don't reuse unique identifiers. If they do, take steps to avoid reuse in less than a year. - -This problem doesn't occur in applications that use sequences or UUIDs. - -### Foreign key constraint conflicts - -Conflicts between a remote transaction being applied and existing local data can also occur for `FOREIGN KEY` (FK) constraints. - -PGD applies changes with `session_replication_role = 'replica'`, so foreign keys aren't rechecked when applying changes. In an active/active environment, this situation can result in FK violations if deletes occur to the referenced table at the same time as inserts into the referencing table. This scenario is similar to an `INSERT`/`DELETE` conflict. - -In single-master Postgres, any `INSERT`/`UPDATE` that refers to a value in the referenced table must wait for `DELETE` operations to finish before they can gain a row-level lock. If a `DELETE` removes a referenced value, then the `INSERT`/`UPDATE` fails the FK check. - -In multi-master PGD. there are no inter-node row-level locks. An `INSERT` on the referencing table doesn't wait behind a `DELETE` on the referenced table, so both actions can occur concurrently. Thus an `INSERT`/`UPDATE` on one node on the referencing table can use a value at the same time as a `DELETE` -on the referenced table on another node. The result, then, is a value in the referencing table that's no longer present in the referenced table. - -In practice, this situation occurs if the `DELETE` operations occurs on referenced tables in separate transactions from `DELETE` operations on referencing tables, which isn't a common operation. - -In a parent-child relationship such as Orders -> OrderItems, it isn't typical to do this. It's more likely to mark an OrderItem as canceled than to remove it completely. For reference/lookup data, it's unusual to completely remove entries at the same time as using those same values for new fact data. - -While dangling FKs are possible, the risk of this in general is very low. Thus PGD doesn't impose a generic solution to cover this case. Once you understand the situation in which this occurs, two solutions are possible. - -The first solution is to restrict the use of FKs to closely related entities that are generally modified from only one node at a time, are infrequently modified, or where the modification's concurrency is -application mediated. This approach avoids any FK violations at the application level. - -The second solution is to add triggers to protect against this case using the PGD-provided functions `bdr.ri_fkey_trigger()` and `bdr.ri_fkey_on_del_trigger()`. When called as `BEFORE` triggers, these functions use `FOREIGN KEY` information to avoid FK anomalies by setting referencing columns to NULL, much as if you had a SET NULL constraint. This approach rechecks all FKs in one trigger, so you need to add only one trigger per table to prevent FK violation. - -As an example, suppose you have two tables: Fact and RefData. Fact has an FK that references RefData. Fact is the referencing table, and RefData is the referenced table. You need to add one trigger to each table. - -Add a trigger that sets columns to NULL in Fact if the referenced row in RefData was already deleted: - -```sql -CREATE TRIGGER bdr_replica_fk_iu_trg - BEFORE INSERT OR UPDATE ON fact - FOR EACH ROW - EXECUTE PROCEDURE bdr.ri_fkey_trigger(); - -ALTER TABLE fact - ENABLE REPLICA TRIGGER bdr_replica_fk_iu_trg; -``` - -Add a trigger that sets columns to NULL in Fact at the time a DELETE occurs on the RefData table: - -```sql -CREATE TRIGGER bdr_replica_fk_d_trg - BEFORE DELETE ON refdata - FOR EACH ROW - EXECUTE PROCEDURE bdr.ri_fkey_on_del_trigger(); - -ALTER TABLE refdata - ENABLE REPLICA TRIGGER bdr_replica_fk_d_trg; -``` - -Adding both triggers avoids dangling foreign keys. - -### TRUNCATE conflicts - -`TRUNCATE` behaves similarly to a `DELETE` of all rows but performs this action by physically removing the table data rather than row-by-row deletion. As a result, row-level conflict handling isn't available, so `TRUNCATE` commands don't generate conflicts with other DML actions, even when there's a clear conflict. - -As a result, the ordering of replay can cause divergent changes if another DML is executed concurrently on other nodes to the `TRUNCATE`. - -You can take one of the following actions: - -- Ensure `TRUNCATE` isn't executed alongside other concurrent DML. Rely on [LiveCompare](/livecompare/latest) to highlight any such inconsistency. - -- Replace `TRUNCATE` with a `DELETE` statement with no `WHERE` clause. This approach is likely to have poor performance on larger tables. - -- Set `bdr.truncate_locking = 'on'` to set the `TRUNCATE` command’s locking behavior. This setting determines whether `TRUNCATE` obeys the `bdr.ddl_locking` setting. This isn't the default behavior for `TRUNCATE` since it requires all nodes to be up. This configuration might not be possible or wanted in all cases. - -### Exclusion constraint conflicts - -PGD doesn't support exclusion constraints and prevents their creation. - -If an existing standalone database is converted to a PGD database, then drop all exclusion constraints manually. - -In a distributed asynchronous system, you can't ensure that no set of rows that violate the constraint exists because all transactions on different nodes are fully isolated. Exclusion constraints lead to replay deadlocks where replay can't progress from any node to any other node because of exclusion constraint violations. - -If you force PGD to create an exclusion constraint, or you don't drop existing ones when converting a standalone database to PGD, expect replication to break. To get it to progress again, remove or alter the -local tuples that an incoming remote tuple conflicts with so that the remote transaction can be applied. - -### Data conflicts for roles and tablespace differences - -Conflicts can also arise where nodes have global (Postgres-system-wide) data, like roles, that differ. This conflict can result in operations—mainly `DDL`—that can run successfully and commit on one node but then fail to apply to other nodes. - -For example, node1 might have a user named fred, and that user wasn't created on node2. If fred on node1 creates a table, the table is replicated with its owner set to fred. When the DDL command is applied to -node2, the DDL fails because there's no user named fred. This failure generates an error in the Postgres logs. - -Administrator intervention is required to resolve this conflict by creating the user fred in the database where PGD is running. You can set `bdr.role_replication = on` to resolve this in future. - -### Lock conflicts and deadlock aborts - -Because PGD writer processes operate much like normal user sessions, they're subject to the usual rules around row and table locking. This can sometimes lead to PGD writer processes waiting on locks held by user transactions or even by each other. - -Relevant locking includes: - -- Explicit table-level locking (`LOCK TABLE ...`) by user sessions -- Explicit row-level locking (`SELECT ... FOR UPDATE/FOR SHARE`) by user sessions -- Implicit locking because of row `UPDATE`, `INSERT`, or `DELETE` operations, either from local activity or from replication from other nodes - -A PGD writer process can deadlock with a user transaction, where the user transaction is waiting on a lock held by the writer process and vice versa. Two writer processes can also deadlock with each other. Postgres's deadlock detector steps in and terminates one of the problem transactions. If the PGD writer process is terminated, it retries and generally succeeds. - -All these issues are transient and generally require no administrator action. If a writer process is stuck for a long time behind a lock on an idle user session, the administrator can terminate the user session to get replication flowing again. However, this is no different from a user holding a long lock that impacts another user session. - -Use of the [log_lock_waits](https://www.postgresql.org/docs/current/runtime-config-logging.html#GUC-LOG-LOCK-WAITS) facility in Postgres can help identify locking related replay stalls. - -### Divergent conflicts - -Divergent conflicts arise when data that should be the same on different nodes differs unexpectedly. Divergent conflicts shouldn't occur, but not all such conflicts can be reliably prevented at the time of writing. - -Changing the `PRIMARY KEY` of a row can lead to a divergent conflict if another node changes the key of the same row before all nodes have replayed the change. Avoid changing primary keys, or change them only on one designated node. - -Divergent conflicts involving row data generally require administrator action to manually adjust the data on one of the nodes to be consistent with the other one. Such conflicts don't arise so long as you use PGD as documented and avoid settings or functions marked as unsafe. - -The administrator must manually resolve such conflicts. You might need to use the advanced options such as `bdr.ddl_replication` and `bdr.ddl_locking` depending on the nature of the conflict. However, careless use of these options can make things much worse and create a conflict that generic instructions can't address. - -### TOAST support details - -Postgres uses out-of-line storage for larger columns called [TOAST](https://www.postgresql.org/docs/current/storage-toast.html). - -The TOAST values handling in logical decoding (which PGD is built on top of) and logical replication is different from inline data stored as part of the main row in the table. - -The TOAST value is logged into the transaction log (WAL) only if the value changed. This can cause problems, especially when handling UPDATE conflicts, because an `UPDATE` statement that didn't change a value of a toasted column produces a row without that column. As mentioned in [INSERT/UPDATE conflicts](#insertupdate-conflicts), PGD reports an error if an `update_missing` conflict is resolved using `insert_or_error` and there are missing TOAST columns. - -However, more subtle issues than this one occur in case of concurrent workloads with asynchronous replication. (Eager transactions aren't affected.) Imagine, for example, the following workload on an EDB Postgres Distributed cluster with three nodes called A, B, and C: - -1. On node A: txn A1 does an UPDATE SET col1 = 'toast data...' and commits first. -2. On node B: txn B1 does UPDATE SET other_column = 'anything else'; and commits after A1. -3. On node C: the connection to node A lags behind. -4. On node C: txn B1 is applied first, it misses the TOASTed column in col1, but gets applied without conflict. -5. On node C: txn A1 conflicts (on update_origin_change) and is skipped. -6. Node C misses the toasted data from A1 forever. - -This scenario isn't usually a problem when using PGD. (It is when using either built-in logical replication or plain pglogical for multi-master.) PGD adds its own logging of TOAST columns when it detects a local `UPDATE` to a row that recently replicated a TOAST column modification and the local -`UPDATE` isn't modifying the TOAST. Thus PGD prevents any inconsistency for toasted data across different nodes. This situation causes increased WAL logging when updates occur on multiple nodes, that is, when origin changes for a tuple. Additional WAL overhead is zero if all updates are made from a single node, -as is normally the case with PGD AlwaysOn architecture. - -!!! Note - Running `VACUUM FULL` or `CLUSTER` on just the TOAST table without doing same on the main table removes metadata needed for the extra logging to work. This means that, for a short period after such a statement, the protection against these concurrency issues isn't present. - -!!! Warning - The additional WAL logging of TOAST is done using the `BEFORE UPDATE` trigger on standard Postgres. This trigger must be sorted alphabetically last based on trigger name among all `BEFORE UPDATE` triggers on the table. It's prefixed with `zzzz_bdr_` to make this easier, but make sure - you don't create any trigger with a name that sorts after it. Otherwise you won't have the protection against the concurrency issues. - -For the `insert_or_error` conflict resolution, the use of `REPLICA IDENTITY FULL` is still required. - -None of these problems associated with toasted columns affect tables with `REPLICA IDENTITY FULL`. This setting always logs a toasted value as part of the key since the whole row is considered to be part of the key. PGD can reconstruct the new row, filling the missing data from the key row. As a result, using `REPLICA IDENTITY FULL` can increase WAL size significantly. - -## Avoiding or tolerating conflicts - -In most cases, you can design the application to avoid or tolerate conflicts. - -Conflicts can happen only if things are happening at the same time on multiple nodes. The simplest way to avoid conflicts is to only ever write to one node or to only ever write to a specific row in a specific way from one specific node at a time. - -This avoidance happens naturally in many applications. For example, many consumer applications allow only the owning user to change data, such as changing the default billing address on an account. Such data changes seldom have update conflicts. - -You might make a change just before a node goes down, so the change seems to be lost. You might then make the same change again, leading to two updates on different nodes. When the down node comes back up, it tries to send the older change to other nodes. It's rejected because the last update of the data is kept. - -For `INSERT`/`INSERT` conflicts, use [global sequences](../sequences) to prevent this type of conflict. - -For applications that assign relationships between objects, such as a room-booking application, applying `update_if_newer` might not give an acceptable business outcome. That is, it isn't useful to confirm to two people separately that they have booked the same room. The simplest resolution is to use Eager Replication to ensure that only one booking succeeds. More complex ways might be possible depending on the application. For example, you can assign 100 seats to each node and allow those to be booked by a writer on that node. But if none are available locally, use a distributed locking scheme or Eager Replication after most seats are reserved. - -Another technique for ensuring certain types of updates occur only from one specific node is to route different types of transactions through different nodes. For example: - -- Receiving parcels on one node but delivering parcels using another node -- A service application where orders are input on one node and work is prepared on a second node and then served back to customers on another - -Frequently, the best course is to allow conflicts to occur and design the application to work with PGD's conflict resolution mechanisms to cope with the conflict. - -## Conflict detection - -PGD provides these mechanisms for conflict detection: - -- [Origin conflict detection](#origin-conflict-detection) (default) -- [Row version conflict detection](#row-version-conflict-detection) -- [Column-level conflict detection](column-level-conflicts) - -### Origin conflict detection - -Origin conflict detection uses and relies on commit timestamps as recorded on the node the transaction originates from. This requires clocks to be in sync to work correctly or to be within a tolerance of the fastest message between two nodes. If this isn't the case, conflict resolution tends to favor the node that's further ahead. You can manage clock skew between nodes using the parameters `bdr.maximum_clock_skew` and `bdr.maximum_clock_skew_action`. - -Row origins are available only if `track_commit_timestamp = on`. - -Conflicts are first detected based on whether the replication origin changed, so conflict triggers are called in situations that might not turn out to be conflicts. Hence, this mechanism isn't precise, since it can generate false-positive conflicts. - -Origin info is available only up to the point where a row is frozen. Updates arriving for a row after it was frozen don't raise a conflict so are applied in all cases. This is the normal case when adding a new node by `bdr_init_physical`, so raising conflicts causes many false-positive results in that case. - -A node that was offline that reconnects and begins sending data changes can cause divergent -errors if the newly arrived updates are older than the frozen rows that they update. Inserts and deletes aren't affected by this situation. - -We suggest that you don't leave down nodes for extended outages, as discussed in [Node restart and down node recovery](../node_management). - -On EDB Postgres Extended Server and EDB Postgres Advanced Server, PGD holds back the freezing of rows while a node is down. This mechanism handles this situation gracefully so you don't need to change parameter settings. - -On other variants of Postgres, you might need to manage this situation with some care. - -Freezing normally occurs when a row being vacuumed is older than `vacuum_freeze_min_age` xids from the current xid, which means that you need to configure suitably high values for these parameters: - -- `vacuum_freeze_min_age` -- `vacuum_freeze_table_age` -- `autovacuum_freeze_max_age` - -Choose values based on the transaction rate, giving a grace period of downtime before removing any conflict data from the database node. For example, when `vacuum_freeze_min_age` is set to 500 million, a node performing 1000 TPS can be down for just over 5.5 days before conflict data is removed. The CommitTS data structure takes on-disk space of 5 GB with that setting, so lower transaction rate systems can benefit from lower settings. - -Initially recommended settings are: - -```sql -# 1 billion = 10GB -autovacuum_freeze_max_age = 1000000000 - -vacuum_freeze_min_age = 500000000 - -# 90% of autovacuum_freeze_max_age -vacuum_freeze_table_age = 900000000 -``` - -Note that: - -- You can set `autovacuum_freeze_max_age` only at node start. -- You can set `vacuum_freeze_min_age`, so using a low value freezes rows early and can result in conflicts being ignored. You can also set `autovacuum_freeze_min_age` and `toast.autovacuum_freeze_min_age` for individual tables. -- Running the `CLUSTER` or `VACUUM FREEZE` commands also freezes rows early and can result in conflicts being ignored. - -### Row version conflict detection - -PGD provides the option to use row versioning and make conflict detection independent of the nodes' system clock. - -Row version conflict detection requires that you enable two things. If any of these steps aren't performed correctly then [origin conflict detection](#origin-conflict-detection) is used. - -- Enable `REPLICA IDENTITY FULL` on all tables that use row version conflict detection. - -- Enable row version tracking on the table by using `bdr.alter_table_conflict_detection`. This function adds a column with a name you specify and an `UPDATE` trigger that manages the new column value. The column is created as `INTEGER` type. - -Although the counter is incremented only on `UPDATE`, this technique allows conflict detection for both `UPDATE` and `DELETE`. - -This approach resembles Lamport timestamps and fully prevents the ABA problem for conflict detection. - -!!! Note - The row-level conflict resolution is still handled based on the [conflict resolution](#conflict-resolution) configuration even with row versioning. The way the row version is generated is useful only for detecting conflicts. Don't rely on it as authoritative information about which - version of row is newer. - -To determine the current conflict resolution strategy used for a specific table, refer to the column `conflict_detection` of the view `bdr.tables`. - -### `bdr.alter_table_conflict_detection` - -Allows the table owner to change how conflict detection works for a given table. - -#### Synopsis - -```sql -bdr.alter_table_conflict_detection(relation regclass, - method text, - column_name name DEFAULT NULL) -``` - -#### Parameters - -- `relation` — Name of the relation for which to set the new conflict detection method. -- `method` — The conflict detection method to use. -- `column_name` — The column to use for storing the column detection data. This can be skipped, in which case the column name is chosen based on the conflict detection method. The `row_origin` method doesn't require an extra column for metadata storage. - -The recognized methods for conflict detection are: - -- `row_origin` — Origin of the previous change made on the tuple (see [Origin conflict detection](#origin-conflict-detection)). This is the only method supported that doesn't require an extra column in the table. -- `row_version` — Row version column (see [Row version conflict detection](#row-version-conflict-detection)). -- `column_commit_timestamp` — Per-column commit timestamps (described in [CLCD](column-level-conflicts)). -- `column_modify_timestamp` — Per-column modification timestamp (described in [CLCD](column-level-conflicts)). - -#### Notes - -For more information about the difference between `column_commit_timestamp` and `column_modify_timestamp` conflict detection methods, see [Current versus commit timestamp](column-level-conflicts#current-versus-commit-timestamp). - -This function uses the same replication mechanism as `DDL` statements. This means the replication is affected by the [ddl filters](../repsets#ddl-replication-filtering) configuration. - -The function takes a `DML` global lock on the relation for which column-level conflict resolution is being enabled. - -This function is transactional. You can roll back the effects with the `ROLLBACK` of the transaction, and the changes are visible to the current transaction. - -Only the owner of the `relation` can execute the `bdr.alter_table_conflict_detection` function unless `bdr.backwards_compatibility` is set to 30618 or less. - -!!! Warning - When changing the conflict detection method from one that uses an extra column to store metadata, that column is dropped. - -!!! Warning - This function disables CAMO and gives a warning, as long as warnings aren't disabled with `bdr.camo_enable_client_warnings`. - -### List of conflict types - -PGD recognizes the following conflict types, which can be used as the `conflict_type` parameter: - -- `insert_exists` — An incoming insert conflicts with an existing row by way of a primary key or a unique key/index. -- `update_differing` — An incoming update's key row differs from a local row. This can happen only when using [row version conflict detection](#row-version-conflict-detection). -- `update_origin_change` — An incoming update is modifying a row that was last changed by a different node. -- `update_missing` — An incoming update is trying to modify a row that doesn't exist. -- `update_recently_deleted` — An incoming update is trying to modify a row that was recently deleted. -- `update_pkey_exists` — An incoming update has modified the `PRIMARY KEY` to a value that already exists on the node that's applying the change. -- `multiple_unique_conflicts` — The incoming row conflicts with multiple UNIQUE constraints/indexes in the target table. -- `delete_recently_updated` — An incoming delete with an older commit timestamp than the most recent update of the row on the current node or when using [row version conflict detection](#row-version-conflict-detection). -- `delete_missing` — An incoming delete is trying to remove a row that doesn't exist. -- `target_column_missing` — The target table is missing one or more columns present in the incoming row. -- `source_column_missing` — The incoming row is missing one or more columns that are present in the target table. -- `target_table_missing` — The target table is missing. -- `apply_error_ddl` — An error was thrown by Postgres when applying a replicated DDL command. - -## Conflict resolution - -Most conflicts can be resolved automatically. PGD defaults to a last-update-wins mechanism or, more accurately, the `update_if_newer` conflict resolver. This mechanism retains the most recently inserted or changed row of the two conflicting ones based on the same commit timestamps used for conflict detection. The behavior in certain corner-case scenarios depends on the settings used for `bdr.create_node_group` and -alternatively for `bdr.alter_node_group`. - -PGD lets you override the default behavior of conflict resolution by using the following function. - -### bdr.alter_node_set_conflict_resolver - -This function sets the behavior of conflict resolution on a given node. - -#### Synopsis - -```sql -bdr.alter_node_set_conflict_resolver(node_name text, - conflict_type text, - conflict_resolver text) -``` - -#### Parameters - -- `node_name` — Name of the node that's being changed. -- `conflict_type` — Conflict type for which to apply the setting (see [List of conflict types](#list-of-conflict-types)). -- `conflict_resolver` — Resolver to use for the given conflict type (see [List of conflict resolvers](#list-of-conflict-resolvers)). - -#### Notes - -Currently you can change only the local node. The function call isn't replicated. If you want to change settings on multiple nodes, you must run the function on each of them. - -The configuration change made by this function overrides any default behavior of conflict resolutions specified by `bdr.create_node_group` or `bdr.alter_node_group`. - -This function is transactional. You can roll back the changes, and they are visible to the current transaction. - -### List of conflict resolvers - -Several conflict resolvers are available in PGD, with differing coverages of the conflict types they can handle: - -- `error` — Throws error and stops replication. Can be used for any conflict type. -- `skip` — Skips processing the remote change and continues replication with the next change. Can be used for `insert_exists`, `update_differing`, `update_origin_change`, `update_missing`, `update_recently_deleted`, `update_pkey_exists`, `delete_recently_updated`, `delete_missing`, `target_table_missing`, `target_column_missing`, and `source_column_missing` conflict types. -- `skip_if_recently_dropped` — Skips the remote change if it's for a table that doesn't exist downstream because it was recently (within one day) dropped on the downstream. Throw an error otherwise. Can be used for the `target_table_missing` conflict type. `skip_if_recently_dropped` conflict -resolver can pose challenges if a table with the same name is re-created shortly after it's dropped. In that case, one of the nodes might see the DMLs on the re-created table before it sees the DDL to re-create the table. It then incorrectly skips the remote data, assuming that the table is recently dropped, and causes data loss. We hence recommend that you don't reuse the object names immediately after they're dropped along with this conflict resolver. -- `skip_transaction` — Skips the whole transaction that generated the conflict. Can be used for `apply_error_ddl` conflict. -- `update_if_newer` — Updates if the remote row was committed later (as determined by the wall clock of the originating node) than the conflicting local row. If the timestamps are same, the node id is used as a tie-breaker to ensure that same row is picked on all nodes (higher nodeid wins). Can be used for `insert_exists`, `update_differing`, `update_origin_change`, and `update_pkey_exists` conflict types. -- `update` — Always performs the replicated action. Can be used for `insert_exists` (turns the `INSERT` into `UPDATE`), `update_differing`, `update_origin_change`, `update_pkey_exists`, and `delete_recently_updated` (performs the delete). -- `insert_or_skip` — Tries to build a new row from available information sent by the origin and INSERT it. If there isn't enough information available to build a full row, skips the change. Can be used for `update_missing` and `update_recently_deleted` conflict types. -- `insert_or_error` — Tries to build new row from available information sent by origin and insert it. If there isn't enough information available to build full row, throws an error and stops the replication. Can be used for `update_missing` and `update_recently_deleted` conflict types. -- `ignore` — Ignores any missing target column and continues processing. Can be used for the `target_column_missing` conflict type. -- `ignore_if_null` — Ignores a missing target column if the extra column in the remote row contains a NULL value. Otherwise, throws an error and stops replication. Can be used for the `target_column_missing` conflict type. -- `use_default_value` — Fills the missing column value with the default (including NULL if that's the column default) and continues processing. Any error while processing the default or violation of constraints (that is, NULL default on NOT NULL column) stops replication. Can be used for the `source_column_missing` conflict type. - -The `insert_exists`, `update_differing`, `update_origin_change`, `update_missing`, `multiple_unique_conflicts`, `update_recently_deleted`, `update_pkey_exists`, `delete_recently_updated`, and `delete_missing` conflict types can also be resolved by user-defined logic using -[Conflict triggers](../striggers). - -This matrix helps you individuate the conflict types the conflict resolvers can handle. - -| | insert_exists | update_differing | update_origin_change | update_missing | update_recently_deleted | update_pkey_exists | delete_recently_updated | delete_missing | target_column_missing | source_column_missing | target_table_missing | multiple_unique_conflicts | -| :----------------------- | ------------- | ---------------- | -------------------- | -------------- | ----------------------- | ------------------ | ----------------------- | -------------- | --------------------- | --------------------- | -------------------- | ------------------------- | -| error | X | X | X | X | X | X | X | X | X | X | X | X | -| skip | X | X | X | X | X | X | X | X | X | X | X | X | -| skip_if_recently_dropped | | | | | | | | | | | X | | -| update_if_newer | X | X | X | | | X | | | | | | | -| update | X | X | X | | | X | X | | | | | X | -| insert_or_skip | | | | X | X | | | | | | | | -| insert_or_error | | | | X | X | | | | | | | | -| ignore | | | | | | | | | X | | | | -| ignore_if_null | | | | | | | | | X | | | | -| use_default_value | | | | | | | | | | X | | | -| conflict_trigger | X | X | X | X | X | X | X | X | | | | X | - -### Default conflict resolvers - -| Conflict type | Resolver | -| ------------------------- | ------------------------ | -| insert_exists | update_if_newer | -| update_differing | update_if_newer | -| update_origin_change | update_if_newer | -| update_missing | insert_or_skip | -| update_recently_deleted | skip | -| update_pkey_exists | update_if_newer | -| multiple_unique_conflicts | error | -| delete_recently_updated | skip | -| delete_missing | skip | -| target_column_missing | ignore_if_null | -| source_column_missing | use_default_value | -| target_table_missing (see note) | skip_if_recently_dropped | -| apply_error_ddl | error | - -
- -!!! note target_table_missing -This conflict type isn't detected on community Postgresql. If the target table is missing, it causes an error and halts replication. -EDB Postgres servers detect and handle missing target tables and can invoke the resolver. -!!! - -### List of conflict resolutions - -The conflict resolution represents the kind of resolution chosen by the conflict resolver and corresponds to the specific action that was taken to resolve the conflict. - -The following conflict resolutions are currently supported for the `conflict_resolution` parameter: - -- `apply_remote` — The remote (incoming) row was applied. -- `skip` — Processing of the row was skipped (no change was made locally). -- `merge` — A new row was created, merging information from remote and local row. -- `user` — User code (a conflict trigger) produced the row that was written to the target table. - -## Conflict logging - -To ease diagnosing and handling multi-master conflicts, PGD, by default, logs every conflict into the `bdr.conflict_history` table. You can change this behavior with more granularity with the following functions. - -### bdr.alter_node_set_log_config - -Set the conflict logging configuration for a node. - -#### Synopsis - -```sql -bdr.alter_node_set_log_config(node_name text, - log_to_file bool DEFAULT true, - log_to_table bool DEFAULT true, - conflict_type text[] DEFAULT NULL, - conflict_resolution text[] DEFAULT NULL) -``` - -#### Parameters - -- `node_name` — Name of the node that's being changed. -- `log_to_file` — Whether to log to the node log file. -- `log_to_table` — Whether to log to the `bdr.conflict_history` table. -- `conflict_type` — Conflict types to log. NULL (the default) means all. -- `conflict_resolution` — Conflict resolutions to log. NULL (the default) means all. - -#### Notes - -You can change only the local node. The function call isn't replicated. If you want to change settings on multiple nodes, you must run the function on each of them. - -This function is transactional. You can roll back the changes, and they're visible to the current transaction. - -#### Listing conflict logging configurations - -The view `bdr.node_log_config` shows all the logging configurations. It lists the name of the logging configuration, where it logs, and the conflict type and resolution it logs. - -#### Logging conflicts to a table - -If `log_to_table` is set to true, conflicts are logged to a table. The target table for conflict logging is `bdr.conflict_history`. - -This table is range partitioned on the column `local_time`. The table is managed by autopartition. By default, a new partition is created for every day, and conflicts of the last one month are maintained. After that, the old partitions are dropped automatically. Autopartition creates between 7 and 14 partitions in advance. bdr_superuser can change these defaults. - -Since conflicts generated for all tables managed by PGD are logged to this table, it's important to ensure that only legitimate users can read the conflicted data. PGD does this by defining ROW LEVEL SECURITY policies on the `bdr.conflict_history` table. Only owners of the tables are allowed to read conflicts on the respective tables. If the underlying tables have RLS policies defined, enabled, and enforced, then even owners can't read the conflicts. RLS policies created with the FORCE option also apply to owners of the table. In that case, some or all rows in the underlying table might not be readable even to the owner. So PGD also enforces a stricter policy on the conflict log table. - -The default role `bdr_read_all_conflicts` can be granted to users who need to see all conflict details logged to the `bdr.conflict_history` table without also granting them `bdr_superuser` role. - -The default role `bdr_read_all_stats` has access to a catalog view called `bdr.conflict_history_summary`, which doesn't contain user data, allowing monitoring of any conflicts logged. - -### Conflict reporting - -You can summarize conflicts logged to tables in reports. Reports allow application owners to identify, understand, and resolve conflicts and introduce application changes to prevent them. - -```sql -SELECT nspname, relname -, date_trunc('day', local_time) :: date AS date -, count(*) -FROM bdr.conflict_history -WHERE local_time > date_trunc('day', current_timestamp) -GROUP BY 1,2,3 -ORDER BY 1,2; - - nspname | relname | date | count ----------+---------+------------+------- - my_app | test | 2019-04-05 | 1 -(1 row) -``` - -## Data verification with LiveCompare - -LiveCompare is a utility program designed to compare any two databases to verify that they are identical. - -LiveCompare is included as part of the PGD stack and can be aimed at any pair of PGD nodes. By default, it compares all replicated tables and reports differences. LiveCompare also works with non-PGD data sources such as Postgres and Oracle. - -You can also use LiveCompare to continuously monitor incoming rows. You can stop and start it without losing context information, so you can run it at convenient times. - -LiveCompare allows concurrent checking of multiple tables. You can configure it to allow checking of a few tables or just a section of rows in a table. Checks are performed by first comparing whole -row hashes. If different, LiveCompare then compares whole rows. LiveCompare avoids overheads by comparing rows in useful-sized batches. - -If differences are found, they can be rechecked over time, allowing for the delays of eventual consistency. - -See the [LiveCompare](/livecompare/latest/) documentation for further details. diff --git a/product_docs/docs/pgd/5/reference/conflicts.mdx b/product_docs/docs/pgd/5/reference/conflicts.mdx new file mode 100644 index 00000000000..7bcc6c5b0cf --- /dev/null +++ b/product_docs/docs/pgd/5/reference/conflicts.mdx @@ -0,0 +1,16 @@ +--- +title: Conflict settings +indexdepth: 2 +--- + +## bdr.alter_table_conflict_detection + +Allows the table owner to change how conflict detection works for a given table. + +### Synopsis + +```sql +bdr.alter_table_conflict_detection(relation regclass, + method text, + column_name name DEFAULT NULL) +``` \ No newline at end of file diff --git a/product_docs/docs/pgd/5/reference/index.json b/product_docs/docs/pgd/5/reference/index.json index 0a844545bb9..d93f618187e 100644 --- a/product_docs/docs/pgd/5/reference/index.json +++ b/product_docs/docs/pgd/5/reference/index.json @@ -318,5 +318,6 @@ "bdrshow_writers": "/pgd/latest/reference/functions-internal#bdrshow_writers", "bdrtaskmgr_set_leader": "/pgd/latest/reference/functions-internal#bdrtaskmgr_set_leader", "bdrtaskmgr_get_last_completed_workitem": "/pgd/latest/reference/functions-internal#bdrtaskmgr_get_last_completed_workitem", - "bdrtaskmgr_work_queue_check_status": "/pgd/latest/reference/functions-internal#bdrtaskmgr_work_queue_check_status" + "bdrtaskmgr_work_queue_check_status": "/pgd/latest/reference/functions-internal#bdrtaskmgr_work_queue_check_status", + "bdralter_table_conflict_detection": "/pgd/latest/reference/conflicts#bdralter_table_conflict_detection" } \ No newline at end of file diff --git a/product_docs/docs/pgd/5/reference/index.mdx b/product_docs/docs/pgd/5/reference/index.mdx index 8ee2d810193..bd8033606f2 100644 --- a/product_docs/docs/pgd/5/reference/index.mdx +++ b/product_docs/docs/pgd/5/reference/index.mdx @@ -434,3 +434,7 @@ The reference section is a definitive listing of all functions, views, and comma * [`bdr.taskmgr_set_leader`](functions-internal#bdrtaskmgr_set_leader) * [`bdr.taskmgr_get_last_completed_workitem`](functions-internal#bdrtaskmgr_get_last_completed_workitem) * [`bdr.taskmgr_work_queue_check_status`](functions-internal#bdrtaskmgr_work_queue_check_status) + + +## [Conflict settings](conflicts) + * [bdr.alter_table_conflict_detection](conflicts#bdralter_table_conflict_detection) From b9a974a424c1a532825ace1b0590d93fe24aeec9 Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Tue, 25 Jun 2024 16:36:46 -0400 Subject: [PATCH 04/24] First command and list to reference. --- .../docs/pgd/5/reference/conflicts.mdx | 52 ++++++++++++++++++- product_docs/docs/pgd/5/reference/index.json | 3 +- product_docs/docs/pgd/5/reference/index.mdx | 1 + 3 files changed, 54 insertions(+), 2 deletions(-) diff --git a/product_docs/docs/pgd/5/reference/conflicts.mdx b/product_docs/docs/pgd/5/reference/conflicts.mdx index 7bcc6c5b0cf..ee8ec11cf63 100644 --- a/product_docs/docs/pgd/5/reference/conflicts.mdx +++ b/product_docs/docs/pgd/5/reference/conflicts.mdx @@ -13,4 +13,54 @@ Allows the table owner to change how conflict detection works for a given table. bdr.alter_table_conflict_detection(relation regclass, method text, column_name name DEFAULT NULL) -``` \ No newline at end of file +``` + +### Parameters + +- `relation` — Name of the relation for which to set the new conflict detection method. +- `method` — The conflict detection method to use. +- `column_name` — The column to use for storing the column detection data. This can be skipped, in which case the column name is chosen based on the conflict detection method. The `row_origin` method doesn't require an extra column for metadata storage. + +The recognized methods for conflict detection are: + +- `row_origin` — Origin of the previous change made on the tuple (see [Origin conflict detection](#origin-conflict-detection)). This is the only method supported that doesn't require an extra column in the table. +- `row_version` — Row version column (see [Row version conflict detection](#row-version-conflict-detection)). +- `column_commit_timestamp` — Per-column commit timestamps (described in [CLCD](column-level-conflicts)). +- `column_modify_timestamp` — Per-column modification timestamp (described in [CLCD](column-level-conflicts)). + +### Notes + +For more information about the difference between `column_commit_timestamp` and `column_modify_timestamp` conflict detection methods, see [Current versus commit timestamp](column-level-conflicts#current-versus-commit-timestamp). + +This function uses the same replication mechanism as `DDL` statements. This means the replication is affected by the [ddl filters](../repsets#ddl-replication-filtering) configuration. + +The function takes a `DML` global lock on the relation for which column-level conflict resolution is being enabled. + +This function is transactional. You can roll back the effects with the `ROLLBACK` of the transaction, and the changes are visible to the current transaction. + +Only the owner of the `relation` can execute the `bdr.alter_table_conflict_detection` function unless `bdr.backwards_compatibility` is set to 30618 or less. + +!!! Warning + When changing the conflict detection method from one that uses an extra column to store metadata, that column is dropped. + +!!! Warning + This function disables CAMO and gives a warning, as long as warnings aren't disabled with `bdr.camo_enable_client_warnings`. + +## List of conflict types + +PGD recognizes the following conflict types, which can be used as the `conflict_type` parameter: + +- `insert_exists` — An incoming insert conflicts with an existing row by way of a primary key or a unique key/index. +- `update_differing` — An incoming update's key row differs from a local row. This can happen only when using [row version conflict detection](#row-version-conflict-detection). +- `update_origin_change` — An incoming update is modifying a row that was last changed by a different node. +- `update_missing` — An incoming update is trying to modify a row that doesn't exist. +- `update_recently_deleted` — An incoming update is trying to modify a row that was recently deleted. +- `update_pkey_exists` — An incoming update has modified the `PRIMARY KEY` to a value that already exists on the node that's applying the change. +- `multiple_unique_conflicts` — The incoming row conflicts with multiple UNIQUE constraints/indexes in the target table. +- `delete_recently_updated` — An incoming delete with an older commit timestamp than the most recent update of the row on the current node or when using [row version conflict detection](#row-version-conflict-detection). +- `delete_missing` — An incoming delete is trying to remove a row that doesn't exist. +- `target_column_missing` — The target table is missing one or more columns present in the incoming row. +- `source_column_missing` — The incoming row is missing one or more columns that are present in the target table. +- `target_table_missing` — The target table is missing. +- `apply_error_ddl` — An error was thrown by Postgres when applying a replicated DDL command. + diff --git a/product_docs/docs/pgd/5/reference/index.json b/product_docs/docs/pgd/5/reference/index.json index d93f618187e..279ced2caff 100644 --- a/product_docs/docs/pgd/5/reference/index.json +++ b/product_docs/docs/pgd/5/reference/index.json @@ -319,5 +319,6 @@ "bdrtaskmgr_set_leader": "/pgd/latest/reference/functions-internal#bdrtaskmgr_set_leader", "bdrtaskmgr_get_last_completed_workitem": "/pgd/latest/reference/functions-internal#bdrtaskmgr_get_last_completed_workitem", "bdrtaskmgr_work_queue_check_status": "/pgd/latest/reference/functions-internal#bdrtaskmgr_work_queue_check_status", - "bdralter_table_conflict_detection": "/pgd/latest/reference/conflicts#bdralter_table_conflict_detection" + "bdralter_table_conflict_detection": "/pgd/latest/reference/conflicts#bdralter_table_conflict_detection", + "list-of-conflict-types": "/pgd/latest/reference/conflicts#list-of-conflict-types" } \ No newline at end of file diff --git a/product_docs/docs/pgd/5/reference/index.mdx b/product_docs/docs/pgd/5/reference/index.mdx index bd8033606f2..e87ed4d8aae 100644 --- a/product_docs/docs/pgd/5/reference/index.mdx +++ b/product_docs/docs/pgd/5/reference/index.mdx @@ -438,3 +438,4 @@ The reference section is a definitive listing of all functions, views, and comma ## [Conflict settings](conflicts) * [bdr.alter_table_conflict_detection](conflicts#bdralter_table_conflict_detection) + * [List of conflict types](conflicts#list-of-conflict-types) From ecce35e65e1267db01c817f9e3ecbe265866301b Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Tue, 25 Jun 2024 16:49:37 -0400 Subject: [PATCH 05/24] Main structure in place for new conflict reference page. --- .../docs/pgd/5/reference/conflicts.mdx | 173 ++++++++++++++++++ product_docs/docs/pgd/5/reference/index.json | 11 +- product_docs/docs/pgd/5/reference/index.mdx | 13 +- 3 files changed, 194 insertions(+), 3 deletions(-) diff --git a/product_docs/docs/pgd/5/reference/conflicts.mdx b/product_docs/docs/pgd/5/reference/conflicts.mdx index ee8ec11cf63..41998b84734 100644 --- a/product_docs/docs/pgd/5/reference/conflicts.mdx +++ b/product_docs/docs/pgd/5/reference/conflicts.mdx @@ -3,6 +3,8 @@ title: Conflict settings indexdepth: 2 --- +# Conflict detection + ## bdr.alter_table_conflict_detection Allows the table owner to change how conflict detection works for a given table. @@ -64,3 +66,174 @@ PGD recognizes the following conflict types, which can be used as the `conflict_ - `target_table_missing` — The target table is missing. - `apply_error_ddl` — An error was thrown by Postgres when applying a replicated DDL command. +# Conflict resolution + +Most conflicts can be resolved automatically. PGD defaults to a last-update-wins mechanism or, more accurately, the `update_if_newer` conflict resolver. This mechanism retains the most recently inserted or changed row of the two conflicting ones based on the same commit timestamps used for conflict detection. The behavior in certain corner-case scenarios depends on the settings used for `bdr.create_node_group` and +alternatively for `bdr.alter_node_group`. + +PGD lets you override the default behavior of conflict resolution by using the following function. + +## bdr.alter_node_set_conflict_resolver + +This function sets the behavior of conflict resolution on a given node. + +### Synopsis + +```sql +bdr.alter_node_set_conflict_resolver(node_name text, + conflict_type text, + conflict_resolver text) +``` + +### Parameters + +- `node_name` — Name of the node that's being changed. +- `conflict_type` — Conflict type for which to apply the setting (see [List of conflict types](#list-of-conflict-types)). +- `conflict_resolver` — Resolver to use for the given conflict type (see [List of conflict resolvers](#list-of-conflict-resolvers)). + +### Notes + +Currently you can change only the local node. The function call isn't replicated. If you want to change settings on multiple nodes, you must run the function on each of them. + +The configuration change made by this function overrides any default behavior of conflict resolutions specified by `bdr.create_node_group` or `bdr.alter_node_group`. + +This function is transactional. You can roll back the changes, and they are visible to the current transaction. + +## List of conflict resolvers + +Several conflict resolvers are available in PGD, with differing coverages of the conflict types they can handle: + +- `error` — Throws error and stops replication. Can be used for any conflict type. +- `skip` — Skips processing the remote change and continues replication with the next change. Can be used for `insert_exists`, `update_differing`, `update_origin_change`, `update_missing`, `update_recently_deleted`, `update_pkey_exists`, `delete_recently_updated`, `delete_missing`, `target_table_missing`, `target_column_missing`, and `source_column_missing` conflict types. +- `skip_if_recently_dropped` — Skips the remote change if it's for a table that doesn't exist downstream because it was recently (within one day) dropped on the downstream. Throw an error otherwise. Can be used for the `target_table_missing` conflict type. `skip_if_recently_dropped` conflict +resolver can pose challenges if a table with the same name is re-created shortly after it's dropped. In that case, one of the nodes might see the DMLs on the re-created table before it sees the DDL to re-create the table. It then incorrectly skips the remote data, assuming that the table is recently dropped, and causes data loss. We hence recommend that you don't reuse the object names immediately after they're dropped along with this conflict resolver. +- `skip_transaction` — Skips the whole transaction that generated the conflict. Can be used for `apply_error_ddl` conflict. +- `update_if_newer` — Updates if the remote row was committed later (as determined by the wall clock of the originating node) than the conflicting local row. If the timestamps are same, the node id is used as a tie-breaker to ensure that same row is picked on all nodes (higher nodeid wins). Can be used for `insert_exists`, `update_differing`, `update_origin_change`, and `update_pkey_exists` conflict types. +- `update` — Always performs the replicated action. Can be used for `insert_exists` (turns the `INSERT` into `UPDATE`), `update_differing`, `update_origin_change`, `update_pkey_exists`, and `delete_recently_updated` (performs the delete). +- `insert_or_skip` — Tries to build a new row from available information sent by the origin and INSERT it. If there isn't enough information available to build a full row, skips the change. Can be used for `update_missing` and `update_recently_deleted` conflict types. +- `insert_or_error` — Tries to build new row from available information sent by origin and insert it. If there isn't enough information available to build full row, throws an error and stops the replication. Can be used for `update_missing` and `update_recently_deleted` conflict types. +- `ignore` — Ignores any missing target column and continues processing. Can be used for the `target_column_missing` conflict type. +- `ignore_if_null` — Ignores a missing target column if the extra column in the remote row contains a NULL value. Otherwise, throws an error and stops replication. Can be used for the `target_column_missing` conflict type. +- `use_default_value` — Fills the missing column value with the default (including NULL if that's the column default) and continues processing. Any error while processing the default or violation of constraints (that is, NULL default on NOT NULL column) stops replication. Can be used for the `source_column_missing` conflict type. + +The `insert_exists`, `update_differing`, `update_origin_change`, `update_missing`, `multiple_unique_conflicts`, `update_recently_deleted`, `update_pkey_exists`, `delete_recently_updated`, and `delete_missing` conflict types can also be resolved by user-defined logic using +[Conflict triggers](../striggers). + +This matrix helps you individuate the conflict types the conflict resolvers can handle. + +| | insert_exists | update_differing | update_origin_change | update_missing | update_recently_deleted | update_pkey_exists | delete_recently_updated | delete_missing | target_column_missing | source_column_missing | target_table_missing | multiple_unique_conflicts | +| :----------------------- | ------------- | ---------------- | -------------------- | -------------- | ----------------------- | ------------------ | ----------------------- | -------------- | --------------------- | --------------------- | -------------------- | ------------------------- | +| error | X | X | X | X | X | X | X | X | X | X | X | X | +| skip | X | X | X | X | X | X | X | X | X | X | X | X | +| skip_if_recently_dropped | | | | | | | | | | | X | | +| update_if_newer | X | X | X | | | X | | | | | | | +| update | X | X | X | | | X | X | | | | | X | +| insert_or_skip | | | | X | X | | | | | | | | +| insert_or_error | | | | X | X | | | | | | | | +| ignore | | | | | | | | | X | | | | +| ignore_if_null | | | | | | | | | X | | | | +| use_default_value | | | | | | | | | | X | | | +| conflict_trigger | X | X | X | X | X | X | X | X | | | | X | + +## Default conflict resolvers + +| Conflict type | Resolver | +| ------------------------- | ------------------------ | +| insert_exists | update_if_newer | +| update_differing | update_if_newer | +| update_origin_change | update_if_newer | +| update_missing | insert_or_skip | +| update_recently_deleted | skip | +| update_pkey_exists | update_if_newer | +| multiple_unique_conflicts | error | +| delete_recently_updated | skip | +| delete_missing | skip | +| target_column_missing | ignore_if_null | +| source_column_missing | use_default_value | +| target_table_missing (see note) | skip_if_recently_dropped | +| apply_error_ddl | error | + +
+ +!!! note target_table_missing +This conflict type isn't detected on community Postgresql. If the target table is missing, it causes an error and halts replication. +EDB Postgres servers detect and handle missing target tables and can invoke the resolver. +!!! + +## List of conflict resolutions + +The conflict resolution represents the kind of resolution chosen by the conflict resolver and corresponds to the specific action that was taken to resolve the conflict. + +The following conflict resolutions are currently supported for the `conflict_resolution` parameter: + +- `apply_remote` — The remote (incoming) row was applied. +- `skip` — Processing of the row was skipped (no change was made locally). +- `merge` — A new row was created, merging information from remote and local row. +- `user` — User code (a conflict trigger) produced the row that was written to the target table. + +# Conflict logging + +To ease diagnosing and handling multi-master conflicts, PGD, by default, logs every conflict into the `bdr.conflict_history` table. You can change this behavior with more granularity with the following functions. + +## bdr.alter_node_set_log_config + +Set the conflict logging configuration for a node. + +### Synopsis + +```sql +bdr.alter_node_set_log_config(node_name text, + log_to_file bool DEFAULT true, + log_to_table bool DEFAULT true, + conflict_type text[] DEFAULT NULL, + conflict_resolution text[] DEFAULT NULL) +``` + +### Parameters + +- `node_name` — Name of the node that's being changed. +- `log_to_file` — Whether to log to the node log file. +- `log_to_table` — Whether to log to the `bdr.conflict_history` table. +- `conflict_type` — Conflict types to log. NULL (the default) means all. +- `conflict_resolution` — Conflict resolutions to log. NULL (the default) means all. + +### Notes + +You can change only the local node. The function call isn't replicated. If you want to change settings on multiple nodes, you must run the function on each of them. + +This function is transactional. You can roll back the changes, and they're visible to the current transaction. + +### Listing conflict logging configurations + +The view `bdr.node_log_config` shows all the logging configurations. It lists the name of the logging configuration, where it logs, and the conflict type and resolution it logs. + +### Logging conflicts to a table + +If `log_to_table` is set to true, conflicts are logged to a table. The target table for conflict logging is `bdr.conflict_history`. + +This table is range partitioned on the column `local_time`. The table is managed by autopartition. By default, a new partition is created for every day, and conflicts of the last one month are maintained. After that, the old partitions are dropped automatically. Autopartition creates between 7 and 14 partitions in advance. bdr_superuser can change these defaults. + +Since conflicts generated for all tables managed by PGD are logged to this table, it's important to ensure that only legitimate users can read the conflicted data. PGD does this by defining ROW LEVEL SECURITY policies on the `bdr.conflict_history` table. Only owners of the tables are allowed to read conflicts on the respective tables. If the underlying tables have RLS policies defined, enabled, and enforced, then even owners can't read the conflicts. RLS policies created with the FORCE option also apply to owners of the table. In that case, some or all rows in the underlying table might not be readable even to the owner. So PGD also enforces a stricter policy on the conflict log table. + +The default role `bdr_read_all_conflicts` can be granted to users who need to see all conflict details logged to the `bdr.conflict_history` table without also granting them `bdr_superuser` role. + +The default role `bdr_read_all_stats` has access to a catalog view called `bdr.conflict_history_summary`, which doesn't contain user data, allowing monitoring of any conflicts logged. + +## Conflict reporting + +You can summarize conflicts logged to tables in reports. Reports allow application owners to identify, understand, and resolve conflicts and introduce application changes to prevent them. + +```sql +SELECT nspname, relname +, date_trunc('day', local_time) :: date AS date +, count(*) +FROM bdr.conflict_history +WHERE local_time > date_trunc('day', current_timestamp) +GROUP BY 1,2,3 +ORDER BY 1,2; + + nspname | relname | date | count +---------+---------+------------+------- + my_app | test | 2019-04-05 | 1 +(1 row) +``` \ No newline at end of file diff --git a/product_docs/docs/pgd/5/reference/index.json b/product_docs/docs/pgd/5/reference/index.json index 279ced2caff..3131d9bd4ef 100644 --- a/product_docs/docs/pgd/5/reference/index.json +++ b/product_docs/docs/pgd/5/reference/index.json @@ -319,6 +319,15 @@ "bdrtaskmgr_set_leader": "/pgd/latest/reference/functions-internal#bdrtaskmgr_set_leader", "bdrtaskmgr_get_last_completed_workitem": "/pgd/latest/reference/functions-internal#bdrtaskmgr_get_last_completed_workitem", "bdrtaskmgr_work_queue_check_status": "/pgd/latest/reference/functions-internal#bdrtaskmgr_work_queue_check_status", + "conflict-detection": "/pgd/latest/reference/conflicts#conflict-detection", "bdralter_table_conflict_detection": "/pgd/latest/reference/conflicts#bdralter_table_conflict_detection", - "list-of-conflict-types": "/pgd/latest/reference/conflicts#list-of-conflict-types" + "list-of-conflict-types": "/pgd/latest/reference/conflicts#list-of-conflict-types", + "conflict-resolution": "/pgd/latest/reference/conflicts#conflict-resolution", + "bdralter_node_set_conflict_resolver": "/pgd/latest/reference/conflicts#bdralter_node_set_conflict_resolver", + "list-of-conflict-resolvers": "/pgd/latest/reference/conflicts#list-of-conflict-resolvers", + "default-conflict-resolvers": "/pgd/latest/reference/conflicts#default-conflict-resolvers", + "list-of-conflict-resolutions": "/pgd/latest/reference/conflicts#list-of-conflict-resolutions", + "conflict-logging": "/pgd/latest/reference/conflicts#conflict-logging", + "bdralter_node_set_log_config": "/pgd/latest/reference/conflicts#bdralter_node_set_log_config", + "conflict-reporting": "/pgd/latest/reference/conflicts#conflict-reporting" } \ No newline at end of file diff --git a/product_docs/docs/pgd/5/reference/index.mdx b/product_docs/docs/pgd/5/reference/index.mdx index e87ed4d8aae..a52198c69b7 100644 --- a/product_docs/docs/pgd/5/reference/index.mdx +++ b/product_docs/docs/pgd/5/reference/index.mdx @@ -437,5 +437,14 @@ The reference section is a definitive listing of all functions, views, and comma ## [Conflict settings](conflicts) - * [bdr.alter_table_conflict_detection](conflicts#bdralter_table_conflict_detection) - * [List of conflict types](conflicts#list-of-conflict-types) + * [Conflict detection](conflicts#conflict-detection) + * [bdr.alter_table_conflict_detection](conflicts#bdralter_table_conflict_detection) + * [List of conflict types](conflicts#list-of-conflict-types) + * [Conflict resolution](conflicts#conflict-resolution) + * [bdr.alter_node_set_conflict_resolver](conflicts#bdralter_node_set_conflict_resolver) + * [List of conflict resolvers](conflicts#list-of-conflict-resolvers) + * [Default conflict resolvers](conflicts#default-conflict-resolvers) + * [List of conflict resolutions](conflicts#list-of-conflict-resolutions) + * [Conflict logging](conflicts#conflict-logging) + * [bdr.alter_node_set_log_config](conflicts#bdralter_node_set_log_config) + * [Conflict reporting](conflicts#conflict-reporting) From 968b02a1c59715d9616b1369fb3c735a74087d0e Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Wed, 26 Jun 2024 13:06:42 -0400 Subject: [PATCH 06/24] Extracted old reference material from main Consistency pages so it is just in reference section now. --- .../conflicts/03_conflict_detection.mdx | 61 +---------- .../conflicts/04_conflict_resolution.mdx | 101 +----------------- .../conflicts/05_conflict_logging.mdx | 46 +------- .../docs/pgd/5/reference/conflicts.mdx | 19 +--- product_docs/docs/pgd/5/reference/index.json | 3 +- product_docs/docs/pgd/5/reference/index.mdx | 1 - 6 files changed, 5 insertions(+), 226 deletions(-) diff --git a/product_docs/docs/pgd/5/consistency/conflicts/03_conflict_detection.mdx b/product_docs/docs/pgd/5/consistency/conflicts/03_conflict_detection.mdx index b7198adf61d..cdf6dbaf6e2 100644 --- a/product_docs/docs/pgd/5/consistency/conflicts/03_conflict_detection.mdx +++ b/product_docs/docs/pgd/5/consistency/conflicts/03_conflict_detection.mdx @@ -77,63 +77,4 @@ This approach resembles Lamport timestamps and fully prevents the ABA problem fo To determine the current conflict resolution strategy used for a specific table, refer to the column `conflict_detection` of the view `bdr.tables`. -## bdr.alter_table_conflict_detection - -Allows the table owner to change how conflict detection works for a given table. - -### Synopsis - -```sql -bdr.alter_table_conflict_detection(relation regclass, - method text, - column_name name DEFAULT NULL) -``` - -### Parameters - -- `relation` — Name of the relation for which to set the new conflict detection method. -- `method` — The conflict detection method to use. -- `column_name` — The column to use for storing the column detection data. This can be skipped, in which case the column name is chosen based on the conflict detection method. The `row_origin` method doesn't require an extra column for metadata storage. - -The recognized methods for conflict detection are: - -- `row_origin` — Origin of the previous change made on the tuple (see [Origin conflict detection](#origin-conflict-detection)). This is the only method supported that doesn't require an extra column in the table. -- `row_version` — Row version column (see [Row version conflict detection](#row-version-conflict-detection)). -- `column_commit_timestamp` — Per-column commit timestamps (described in [CLCD](column-level-conflicts)). -- `column_modify_timestamp` — Per-column modification timestamp (described in [CLCD](column-level-conflicts)). - -### Notes - -For more information about the difference between `column_commit_timestamp` and `column_modify_timestamp` conflict detection methods, see [Current versus commit timestamp](column-level-conflicts#current-versus-commit-timestamp). - -This function uses the same replication mechanism as `DDL` statements. This means the replication is affected by the [ddl filters](../repsets#ddl-replication-filtering) configuration. - -The function takes a `DML` global lock on the relation for which column-level conflict resolution is being enabled. - -This function is transactional. You can roll back the effects with the `ROLLBACK` of the transaction, and the changes are visible to the current transaction. - -Only the owner of the `relation` can execute the `bdr.alter_table_conflict_detection` function unless `bdr.backwards_compatibility` is set to 30618 or less. - -!!! Warning - When changing the conflict detection method from one that uses an extra column to store metadata, that column is dropped. - -!!! Warning - This function disables CAMO and gives a warning, as long as warnings aren't disabled with `bdr.camo_enable_client_warnings`. - -## List of conflict types - -PGD recognizes the following conflict types, which can be used as the `conflict_type` parameter: - -- `insert_exists` — An incoming insert conflicts with an existing row by way of a primary key or a unique key/index. -- `update_differing` — An incoming update's key row differs from a local row. This can happen only when using [row version conflict detection](#row-version-conflict-detection). -- `update_origin_change` — An incoming update is modifying a row that was last changed by a different node. -- `update_missing` — An incoming update is trying to modify a row that doesn't exist. -- `update_recently_deleted` — An incoming update is trying to modify a row that was recently deleted. -- `update_pkey_exists` — An incoming update has modified the `PRIMARY KEY` to a value that already exists on the node that's applying the change. -- `multiple_unique_conflicts` — The incoming row conflicts with multiple UNIQUE constraints/indexes in the target table. -- `delete_recently_updated` — An incoming delete with an older commit timestamp than the most recent update of the row on the current node or when using [row version conflict detection](#row-version-conflict-detection). -- `delete_missing` — An incoming delete is trying to remove a row that doesn't exist. -- `target_column_missing` — The target table is missing one or more columns present in the incoming row. -- `source_column_missing` — The incoming row is missing one or more columns that are present in the target table. -- `target_table_missing` — The target table is missing. -- `apply_error_ddl` — An error was thrown by Postgres when applying a replicated DDL command. \ No newline at end of file +To change the current conflict detection strategy, use [brd.alter_table_conflict_detection](../../reference/conflicts/#brd.alter_table_conflict_detection). \ No newline at end of file diff --git a/product_docs/docs/pgd/5/consistency/conflicts/04_conflict_resolution.mdx b/product_docs/docs/pgd/5/consistency/conflicts/04_conflict_resolution.mdx index ec866eaabf8..5820c6b4b0f 100644 --- a/product_docs/docs/pgd/5/consistency/conflicts/04_conflict_resolution.mdx +++ b/product_docs/docs/pgd/5/consistency/conflicts/04_conflict_resolution.mdx @@ -7,103 +7,4 @@ deepToC: true Most conflicts can be resolved automatically. PGD defaults to a last-update-wins mechanism or, more accurately, the `update_if_newer` conflict resolver. This mechanism retains the most recently inserted or changed row of the two conflicting ones based on the same commit timestamps used for conflict detection. The behavior in certain corner-case scenarios depends on the settings used for `bdr.create_node_group` and alternatively for `bdr.alter_node_group`. -PGD lets you override the default behavior of conflict resolution by using the following function. - -## bdr.alter_node_set_conflict_resolver - -This function sets the behavior of conflict resolution on a given node. - -### Synopsis - -```sql -bdr.alter_node_set_conflict_resolver(node_name text, - conflict_type text, - conflict_resolver text) -``` - -### Parameters - -- `node_name` — Name of the node that's being changed. -- `conflict_type` — Conflict type for which to apply the setting (see [List of conflict types](#list-of-conflict-types)). -- `conflict_resolver` — Resolver to use for the given conflict type (see [List of conflict resolvers](#list-of-conflict-resolvers)). - -### Notes - -Currently you can change only the local node. The function call isn't replicated. If you want to change settings on multiple nodes, you must run the function on each of them. - -The configuration change made by this function overrides any default behavior of conflict resolutions specified by `bdr.create_node_group` or `bdr.alter_node_group`. - -This function is transactional. You can roll back the changes, and they are visible to the current transaction. - -## List of conflict resolvers - -Several conflict resolvers are available in PGD, with differing coverages of the conflict types they can handle: - -- `error` — Throws error and stops replication. Can be used for any conflict type. -- `skip` — Skips processing the remote change and continues replication with the next change. Can be used for `insert_exists`, `update_differing`, `update_origin_change`, `update_missing`, `update_recently_deleted`, `update_pkey_exists`, `delete_recently_updated`, `delete_missing`, `target_table_missing`, `target_column_missing`, and `source_column_missing` conflict types. -- `skip_if_recently_dropped` — Skips the remote change if it's for a table that doesn't exist downstream because it was recently (within one day) dropped on the downstream. Throw an error otherwise. Can be used for the `target_table_missing` conflict type. `skip_if_recently_dropped` conflict -resolver can pose challenges if a table with the same name is re-created shortly after it's dropped. In that case, one of the nodes might see the DMLs on the re-created table before it sees the DDL to re-create the table. It then incorrectly skips the remote data, assuming that the table is recently dropped, and causes data loss. We hence recommend that you don't reuse the object names immediately after they're dropped along with this conflict resolver. -- `skip_transaction` — Skips the whole transaction that generated the conflict. Can be used for `apply_error_ddl` conflict. -- `update_if_newer` — Updates if the remote row was committed later (as determined by the wall clock of the originating node) than the conflicting local row. If the timestamps are same, the node id is used as a tie-breaker to ensure that same row is picked on all nodes (higher nodeid wins). Can be used for `insert_exists`, `update_differing`, `update_origin_change`, and `update_pkey_exists` conflict types. -- `update` — Always performs the replicated action. Can be used for `insert_exists` (turns the `INSERT` into `UPDATE`), `update_differing`, `update_origin_change`, `update_pkey_exists`, and `delete_recently_updated` (performs the delete). -- `insert_or_skip` — Tries to build a new row from available information sent by the origin and INSERT it. If there isn't enough information available to build a full row, skips the change. Can be used for `update_missing` and `update_recently_deleted` conflict types. -- `insert_or_error` — Tries to build new row from available information sent by origin and insert it. If there isn't enough information available to build full row, throws an error and stops the replication. Can be used for `update_missing` and `update_recently_deleted` conflict types. -- `ignore` — Ignores any missing target column and continues processing. Can be used for the `target_column_missing` conflict type. -- `ignore_if_null` — Ignores a missing target column if the extra column in the remote row contains a NULL value. Otherwise, throws an error and stops replication. Can be used for the `target_column_missing` conflict type. -- `use_default_value` — Fills the missing column value with the default (including NULL if that's the column default) and continues processing. Any error while processing the default or violation of constraints (that is, NULL default on NOT NULL column) stops replication. Can be used for the `source_column_missing` conflict type. - -The `insert_exists`, `update_differing`, `update_origin_change`, `update_missing`, `multiple_unique_conflicts`, `update_recently_deleted`, `update_pkey_exists`, `delete_recently_updated`, and `delete_missing` conflict types can also be resolved by user-defined logic using -[Conflict triggers](../striggers). - -This matrix helps you individuate the conflict types the conflict resolvers can handle. - -| | insert_exists | update_differing | update_origin_change | update_missing | update_recently_deleted | update_pkey_exists | delete_recently_updated | delete_missing | target_column_missing | source_column_missing | target_table_missing | multiple_unique_conflicts | -| :----------------------- | ------------- | ---------------- | -------------------- | -------------- | ----------------------- | ------------------ | ----------------------- | -------------- | --------------------- | --------------------- | -------------------- | ------------------------- | -| error | X | X | X | X | X | X | X | X | X | X | X | X | -| skip | X | X | X | X | X | X | X | X | X | X | X | X | -| skip_if_recently_dropped | | | | | | | | | | | X | | -| update_if_newer | X | X | X | | | X | | | | | | | -| update | X | X | X | | | X | X | | | | | X | -| insert_or_skip | | | | X | X | | | | | | | | -| insert_or_error | | | | X | X | | | | | | | | -| ignore | | | | | | | | | X | | | | -| ignore_if_null | | | | | | | | | X | | | | -| use_default_value | | | | | | | | | | X | | | -| conflict_trigger | X | X | X | X | X | X | X | X | | | | X | - -## Default conflict resolvers - -| Conflict type | Resolver | -| ------------------------- | ------------------------ | -| insert_exists | update_if_newer | -| update_differing | update_if_newer | -| update_origin_change | update_if_newer | -| update_missing | insert_or_skip | -| update_recently_deleted | skip | -| update_pkey_exists | update_if_newer | -| multiple_unique_conflicts | error | -| delete_recently_updated | skip | -| delete_missing | skip | -| target_column_missing | ignore_if_null | -| source_column_missing | use_default_value | -| target_table_missing (see note) | skip_if_recently_dropped | -| apply_error_ddl | error | - -
- -!!! note target_table_missing -This conflict type isn't detected on community Postgresql. If the target table is missing, it causes an error and halts replication. -EDB Postgres servers detect and handle missing target tables and can invoke the resolver. -!!! - -## List of conflict resolutions - -The conflict resolution represents the kind of resolution chosen by the conflict resolver and corresponds to the specific action that was taken to resolve the conflict. - -The following conflict resolutions are currently supported for the `conflict_resolution` parameter: - -- `apply_remote` — The remote (incoming) row was applied. -- `skip` — Processing of the row was skipped (no change was made locally). -- `merge` — A new row was created, merging information from remote and local row. -- `user` — User code (a conflict trigger) produced the row that was written to the target table. - +PGD lets you override the default behavior of conflict resolution by using [bdr.alter_node_set_conflict_resolver](../../reference/conflicts/#bdr.alter_node_set_conflict_resolver). \ No newline at end of file diff --git a/product_docs/docs/pgd/5/consistency/conflicts/05_conflict_logging.mdx b/product_docs/docs/pgd/5/consistency/conflicts/05_conflict_logging.mdx index d9755312dcb..5b55718e621 100644 --- a/product_docs/docs/pgd/5/consistency/conflicts/05_conflict_logging.mdx +++ b/product_docs/docs/pgd/5/consistency/conflicts/05_conflict_logging.mdx @@ -4,51 +4,7 @@ Description: How PGD logs database conflicts. deepToC: true --- -To ease diagnosing and handling multi-master conflicts, PGD, by default, logs every conflict into the `bdr.conflict_history` table. You can change this behavior with more granularity with the following functions. - -## bdr.alter_node_set_log_config - -Set the conflict logging configuration for a node. - -### Synopsis - -```sql -bdr.alter_node_set_log_config(node_name text, - log_to_file bool DEFAULT true, - log_to_table bool DEFAULT true, - conflict_type text[] DEFAULT NULL, - conflict_resolution text[] DEFAULT NULL) -``` - -### Parameters - -- `node_name` — Name of the node that's being changed. -- `log_to_file` — Whether to log to the node log file. -- `log_to_table` — Whether to log to the `bdr.conflict_history` table. -- `conflict_type` — Conflict types to log. NULL (the default) means all. -- `conflict_resolution` — Conflict resolutions to log. NULL (the default) means all. - -### Notes - -You can change only the local node. The function call isn't replicated. If you want to change settings on multiple nodes, you must run the function on each of them. - -This function is transactional. You can roll back the changes, and they're visible to the current transaction. - -### Listing conflict logging configurations - -The view `bdr.node_log_config` shows all the logging configurations. It lists the name of the logging configuration, where it logs, and the conflict type and resolution it logs. - -### Logging conflicts to a table - -If `log_to_table` is set to true, conflicts are logged to a table. The target table for conflict logging is `bdr.conflict_history`. - -This table is range partitioned on the column `local_time`. The table is managed by autopartition. By default, a new partition is created for every day, and conflicts of the last one month are maintained. After that, the old partitions are dropped automatically. Autopartition creates between 7 and 14 partitions in advance. bdr_superuser can change these defaults. - -Since conflicts generated for all tables managed by PGD are logged to this table, it's important to ensure that only legitimate users can read the conflicted data. PGD does this by defining ROW LEVEL SECURITY policies on the `bdr.conflict_history` table. Only owners of the tables are allowed to read conflicts on the respective tables. If the underlying tables have RLS policies defined, enabled, and enforced, then even owners can't read the conflicts. RLS policies created with the FORCE option also apply to owners of the table. In that case, some or all rows in the underlying table might not be readable even to the owner. So PGD also enforces a stricter policy on the conflict log table. - -The default role `bdr_read_all_conflicts` can be granted to users who need to see all conflict details logged to the `bdr.conflict_history` table without also granting them `bdr_superuser` role. - -The default role `bdr_read_all_stats` has access to a catalog view called `bdr.conflict_history_summary`, which doesn't contain user data, allowing monitoring of any conflicts logged. +To ease diagnosing and handling multi-master conflicts, PGD, by default, logs every conflict into the `bdr.conflict_history` table. You can change this behavior with more granularity using [bdr.alter_node_set_log_config](../../reference/conflicts/bdr.alter_node_set_log_config). ## Conflict reporting diff --git a/product_docs/docs/pgd/5/reference/conflicts.mdx b/product_docs/docs/pgd/5/reference/conflicts.mdx index 41998b84734..4780117513d 100644 --- a/product_docs/docs/pgd/5/reference/conflicts.mdx +++ b/product_docs/docs/pgd/5/reference/conflicts.mdx @@ -1,6 +1,7 @@ --- title: Conflict settings indexdepth: 2 +deepToC: true --- # Conflict detection @@ -218,22 +219,4 @@ Since conflicts generated for all tables managed by PGD are logged to this table The default role `bdr_read_all_conflicts` can be granted to users who need to see all conflict details logged to the `bdr.conflict_history` table without also granting them `bdr_superuser` role. The default role `bdr_read_all_stats` has access to a catalog view called `bdr.conflict_history_summary`, which doesn't contain user data, allowing monitoring of any conflicts logged. - -## Conflict reporting - -You can summarize conflicts logged to tables in reports. Reports allow application owners to identify, understand, and resolve conflicts and introduce application changes to prevent them. - -```sql -SELECT nspname, relname -, date_trunc('day', local_time) :: date AS date -, count(*) -FROM bdr.conflict_history -WHERE local_time > date_trunc('day', current_timestamp) -GROUP BY 1,2,3 -ORDER BY 1,2; - - nspname | relname | date | count ----------+---------+------------+------- - my_app | test | 2019-04-05 | 1 -(1 row) ``` \ No newline at end of file diff --git a/product_docs/docs/pgd/5/reference/index.json b/product_docs/docs/pgd/5/reference/index.json index 3131d9bd4ef..f6fd0bd3056 100644 --- a/product_docs/docs/pgd/5/reference/index.json +++ b/product_docs/docs/pgd/5/reference/index.json @@ -328,6 +328,5 @@ "default-conflict-resolvers": "/pgd/latest/reference/conflicts#default-conflict-resolvers", "list-of-conflict-resolutions": "/pgd/latest/reference/conflicts#list-of-conflict-resolutions", "conflict-logging": "/pgd/latest/reference/conflicts#conflict-logging", - "bdralter_node_set_log_config": "/pgd/latest/reference/conflicts#bdralter_node_set_log_config", - "conflict-reporting": "/pgd/latest/reference/conflicts#conflict-reporting" + "bdralter_node_set_log_config": "/pgd/latest/reference/conflicts#bdralter_node_set_log_config" } \ No newline at end of file diff --git a/product_docs/docs/pgd/5/reference/index.mdx b/product_docs/docs/pgd/5/reference/index.mdx index a52198c69b7..b41777d2f4c 100644 --- a/product_docs/docs/pgd/5/reference/index.mdx +++ b/product_docs/docs/pgd/5/reference/index.mdx @@ -447,4 +447,3 @@ The reference section is a definitive listing of all functions, views, and comma * [List of conflict resolutions](conflicts#list-of-conflict-resolutions) * [Conflict logging](conflicts#conflict-logging) * [bdr.alter_node_set_log_config](conflicts#bdralter_node_set_log_config) - * [Conflict reporting](conflicts#conflict-reporting) From 546ec8319689dc4b72f33473c7bf14c535c79784 Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Wed, 26 Jun 2024 16:14:21 -0400 Subject: [PATCH 07/24] Fixed links. --- .../docs/pgd/5/consistency/conflicts/03_conflict_detection.mdx | 2 +- .../docs/pgd/5/consistency/conflicts/04_conflict_resolution.mdx | 2 +- .../docs/pgd/5/consistency/conflicts/05_conflict_logging.mdx | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/product_docs/docs/pgd/5/consistency/conflicts/03_conflict_detection.mdx b/product_docs/docs/pgd/5/consistency/conflicts/03_conflict_detection.mdx index cdf6dbaf6e2..61530bd82f3 100644 --- a/product_docs/docs/pgd/5/consistency/conflicts/03_conflict_detection.mdx +++ b/product_docs/docs/pgd/5/consistency/conflicts/03_conflict_detection.mdx @@ -77,4 +77,4 @@ This approach resembles Lamport timestamps and fully prevents the ABA problem fo To determine the current conflict resolution strategy used for a specific table, refer to the column `conflict_detection` of the view `bdr.tables`. -To change the current conflict detection strategy, use [brd.alter_table_conflict_detection](../../reference/conflicts/#brd.alter_table_conflict_detection). \ No newline at end of file +To change the current conflict detection strategy, use [bdr.alter_table_conflict_detection](../../reference/conflicts/#bdralter_table_conflict_detection). \ No newline at end of file diff --git a/product_docs/docs/pgd/5/consistency/conflicts/04_conflict_resolution.mdx b/product_docs/docs/pgd/5/consistency/conflicts/04_conflict_resolution.mdx index 5820c6b4b0f..c6fe8a58d04 100644 --- a/product_docs/docs/pgd/5/consistency/conflicts/04_conflict_resolution.mdx +++ b/product_docs/docs/pgd/5/consistency/conflicts/04_conflict_resolution.mdx @@ -7,4 +7,4 @@ deepToC: true Most conflicts can be resolved automatically. PGD defaults to a last-update-wins mechanism or, more accurately, the `update_if_newer` conflict resolver. This mechanism retains the most recently inserted or changed row of the two conflicting ones based on the same commit timestamps used for conflict detection. The behavior in certain corner-case scenarios depends on the settings used for `bdr.create_node_group` and alternatively for `bdr.alter_node_group`. -PGD lets you override the default behavior of conflict resolution by using [bdr.alter_node_set_conflict_resolver](../../reference/conflicts/#bdr.alter_node_set_conflict_resolver). \ No newline at end of file +PGD lets you override the default behavior of conflict resolution by using [bdr.alter_node_set_conflict_resolver](../../reference/conflicts/#bdralter_node_set_conflict_resolver). \ No newline at end of file diff --git a/product_docs/docs/pgd/5/consistency/conflicts/05_conflict_logging.mdx b/product_docs/docs/pgd/5/consistency/conflicts/05_conflict_logging.mdx index 5b55718e621..f5e548428c0 100644 --- a/product_docs/docs/pgd/5/consistency/conflicts/05_conflict_logging.mdx +++ b/product_docs/docs/pgd/5/consistency/conflicts/05_conflict_logging.mdx @@ -4,7 +4,7 @@ Description: How PGD logs database conflicts. deepToC: true --- -To ease diagnosing and handling multi-master conflicts, PGD, by default, logs every conflict into the `bdr.conflict_history` table. You can change this behavior with more granularity using [bdr.alter_node_set_log_config](../../reference/conflicts/bdr.alter_node_set_log_config). +To ease diagnosing and handling multi-master conflicts, PGD, by default, logs every conflict into the `bdr.conflict_history` table. You can change this behavior with more granularity using [bdr.alter_node_set_log_config](../../reference/conflicts/#bdralter_node_set_log_config). ## Conflict reporting From 26e5e85eb8bdc7b497df1c43c388dbc08c337505 Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Wed, 26 Jun 2024 16:15:51 -0400 Subject: [PATCH 08/24] Small change. --- .../pgd/5/consistency/conflicts/04_conflict_resolution.mdx | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/product_docs/docs/pgd/5/consistency/conflicts/04_conflict_resolution.mdx b/product_docs/docs/pgd/5/consistency/conflicts/04_conflict_resolution.mdx index c6fe8a58d04..c156aa021e1 100644 --- a/product_docs/docs/pgd/5/consistency/conflicts/04_conflict_resolution.mdx +++ b/product_docs/docs/pgd/5/consistency/conflicts/04_conflict_resolution.mdx @@ -4,7 +4,6 @@ Description: How PGD resolves database conflicts. deepToC: true --- -Most conflicts can be resolved automatically. PGD defaults to a last-update-wins mechanism or, more accurately, the `update_if_newer` conflict resolver. This mechanism retains the most recently inserted or changed row of the two conflicting ones based on the same commit timestamps used for conflict detection. The behavior in certain corner-case scenarios depends on the settings used for `bdr.create_node_group` and -alternatively for `bdr.alter_node_group`. +Most conflicts can be resolved automatically. PGD defaults to a last-update-wins mechanism or, more accurately, the `update_if_newer` conflict resolver. This mechanism retains the most recently inserted or changed row of the two conflicting ones based on the same commit timestamps used for conflict detection. The behavior in certain corner-case scenarios depends on the settings used for `bdr.create_node_group` and alternatively for `bdr.alter_node_group`. PGD lets you override the default behavior of conflict resolution by using [bdr.alter_node_set_conflict_resolver](../../reference/conflicts/#bdralter_node_set_conflict_resolver). \ No newline at end of file From db5817d4807b458dca22b86b45097bf6817cfc70 Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Thu, 27 Jun 2024 13:56:17 -0400 Subject: [PATCH 09/24] Update product_docs/docs/pgd/5/reference/conflicts.mdx Co-authored-by: Dj Walker-Morgan <126472455+djw-m@users.noreply.github.com> --- product_docs/docs/pgd/5/reference/conflicts.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/product_docs/docs/pgd/5/reference/conflicts.mdx b/product_docs/docs/pgd/5/reference/conflicts.mdx index 4780117513d..55ba4a93a53 100644 --- a/product_docs/docs/pgd/5/reference/conflicts.mdx +++ b/product_docs/docs/pgd/5/reference/conflicts.mdx @@ -74,7 +74,7 @@ alternatively for `bdr.alter_node_group`. PGD lets you override the default behavior of conflict resolution by using the following function. -## bdr.alter_node_set_conflict_resolver +## `bdr.alter_node_set_conflict_resolver` This function sets the behavior of conflict resolution on a given node. From 9814cfb0c194e43c2c8e231b1fad5710022b9f22 Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Thu, 27 Jun 2024 13:56:29 -0400 Subject: [PATCH 10/24] Update product_docs/docs/pgd/5/reference/conflicts.mdx Co-authored-by: Dj Walker-Morgan <126472455+djw-m@users.noreply.github.com> --- product_docs/docs/pgd/5/reference/conflicts.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/product_docs/docs/pgd/5/reference/conflicts.mdx b/product_docs/docs/pgd/5/reference/conflicts.mdx index 55ba4a93a53..30e15226967 100644 --- a/product_docs/docs/pgd/5/reference/conflicts.mdx +++ b/product_docs/docs/pgd/5/reference/conflicts.mdx @@ -6,7 +6,7 @@ deepToC: true # Conflict detection -## bdr.alter_table_conflict_detection +## `bdr.alter_table_conflict_detection` Allows the table owner to change how conflict detection works for a given table. From d100212f95c0d03eaff8b50f87ea3041d86c704c Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Thu, 27 Jun 2024 14:33:16 -0400 Subject: [PATCH 11/24] Fix bad links --- .../docs/pgd/5/reference/conflicts.mdx | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-) diff --git a/product_docs/docs/pgd/5/reference/conflicts.mdx b/product_docs/docs/pgd/5/reference/conflicts.mdx index 30e15226967..66155649ed7 100644 --- a/product_docs/docs/pgd/5/reference/conflicts.mdx +++ b/product_docs/docs/pgd/5/reference/conflicts.mdx @@ -26,16 +26,16 @@ bdr.alter_table_conflict_detection(relation regclass, The recognized methods for conflict detection are: -- `row_origin` — Origin of the previous change made on the tuple (see [Origin conflict detection](#origin-conflict-detection)). This is the only method supported that doesn't require an extra column in the table. -- `row_version` — Row version column (see [Row version conflict detection](#row-version-conflict-detection)). -- `column_commit_timestamp` — Per-column commit timestamps (described in [CLCD](column-level-conflicts)). -- `column_modify_timestamp` — Per-column modification timestamp (described in [CLCD](column-level-conflicts)). +- `row_origin` — Origin of the previous change made on the tuple (see [Origin conflict detection](../consistency/conflicts/03_conflict_detection/#origin-conflict-detection)). This is the only method supported that doesn't require an extra column in the table. +- `row_version` — Row version column (see [Row version conflict detection](../consistency/conflicts/03_conflict_detection/#row-version-conflict-detection)). +- `column_commit_timestamp` — Per-column commit timestamps (described in [CLCD](../consistency/column-level-conflicts)). +- `column_modify_timestamp` — Per-column modification timestamp (described in [CLCD](../consistency/column-level-conflicts)). ### Notes -For more information about the difference between `column_commit_timestamp` and `column_modify_timestamp` conflict detection methods, see [Current versus commit timestamp](column-level-conflicts#current-versus-commit-timestamp). +For more information about the difference between `column_commit_timestamp` and `column_modify_timestamp` conflict detection methods, see [Current versus commit timestamp](../consistency/column-level-conflicts/#current-versus-commit-timestamp). -This function uses the same replication mechanism as `DDL` statements. This means the replication is affected by the [ddl filters](../repsets#ddl-replication-filtering) configuration. +This function uses the same replication mechanism as `DDL` statements. This means the replication is affected by the [ddl filters](../repsets/#ddl-replication-filtering) configuration. The function takes a `DML` global lock on the relation for which column-level conflict resolution is being enabled. @@ -54,13 +54,13 @@ Only the owner of the `relation` can execute the `bdr.alter_table_conflict_detec PGD recognizes the following conflict types, which can be used as the `conflict_type` parameter: - `insert_exists` — An incoming insert conflicts with an existing row by way of a primary key or a unique key/index. -- `update_differing` — An incoming update's key row differs from a local row. This can happen only when using [row version conflict detection](#row-version-conflict-detection). +- `update_differing` — An incoming update's key row differs from a local row. This can happen only when using [row version conflict detection](../consistency/conflicts/03_conflict_detection/#row-version-conflict-detection). - `update_origin_change` — An incoming update is modifying a row that was last changed by a different node. - `update_missing` — An incoming update is trying to modify a row that doesn't exist. - `update_recently_deleted` — An incoming update is trying to modify a row that was recently deleted. - `update_pkey_exists` — An incoming update has modified the `PRIMARY KEY` to a value that already exists on the node that's applying the change. - `multiple_unique_conflicts` — The incoming row conflicts with multiple UNIQUE constraints/indexes in the target table. -- `delete_recently_updated` — An incoming delete with an older commit timestamp than the most recent update of the row on the current node or when using [row version conflict detection](#row-version-conflict-detection). +- `delete_recently_updated` — An incoming delete with an older commit timestamp than the most recent update of the row on the current node or when using [row version conflict detection](../consistency/conflicts/03_conflict_detection/#row-version-conflict-detection). - `delete_missing` — An incoming delete is trying to remove a row that doesn't exist. - `target_column_missing` — The target table is missing one or more columns present in the incoming row. - `source_column_missing` — The incoming row is missing one or more columns that are present in the target table. @@ -69,8 +69,7 @@ PGD recognizes the following conflict types, which can be used as the `conflict_ # Conflict resolution -Most conflicts can be resolved automatically. PGD defaults to a last-update-wins mechanism or, more accurately, the `update_if_newer` conflict resolver. This mechanism retains the most recently inserted or changed row of the two conflicting ones based on the same commit timestamps used for conflict detection. The behavior in certain corner-case scenarios depends on the settings used for `bdr.create_node_group` and -alternatively for `bdr.alter_node_group`. +Most conflicts can be resolved automatically. PGD defaults to a last-update-wins mechanism or, more accurately, the `update_if_newer` conflict resolver. This mechanism retains the most recently inserted or changed row of the two conflicting ones based on the same commit timestamps used for conflict detection. The behavior in certain corner-case scenarios depends on the settings used for `bdr.create_node_group` and alternatively for `bdr.alter_node_group`. PGD lets you override the default behavior of conflict resolution by using the following function. From 273852d806ba025c1015e06ceec3f00ade368711 Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Thu, 27 Jun 2024 14:47:03 -0400 Subject: [PATCH 12/24] Reference index.mdx changes. --- product_docs/docs/pgd/5/reference/index.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/product_docs/docs/pgd/5/reference/index.mdx b/product_docs/docs/pgd/5/reference/index.mdx index b41777d2f4c..6c067563fe6 100644 --- a/product_docs/docs/pgd/5/reference/index.mdx +++ b/product_docs/docs/pgd/5/reference/index.mdx @@ -438,10 +438,10 @@ The reference section is a definitive listing of all functions, views, and comma ## [Conflict settings](conflicts) * [Conflict detection](conflicts#conflict-detection) - * [bdr.alter_table_conflict_detection](conflicts#bdralter_table_conflict_detection) + * [`bdr.alter_table_conflict_detection`](conflicts#bdralter_table_conflict_detection) * [List of conflict types](conflicts#list-of-conflict-types) * [Conflict resolution](conflicts#conflict-resolution) - * [bdr.alter_node_set_conflict_resolver](conflicts#bdralter_node_set_conflict_resolver) + * [`bdr.alter_node_set_conflict_resolver`](conflicts#bdralter_node_set_conflict_resolver) * [List of conflict resolvers](conflicts#list-of-conflict-resolvers) * [Default conflict resolvers](conflicts#default-conflict-resolvers) * [List of conflict resolutions](conflicts#list-of-conflict-resolutions) From 004b5a9b789017c141e128744789bf22a892a9ce Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Tue, 2 Jul 2024 13:24:53 -0400 Subject: [PATCH 13/24] Integrated fixed links. --- .../5/consistency/conflicts/01_types_of_conflict.mdx | 12 ++++++------ .../conflicts/02_avoid_tolerate_conflicts.mdx | 2 +- .../consistency/conflicts/03_conflict_detection.mdx | 10 +++++----- .../docs/pgd/5/consistency/conflicts/index.mdx | 4 ++-- 4 files changed, 14 insertions(+), 14 deletions(-) diff --git a/product_docs/docs/pgd/5/consistency/conflicts/01_types_of_conflict.mdx b/product_docs/docs/pgd/5/consistency/conflicts/01_types_of_conflict.mdx index 66bb791ed45..a8fb57f1d84 100644 --- a/product_docs/docs/pgd/5/consistency/conflicts/01_types_of_conflict.mdx +++ b/product_docs/docs/pgd/5/consistency/conflicts/01_types_of_conflict.mdx @@ -25,11 +25,11 @@ The most common conflict, `INSERT`/`INSERT`, arises where `INSERT` operations on PGD handles this situation by retaining the most recently inserted tuple of the two according to the originating node's timestamps. (A user-defined conflict handler can override this behavior.) -This conflict generates the `insert_exists` conflict type, which is by default resolved by choosing the newer row, based on commit time, and keeping only that one (`update_if_newer` resolver). You can configure other resolvers. See [Conflict resolution](#conflict-resolution) for details. +This conflict generates the `insert_exists` conflict type, which is by default resolved by choosing the newer row, based on commit time, and keeping only that one (`update_if_newer` resolver). You can configure other resolvers. See [Conflict resolution](04_conflict_resolution) for details. To resolve this conflict type, you can also use column-level conflict resolution and user-defined conflict triggers. -You can effectively eliminate this type of conflict by using [global sequences](../sequences). +You can effectively eliminate this type of conflict by using [global sequences](../../sequences/#pgd-global-sequences). ### INSERT operations that violate multiple UNIQUE constraints @@ -46,7 +46,7 @@ You can also define a different behavior using a conflict trigger. Where two concurrent `UPDATE` operations on different nodes change the same tuple but not its `PRIMARY KEY`, an `UPDATE`/`UPDATE` conflict can occur on replay. -These can generate different conflict kinds based on the configuration and situation. If the table is configured with [row version conflict detection](#row-version-conflict-detection), then the original (key) row is compared with the local row. If they're different, the `update_differing` conflict is generated. When using [origin conflict detection](#origin-conflict-detection), the origin of the row is checked. (The origin is the node that the current local row came from.) If that changed, the `update_origin_change` conflict is generated. In all other cases, the `UPDATE` is normally applied without generating a conflict. +These can generate different conflict kinds based on the configuration and situation. If the table is configured with [row version conflict detection](03_conflict_detection/#row-version-conflict-detection), then the original (key) row is compared with the local row. If they're different, the `update_differing` conflict is generated. When using [origin conflict detection](03_conflict_detection/#origin-conflict-detection), the origin of the row is checked. (The origin is the node that the current local row came from.) If that changed, the `update_origin_change` conflict is generated. In all other cases, the `UPDATE` is normally applied without generating a conflict. Both of these conflicts are resolved the same way as `insert_exists`, described in [INSERT/INSERT conflicts](#insertinsert-conflicts). @@ -175,12 +175,12 @@ PGD supports deferred unique constraints. If a transaction can commit on the sou One node can update a row that another node deletes at ths same time. In this case an `UPDATE`/`DELETE` conflict can occur on replay. -If the deleted row is still detectable (the deleted row wasn't removed by `VACUUM`), the `update_recently_deleted` conflict is generated. By default, the `UPDATE` is skipped, but you can configure the resolution for this. See [Conflict resolution](#conflict-resolution) for details. +If the deleted row is still detectable (the deleted row wasn't removed by `VACUUM`), the `update_recently_deleted` conflict is generated. By default, the `UPDATE` is skipped, but you can configure the resolution for this. See [Conflict resolution](04_conflict_resolution) for details. The database can clean up the deleted row by the time the `UPDATE` is received in case the local node is lagging behind in replication. In this case, PGD can't differentiate between `UPDATE`/`DELETE` conflicts and [INSERT/UPDATE conflicts](#insertupdate-conflicts). It generates the `update_missing` conflict. Another type of conflicting `DELETE` and `UPDATE` is a `DELETE` that comes after the row was updated locally. In this situation, the outcome depends on the type of conflict detection used. When using the -default, [origin conflict detection](#origin-conflict-detection), no conflict is detected, leading to the `DELETE` being applied and the row removed. If you enable [row version conflict detection](#row-version-conflict-detection), a `delete_recently_updated` conflict is generated. The default resolution for a `delete_recently_updated` conflict is to `skip` the deletion. However, you can configure the resolution or a conflict trigger can be configured to handle it. +default, [origin conflict detection](03_conflict_detection/#origin-conflict-detection), no conflict is detected, leading to the `DELETE` being applied and the row removed. If you enable [row version conflict detection](03_conflict_detection/#row-version-conflict-detection), a `delete_recently_updated` conflict is generated. The default resolution for a `delete_recently_updated` conflict is to `skip` the deletion. However, you can configure the resolution or a conflict trigger can be configured to handle it. ### INSERT/UPDATE conflicts @@ -231,7 +231,7 @@ Using `insert_or_error` (or in some cases the `insert_or_skip` conflict resolver If these are problems, we recommend tuning freezing settings for a table or database so that they're correctly detected as `update_recently_deleted`. -Another alternative is to use [Eager Replication](eager) to prevent these conflicts. +Another alternative is to use [Eager Replication](../eager) to prevent these conflicts. `INSERT`/`DELETE` conflicts can also occur with three or more nodes. Such a conflict is identical to `INSERT`/`UPDATE` except with the `UPDATE` replaced by a `DELETE`. This can result in a `delete_missing` conflict. diff --git a/product_docs/docs/pgd/5/consistency/conflicts/02_avoid_tolerate_conflicts.mdx b/product_docs/docs/pgd/5/consistency/conflicts/02_avoid_tolerate_conflicts.mdx index e75ab220747..66b3f8fb484 100644 --- a/product_docs/docs/pgd/5/consistency/conflicts/02_avoid_tolerate_conflicts.mdx +++ b/product_docs/docs/pgd/5/consistency/conflicts/02_avoid_tolerate_conflicts.mdx @@ -12,7 +12,7 @@ This avoidance happens naturally in many applications. For example, many consume You might make a change just before a node goes down, so the change seems to be lost. You might then make the same change again, leading to two updates on different nodes. When the down node comes back up, it tries to send the older change to other nodes. It's rejected because the last update of the data is kept. -For `INSERT`/`INSERT` conflicts, use [global sequences](../sequences) to prevent this type of conflict. +For `INSERT`/`INSERT` conflicts, use [global sequences](../../sequences//#pgd-global-sequences) to prevent this type of conflict. For applications that assign relationships between objects, such as a room-booking application, applying `update_if_newer` might not give an acceptable business outcome. That is, it isn't useful to confirm to two people separately that they have booked the same room. The simplest resolution is to use Eager Replication to ensure that only one booking succeeds. More complex ways might be possible depending on the application. For example, you can assign 100 seats to each node and allow those to be booked by a writer on that node. But if none are available locally, use a distributed locking scheme or Eager Replication after most seats are reserved. diff --git a/product_docs/docs/pgd/5/consistency/conflicts/03_conflict_detection.mdx b/product_docs/docs/pgd/5/consistency/conflicts/03_conflict_detection.mdx index 61530bd82f3..d67924ba579 100644 --- a/product_docs/docs/pgd/5/consistency/conflicts/03_conflict_detection.mdx +++ b/product_docs/docs/pgd/5/consistency/conflicts/03_conflict_detection.mdx @@ -8,7 +8,7 @@ PGD provides these mechanisms for conflict detection: - [Origin conflict detection](#origin-conflict-detection) (default) - [Row version conflict detection](#row-version-conflict-detection) -- [Column-level conflict detection](column-level-conflicts) +- [Column-level conflict detection](../column-level-conflicts) ## Origin conflict detection @@ -23,7 +23,7 @@ Origin info is available only up to the point where a row is frozen. Updates arr A node that was offline that reconnects and begins sending data changes can cause divergent errors if the newly arrived updates are older than the frozen rows that they update. Inserts and deletes aren't affected by this situation. -We suggest that you don't leave down nodes for extended outages, as discussed in [Node restart and down node recovery](../node_management). +We suggest that you don't leave down nodes for extended outages, as discussed in [Node restart and down node recovery](../../node_management/node_recovery/). On EDB Postgres Extended Server and EDB Postgres Advanced Server, PGD holds back the freezing of rows while a node is down. This mechanism handles this situation gracefully so you don't need to change parameter settings. @@ -37,7 +37,7 @@ Freezing normally occurs when a row being vacuumed is older than `vacuum_freeze_ Choose values based on the transaction rate, giving a grace period of downtime before removing any conflict data from the database node. For example, when `vacuum_freeze_min_age` is set to 500 million, a node performing 1000 TPS can be down for just over 5.5 days before conflict data is removed. The CommitTS data structure takes on-disk space of 5 GB with that setting, so lower transaction rate systems can benefit from lower settings. -Initially recommended settings are: +Initially, recommended settings are: ```sql # 1 billion = 10GB @@ -72,9 +72,9 @@ Although the counter is incremented only on `UPDATE`, this technique allows conf This approach resembles Lamport timestamps and fully prevents the ABA problem for conflict detection. !!! Note - The row-level conflict resolution is still handled based on the [conflict resolution](#conflict-resolution) configuration even with row versioning. The way the row version is generated is useful only for detecting conflicts. Don't rely on it as authoritative information about which + The row-level conflict resolution is still handled based on the [conflict resolution](04_conflict_resolution) configuration even with row versioning. The way the row version is generated is useful only for detecting conflicts. Don't rely on it as authoritative information about which version of row is newer. -To determine the current conflict resolution strategy used for a specific table, refer to the column `conflict_detection` of the view `bdr.tables`. +To determine the current conflict detection strategy used for a specific table, refer to the column `conflict_detection` of the view `bdr.tables`. To change the current conflict detection strategy, use [bdr.alter_table_conflict_detection](../../reference/conflicts/#bdralter_table_conflict_detection). \ No newline at end of file diff --git a/product_docs/docs/pgd/5/consistency/conflicts/index.mdx b/product_docs/docs/pgd/5/consistency/conflicts/index.mdx index 96ee2966780..33a271c6bb1 100644 --- a/product_docs/docs/pgd/5/consistency/conflicts/index.mdx +++ b/product_docs/docs/pgd/5/consistency/conflicts/index.mdx @@ -11,9 +11,9 @@ PGD to provide the application a range of choices for how to resolve conflicts. By default, conflicts are resolved at the row level. When changes from two nodes conflict, PGD picks either the local or remote tuple and the discards the other. For example, the commit timestamps might be compared for the two conflicting changes and the newer one kept. This approach ensures that all nodes converge to the same result and establishes commit-order-like semantics on the whole cluster. -Conflict handling is configurable, as described in [Conflict resolution](#conflict-resolution). PGD can detect conflicts and handle them differently for each table using conflict triggers, described in [Stream triggers](../striggers). +Conflict handling is configurable, as described in [Conflict resolution](04_conflict_resolution). PGD can detect conflicts and handle them differently for each table using conflict triggers, described in [Stream triggers](../../striggers). -Column-level conflict detection and resolution is available with PGD, described in [CLCD](column-level-conflicts). +Column-level conflict detection and resolution is available with PGD, described in [CLCD](../column-level-conflicts). By default, all conflicts are logged to `bdr.conflict_history`. If conflicts are possible, then table owners must monitor for them and analyze how to avoid them or make plans to handle them regularly as an application task. The [LiveCompare](/livecompare/latest) tool is also available to scan regularly for divergence. From 3d5eefe141154aa497f3a5ba9a4eb44e3fb022b8 Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Tue, 2 Jul 2024 13:35:29 -0400 Subject: [PATCH 14/24] Added overview page and switched avoiding with types. --- .../pgd/5/consistency/conflicts/00_conflicts_overview.mdx | 6 ++++++ ...lerate_conflicts.mdx => 01_avoid_tolerate_conflicts.mdx} | 0 .../{01_types_of_conflict.mdx => 02_types_of_conflict.mdx} | 0 3 files changed, 6 insertions(+) create mode 100644 product_docs/docs/pgd/5/consistency/conflicts/00_conflicts_overview.mdx rename product_docs/docs/pgd/5/consistency/conflicts/{02_avoid_tolerate_conflicts.mdx => 01_avoid_tolerate_conflicts.mdx} (100%) rename product_docs/docs/pgd/5/consistency/conflicts/{01_types_of_conflict.mdx => 02_types_of_conflict.mdx} (100%) diff --git a/product_docs/docs/pgd/5/consistency/conflicts/00_conflicts_overview.mdx b/product_docs/docs/pgd/5/consistency/conflicts/00_conflicts_overview.mdx new file mode 100644 index 00000000000..8c7464083ab --- /dev/null +++ b/product_docs/docs/pgd/5/consistency/conflicts/00_conflicts_overview.mdx @@ -0,0 +1,6 @@ +--- +title: Overview +Description: Conflicts section overview. +deepToC: true +--- + diff --git a/product_docs/docs/pgd/5/consistency/conflicts/02_avoid_tolerate_conflicts.mdx b/product_docs/docs/pgd/5/consistency/conflicts/01_avoid_tolerate_conflicts.mdx similarity index 100% rename from product_docs/docs/pgd/5/consistency/conflicts/02_avoid_tolerate_conflicts.mdx rename to product_docs/docs/pgd/5/consistency/conflicts/01_avoid_tolerate_conflicts.mdx diff --git a/product_docs/docs/pgd/5/consistency/conflicts/01_types_of_conflict.mdx b/product_docs/docs/pgd/5/consistency/conflicts/02_types_of_conflict.mdx similarity index 100% rename from product_docs/docs/pgd/5/consistency/conflicts/01_types_of_conflict.mdx rename to product_docs/docs/pgd/5/consistency/conflicts/02_types_of_conflict.mdx From 4e2eb87a310c7b6eaab80ae98f78cadabaa01ec2 Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Tue, 2 Jul 2024 14:18:32 -0400 Subject: [PATCH 15/24] Added index content. --- .../conflicts/00_conflicts_overview.mdx | 28 +++++++++++++++++++ .../pgd/5/consistency/conflicts/index.mdx | 28 +++++-------------- 2 files changed, 35 insertions(+), 21 deletions(-) diff --git a/product_docs/docs/pgd/5/consistency/conflicts/00_conflicts_overview.mdx b/product_docs/docs/pgd/5/consistency/conflicts/00_conflicts_overview.mdx index 8c7464083ab..0ceba54841e 100644 --- a/product_docs/docs/pgd/5/consistency/conflicts/00_conflicts_overview.mdx +++ b/product_docs/docs/pgd/5/consistency/conflicts/00_conflicts_overview.mdx @@ -4,3 +4,31 @@ Description: Conflicts section overview. deepToC: true --- +EDB Postgres Distributed is an active/active or multi-master DBMS. If used asynchronously, writes to the same or related rows from multiple different nodes can result in data conflicts when using standard data types. + +Conflicts aren't errors. In most cases, they are events that PGD can detect and resolve as they occur. Resolving them depends on the nature of the application and the meaning of the data, so it's important for +PGD to provide the application a range of choices for how to resolve conflicts. + +By default, conflicts are resolved at the row level. When changes from two nodes conflict, PGD picks either the local or remote tuple and the discards the other. For example, the commit timestamps might be compared for the two conflicting changes and the newer one kept. This approach ensures that all nodes converge to the same result and establishes commit-order-like semantics on the whole cluster. + +Conflict handling is configurable, as described in [Conflict resolution](04_conflict_resolution). PGD can detect conflicts and handle them differently for each table using conflict triggers, described in [Stream triggers](../../striggers). + +Column-level conflict detection and resolution is available with PGD, described in [CLCD](../column-level-conflicts). + +By default, all conflicts are logged to `bdr.conflict_history`. If conflicts are possible, then table owners must monitor for them and analyze how to avoid them or make plans to handle them regularly as an application task. The [LiveCompare](/livecompare/latest) tool is also available to scan regularly for divergence. + +Some clustering systems use distributed lock mechanisms to prevent concurrent access to data. These can perform reasonably when servers are very close to each other but can't support geographically distributed applications where very low latency is critical for acceptable performance. + +Distributed locking is essentially a pessimistic approach. PGD advocates an optimistic approach, which is to avoid conflicts where possible but allow some types of conflicts to occur and resolve them when they arise. + +## How conflicts happen + +Inter-node conflicts arise as a result of sequences of events that can't happen if all the involved transactions happen concurrently on the same node. Because the nodes exchange changes only after the transactions commit, each transaction is individually valid on the node it committed on. It isn't +valid if applied on another node that did other conflicting work at the same time. + +Since PGD replication essentially replays the transaction on the other nodes, the replay operation can fail if there's a conflict between a transaction being applied and a transaction that was committed on the receiving node. + +Most conflicts can't happen when all transactions run on a single node because Postgres has inter-transaction communication mechanisms to prevent it. Examples of these mechanisms are `UNIQUE` indexes, `SEQUENCE` operations, row and relation locking, and `SERIALIZABLE` dependency tracking. All of these mechanisms are ways to communicate between ongoing transactions to prevent undesirable concurrency +issues. + +PGD doesn't have a distributed transaction manager or lock manager. That's part of why it performs well with latency and network partitions. As a result, transactions on different nodes execute entirely independently from each other when using the default, which is lazy replication. Less independence between nodes can avoid conflicts altogether, which is why PGD also offers Eager Replication for when this is important. \ No newline at end of file diff --git a/product_docs/docs/pgd/5/consistency/conflicts/index.mdx b/product_docs/docs/pgd/5/consistency/conflicts/index.mdx index 33a271c6bb1..51b6e97caa9 100644 --- a/product_docs/docs/pgd/5/consistency/conflicts/index.mdx +++ b/product_docs/docs/pgd/5/consistency/conflicts/index.mdx @@ -4,31 +4,17 @@ redirects: - /pgd/latest/bdr/conflicts/ --- -EDB Postgres Distributed is an active/active or multi-master DBMS. If used asynchronously, writes to the same or related rows from multiple different nodes can result in data conflicts when using standard data types. +- [Overview](00_conflicts_overview) introduces the idea of conflicts in PGD and explains how they can happen. -Conflicts aren't errors. In most cases, they are events that PGD can detect and resolve as they occur. Resolving them depends on the nature of the application and the meaning of the data, so it's important for -PGD to provide the application a range of choices for how to resolve conflicts. +- [Avoiding or tolerating conflicts](01_avoid_tolerate_conflicts) discusses strategies for avoiding or tolerating conflicts in PGD/ -By default, conflicts are resolved at the row level. When changes from two nodes conflict, PGD picks either the local or remote tuple and the discards the other. For example, the commit timestamps might be compared for the two conflicting changes and the newer one kept. This approach ensures that all nodes converge to the same result and establishes commit-order-like semantics on the whole cluster. +- [Types of conflicts](02_types_of_conflict) lists and discusses the various sorts of conflicts you might run across in PGD. -Conflict handling is configurable, as described in [Conflict resolution](04_conflict_resolution). PGD can detect conflicts and handle them differently for each table using conflict triggers, described in [Stream triggers](../../striggers). +- [Conflict detection](03_conflict_detection) introduces the mechanisms PGD provides for conflict detection. -Column-level conflict detection and resolution is available with PGD, described in [CLCD](../column-level-conflicts). +- [Conflict resolution](04_conflict_resolution) explains how PGD resolves conflicts and how you can change the default behavior. -By default, all conflicts are logged to `bdr.conflict_history`. If conflicts are possible, then table owners must monitor for them and analyze how to avoid them or make plans to handle them regularly as an application task. The [LiveCompare](/livecompare/latest) tool is also available to scan regularly for divergence. +- [Conflict logging](05_conflict_logging) points out where PGD keeps conflict logs and explains how you can perform conflict reporting. -Some clustering systems use distributed lock mechanisms to prevent concurrent access to data. These can perform reasonably when servers are very close to each other but can't support geographically distributed applications where very low latency is critical for acceptable performance. +- [Data verification with LiveCompare](06_live_compare) explains how LiveCompare can help keep data consistent by pointing out conflicts as they arise. -Distributed locking is essentially a pessimistic approach. PGD advocates an optimistic approach, which is to avoid conflicts where possible but allow some types of conflicts to occur and resolve them when they arise. - -## How conflicts happen - -Inter-node conflicts arise as a result of sequences of events that can't happen if all the involved transactions happen concurrently on the same node. Because the nodes exchange changes only after the transactions commit, each transaction is individually valid on the node it committed on. It isn't -valid if applied on another node that did other conflicting work at the same time. - -Since PGD replication essentially replays the transaction on the other nodes, the replay operation can fail if there's a conflict between a transaction being applied and a transaction that was committed on the receiving node. - -Most conflicts can't happen when all transactions run on a single node because Postgres has inter-transaction communication mechanisms to prevent it. Examples of these mechanisms are `UNIQUE` indexes, `SEQUENCE` operations, row and relation locking, and `SERIALIZABLE` dependency tracking. All of these mechanisms are ways to communicate between ongoing transactions to prevent undesirable concurrency -issues. - -PGD doesn't have a distributed transaction manager or lock manager. That's part of why it performs well with latency and network partitions. As a result, transactions on different nodes execute entirely independently from each other when using the default, which is lazy replication. Less independence between nodes can avoid conflicts altogether, which is why PGD also offers Eager Replication for when this is important. \ No newline at end of file From b246ef5446e6077d7ef4cf4da42cb7478d9520c2 Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Tue, 2 Jul 2024 14:27:28 -0400 Subject: [PATCH 16/24] Small typos --- .../pgd/5/consistency/conflicts/00_conflicts_overview.mdx | 4 ++-- product_docs/docs/pgd/5/consistency/conflicts/index.mdx | 3 +-- 2 files changed, 3 insertions(+), 4 deletions(-) diff --git a/product_docs/docs/pgd/5/consistency/conflicts/00_conflicts_overview.mdx b/product_docs/docs/pgd/5/consistency/conflicts/00_conflicts_overview.mdx index 0ceba54841e..8d648c87bea 100644 --- a/product_docs/docs/pgd/5/consistency/conflicts/00_conflicts_overview.mdx +++ b/product_docs/docs/pgd/5/consistency/conflicts/00_conflicts_overview.mdx @@ -7,13 +7,13 @@ deepToC: true EDB Postgres Distributed is an active/active or multi-master DBMS. If used asynchronously, writes to the same or related rows from multiple different nodes can result in data conflicts when using standard data types. Conflicts aren't errors. In most cases, they are events that PGD can detect and resolve as they occur. Resolving them depends on the nature of the application and the meaning of the data, so it's important for -PGD to provide the application a range of choices for how to resolve conflicts. +PGD to provide the application with a range of choices for how to resolve conflicts. By default, conflicts are resolved at the row level. When changes from two nodes conflict, PGD picks either the local or remote tuple and the discards the other. For example, the commit timestamps might be compared for the two conflicting changes and the newer one kept. This approach ensures that all nodes converge to the same result and establishes commit-order-like semantics on the whole cluster. Conflict handling is configurable, as described in [Conflict resolution](04_conflict_resolution). PGD can detect conflicts and handle them differently for each table using conflict triggers, described in [Stream triggers](../../striggers). -Column-level conflict detection and resolution is available with PGD, described in [CLCD](../column-level-conflicts). +Column-level conflict detection and resolution is available with PGD, as described in [CLCD](../column-level-conflicts). By default, all conflicts are logged to `bdr.conflict_history`. If conflicts are possible, then table owners must monitor for them and analyze how to avoid them or make plans to handle them regularly as an application task. The [LiveCompare](/livecompare/latest) tool is also available to scan regularly for divergence. diff --git a/product_docs/docs/pgd/5/consistency/conflicts/index.mdx b/product_docs/docs/pgd/5/consistency/conflicts/index.mdx index 51b6e97caa9..986d588d99a 100644 --- a/product_docs/docs/pgd/5/consistency/conflicts/index.mdx +++ b/product_docs/docs/pgd/5/consistency/conflicts/index.mdx @@ -16,5 +16,4 @@ redirects: - [Conflict logging](05_conflict_logging) points out where PGD keeps conflict logs and explains how you can perform conflict reporting. -- [Data verification with LiveCompare](06_live_compare) explains how LiveCompare can help keep data consistent by pointing out conflicts as they arise. - +- [Data verification with LiveCompare](06_live_compare) explains how LiveCompare can help keep data consistent by pointing out conflicts as they arise. \ No newline at end of file From 532148e0ed906e763a64dcac4063a5fb32188f53 Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Wed, 3 Jul 2024 09:37:55 -0400 Subject: [PATCH 17/24] Update product_docs/docs/pgd/5/consistency/conflicts/index.mdx Co-authored-by: gvasquezvargas --- product_docs/docs/pgd/5/consistency/conflicts/index.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/product_docs/docs/pgd/5/consistency/conflicts/index.mdx b/product_docs/docs/pgd/5/consistency/conflicts/index.mdx index 986d588d99a..16f86c0ae9c 100644 --- a/product_docs/docs/pgd/5/consistency/conflicts/index.mdx +++ b/product_docs/docs/pgd/5/consistency/conflicts/index.mdx @@ -6,7 +6,7 @@ redirects: - [Overview](00_conflicts_overview) introduces the idea of conflicts in PGD and explains how they can happen. -- [Avoiding or tolerating conflicts](01_avoid_tolerate_conflicts) discusses strategies for avoiding or tolerating conflicts in PGD/ +- [Avoiding or tolerating conflicts](01_avoid_tolerate_conflicts) discusses strategies for avoiding or tolerating conflicts in PGD. - [Types of conflicts](02_types_of_conflict) lists and discusses the various sorts of conflicts you might run across in PGD. From 908b87c3f28ad6a55db385b492fed0b0426353ac Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Wed, 3 Jul 2024 11:29:02 -0400 Subject: [PATCH 18/24] Split out functions in their own file and put all the lists in their own file. --- .../pgd/5/reference/conflict_functions.mdx | 118 ++++++++++++++++++ .../docs/pgd/5/reference/conflicts.mdx | 117 +---------------- product_docs/docs/pgd/5/reference/index.json | 13 +- product_docs/docs/pgd/5/reference/index.mdx | 15 +-- 4 files changed, 127 insertions(+), 136 deletions(-) create mode 100644 product_docs/docs/pgd/5/reference/conflict_functions.mdx diff --git a/product_docs/docs/pgd/5/reference/conflict_functions.mdx b/product_docs/docs/pgd/5/reference/conflict_functions.mdx new file mode 100644 index 00000000000..930d320e088 --- /dev/null +++ b/product_docs/docs/pgd/5/reference/conflict_functions.mdx @@ -0,0 +1,118 @@ +--- +title: Conflict functions +indexdepth: 2 +deepToC: true +--- + +## `bdr.alter_table_conflict_detection` + +Allows the table owner to change how conflict detection works for a given table. + +### Synopsis + +```sql +bdr.alter_table_conflict_detection(relation regclass, + method text, + column_name name DEFAULT NULL) +``` + +### Parameters + +- `relation` — Name of the relation for which to set the new conflict detection method. +- `method` — The conflict detection method to use. +- `column_name` — The column to use for storing the column detection data. This can be skipped, in which case the column name is chosen based on the conflict detection method. The `row_origin` method doesn't require an extra column for metadata storage. + +The recognized methods for conflict detection are: + +- `row_origin` — Origin of the previous change made on the tuple (see [Origin conflict detection](../consistency/conflicts/03_conflict_detection/#origin-conflict-detection)). This is the only method supported that doesn't require an extra column in the table. +- `row_version` — Row version column (see [Row version conflict detection](../consistency/conflicts/03_conflict_detection/#row-version-conflict-detection)). +- `column_commit_timestamp` — Per-column commit timestamps (described in [CLCD](../consistency/column-level-conflicts)). +- `column_modify_timestamp` — Per-column modification timestamp (described in [CLCD](../consistency/column-level-conflicts)). + +### Notes + +For more information about the difference between `column_commit_timestamp` and `column_modify_timestamp` conflict detection methods, see [Current versus commit timestamp](../consistency/column-level-conflicts/#current-versus-commit-timestamp). + +This function uses the same replication mechanism as `DDL` statements. This means the replication is affected by the [ddl filters](../repsets/#ddl-replication-filtering) configuration. + +The function takes a `DML` global lock on the relation for which column-level conflict resolution is being enabled. + +This function is transactional. You can roll back the effects with the `ROLLBACK` of the transaction, and the changes are visible to the current transaction. + +Only the owner of the `relation` can execute the `bdr.alter_table_conflict_detection` function unless `bdr.backwards_compatibility` is set to 30618 or less. + +!!! Warning + When changing the conflict detection method from one that uses an extra column to store metadata, that column is dropped. + +!!! Warning + This function disables CAMO and gives a warning, as long as warnings aren't disabled with `bdr.camo_enable_client_warnings`. + +## `bdr.alter_node_set_conflict_resolver` + +This function sets the behavior of conflict resolution on a given node. + +### Synopsis + +```sql +bdr.alter_node_set_conflict_resolver(node_name text, + conflict_type text, + conflict_resolver text) +``` + +### Parameters + +- `node_name` — Name of the node that's being changed. +- `conflict_type` — Conflict type for which to apply the setting (see [List of conflict types](#list-of-conflict-types)). +- `conflict_resolver` — Resolver to use for the given conflict type (see [List of conflict resolvers](#list-of-conflict-resolvers)). + +### Notes + +Currently you can change only the local node. The function call isn't replicated. If you want to change settings on multiple nodes, you must run the function on each of them. + +The configuration change made by this function overrides any default behavior of conflict resolutions specified by `bdr.create_node_group` or `bdr.alter_node_group`. + +This function is transactional. You can roll back the changes, and they are visible to the current transaction. + +## `bdr.alter_node_set_log_config` + +Set the conflict logging configuration for a node. + +### Synopsis + +```sql +bdr.alter_node_set_log_config(node_name text, + log_to_file bool DEFAULT true, + log_to_table bool DEFAULT true, + conflict_type text[] DEFAULT NULL, + conflict_resolution text[] DEFAULT NULL) +``` + +### Parameters + +- `node_name` — Name of the node that's being changed. +- `log_to_file` — Whether to log to the node log file. +- `log_to_table` — Whether to log to the `bdr.conflict_history` table. +- `conflict_type` — Conflict types to log. NULL (the default) means all. +- `conflict_resolution` — Conflict resolutions to log. NULL (the default) means all. + +### Notes + +You can change only the local node. The function call isn't replicated. If you want to change settings on multiple nodes, you must run the function on each of them. + +This function is transactional. You can roll back the changes, and they're visible to the current transaction. + +### Listing conflict logging configurations + +The view `bdr.node_log_config` shows all the logging configurations. It lists the name of the logging configuration, where it logs, and the conflict type and resolution it logs. + +### Logging conflicts to a table + +If `log_to_table` is set to true, conflicts are logged to a table. The target table for conflict logging is `bdr.conflict_history`. + +This table is range partitioned on the column `local_time`. The table is managed by autopartition. By default, a new partition is created for every day, and conflicts of the last one month are maintained. After that, the old partitions are dropped automatically. Autopartition creates between 7 and 14 partitions in advance. bdr_superuser can change these defaults. + +Since conflicts generated for all tables managed by PGD are logged to this table, it's important to ensure that only legitimate users can read the conflicted data. PGD does this by defining ROW LEVEL SECURITY policies on the `bdr.conflict_history` table. Only owners of the tables are allowed to read conflicts on the respective tables. If the underlying tables have RLS policies defined, enabled, and enforced, then even owners can't read the conflicts. RLS policies created with the FORCE option also apply to owners of the table. In that case, some or all rows in the underlying table might not be readable even to the owner. So PGD also enforces a stricter policy on the conflict log table. + +The default role `bdr_read_all_conflicts` can be granted to users who need to see all conflict details logged to the `bdr.conflict_history` table without also granting them `bdr_superuser` role. + +The default role `bdr_read_all_stats` has access to a catalog view called `bdr.conflict_history_summary`, which doesn't contain user data, allowing monitoring of any conflicts logged. diff --git a/product_docs/docs/pgd/5/reference/conflicts.mdx b/product_docs/docs/pgd/5/reference/conflicts.mdx index 66155649ed7..eef93f40862 100644 --- a/product_docs/docs/pgd/5/reference/conflicts.mdx +++ b/product_docs/docs/pgd/5/reference/conflicts.mdx @@ -1,54 +1,11 @@ --- -title: Conflict settings +title: Conflicts indexdepth: 2 deepToC: true --- # Conflict detection -## `bdr.alter_table_conflict_detection` - -Allows the table owner to change how conflict detection works for a given table. - -### Synopsis - -```sql -bdr.alter_table_conflict_detection(relation regclass, - method text, - column_name name DEFAULT NULL) -``` - -### Parameters - -- `relation` — Name of the relation for which to set the new conflict detection method. -- `method` — The conflict detection method to use. -- `column_name` — The column to use for storing the column detection data. This can be skipped, in which case the column name is chosen based on the conflict detection method. The `row_origin` method doesn't require an extra column for metadata storage. - -The recognized methods for conflict detection are: - -- `row_origin` — Origin of the previous change made on the tuple (see [Origin conflict detection](../consistency/conflicts/03_conflict_detection/#origin-conflict-detection)). This is the only method supported that doesn't require an extra column in the table. -- `row_version` — Row version column (see [Row version conflict detection](../consistency/conflicts/03_conflict_detection/#row-version-conflict-detection)). -- `column_commit_timestamp` — Per-column commit timestamps (described in [CLCD](../consistency/column-level-conflicts)). -- `column_modify_timestamp` — Per-column modification timestamp (described in [CLCD](../consistency/column-level-conflicts)). - -### Notes - -For more information about the difference between `column_commit_timestamp` and `column_modify_timestamp` conflict detection methods, see [Current versus commit timestamp](../consistency/column-level-conflicts/#current-versus-commit-timestamp). - -This function uses the same replication mechanism as `DDL` statements. This means the replication is affected by the [ddl filters](../repsets/#ddl-replication-filtering) configuration. - -The function takes a `DML` global lock on the relation for which column-level conflict resolution is being enabled. - -This function is transactional. You can roll back the effects with the `ROLLBACK` of the transaction, and the changes are visible to the current transaction. - -Only the owner of the `relation` can execute the `bdr.alter_table_conflict_detection` function unless `bdr.backwards_compatibility` is set to 30618 or less. - -!!! Warning - When changing the conflict detection method from one that uses an extra column to store metadata, that column is dropped. - -!!! Warning - This function disables CAMO and gives a warning, as long as warnings aren't disabled with `bdr.camo_enable_client_warnings`. - ## List of conflict types PGD recognizes the following conflict types, which can be used as the `conflict_type` parameter: @@ -73,32 +30,6 @@ Most conflicts can be resolved automatically. PGD defaults to a last-update-wins PGD lets you override the default behavior of conflict resolution by using the following function. -## `bdr.alter_node_set_conflict_resolver` - -This function sets the behavior of conflict resolution on a given node. - -### Synopsis - -```sql -bdr.alter_node_set_conflict_resolver(node_name text, - conflict_type text, - conflict_resolver text) -``` - -### Parameters - -- `node_name` — Name of the node that's being changed. -- `conflict_type` — Conflict type for which to apply the setting (see [List of conflict types](#list-of-conflict-types)). -- `conflict_resolver` — Resolver to use for the given conflict type (see [List of conflict resolvers](#list-of-conflict-resolvers)). - -### Notes - -Currently you can change only the local node. The function call isn't replicated. If you want to change settings on multiple nodes, you must run the function on each of them. - -The configuration change made by this function overrides any default behavior of conflict resolutions specified by `bdr.create_node_group` or `bdr.alter_node_group`. - -This function is transactional. You can roll back the changes, and they are visible to the current transaction. - ## List of conflict resolvers Several conflict resolvers are available in PGD, with differing coverages of the conflict types they can handle: @@ -173,49 +104,5 @@ The following conflict resolutions are currently supported for the `conflict_res # Conflict logging -To ease diagnosing and handling multi-master conflicts, PGD, by default, logs every conflict into the `bdr.conflict_history` table. You can change this behavior with more granularity with the following functions. - -## bdr.alter_node_set_log_config - -Set the conflict logging configuration for a node. - -### Synopsis - -```sql -bdr.alter_node_set_log_config(node_name text, - log_to_file bool DEFAULT true, - log_to_table bool DEFAULT true, - conflict_type text[] DEFAULT NULL, - conflict_resolution text[] DEFAULT NULL) -``` - -### Parameters - -- `node_name` — Name of the node that's being changed. -- `log_to_file` — Whether to log to the node log file. -- `log_to_table` — Whether to log to the `bdr.conflict_history` table. -- `conflict_type` — Conflict types to log. NULL (the default) means all. -- `conflict_resolution` — Conflict resolutions to log. NULL (the default) means all. - -### Notes - -You can change only the local node. The function call isn't replicated. If you want to change settings on multiple nodes, you must run the function on each of them. - -This function is transactional. You can roll back the changes, and they're visible to the current transaction. - -### Listing conflict logging configurations - -The view `bdr.node_log_config` shows all the logging configurations. It lists the name of the logging configuration, where it logs, and the conflict type and resolution it logs. - -### Logging conflicts to a table - -If `log_to_table` is set to true, conflicts are logged to a table. The target table for conflict logging is `bdr.conflict_history`. - -This table is range partitioned on the column `local_time`. The table is managed by autopartition. By default, a new partition is created for every day, and conflicts of the last one month are maintained. After that, the old partitions are dropped automatically. Autopartition creates between 7 and 14 partitions in advance. bdr_superuser can change these defaults. - -Since conflicts generated for all tables managed by PGD are logged to this table, it's important to ensure that only legitimate users can read the conflicted data. PGD does this by defining ROW LEVEL SECURITY policies on the `bdr.conflict_history` table. Only owners of the tables are allowed to read conflicts on the respective tables. If the underlying tables have RLS policies defined, enabled, and enforced, then even owners can't read the conflicts. RLS policies created with the FORCE option also apply to owners of the table. In that case, some or all rows in the underlying table might not be readable even to the owner. So PGD also enforces a stricter policy on the conflict log table. - -The default role `bdr_read_all_conflicts` can be granted to users who need to see all conflict details logged to the `bdr.conflict_history` table without also granting them `bdr_superuser` role. +To ease diagnosing and handling multi-master conflicts, PGD, by default, logs every conflict into the `bdr.conflict_history` table. You can change this behavior with more granularity using [bdr.alter_node_set_log_config](conflict_functions/#bdralter_node_set_log_config). -The default role `bdr_read_all_stats` has access to a catalog view called `bdr.conflict_history_summary`, which doesn't contain user data, allowing monitoring of any conflicts logged. -``` \ No newline at end of file diff --git a/product_docs/docs/pgd/5/reference/index.json b/product_docs/docs/pgd/5/reference/index.json index f6fd0bd3056..e1f4640e238 100644 --- a/product_docs/docs/pgd/5/reference/index.json +++ b/product_docs/docs/pgd/5/reference/index.json @@ -319,14 +319,7 @@ "bdrtaskmgr_set_leader": "/pgd/latest/reference/functions-internal#bdrtaskmgr_set_leader", "bdrtaskmgr_get_last_completed_workitem": "/pgd/latest/reference/functions-internal#bdrtaskmgr_get_last_completed_workitem", "bdrtaskmgr_work_queue_check_status": "/pgd/latest/reference/functions-internal#bdrtaskmgr_work_queue_check_status", - "conflict-detection": "/pgd/latest/reference/conflicts#conflict-detection", - "bdralter_table_conflict_detection": "/pgd/latest/reference/conflicts#bdralter_table_conflict_detection", - "list-of-conflict-types": "/pgd/latest/reference/conflicts#list-of-conflict-types", - "conflict-resolution": "/pgd/latest/reference/conflicts#conflict-resolution", - "bdralter_node_set_conflict_resolver": "/pgd/latest/reference/conflicts#bdralter_node_set_conflict_resolver", - "list-of-conflict-resolvers": "/pgd/latest/reference/conflicts#list-of-conflict-resolvers", - "default-conflict-resolvers": "/pgd/latest/reference/conflicts#default-conflict-resolvers", - "list-of-conflict-resolutions": "/pgd/latest/reference/conflicts#list-of-conflict-resolutions", - "conflict-logging": "/pgd/latest/reference/conflicts#conflict-logging", - "bdralter_node_set_log_config": "/pgd/latest/reference/conflicts#bdralter_node_set_log_config" + "bdralter_table_conflict_detection": "/pgd/latest/reference/conflict_functions#bdralter_table_conflict_detection", + "bdralter_node_set_conflict_resolver": "/pgd/latest/reference/conflict_functions#bdralter_node_set_conflict_resolver", + "bdralter_node_set_log_config": "/pgd/latest/reference/conflict_functions#bdralter_node_set_log_config" } \ No newline at end of file diff --git a/product_docs/docs/pgd/5/reference/index.mdx b/product_docs/docs/pgd/5/reference/index.mdx index 6c067563fe6..6ce731e3436 100644 --- a/product_docs/docs/pgd/5/reference/index.mdx +++ b/product_docs/docs/pgd/5/reference/index.mdx @@ -436,14 +436,7 @@ The reference section is a definitive listing of all functions, views, and comma * [`bdr.taskmgr_work_queue_check_status`](functions-internal#bdrtaskmgr_work_queue_check_status) -## [Conflict settings](conflicts) - * [Conflict detection](conflicts#conflict-detection) - * [`bdr.alter_table_conflict_detection`](conflicts#bdralter_table_conflict_detection) - * [List of conflict types](conflicts#list-of-conflict-types) - * [Conflict resolution](conflicts#conflict-resolution) - * [`bdr.alter_node_set_conflict_resolver`](conflicts#bdralter_node_set_conflict_resolver) - * [List of conflict resolvers](conflicts#list-of-conflict-resolvers) - * [Default conflict resolvers](conflicts#default-conflict-resolvers) - * [List of conflict resolutions](conflicts#list-of-conflict-resolutions) - * [Conflict logging](conflicts#conflict-logging) - * [bdr.alter_node_set_log_config](conflicts#bdralter_node_set_log_config) +## [Conflict functions](conflict_functions) + * [`bdr.alter_table_conflict_detection`](conflict_functions#bdralter_table_conflict_detection) + * [`bdr.alter_node_set_conflict_resolver`](conflict_functions#bdralter_node_set_conflict_resolver) + * [`bdr.alter_node_set_log_config`](conflict_functions#bdralter_node_set_log_config) From 2be59ff137cb9b5d7c66a1a3925aaffd108c1a70 Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Wed, 3 Jul 2024 12:40:57 -0400 Subject: [PATCH 19/24] dropped deepToCs --- product_docs/docs/pgd/5/reference/conflict_functions.mdx | 1 - product_docs/docs/pgd/5/reference/conflicts.mdx | 1 - 2 files changed, 2 deletions(-) diff --git a/product_docs/docs/pgd/5/reference/conflict_functions.mdx b/product_docs/docs/pgd/5/reference/conflict_functions.mdx index 930d320e088..7d434446382 100644 --- a/product_docs/docs/pgd/5/reference/conflict_functions.mdx +++ b/product_docs/docs/pgd/5/reference/conflict_functions.mdx @@ -1,7 +1,6 @@ --- title: Conflict functions indexdepth: 2 -deepToC: true --- ## `bdr.alter_table_conflict_detection` diff --git a/product_docs/docs/pgd/5/reference/conflicts.mdx b/product_docs/docs/pgd/5/reference/conflicts.mdx index eef93f40862..ecd776903ba 100644 --- a/product_docs/docs/pgd/5/reference/conflicts.mdx +++ b/product_docs/docs/pgd/5/reference/conflicts.mdx @@ -1,7 +1,6 @@ --- title: Conflicts indexdepth: 2 -deepToC: true --- # Conflict detection From 1c4d51fd133f64d26b2f41c4eb0d8c5bf208fcd3 Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Wed, 3 Jul 2024 13:19:50 -0400 Subject: [PATCH 20/24] Fixed broken links. --- .../pgd/5/consistency/conflicts/01_avoid_tolerate_conflicts.mdx | 2 +- .../docs/pgd/5/consistency/conflicts/03_conflict_detection.mdx | 2 +- .../docs/pgd/5/consistency/conflicts/04_conflict_resolution.mdx | 2 +- .../docs/pgd/5/consistency/conflicts/05_conflict_logging.mdx | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/product_docs/docs/pgd/5/consistency/conflicts/01_avoid_tolerate_conflicts.mdx b/product_docs/docs/pgd/5/consistency/conflicts/01_avoid_tolerate_conflicts.mdx index 66b3f8fb484..a963aebb0ab 100644 --- a/product_docs/docs/pgd/5/consistency/conflicts/01_avoid_tolerate_conflicts.mdx +++ b/product_docs/docs/pgd/5/consistency/conflicts/01_avoid_tolerate_conflicts.mdx @@ -12,7 +12,7 @@ This avoidance happens naturally in many applications. For example, many consume You might make a change just before a node goes down, so the change seems to be lost. You might then make the same change again, leading to two updates on different nodes. When the down node comes back up, it tries to send the older change to other nodes. It's rejected because the last update of the data is kept. -For `INSERT`/`INSERT` conflicts, use [global sequences](../../sequences//#pgd-global-sequences) to prevent this type of conflict. +For `INSERT`/`INSERT` conflicts, use [global sequences](../../sequences/#pgd-global-sequences) to prevent this type of conflict. For applications that assign relationships between objects, such as a room-booking application, applying `update_if_newer` might not give an acceptable business outcome. That is, it isn't useful to confirm to two people separately that they have booked the same room. The simplest resolution is to use Eager Replication to ensure that only one booking succeeds. More complex ways might be possible depending on the application. For example, you can assign 100 seats to each node and allow those to be booked by a writer on that node. But if none are available locally, use a distributed locking scheme or Eager Replication after most seats are reserved. diff --git a/product_docs/docs/pgd/5/consistency/conflicts/03_conflict_detection.mdx b/product_docs/docs/pgd/5/consistency/conflicts/03_conflict_detection.mdx index d67924ba579..fda6f616dfe 100644 --- a/product_docs/docs/pgd/5/consistency/conflicts/03_conflict_detection.mdx +++ b/product_docs/docs/pgd/5/consistency/conflicts/03_conflict_detection.mdx @@ -77,4 +77,4 @@ This approach resembles Lamport timestamps and fully prevents the ABA problem fo To determine the current conflict detection strategy used for a specific table, refer to the column `conflict_detection` of the view `bdr.tables`. -To change the current conflict detection strategy, use [bdr.alter_table_conflict_detection](../../reference/conflicts/#bdralter_table_conflict_detection). \ No newline at end of file +To change the current conflict detection strategy, use [bdr.alter_table_conflict_detection](../../reference/conflict_functions/#bdralter_table_conflict_detection). \ No newline at end of file diff --git a/product_docs/docs/pgd/5/consistency/conflicts/04_conflict_resolution.mdx b/product_docs/docs/pgd/5/consistency/conflicts/04_conflict_resolution.mdx index c156aa021e1..f7406c36d60 100644 --- a/product_docs/docs/pgd/5/consistency/conflicts/04_conflict_resolution.mdx +++ b/product_docs/docs/pgd/5/consistency/conflicts/04_conflict_resolution.mdx @@ -6,4 +6,4 @@ deepToC: true Most conflicts can be resolved automatically. PGD defaults to a last-update-wins mechanism or, more accurately, the `update_if_newer` conflict resolver. This mechanism retains the most recently inserted or changed row of the two conflicting ones based on the same commit timestamps used for conflict detection. The behavior in certain corner-case scenarios depends on the settings used for `bdr.create_node_group` and alternatively for `bdr.alter_node_group`. -PGD lets you override the default behavior of conflict resolution by using [bdr.alter_node_set_conflict_resolver](../../reference/conflicts/#bdralter_node_set_conflict_resolver). \ No newline at end of file +PGD lets you override the default behavior of conflict resolution by using [bdr.alter_node_set_conflict_resolver](../../reference/conflict_functions/#bdralter_node_set_conflict_resolver). \ No newline at end of file diff --git a/product_docs/docs/pgd/5/consistency/conflicts/05_conflict_logging.mdx b/product_docs/docs/pgd/5/consistency/conflicts/05_conflict_logging.mdx index f5e548428c0..bb0ff2a6414 100644 --- a/product_docs/docs/pgd/5/consistency/conflicts/05_conflict_logging.mdx +++ b/product_docs/docs/pgd/5/consistency/conflicts/05_conflict_logging.mdx @@ -4,7 +4,7 @@ Description: How PGD logs database conflicts. deepToC: true --- -To ease diagnosing and handling multi-master conflicts, PGD, by default, logs every conflict into the `bdr.conflict_history` table. You can change this behavior with more granularity using [bdr.alter_node_set_log_config](../../reference/conflicts/#bdralter_node_set_log_config). +To ease diagnosing and handling multi-master conflicts, PGD, by default, logs every conflict into the `bdr.conflict_history` table. You can change this behavior with more granularity using [bdr.alter_node_set_log_config](../../reference/conflict_functions/#bdralter_node_set_log_config). ## Conflict reporting From b70318d851abda3c9df1c2fc8cbd84aedc288b87 Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Tue, 9 Jul 2024 14:00:31 -0400 Subject: [PATCH 21/24] Merged avoiding conflicts section into overview. --- .../conflicts/00_conflicts_overview.mdx | 23 +++++++++++++++++- .../conflicts/01_avoid_tolerate_conflicts.mdx | 24 ------------------- 2 files changed, 22 insertions(+), 25 deletions(-) delete mode 100644 product_docs/docs/pgd/5/consistency/conflicts/01_avoid_tolerate_conflicts.mdx diff --git a/product_docs/docs/pgd/5/consistency/conflicts/00_conflicts_overview.mdx b/product_docs/docs/pgd/5/consistency/conflicts/00_conflicts_overview.mdx index 8d648c87bea..254886402e6 100644 --- a/product_docs/docs/pgd/5/consistency/conflicts/00_conflicts_overview.mdx +++ b/product_docs/docs/pgd/5/consistency/conflicts/00_conflicts_overview.mdx @@ -31,4 +31,25 @@ Since PGD replication essentially replays the transaction on the other nodes, th Most conflicts can't happen when all transactions run on a single node because Postgres has inter-transaction communication mechanisms to prevent it. Examples of these mechanisms are `UNIQUE` indexes, `SEQUENCE` operations, row and relation locking, and `SERIALIZABLE` dependency tracking. All of these mechanisms are ways to communicate between ongoing transactions to prevent undesirable concurrency issues. -PGD doesn't have a distributed transaction manager or lock manager. That's part of why it performs well with latency and network partitions. As a result, transactions on different nodes execute entirely independently from each other when using the default, which is lazy replication. Less independence between nodes can avoid conflicts altogether, which is why PGD also offers Eager Replication for when this is important. \ No newline at end of file +PGD doesn't have a distributed transaction manager or lock manager. That's part of why it performs well with latency and network partitions. As a result, transactions on different nodes execute entirely independently from each other when using the default, which is lazy replication. Less independence between nodes can avoid conflicts altogether, which is why PGD also offers Eager Replication for when this is important. + +## Avoiding or tolerating conflicts + +In most cases, you can design the application to avoid or tolerate conflicts. + +Conflicts can happen only if things are happening at the same time on multiple nodes. The simplest way to avoid conflicts is to only ever write to one node or to only ever write to a specific row in a specific way from one specific node at a time. + +This avoidance happens naturally in many applications. For example, many consumer applications allow only the owning user to change data, such as changing the default billing address on an account. Such data changes seldom have update conflicts. + +You might make a change just before a node goes down, so the change seems to be lost. You might then make the same change again, leading to two updates on different nodes. When the down node comes back up, it tries to send the older change to other nodes. It's rejected because the last update of the data is kept. + +For `INSERT`/`INSERT` conflicts, use [global sequences](../../sequences/#pgd-global-sequences) to prevent this type of conflict. + +For applications that assign relationships between objects, such as a room-booking application, applying `update_if_newer` might not give an acceptable business outcome. That is, it isn't useful to confirm to two people separately that they have booked the same room. The simplest resolution is to use Eager Replication to ensure that only one booking succeeds. More complex ways might be possible depending on the application. For example, you can assign 100 seats to each node and allow those to be booked by a writer on that node. But if none are available locally, use a distributed locking scheme or Eager Replication after most seats are reserved. + +Another technique for ensuring certain types of updates occur only from one specific node is to route different types of transactions through different nodes. For example: + +- Receiving parcels on one node but delivering parcels using another node +- A service application where orders are input on one node and work is prepared on a second node and then served back to customers on another + +Frequently, the best course is to allow conflicts to occur and design the application to work with PGD's conflict resolution mechanisms to cope with the conflict. \ No newline at end of file diff --git a/product_docs/docs/pgd/5/consistency/conflicts/01_avoid_tolerate_conflicts.mdx b/product_docs/docs/pgd/5/consistency/conflicts/01_avoid_tolerate_conflicts.mdx deleted file mode 100644 index a963aebb0ab..00000000000 --- a/product_docs/docs/pgd/5/consistency/conflicts/01_avoid_tolerate_conflicts.mdx +++ /dev/null @@ -1,24 +0,0 @@ ---- -title: Avoiding or tolerating conflicts -Description: How to avoid database conflicts. -deepToC: true ---- - -In most cases, you can design the application to avoid or tolerate conflicts. - -Conflicts can happen only if things are happening at the same time on multiple nodes. The simplest way to avoid conflicts is to only ever write to one node or to only ever write to a specific row in a specific way from one specific node at a time. - -This avoidance happens naturally in many applications. For example, many consumer applications allow only the owning user to change data, such as changing the default billing address on an account. Such data changes seldom have update conflicts. - -You might make a change just before a node goes down, so the change seems to be lost. You might then make the same change again, leading to two updates on different nodes. When the down node comes back up, it tries to send the older change to other nodes. It's rejected because the last update of the data is kept. - -For `INSERT`/`INSERT` conflicts, use [global sequences](../../sequences/#pgd-global-sequences) to prevent this type of conflict. - -For applications that assign relationships between objects, such as a room-booking application, applying `update_if_newer` might not give an acceptable business outcome. That is, it isn't useful to confirm to two people separately that they have booked the same room. The simplest resolution is to use Eager Replication to ensure that only one booking succeeds. More complex ways might be possible depending on the application. For example, you can assign 100 seats to each node and allow those to be booked by a writer on that node. But if none are available locally, use a distributed locking scheme or Eager Replication after most seats are reserved. - -Another technique for ensuring certain types of updates occur only from one specific node is to route different types of transactions through different nodes. For example: - -- Receiving parcels on one node but delivering parcels using another node -- A service application where orders are input on one node and work is prepared on a second node and then served back to customers on another - -Frequently, the best course is to allow conflicts to occur and design the application to work with PGD's conflict resolution mechanisms to cope with the conflict. \ No newline at end of file From 2242aae8f36c877b0624106c2167878495a7c871 Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Tue, 9 Jul 2024 14:01:29 -0400 Subject: [PATCH 22/24] Update product_docs/docs/pgd/5/consistency/conflicts/index.mdx Co-authored-by: Dj Walker-Morgan <126472455+djw-m@users.noreply.github.com> --- product_docs/docs/pgd/5/consistency/conflicts/index.mdx | 2 ++ 1 file changed, 2 insertions(+) diff --git a/product_docs/docs/pgd/5/consistency/conflicts/index.mdx b/product_docs/docs/pgd/5/consistency/conflicts/index.mdx index 16f86c0ae9c..4b13805d118 100644 --- a/product_docs/docs/pgd/5/consistency/conflicts/index.mdx +++ b/product_docs/docs/pgd/5/consistency/conflicts/index.mdx @@ -3,7 +3,9 @@ title: Conflicts redirects: - /pgd/latest/bdr/conflicts/ --- +EDB Postgres Distributed is an active/active or multi-master DBMS. If used asynchronously, writes to the same or related rows from multiple different nodes can result in data conflicts when using standard data types. +Conflicts aren't errors. In most cases, they are events that PGD can detect and resolve as they occur. This section introduces the PGD functionality that allows you to manage that detection and resolution. - [Overview](00_conflicts_overview) introduces the idea of conflicts in PGD and explains how they can happen. - [Avoiding or tolerating conflicts](01_avoid_tolerate_conflicts) discusses strategies for avoiding or tolerating conflicts in PGD. From 5cb5f65853f470f20efcca580f5a04e3b9f94148 Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Tue, 9 Jul 2024 14:01:40 -0400 Subject: [PATCH 23/24] Update product_docs/docs/pgd/5/consistency/conflicts/00_conflicts_overview.mdx Co-authored-by: Dj Walker-Morgan <126472455+djw-m@users.noreply.github.com> --- .../docs/pgd/5/consistency/conflicts/00_conflicts_overview.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/product_docs/docs/pgd/5/consistency/conflicts/00_conflicts_overview.mdx b/product_docs/docs/pgd/5/consistency/conflicts/00_conflicts_overview.mdx index 254886402e6..da19837bec6 100644 --- a/product_docs/docs/pgd/5/consistency/conflicts/00_conflicts_overview.mdx +++ b/product_docs/docs/pgd/5/consistency/conflicts/00_conflicts_overview.mdx @@ -15,7 +15,7 @@ Conflict handling is configurable, as described in [Conflict resolution](04_conf Column-level conflict detection and resolution is available with PGD, as described in [CLCD](../column-level-conflicts). -By default, all conflicts are logged to `bdr.conflict_history`. If conflicts are possible, then table owners must monitor for them and analyze how to avoid them or make plans to handle them regularly as an application task. The [LiveCompare](/livecompare/latest) tool is also available to scan regularly for divergence. +By default, all conflicts are logged to [`bdr.conflict_history`](/pgd/latest/reference/catalogs-visible/#bdrconflict_history). If conflicts are possible, then table owners must monitor for them and analyze how to avoid them or make plans to handle them regularly as an application task. The [LiveCompare](/livecompare/latest) tool is also available to scan regularly for divergence. Some clustering systems use distributed lock mechanisms to prevent concurrent access to data. These can perform reasonably when servers are very close to each other but can't support geographically distributed applications where very low latency is critical for acceptable performance. From 89fcbafd77312640a2931c590e36d5faceeb64ab Mon Sep 17 00:00:00 2001 From: Josh Earlenbaugh Date: Tue, 9 Jul 2024 14:07:03 -0400 Subject: [PATCH 24/24] add whitespace for easier reading in code --- product_docs/docs/pgd/5/consistency/conflicts/index.mdx | 1 + 1 file changed, 1 insertion(+) diff --git a/product_docs/docs/pgd/5/consistency/conflicts/index.mdx b/product_docs/docs/pgd/5/consistency/conflicts/index.mdx index 4b13805d118..952c714b880 100644 --- a/product_docs/docs/pgd/5/consistency/conflicts/index.mdx +++ b/product_docs/docs/pgd/5/consistency/conflicts/index.mdx @@ -6,6 +6,7 @@ redirects: EDB Postgres Distributed is an active/active or multi-master DBMS. If used asynchronously, writes to the same or related rows from multiple different nodes can result in data conflicts when using standard data types. Conflicts aren't errors. In most cases, they are events that PGD can detect and resolve as they occur. This section introduces the PGD functionality that allows you to manage that detection and resolution. + - [Overview](00_conflicts_overview) introduces the idea of conflicts in PGD and explains how they can happen. - [Avoiding or tolerating conflicts](01_avoid_tolerate_conflicts) discusses strategies for avoiding or tolerating conflicts in PGD.