diff --git a/product_docs/docs/bdr/4.0/appusage.mdx b/product_docs/docs/bdr/4.0/appusage.mdx
index 0b5febe6ea4..cd2a3989b89 100644
--- a/product_docs/docs/bdr/4.0/appusage.mdx
+++ b/product_docs/docs/bdr/4.0/appusage.mdx
@@ -361,9 +361,9 @@ application.
### pgbench with CAMO/Failover options
-pgbench has been extended to allow users to run failover tests while
-using CAMO or regular BDR deployments. The following new options have
-been added:
+In EDB Postgres Extended, the pgbench has been extended to allow users to
+run failover tests while using CAMO or regular BDR deployments. The following
+new options have been added:
```
-m, --mode=regular|camo|failover
@@ -624,9 +624,9 @@ BDR cannot currently perform conflict resolution where the PRIMARY KEY is change
by an UPDATE operation. It is permissible to update the primary key, but you must
ensure that no conflict with existing values is possible.
-When running on EDB Postgres Extended, BDR provides the following configuration
-parameter to assess how frequently the primary key/replica identity of any table
-is being subjected to update operations.
+BDR provides the following configuration parameter to assess how frequently
+the primary key/replica identity of any table is being subjected to update
+operations.
Note that these configuration parameters must only be used for assessment only.
They can be used on a single node BDR instance, but must not be used on a production
@@ -659,8 +659,8 @@ Because BDR writer processes operate much like normal user sessions, they are su
the usual rules around row and table locking. This can sometimes lead to BDR writer
processes waiting on locks held by user transactions, or even by each other.
-When running on EDB Postgres Extended, BDR provides the following
-configuration parameter to assess if the application is taking explicit locks.
+BDR provides the following configuration parameter to assess if the application
+is taking explicit locks.
```sql
bdr.assess_lock_statement = IGNORE (default) | LOG | WARNING | ERROR
diff --git a/product_docs/docs/bdr/4.0/camo_clients.mdx b/product_docs/docs/bdr/4.0/camo_clients.mdx
index eafc0d41337..5a548885d8c 100644
--- a/product_docs/docs/bdr/4.0/camo_clients.mdx
+++ b/product_docs/docs/bdr/4.0/camo_clients.mdx
@@ -1,5 +1,5 @@
---
-navTitle: 'Appendix F: CAMO Reference Clients'
+navTitle: 'Appendix E: CAMO Reference Clients'
title: CAMO Clients
originalFilePath: camo_clients.md
diff --git a/product_docs/docs/bdr/4.0/catalogs.mdx b/product_docs/docs/bdr/4.0/catalogs.mdx
index 19da9e31a07..ac989ed56cb 100644
--- a/product_docs/docs/bdr/4.0/catalogs.mdx
+++ b/product_docs/docs/bdr/4.0/catalogs.mdx
@@ -1043,17 +1043,22 @@ Specific information about BDR writer processes.
#### `bdr.writers` Columns
-| Name | Type | Description |
-| --------------------- | ----------- | ----------------------------------------------------------------- |
-| sub_name | name | Name of the subscription |
-| pid | int | Process Id of the worker process |
-| syncing_rel | int | Oid of the relation being synchronized (if any) |
-| streaming_allowed | text | Can this writer be target of direct to writer streaming |
-| is_streaming | bool | Is there transaction being streamed to this writer |
-| remote_xid | xid | Remote transaction id of the transaction being processed (if any) |
-| remote_commit_lsn | pg_lsn | LSN of last commit processed |
-| commit_queue_position | int | Position in the internal commit queue |
-| commit_timestamp | timestamptz | Timestamp of last commit processed |
+| Name | Type | Description |
+| --------------------- | ----------- | ------------------------------------------------------------------ |
+| sub_name | name | Name of the subscription |
+| pid | int | Process Id of the worker process |
+| syncing_rel | int | Oid of the relation being synchronized (if any) |
+| streaming_allowed | text | Can this writer be target of direct to writer streaming |
+| is_streaming | bool | Is there transaction being streamed to this writer |
+| remote_xid | xid | Remote transaction id of the transaction being processed (if any) |
+| remote_commit_lsn | pg_lsn | LSN of last commit processed |
+| commit_queue_position | int | Position in the internal commit queue |
+| commit_timestamp | timestamptz | Timestamp of last commit processed |
+| nxacts | bigint | Number of transactions processed by this writer |
+| ncommits | bigint | Number of transactions committed by this writer |
+| naborts | bigint | Number of transactions aborted by this writer |
+| nstream_file | bigint | Number of streamed-to-file transactions processed by this writer |
+| nstream_writer | bigint | Number of streamed-to-writer transactions processed by this writer |
### `bdr.worker_tasks`
diff --git a/product_docs/docs/bdr/4.0/column-level-conflicts.mdx b/product_docs/docs/bdr/4.0/column-level-conflicts.mdx
index fe8ec859c72..2a3628743eb 100644
--- a/product_docs/docs/bdr/4.0/column-level-conflicts.mdx
+++ b/product_docs/docs/bdr/4.0/column-level-conflicts.mdx
@@ -60,10 +60,6 @@ Column-level conflict resolution requires the table to have `REPLICA
IDENTITY FULL`. The `bdr.alter_table_conflict_detection` function does check
that, and will fail with an error otherwise.
-!!! Note
- This feature is currently only available on EDB Postgres Extended and
- EDB Postgres Advanced.
-
## Enabling and Disabling Column-Level Conflict Resolution
The Column-Level Conflict Resolution is managed by the
diff --git a/product_docs/docs/bdr/4.0/configuration.mdx b/product_docs/docs/bdr/4.0/configuration.mdx
index ee97126ba22..296503dc09c 100644
--- a/product_docs/docs/bdr/4.0/configuration.mdx
+++ b/product_docs/docs/bdr/4.0/configuration.mdx
@@ -68,23 +68,6 @@ Applications may also wish to set these parameters. Please see chapter on
in a similar way to [physical replication](https://www.postgresql.org/docs/11/runtime-config-wal.html#GUC-SYNCHRONOUS-COMMIT).
- `synchronous_standby_names` - same as above
-## EDB PG Extended and EDB PG Advanced Settings for BDR
-
-The following Postgres settings need to be considered for commit at
-most once (CAMO), a feature that is only available for BDR in
-combination with PG Extended. Some of these are only available in
-PG Extended; others already exist in the community version, but only
-become relevant with BDR in combination with CAMO.
-
-- `synchronous_replication_availability` - Can optionally be `async`
- for increased availability by allowing a node to continue and
- commit after its CAMO partner got disconnected. Under the default
- value of `wait`, the node will wait indefinitely, and proceed to
- commit only after the CAMO partner reconnects and sends
- confirmation.
-- `snapshot_timestamp` - Turns on the usage of
- [timestamp-based snapshots](tssnapshots) and sets the timestamp to use.
-
## BDR Specific Settings
There are also BDR specific configuration settings that can be set.
@@ -395,7 +378,6 @@ and receivers don't have a writer id.
local two-phase commit or CAMO transaction, and will prevent all
Eager transactions on the cluster.
May only be set at Postgres server start.
- (EDB Postgres Extended)
### Eager Replication
@@ -407,9 +389,6 @@ and receivers don't have a writer id.
in their commit phase, as a limit for how long to wait for the CAMO
partner.
-!!! Note
- This is only available on EDB Postgres Extended.
-
### Commit at Most Once
- `bdr.enable_camo` - Used to enable and control the CAMO feature.
@@ -430,9 +409,12 @@ and receivers don't have a writer id.
carried out in the database for which CAMO properties cannot be
guaranteed. This is enabled by default. Well-informed users can choose
to disable this to reduce the amount of warnings going into their logs.
-
-!!! Note
- This is only available on EDB Postgres Extended.
+- `synchronous_replication_availability` - Can optionally be `async`
+ for increased availability by allowing a node to continue and
+ commit after its CAMO partner got disconnected. Under the default
+ value of `wait`, the node will wait indefinitely, and proceed to
+ commit only after the CAMO partner reconnects and sends
+ confirmation.
### Transaction streaming
@@ -441,16 +423,15 @@ and receivers don't have a writer id.
Defaults to `auto`. If set to `off`, the subscriber will not request
transaction streaming. If set to one of the other permissible values, the
subscriber will request transaction streaming and the publisher will provide
- this if it supports them (support is available from BDR 4.0 with PostgreSQL
- 14, EDB Postgres Extended 13 and EPAS 14) and configured at group level. For
+ this if it supports them and configured at group level. For
more details, see [Transaction Streaming](transaction-streaming).
### Timestamp-based Snapshots
+- `snapshot_timestamp` - Turns on the usage of
+ [timestamp-based snapshots](tssnapshots) and sets the timestamp to use.
- `bdr.timestamp_snapshot_keep` - For how long to keep valid snapshots for the
timestamp-based snapshot usage (default 0, meaning do not keep past snapshots).
- Also see `snapshot_timestamp` above.
- (EDB Postgres Extended)
### Monitoring and Logging
@@ -510,10 +491,8 @@ and receivers don't have a writer id.
of apply rates and catchup intervals for peer nodes. This information can
be used by protocols like CAMO to estimate the readiness of a
peer node. This parameter is enabled by default.
- (EDB Postgres Extended)
- `bdr.lag_tracker_apply_rate_weight` - We monitor how far behind peer nodes
are in terms of applying WAL from the local node, and calculate a moving
average of the apply rates for the lag tracking. This parameter specifies
how much contribution newer calculated values have in this moving average
calculation. Default value is 0.1.
- (EDB Postgres Extended)
diff --git a/product_docs/docs/bdr/4.0/conflicts.mdx b/product_docs/docs/bdr/4.0/conflicts.mdx
index aff72faf2bc..73411db5c4f 100644
--- a/product_docs/docs/bdr/4.0/conflicts.mdx
+++ b/product_docs/docs/bdr/4.0/conflicts.mdx
@@ -708,13 +708,11 @@ as is normally the case with BDR AlwaysOn architecture.
!!! Warning
The additional WAL logging of TOAST is done using the `BEFORE UPDATE`
- trigger. This trigger must be sorted alphabetically
+ trigger on standard PostgreSQL. This trigger must be sorted alphabetically
last (based on trigger name) among all `BEFORE UPDATE` triggers on the
table. It's prefixed with `zzzz_bdr_` to make this easier, but make sure
you don't create any trigger with name that would sort after it, otherwise
the protection against the concurrency issues will not be present.
- This trigger is not created or used when using BDR with EDB Postgres
- Extended.
For the `insert_or_error` conflict resolution, the use of
`REPLICA IDENTITY FULL` is however still required.
@@ -813,9 +811,9 @@ frozen rows that they update. Inserts and Deletes are not affected by this situa
Users are advised to not leave down nodes for extended outages,
as discussed in [Node Restart and Down Node Recovery](nodes).
-On EDB Postgres Extended, BDR will automatically hold back the freezing of
-rows while a node is down to handle this situation gracefully without the need
-for changing parameter settings.
+On EDB Postgres Extended Server and EDB Postgres Advanced Server, BDR will
+automatically hold back the freezing of rows while a node is down to handle
+this situation gracefully without the need for changing parameter settings.
On other variants of Postgres, users may need to manage this situation with
some care:
diff --git a/product_docs/docs/bdr/4.0/crdt.mdx b/product_docs/docs/bdr/4.0/crdt.mdx
index c87591c7a41..dc67187e1de 100644
--- a/product_docs/docs/bdr/4.0/crdt.mdx
+++ b/product_docs/docs/bdr/4.0/crdt.mdx
@@ -163,10 +163,6 @@ new data types. CRDT types are handled transparently - both `ANALYZE` and
the optimizer work, so estimation and query planning works fine, without
having to do anything else.
-!!! Note
- This feature is currently only available on EDB Postgres Extended and
- EDB Postgres Advanced.
-
## State-based and operation-based CRDTs
Following the notation from [1], we do implement both operation-based
diff --git a/product_docs/docs/bdr/4.0/credits.mdx b/product_docs/docs/bdr/4.0/credits.mdx
deleted file mode 100644
index 56adac8fa3e..00000000000
--- a/product_docs/docs/bdr/4.0/credits.mdx
+++ /dev/null
@@ -1,30 +0,0 @@
----
-navTitle: Credits and License
-title: Credits and Licence
-originalFilePath: credits.md
-
----
-
-BDR has been designed, developed and tested by this team:
-
-- Petr Jelinek
-- Craig Ringer
-- Markus Wanner
-- Pavan Deolasee
-- Tomas Vondra
-- Simon Riggs
-- Nikhil Sontakke
-- Pallavi Sontakke
-- Amruta Deolasee
-- Rahila Syed
-- Ashutosh Bapat
-- Abhijit Menon-Sen
-- Florin Irion
-- Oliver Riggs
-
-Copyright © 2018-2020 2ndQuadrant Ltd
-Copyright © 2021 EnterpriseDB UK Ltd
-
-BDR is provided under EDB usage licenses.
-
-The reproduction of these documents is prohibited.
diff --git a/product_docs/docs/bdr/4.0/ddl.mdx b/product_docs/docs/bdr/4.0/ddl.mdx
index e860d943ede..64aeae2ca5e 100644
--- a/product_docs/docs/bdr/4.0/ddl.mdx
+++ b/product_docs/docs/bdr/4.0/ddl.mdx
@@ -639,7 +639,6 @@ DML lock:
- `ALTER TABLE ... ALTER COLUMN ... SET DEFAULT expression`
- `ALTER TABLE ... ALTER COLUMN ... DROP DEFAULT`
- `ALTER TABLE ... ALTER COLUMN ... TYPE` if it does not require rewrite
- (currently only available on EDB Postgres Extended and EDB Postgres Advanced)
- `ALTER TABLE ... ALTER COLUMN ... SET STATISTICS`
- `ALTER TABLE ... VALIDATE CONSTRAINT`
- `ALTER TABLE ... ATTACH PARTITION`
@@ -811,10 +810,6 @@ could lead to table rewrites lasting long durations.
Also note that the above implicit castable ALTER activity cannot be
performed in transaction blocks.
-!!! Note
- This currently only works on EDB Postgres Extended and EDB Postgres
- Advanced.
-
### ALTER TYPE
Users should note that `ALTER TYPE` is replicated but a Global DML lock is *not*
@@ -846,16 +841,8 @@ allowed on a BDR node.
### CREATE TABLE AS and SELECT INTO
-`CREATE TABLE AS` and `SELECT INTO` are only allowed on EDB Postgres Extended
-and EDB Postgres Advanced and only if any sub-commands are also allowed.
-
-You can instead achieve the same effect using, in case the
-`CREATE TABLE AS` is not supported on your variant of Postgres:
-
-```
-CREATE TABLE mytable;
-INSERT INTO mytable SELECT ... ;
-```
+`CREATE TABLE AS` and `SELECT INTO` are only allowed if all sub-commands are
+also allowed.
### EXPLAIN
@@ -972,8 +959,8 @@ locking.
#### Adding a CONSTRAINT
-Starting BDR 3.7.4, a CHECK and FOREIGN KEY constraint can be added
-without requiring a DML lock. This requires a 2-step process.
+CHECK and FOREIGN KEY constraint can be added without requiring a DML lock.
+This requires a 2-step process.
- `ALTER TABLE ... ADD CONSTRAINT ... NOT VALID`
- `ALTER TABLE ... VALIDATE CONSTRAINT`
@@ -994,10 +981,6 @@ Note that the new facility requires the cluster to run with Raft protocol
version 24 and beyond. If the Raft protocol is not yet upgraded, the old
mechanism will be used, resulting in a DML lock request.
-!!! Note
- This currently only works on EDB Postgres Extended and EDB Postgres
- Advanced.
-
#### Adding a Column
To add a column with a volatile default, run these commands in
diff --git a/product_docs/docs/bdr/4.0/feature-matrix.mdx b/product_docs/docs/bdr/4.0/feature-matrix.mdx
index c9cc2711f93..2cd059161c7 100644
--- a/product_docs/docs/bdr/4.0/feature-matrix.mdx
+++ b/product_docs/docs/bdr/4.0/feature-matrix.mdx
@@ -1,5 +1,5 @@
---
-title: 'Appendix B: Feature Compatibility'
+title: 'Appendix A: Feature Compatibility'
originalFilePath: feature-matrix.md
---
@@ -11,14 +11,15 @@ PostgreSQL, EDB Postgres Extended and EDB Postgres Advanced variants of Postgres
The following table lists features of BDR and whether they are supported by
given variant of Postgres and optionally from which version.
-| Feature | PostgreSQL | EDB Postgres Extended | EDB Postgres Advanced |
-| -------------------------------------- | ---------- | --------------------- | --------------------- |
-| Commit At Most Once (CAMO) | N | Y | 14+ |
-| Eager Replication | N | Y | 14+ |
-| Decoding Worker | N | 13+ | 14+ |
-| Assesment Tooling | N | Y | 14+ |
-| Lag Tracker | N | Y | 14+ |
-| Timestamp Snapshots | N | Y | 14+ |
-| Transaction Streaming | 14+ | 13+ | 14+ |
-| Missing Partition Conflict | N | Y | 14+ |
-| No UPDATE Trigger on tables with TOAST | N | Y | 14+ |
+| Feature | PostgreSQL | EDB Postgres Extended | EDB Postgres Advanced |
+| ----------------------------------------------- | ---------- | --------------------- | --------------------- |
+| Commit At Most Once (CAMO) | N | Y | 14+ |
+| Eager Replication | N | Y | 14+ |
+| Decoding Worker | N | 13+ | 14+ |
+| Assesment Tooling | N | Y | 14+ |
+| Lag Tracker | N | Y | 14+ |
+| Timestamp Snapshots | N | Y | 14+ |
+| Transaction Streaming | 14+ | 13+ | 14+ |
+| Missing Partition Conflict | N | Y | 14+ |
+| No need for UPDATE Trigger on tables with TOAST | N | Y | 14+ |
+| Automatically hold back FREEZE | N | Y | 14+ |
diff --git a/product_docs/docs/bdr/4.0/functions.mdx b/product_docs/docs/bdr/4.0/functions.mdx
index d276c4bdfc4..4d4c3ce899c 100644
--- a/product_docs/docs/bdr/4.0/functions.mdx
+++ b/product_docs/docs/bdr/4.0/functions.mdx
@@ -47,7 +47,7 @@ connected to. This allows an application to figure out what node it
is connected to even behind a transparent proxy.
It is also used in combination with CAMO, see the
-[Connection pools and proxies](camo.md#connection-pools-and-proxies)
+[Connection pools and proxies](CAMO.md#connection-pools-and-proxies)
section.
### bdr.last_committed_lsn
@@ -63,8 +63,166 @@ becomes remotely visible.
As soon as Postgres assigns a transaction id, this parameter is
updated to show the transaction id just assigned, if CAMO is enabled.
-!!! Note
- This is only available on EDB Postgres Extended.
+## Consensus Function
+
+### bdr.consensus_disable
+
+Disables the consensus worker on the local node until server restart or until
+it's re-enabled using `bdr.consensus_enable` (whichever happens first).
+
+!!! Warning
+ Disabling consensus will disable some features of BDR and eventually
+ will impact availability of the BDR cluster if left disabled for prolonged
+ periods of time. This function should only be used in coordination with
+ Technical Support.
+
+### bdr.consensus_enable
+
+Re-enabled disabled consensus worker on local node.
+
+### bdr.consensus_proto_version
+
+Returns currently used consensus protocol version by the local node.
+
+Needed by the BDR group reconfiguration internal mechanisms.
+
+### bdr.consensus_snapshot_export
+
+#### Synopsis
+
+```sql
+bdr.consensus_snapshot_export(version integer DEFAULT NULL)
+```
+
+Generate a new BDR consensus snapshot from the currently committed-and-applied
+state of the local node and return it as bytea.
+
+By default, a snapshot for the highest supported Raft version is
+exported. But that can be overridden by passing an explicit `version`
+number.
+
+The exporting node does not have to be the current Raft leader, nor does it
+need to be completely up to date with the latest state on the leader. However,
+such snapshot might not be accepted by `bdr.consensus_snapshot_import()`
+(see bellow).
+
+The new snapshot is not automatically stored to the local node's
+`bdr.local_consensus_snapshot` table. It's only returned to the caller.
+
+The generated snapshot may be passed to `bdr.consensus_snapshot_import()` on
+any other node(s) in the same BDR nodegroup that is behind the exporting node's
+raft log position.
+
+The local BDR consensus worker must be disabled for this function to work.
+Typical usage is:
+
+```
+ SELECT bdr.bdr_consensus_disable();
+ \copy (SELECT * FROM bdr.consensus_snapshot_export()) TO 'my_node_consensus_snapshot.data'
+ SELECT bdr.bdr_consensus_enable();
+```
+
+While the BDR consensus worker is disabled, DDL locking attempts on the node
+will fail or time out, galloc sequences will not get new values, Eager and CAMO
+transactions will pause or ERROR, and other functionality that needs the distributed
+consensus system will be disrupted. The required downtime is generally very brief.
+
+Depending on the use case, it may be practical to extract a snapshot that
+already exists from the `snapshot` field of the `bdr.local_consensus_snapshot`
+table and use that instead. Doing so does not require that the consensus worker
+be stopped.
+
+### bdr.consensus_snapshot_import
+
+#### Synopsis
+
+```sql
+bdr.consensus_snapshot_import(IN snapshot bytea)
+```
+
+Import a consensus snapshot which was exported by
+`bdr.consensus_snapshot_export()`, usually from another node in the same BDR
+nodegroup.
+
+It's also possible to use a snapshot extracted directly from the `snapshot`
+field of the `bdr.local_consensus_snapshot` table on another node.
+
+This function is useful for resetting a BDR node's catalog state to known good
+state in case of corruption or user mistake.
+
+The snapshot can be imported if the importing node's `apply_index` is less than
+or equal to the snapshot-exporting node's `commit_index` at the time the
+snapshot was generated. See `bdr.get_raft_status()`. A node that cannot accept
+the snapshot because its logs is already too far ahead will raise an `ERROR`
+and make no changes. The imported snapshot does not have to be completely
+up-to-date, as once the snapshot is imported the node will fetch the remaining
+changes from the current leader.
+
+The BDR consensus worker must be disabled on the importing node for this
+function to work. See notes on `bdr.consensus_snapshot_export()` for details.
+
+It's possible to use this to force the local node to generate a new Raft
+snapshot by running:
+
+```
+SELECT bdr.consensus_snapshot_import(bdr.consensus_snapshot_export());
+```
+
+This may also cause it to truncate its Raft logs up to the current
+applied log position.
+
+### bdr.consensus_snapshot_verify
+
+#### Synopsis
+
+```sql
+bdr.consensus_snapshot_verify(IN snapshot bytea)
+```
+
+Verify the given consensus snapshot which was exported by
+`bdr.consensus_snapshot_export()`. The snapshot header contains the
+version with which is was generated and the node will try to verify it
+against the same version.
+
+The snapshot may have been exported on the same node or any other node
+in the cluster. If the node verifying the snapshot does not support the
+version of the exported snapshot, then an error will be raised.
+
+### bdr.get_consensus_status
+
+Returns status information about the current consensus (Raft) worker.
+
+### bdr.get_raft_status
+
+Returns status information about the current consensus (Raft) worker.
+Alias for `bdr.get_consensus_status`.
+
+### bdr.raft_leadership_transfer
+
+#### Synopsis
+
+```sql
+bdr.raft_leadership_transfer(IN node_name text, IN wait_for_completion boolean)
+```
+
+Request the node identified by `node_name` to be the Raft leader. The
+request can be initiated from any of the BDR nodes and it will be
+internally forwarded to the current leader to transfer the leadership to
+the designated node. The designated node must be an ACTIVE BDR node and
+with full voting rights.
+
+If `wait_for_completion` is false, the request is served on
+best-efforts basis. If the node can't become a leader within
+`bdr.raft_election_timeout` period, then some other capable node will
+become the leader again. Also, the leadership can change over the
+period of time per Raft protocol. A true return result only indicates
+the request had been submitted successfully.
+
+If `wait_for_completion` is `true` then the function will wait until
+the given node becomes the new leader and possibly wait infinitely if
+the requested node fails to become Raft leader (e.g. due to network
+issues). It is therefore recommended to always set a `statement_timeout`
+in combination with `wait_for_completion` to prevent an infinite loop.
## Utility Functions
diff --git a/product_docs/docs/bdr/4.0/index.mdx b/product_docs/docs/bdr/4.0/index.mdx
index 33e65a499a4..020688e6742 100644
--- a/product_docs/docs/bdr/4.0/index.mdx
+++ b/product_docs/docs/bdr/4.0/index.mdx
@@ -2,6 +2,7 @@
navTitle: BDR
navigation:
- index
+ - release-notes
- overview
- appusage
- configuration
@@ -26,12 +27,10 @@ navigation:
- catalogs
- monitoring
- functions
- - credits
- - release-notes
- feature-matrix
- isolation_details
- known-issues
- - libraries
+ - licenses
- camo_clients
title: BDR (Bi-Directional Replication)
directoryDefaults:
@@ -61,12 +60,12 @@ standard PG extension. Full compatibility matrix is as follows.
| BDR Version | PostgreSQL | EDB Postgres Extended | EDB Postgres Advanced |
| ----------- | ---------- | --------------------- | --------------------- |
-| 4.0.0 | 14.1 | 14.1 | 14.1 |
-| 4.0.0 | 13.5 | 13.5r1.1.7 | 13.5 |
-| 4.0.0 | 12.9 | 12.9r1.1.9 | 12.9 |
-| 3.7.13.1 | 13.5 | 13.5r1.1.7 | 13.5 |
-| 3.7.13.1 | 12.9 | 12.9r1.1.9 | 12.9 |
-| 3.7.13.1 | 11.14 | 11.14r2.1.9 | 11.14 |
+| 4.0.1 | 14.1 | 14.1 | 14.1 |
+| 4.0.1 | 13.5 | 13.5r1.1.7 | 13.5 |
+| 4.0.1 | 12.9 | 12.9r1.1.9 | 12.9 |
+| 3.7.14 | 13.5 | 13.5r1.1.7 | 13.5 |
+| 3.7.14 | 12.9 | 12.9r1.1.9 | 12.9 |
+| 3.7.14 | 11.14 | 11.14r2.1.9 | 11.14 |
It is important to note that some key BDR features depend on certain core
capabilities being available within the targeted PostgresSQL database server.
diff --git a/product_docs/docs/bdr/4.0/isolation_details.mdx b/product_docs/docs/bdr/4.0/isolation_details.mdx
index 8da168d6af4..794a8bd5704 100644
--- a/product_docs/docs/bdr/4.0/isolation_details.mdx
+++ b/product_docs/docs/bdr/4.0/isolation_details.mdx
@@ -1,7 +1,7 @@
---
-title: 'Appendix C: Conflict Details'
+title: 'Appendix B: Conflict Details'
originalFilePath: isolation/details.md
---
-
+
diff --git a/product_docs/docs/bdr/4.0/known-issues.mdx b/product_docs/docs/bdr/4.0/known-issues.mdx
index d4e198b67a7..d11e2f72a9b 100644
--- a/product_docs/docs/bdr/4.0/known-issues.mdx
+++ b/product_docs/docs/bdr/4.0/known-issues.mdx
@@ -1,5 +1,5 @@
---
-title: 'Appendix D: Known Issues'
+title: 'Appendix C: Known Issues'
originalFilePath: known-issues.md
---
@@ -43,8 +43,9 @@ unique identifier.
starting value for the sequence as an argument to the
`bdr.alter_sequence_set_kind()` function.
-- Upgrades on PG Extended 13 from BDR 3.7.7 are only supported by adding
- new nodes, and **not** through in-place upgrade of the same data directory.
+- Upgrades from BDR 3.7 are only supported by adding new nodes,
+ and **not** through in-place upgrade of the same data directory.
+ This is planned to be resolved in a future 4.0 minor release.
- The `bdr.monitor_local_replslots()` function may return CRITICAL result
saying "There is at least 1 BDR replication slot which is missing" even if
diff --git a/product_docs/docs/bdr/4.0/libraries.mdx b/product_docs/docs/bdr/4.0/licenses.mdx
similarity index 99%
rename from product_docs/docs/bdr/4.0/libraries.mdx
rename to product_docs/docs/bdr/4.0/licenses.mdx
index 56fb687979e..dd98c1f85f1 100644
--- a/product_docs/docs/bdr/4.0/libraries.mdx
+++ b/product_docs/docs/bdr/4.0/licenses.mdx
@@ -1,6 +1,6 @@
---
-title: 'Appendix E: Libraries'
-originalFilePath: libraries.md
+title: 'Appendix D: Licenses'
+originalFilePath: licenses.md
---
diff --git a/product_docs/docs/bdr/4.0/nodes.mdx b/product_docs/docs/bdr/4.0/nodes.mdx
index cef4ea1cd12..9f1d60832a1 100644
--- a/product_docs/docs/bdr/4.0/nodes.mdx
+++ b/product_docs/docs/bdr/4.0/nodes.mdx
@@ -243,9 +243,8 @@ a failover or switchover occurs during this interval, the
streaming standby cannot be promoted to replace its BDR node, as the
group slot and other dependent slots do not exist yet.
-On EDB Postgres Extended and EDB Postgres Advaced, this is resolved
-automatically. The slot sync-up process on the standby solves this by
-invoking a function on the upstream. This function moves the group slot in the
+The slot sync-up process on the standby solves this by invoking a function
+on the upstream. This function moves the group slot in the
entire BDR cluster by performing WAL switches and requesting all BDR
peer nodes to replay their progress updates. The above causes the
group slot to move ahead in a short timespan. This reduces the time
@@ -548,8 +547,6 @@ sub-segment. LCR files are binary and variable sized. The maximum size of an
LCR file can be controlled by `bdr.max_lcr_segment_file_size`, which
defaults to 1GB.
-EDB Postgres Extended 13 and above is required for this feature to work.
-
## Node Restart and Down Node Recovery
BDR is designed to recover from node restart or node disconnection.
@@ -587,8 +584,8 @@ PANIC: could not write to file "pg_wal/xlogtemp.559": No space left on device
In addition, slots for offline nodes also hold back the catalog xmin, preventing
vacuuming of catalog tables.
-On EDB Postgres Extended, offline nodes also hold back freezing of data to
-prevent losing conflict resolution data
+On EDB Postgres Extended Server and EDB Postgres Advanced Server, offline nodes
+also hold back freezing of data to prevent losing conflict resolution data
(see: [Origin Conflict Detection](conflicts)).
Administrators should monitor for node outages (see: [monitoring](monitoring))
@@ -658,8 +655,8 @@ The group slot can:
replicating again
- part nodes from cluster consistently, even if some nodes have not
caught up fully with the parted node
-- hold back the freeze point to avoid missing some conflicts (EDB Postgres Extended)
-- keep the historical snapshot for timestamp based snapshots (EDB Postgres Extended)
+- hold back the freeze point to avoid missing some conflicts
+- keep the historical snapshot for timestamp based snapshots
The group slot is usually inactive, and is only fast-forwarded periodically
in response to Raft progress messages from other nodes.
@@ -694,7 +691,7 @@ of `db20xxxxxxxxxxxxxxxx`, `group20xxxxxxxxxxxxx` and
## Removing a Node From a BDR Group
-Since BDR is designed to recover from extended node outages, you
+Since BDR is designed to recover from extended node outages, you
must explicitly tell the system if you are removing a node
permanently. If you permanently shut down a node and do not tell
the other nodes, then performance will suffer, and eventually
@@ -1145,11 +1142,15 @@ node is PARTED it can not *part* other nodes in the cluster.
!!! Note
If you are *parting* the local node you must set `wait_for_completion`
to false, otherwise it will error.
+ you are *parting* the local node you must set `wait_for_completion`
+ false, otherwise it will error.
!!! Warning
This action is permanent. If you wish to temporarily halt replication
to a node, see `bdr.alter_subscription_disable()`.
-
+ s action is permanent. If you wish to temporarily halt replication
+ a node, see `bdr.alter_subscription_disable()`.
+
#### Synopsis
```sql
@@ -1306,11 +1307,6 @@ The user must specify a data directory. If this data directory is empty,
the `pg_basebackup -X stream` command is used to fill the directory
using a fast block-level copy operation.
-When starting from an empty data directory, if the selective backup option
-is chosen, then only that database will be copied from the source node. The
-excluded databases will be dropped and cleaned up on the new node
-(EDB Postgres Extended).
-
If the specified data directory is non-empty, this will be used as the
base for the new node. If the data directory is already active as a
physical standby node, it is required to stop the standby before running
@@ -1348,7 +1344,8 @@ bdr_init_physical [OPTION] ...
- `-s, --stop` - Stop the server once the initialization is done.
- `-v` - Increase logging verbosity.
- `-L` - Perform selective pg_basebackup when used in conjunction with an
- empty/non-existing data directory (-D option). (EDB Postgres Extended)
+ empty/non-existing data directory (-D option). This is feature of
+ EDB Postgres Extended Server only.
- `-S` - Instead of dropping logical replication subscriptions, just disable
them.
diff --git a/product_docs/docs/bdr/4.0/release-notes.mdx b/product_docs/docs/bdr/4.0/release-notes.mdx
index 49b5fd59ff9..23330b7b0e3 100644
--- a/product_docs/docs/bdr/4.0/release-notes.mdx
+++ b/product_docs/docs/bdr/4.0/release-notes.mdx
@@ -1,9 +1,111 @@
---
-title: 'Appendix A: Release Notes'
+title: Release Notes
originalFilePath: release-notes.md
---
+## BDR 4.0.1
+
+This is a maintenance release for BDR 4.0 which includes minor
+improvements as well as fixes for issues identified in previous
+versions.
+
+### Improvements
+
+- Reduce frequency of CAMO partner connection attempts
+ In case of a failure to connect to a CAMO partner to verify its
+ configuration and check the status of transactions, do not retry
+ immediately (leading to a fully busy pglogical manager process), but
+ throttle down repeated attempts to reconnect and checks to once per
+ minute.
+
+- Implement buffered read for LCR segment file (BDR-1422)
+ Implement LCR segment file buffering so that multiple LCR chunks
+ can be read at a time. This should reduce
+ I/O and improve CPU usage of Wal Senders when using the Decoding Worker.
+
+- Avoid unnecessary LCR segment reads (BDR-1426)
+ BDR now attempts to only read new LCR segments when there is at
+ least one available. This reduces I/O load when Decoding Worker is
+ enabled.
+
+- Performance of COPY replication including the initial COPY during join has
+ been greatly improved for partitioned tables (BDR-1479)
+ For large tables this can improve the load times by order of magnitude or
+ more.
+
+### Resolved Issues
+
+- Fix the parallel apply worker selection (BDR-1761)
+ This makes parallel apply work again. In 4.0.0 parallel apply was never
+ in effect due to this bug.
+
+- Fix Raft snapshot handling of `bdr.camo_pairs` (BDR-1753)
+ The previous release would not correctly propagate changes to the CAMO pair
+ configuration when they were received via Raft snapshot.
+
+- Correctly handle Raft snapshots from BDR 3.7 after upgrades (BDR-1754)
+ Upgrading a CAMO configured cluster taking into account the
+ `bdr.camo_pairs` in the snapshot while still excluding the ability
+ to perform in place upgrade of a cluster (due to upgrade
+ limitations unrelated to CAMO).
+
+- Switch from CAMO to Local Mode only after timeouts (RT74892)
+ Do not use the `catchup_interval` estimate when switching from CAMO
+ protected to Local Mode, as that could induce inadvertent switching
+ due to load spikes. Use the estimate only when switching from Local
+ Mode back to CAMO protected (to prevent toggling forth and back due
+ to lag on the CAMO partner).
+
+- Fix replication set cache invalidation when published replication set list
+ have changed (BDR-1715)
+ In previous versions we could use stale information about which replication
+ sets (and as a result which tables) should be published until the subscription
+ has reconnected.
+
+- Prevent duplicate values generated locally by galloc sequence in high
+ concurrency situations when the new chunk is used (RT76528)
+ The galloc sequence could have temporarily produce duplicate value when
+ switching which chunk is used locally (but not across nodes) if there were
+ multiple sessions waiting for the new value. This is now fixed.
+
+- Address memory leak on streaming transactions (BDR-1479)
+ For large transaction this reduces memory usage and I/O considerably when using
+ the streaming transactions feature. This primarily improves performance of
+ COPY replication.
+
+- Don't leave slot behind after PART_CATCHUP phase of node parting when the
+ catchup source has changed while the node was parting (BDR-1716)
+ When node is being removed (parted) from BDR group, we do so called catchup
+ in order to forward any missing changes from that node between remaining nodes
+ in order to keep the data on all nodes consistent. This requires an additional
+ replication slot to be created temporarily. Normally this replication slot
+ is removed at the end of the catchup phase, however in certain scenarios
+ where we have to change the source node for the changes, this slot could
+ have previously been left behind. From this version, this slot is always
+ correctly removed.
+
+- Ensure that the group slot is moved forward when there is only one node in
+ the BDR group
+ This prevents disk exhaustion due to WAL accumulation when the group is left
+ running with just single BDR node for a prolonged period of time. This is not
+ recommended setup but the WAL accumulation was not intentional.
+
+- Advance Raft protocol version when there is only one node in the BDR group
+ Single node clusters would otherwise always stay on oldest support protocol
+ until another node was added. This could limit available feature set on that
+ single node.
+
+### Upgrades
+
+This release supports upgrading from the following versions of BDR:
+
+- 3.7.14
+- 4.0.0 and higher
+
+Please make sure you read and understand the process and limitations described
+in the [Upgrade Guide](upgrades) before upgrading.
+
## BDR 4.0.0
BDR 4.0 is a new major version of BDR and adopted with this release number is
diff --git a/product_docs/docs/bdr/4.0/repsets.mdx b/product_docs/docs/bdr/4.0/repsets.mdx
index 44dc54d5d13..1a7e8549af5 100644
--- a/product_docs/docs/bdr/4.0/repsets.mdx
+++ b/product_docs/docs/bdr/4.0/repsets.mdx
@@ -278,11 +278,11 @@ transaction.
another node, because this will stop replication on that
node. Should this happen, please unsubscribe the affected node
from that replication set.
- For the same reason, you should not drop a replication set if
+ the same reason, you should not drop a replication set if
there is a join operation in progress, and the node being joined
is a member of that replication set; replication set membership is
only checked at the beginning of the join.
- This happens because the information on replication set usage is
+ s happens because the information on replication set usage is
local to each node, so that it can be configured on a node before
it joins the group.
diff --git a/product_docs/docs/bdr/4.0/scaling.mdx b/product_docs/docs/bdr/4.0/scaling.mdx
index 9065a78b5a2..d3b19c85c2f 100644
--- a/product_docs/docs/bdr/4.0/scaling.mdx
+++ b/product_docs/docs/bdr/4.0/scaling.mdx
@@ -24,9 +24,9 @@ otherwise later executions will alter the definition.
`bdr.autopartition()` does not lock the actual table, it only changes the
definition of when and how new partition maintenance actions will take place.
-`bdr.autopartition()` leverages the EDB Postgres Extended features to allow a
-partition to be attached or detached/dropped without locking the rest of the
-table, the feature to set a new tablespace while allowing SELECT queries.
+`bdr.autopartition()` leverages the features that allow a partition to be
+attached or detached/dropped without locking the rest of the table
+(when the underlaying Postgres version supports it).
An ERROR is raised if the table is not RANGE partitioned or a multi-column
partition key is used.
diff --git a/product_docs/docs/bdr/4.0/striggers.mdx b/product_docs/docs/bdr/4.0/striggers.mdx
index d9bd62fe75e..1df60531a0e 100644
--- a/product_docs/docs/bdr/4.0/striggers.mdx
+++ b/product_docs/docs/bdr/4.0/striggers.mdx
@@ -12,10 +12,6 @@ data processing on the downstream/target node.
Together, these types of triggers are known as Stream Triggers.
-!!! Note
- This feature is currently only available on EDB Postgres Extended and
- EDB Postgres Advanced.
-
Stream Triggers are designed to be trigger-like in syntax, they leverage the
PostgreSQL BEFORE trigger architecture, and are likely to have similar
performance characteristics as PostgreSQL BEFORE Triggers.
diff --git a/product_docs/docs/bdr/4.0/transaction-streaming.mdx b/product_docs/docs/bdr/4.0/transaction-streaming.mdx
index dc2f6e8ab06..2f4b0e49f9c 100644
--- a/product_docs/docs/bdr/4.0/transaction-streaming.mdx
+++ b/product_docs/docs/bdr/4.0/transaction-streaming.mdx
@@ -19,9 +19,6 @@ the transaction commits (or discard them if the transaction aborts). This makes
it possible to apply transactions on on subscribers as soon as the transaction
commits.
-This functionality is available from PostgreSQL 14 (including EPAS 14 and
-EDB Postgres Extended 14) and has been back-ported to EDB Postgres Extended 13.
-
## BDR enhancements
PostgreSQL's built-in transaction streaming, as described above, has the following
@@ -47,7 +44,11 @@ processes on each subscriber, which is leveraged to provide the following enhanc
frequent deadlocks between writers
!!! Note
- Direct streaming to writer is still an experimental feature and must be used with caution. For specifically, it may not work well with conflict resolutions since the commit timestamp of the streaming may not be available (as the transaction may not have yet committed on the origin).
+ Direct streaming to writer is still an experimental feature and must
+ be used with caution. For specifically, it may not work well with
+ conflict resolutions since the commit timestamp of the streaming may not
+ be available (as the transaction may not have yet committed on the
+ origin).
## Configuration
diff --git a/product_docs/docs/bdr/4.0/tssnapshots.mdx b/product_docs/docs/bdr/4.0/tssnapshots.mdx
index e123f8d7487..8dec9fa701d 100644
--- a/product_docs/docs/bdr/4.0/tssnapshots.mdx
+++ b/product_docs/docs/bdr/4.0/tssnapshots.mdx
@@ -4,16 +4,11 @@ originalFilePath: tssnapshots.md
---
-The Timestamp-Based Snapshots feature of PG Extended allows reading data in
-a consistent manner via a user-specified timestamp rather than the usual
-MVCC snapshot. This can be used to access data on different BDR nodes
-at a common point-in-time; for example, as a way to compare data on
-multiple nodes for data quality checking. At this time, this feature does
-not work with write transactions.
-
-!!! Note
- This feature is currently only available on EDB Postgres Extended and
- EDB Postgres Advanced.
+The Timestamp-Based Snapshots allow reading data in a consistent manner via
+a user-specified timestamp rather than the usual MVCC snapshot. This can be
+used to access data on different BDR nodes at a common point-in-time; for
+example, as a way to compare data on multiple nodes for data quality checking.
+At this time, this feature does not work with write transactions.
The use of timestamp-based snapshots are enabled via the `snapshot_timestamp`
parameter; this accepts either a timestamp value or
@@ -29,9 +24,8 @@ SET snapshot_timestamp = '2018-12-08 02:28:30 GMT';
SELECT count(*) FROM customers;
```
-In plain PG Extended, this only works with future timestamps or the above
-mentioned special 'current' value, so it cannot be used for historical queries
-(though that is on the longer-term roadmap).
+Without BDR, this only works with future timestamps or the above mentioned
+special 'current' value, so it cannot be used for historical queries.
BDR works with and improves on that feature in a multi-node environment. Firstly,
BDR will make sure that all connections to other nodes replicated any
diff --git a/product_docs/docs/bdr/4.0/upgrades.mdx b/product_docs/docs/bdr/4.0/upgrades.mdx
index 2d522a46bb3..be569df503e 100644
--- a/product_docs/docs/bdr/4.0/upgrades.mdx
+++ b/product_docs/docs/bdr/4.0/upgrades.mdx
@@ -146,16 +146,14 @@ Configuration of CAMO pairs has changed significantly compared to BDR
BDR system catalog `bdr.camo_pairs`. To upgrade a BDR cluster with
CAMO pairs from 3.7 to 4.0, the following steps need to be performed:
-- Eliminate `bdr.camo_partner_of` and `bdr.camo_origin_for`
- configuration on all nodes.
-- Restart all nodes affected by this change (still using the existing
- BDR version), one at a time. This will temporarily disable CAMO
- and attempting to run transactions with `bdr.enable_camo` set will
- result in warnings.
-- Upgrade the entire BDR cluster, either all nodes at once or using a
- rolling upgrade.
-- Re-configure CAMO via `bdr.add_camo_pair`. This function will
- only work after all nodes in the BDR cluster are upgraded.
+- Add new upgraded nodes to the cluster in a rolling upgrade fashion.
+- Configure the new CAMO pairs with the `bdr.add_camo_pair()` function.
+- Switch the application to the new nodes.
+- Drop the old nodes.
+
+!!!Note
+ `bdr.camo_partner_of` and `bdr.camo_origin_for` configurations are no
+ longer needed in BDR 4.0.
## Upgrade Preparation
@@ -224,6 +222,7 @@ prefix instead.
The upgrade of BDR software on individual nodes happens in-place. There is no
need for backup and restore when upgrading the BDR extension.
+!!!
!!! Warning
This method cannot be currently used for upgrading BDR 3.7 to 4.0. Only