Merge pull request #1878 from EnterpriseDB/release/2021-09-24

Release: 2021-09-24

.gitignore

@@ -82,6 +82,7 @@ product_docs/content_build/
 static/nginx_redirects.generated
 temp_kubernetes/
 temp_bdr/
+temp_pglogical3/
 
 # Track base direnv file
 !.envrc

build-sources.json

@@ -17,6 +17,7 @@
   "odbc_connector": true,
   "pem": true,
   "pgbouncer": true,
+  "pglogical": true,
   "pgpool": true,
   "postgis": true,
   "repmgr": true,

gatsby-config.js

@@ -62,6 +62,7 @@ const sourceToPluginConfig = {
   },
   pem: { name: "pem", path: "product_docs/docs/pem" },
   pgbouncer: { name: "pgbouncer", path: "product_docs/docs/pgbouncer" },
+  pglogical: { name: "pglogical", path: "product_docs/docs/pglogical" },
   pgpool: { name: "pgpool", path: "product_docs/docs/pgpool" },
   postgis: { name: "postgis", path: "product_docs/docs/postgis" },
   repmgr: { name: "repmgr", path: "product_docs/docs/repmgr" },

product_docs/docs/bdr/3.7/appusage.mdx

@@ -329,7 +329,8 @@ between them to read stale data.
 A [queue wait function](functions#bdrwait_for_apply_queue) is
 provided for clients or proxies to prevent such stale reads.
 
-In addition, BDR provides multiple variants for more synchronous
+The synchronous replication features of Postgres are available to BDR
+as well. In addition, BDR provides multiple variants for more synchronous
 replication.  Please refer to the
 [Durability & Performance Options](durability)
 chapter for an overview and comparison of all variants available and
@@ -626,7 +627,7 @@ BDR cannot currently perform conflict resolution where the PRIMARY KEY is change
 by an UPDATE operation. It is permissible to update the primary key, but you must
 ensure that no conflict with existing values is possible.
 
-BDR-EE provides the following configuration
+When running on EDB Postgres Extended, BDR provides the following configuration
 parameter to assess how frequently the primary key/replica identity of any table
 is being subjected to update operations.
 
@@ -661,8 +662,8 @@ Because BDR writer processes operate much like normal user sessions, they are su
 the usual rules around row and table locking. This can sometimes lead to BDR writer
 processes waiting on locks held by user transactions, or even by each other.
 
-BDR-EE provides the following configuration parameter
-to assess if the application is taking explicit locks.
+When running on EDB Postgres Extended, BDR provides the following
+configuration parameter to assess if the application is taking explicit locks.
 
 ```sql
 bdr.assess_lock_statement = IGNORE (default) | LOG | WARNING | ERROR

product_docs/docs/bdr/3.7/column-level-conflicts.mdx

@@ -60,6 +60,10 @@ Column-level conflict resolution requires the table to have `REPLICA
 IDENTITY FULL`. The `bdr.alter_table_conflict_detection` function does check
 that, and will fail with an error otherwise.
 
+!!! Note
+    This feature is currently only available on EDB Postgres Extended and
+    EDB Postgres Advanced.
+
 ## Enabling and Disabling Column-Level Conflict Resolution
 
 The Column-Level Conflict Resolution is managed by the

product_docs/docs/bdr/3.7/configuration.mdx

@@ -25,7 +25,6 @@ which vary according to the size and scale of the cluster.
 -   `logical_decoding_work_mem` - memory buffer size used by logical decoding.
     Transactions larger than this will overflow the buffer and be stored
     temporarily on local disk. Default 64MB, but can be set much higher.
-
 -   `max_worker_processes` - BDR uses background workers for replication
     and maintenance tasks, so there need to be enough worker slots for it to
     work correctly. The formula for the correct minimal number of workers is:
@@ -34,28 +33,25 @@ which vary according to the size and scale of the cluster.
     writer enabled per peer node in the BDR group, for each database.
     Additional worker processes may be needed temporarily when node is being
     removed from a BDR group.
-
 -   `max_wal_senders` - Two needed per every peer node.
-
 -   `max_replication_slots` - Same as `max_wal_senders`.
-
 -   `wal_sender_timeout` and `wal_receiver_timeout` - Determine how
     quickly an origin considers its CAMO partner as disconnected or
     reconnected; see [CAMO Failure Scenarios](camo#failure-scenarios) for
     details.
 
 Note that in normal running for a group with N peer nodes, BDR will require
-N slots/walsenders. During synchronization, BDR will temporarily use another
-N - 1 slots/walsenders, so be careful to set the above parameters high enough
+N slots and WAL senders. During synchronization, BDR will temporarily use another
+N - 1 slots and WAL senders, so be careful to set the above parameters high enough
 to cater for this occasional peak demand.
 
 With parallel apply turned on, the number of slots needs to be increased to
 N slots from above formula \* writers. This is because the `max_replication_slots`
 also sets maximum number of replication origins and some of the functionality
 of parallel apply uses extra origin per writer.
 
-When WAL Decoder is enabled, the WAL decoder process will require one extra 
-replication slot per BDR group.
+When the [Decoding Worker](nodes#decoding-worker) is enabled, this
+process will require one extra replication slot per BDR group.
 
 The general safe recommended value on a 4 node BDR Group with a single database
 is just to set `max_replication_slots` and `max_worker_processes` to something
@@ -71,7 +67,7 @@ Applications may also wish to set these parameters. Please see chapter on
     in a similar way to [physical replication](https://www.postgresql.org/docs/11/runtime-config-wal.html#GUC-SYNCHRONOUS-COMMIT).
 -   `synchronous_standby_names` - same as above
 
-## 2ndQPostgres Settings for BDR
+## 2ndQPostgres/EDB Postgres Extended Settings for BDR
 
 The following Postgres settings need to be considered for commit at
 most once (CAMO), a feature that is only available for BDR in
@@ -304,6 +300,7 @@ Unless noted otherwise, values may be set by any user at any time.
     local two-phase commit or CAMO transaction, and will prevent all
     Eager transactions on the cluster.
     May only be set at Postgres server start.
+    (EDB Postgres Extended)
 
 ### Eager Replication
 
@@ -315,6 +312,9 @@ Unless noted otherwise, values may be set by any user at any time.
     in their commit phase, as a limit for how long to wait for the CAMO
     partner.
 
+!!! Note
+    This is only available on EDB Postgres Extended.
+
 ### Commit at Most Once
 
 -   `bdr.enable_camo` - Used to enable and control the CAMO feature.
@@ -345,11 +345,15 @@ Unless noted otherwise, values may be set by any user at any time.
     guaranteed. This is enabled by default. Well-informed users can choose
     to disable this to reduce the amount of warnings going into their logs.
 
+!!! Note
+    This is only available on EDB Postgres Extended.
+
 ### Timestamp-based Snapshots
 
 -   `bdr.timestamp_snapshot_keep` - For how long to keep valid snapshots for the
     timestamp-based snapshot usage (default 0, meaning do not keep past snapshots).
     Also see `snapshot_timestamp` above.
+    (EDB Postgres Extended)
 
 ### Monitoring and Logging
 
@@ -374,38 +378,38 @@ Unless noted otherwise, values may be set by any user at any time.
     Raft log when doing log compaction (default 100). The value of 0 will disable
     log compaction. **WARNING: If log compaction is disabled, the log will
     grow in size forever.** May only be set at Postgres server start.
-
 -   `bdr.raft_response_timeout` - To account for network failures, the
     Raft consensus protocol implemented will time out requests after a
     certain amount of time.  This timeout defaults to 30 seconds.
-
 -   `bdr.raft_log_min_apply_duration` - To move the state machine
     forward, Raft appends entries to its internal log.  During normal
     operation, appending takes only a few milliseconds.  This poses an
     upper threshold on the duration of that append action, above which
     an `INFO` message is logged.  This may indicate an
     actual problem. Default value of this parameter is 3000 ms.
-
 -   `bdr.raft_log_min_message_duration` - When to log a consensus request.
      Measure round trip time of a bdr consensus request and log an
      `INFO` message if the time exceeds this parameter. Default value
      of this parameter is 5000 ms.
-
+-   `bdr.raft_group_max_connections` - The maximum number of connections
+     across all BDR groups for a Postgres server. These connections carry
+     bdr consensus requests between the groups' nodes. Default value of this
+     parameter is 100 connections. May only be set at Postgres server start.  
 -   `bdr.backwards_compatibility` - Specifies the version to be
     backwards-compatible to, in the same numerical format as used by
     `bdr.bdr_version_num`, e.g. `30618`.  Enables exact behavior of a
     former BDR version, even if this has generally unwanted effects.
     Defaults to the current BDR version. Since this changes from release
     to release, we advise against explicit use within the configuration
     file unless the value is different to the current version.
-
 -   `bdr.track_replication_estimates` - Track replication estimates in terms
     of apply rates and catchup intervals for peer nodes.  This information can
     be used by protocols like CAMO to estimate the readiness of a
     peer node. This parameter is enabled by default.
-
+    (EDB Postgres Extended)
 -   `bdr.lag_tracker_apply_rate_weight` - We monitor how far behind peer nodes
     are in terms of applying WAL from the local node, and calculate a moving
     average of the apply rates for the lag tracking. This parameter specifies
     how much contribution newer calculated values have in this moving average
     calculation. Default value is 0.1.
+    (EDB Postgres Extended)

product_docs/docs/bdr/3.7/conflicts.mdx

@@ -23,14 +23,12 @@ This chapter covers row-level conflicts with standard data types in detail.
 
 Conflict handling is configurable, as described later in this chapter.
 Conflicts can be detected and handled differently for each table using
+conflict triggers, described in the [Stream Triggers](striggers) chapter.
 
-conflict triggers, available with BDR-EE,
-described in the [Stream Triggers](striggers) chapter.
-
-Column-level conflict detection and resolution is available with BDR-EE,
+Column-level conflict detection and resolution is available with BDR,
 described in the [CLCD](column-level-conflicts) chapter.
 
-If you wish to avoid conflicts, you can use these features in BDR-EE
+If you wish to avoid conflicts, you can use these features in BDR.
 
 -   Conflict-free data types (CRDTs) - described in the [CRDT](crdt) chapter.
 -   Eager replication - described in the [Eager Replication](eager) chapter.
@@ -56,8 +54,6 @@ some types of conflict to occur and resolve them when they arise.
     groups that contain at least one 3.7+ node.
     Please use the ones in `bdr` schema that are already present in all BDR versions.
 
-.
-
 ## How conflicts happen
 
 Inter-node conflicts arise as a result of sequences of events that could not
@@ -120,8 +116,8 @@ default resolved by choosing the newer (based on commit time) row and keeping
 only that one (`update_if_newer` resolver). Other resolvers can be configured -
 see [Conflict Resolution] for details.
 
-To resolve this conflict, type in the Enterprise Edition; you can also use
-column-level conflict resolution and user-defined conflict triggers.
+To resolve this conflict type, you can also use column-level conflict
+resolution and user-defined conflict triggers.
 
 This type of conflict can be effectively eliminated by use of
 [Global Sequences](sequences).
@@ -145,6 +141,8 @@ preserve the row with the correct `PRIMARY KEY` and delete the others.
     resolution is to proceed with the insert operation, some of the data will
     always be deleted!
 
+It's also possible to define a different behaviour using a conflict trigger.
+
 #### UPDATE/UPDATE Conflicts
 
 Where two concurrent `UPDATE`s on different nodes change the same tuple
@@ -410,7 +408,6 @@ these conflicts. Note however that enabling this option opens the door for
 
 If these are problems, it's recommended to tune freezing settings for a table
 or database so that they are correctly detected as `update_recently_deleted`.
-This is done automatically in BDR Enterprise Edition.
 
 Another alternative is to use [Eager Replication] to prevent these conflicts.
 
@@ -709,6 +706,16 @@ as is normally the case with BDR AlwaysOn architecture.
     such a statement, the protection against these concurrency issues will not
     be present.
 
+!!! Warning
+    The additional WAL logging of TOAST is done using the `BEFORE UPDATE`
+    trigger. This trigger must be sorted alphabetically
+    last (based on trigger name) among all `BEFORE UPDATE` triggers on the
+    table. It's prefixed with `zzzz_bdr_` to make this easier, but make sure
+    you don't create any trigger with name that would sort after it, otherwise
+    the protection against the concurrency issues will not be present.
+    This trigger is not created or used when using BDR with EDB Postgres
+    Extended.
+
 For the `insert_or_error` conflict resolution, the use of
 `REPLICA IDENTITY FULL` is however still required.
 
@@ -770,13 +777,8 @@ mechanisms to cope with the conflict.
 BDR provides these mechanisms for conflict detection:
 
 -   [Origin Conflict Detection] \(default)
-
 -   [Row Version Conflict Detection]
-
 -   [Column-Level Conflict Detection](column-level-conflicts)
-    .
-
-as well as other mechanisms when using BDR-EE.
 
 ### Origin Conflict Detection
 
@@ -811,11 +813,51 @@ frozen rows that they update. Inserts and Deletes are not affected by this situa
 Users are advised to not leave down nodes for extended outages,
 as discussed in [Node Restart and Down Node Recovery](nodes).
 
-To handle this situation gracefully, BDR-EE will automatically hold
-back the freezing of rows while a node is down.
+On EDB Postgres Extended, BDR will automatically hold back the freezing of
+rows while a node is down to handle this situation gracefully without the need
+for changing parameter settings.
+
+On other variants of Postgres, users may need to manage this situation with
+some care:
+
+Freezing normally occurs when a row being vacuumed is older than
+`vacuum_freeze_min_age` xids from the current xid, which means that you
+need to configure suitably high values for these parameters:
+
+-   vacuum_freeze_min_age
+-   vacuum_freeze_table_age
+-   autovacuum_freeze_max_age
 
-No changes to parameter settings are required.
-.
+Values should be chosen based upon the transaction rate, giving
+a grace period of downtime before any conflict data is removed
+from the database server. For example, a node performing
+1000 TPS could be down for just over 5.5 days before conflict
+data is removed, when vacuum_freeze_min_age is set to 500 million.
+The CommitTS datastructure will take on-disk space of 5 GB with
+that setting, so lower transaction rate systems may benefit from
+lower settings.
+
+Initially recommended settings would be:
+
+```sql
+# 1 billion = 10GB
+autovacuum_freeze_max_age = 1000000000
+
+vacuum_freeze_min_age = 500000000
+
+# 90% of autovacuum_freeze_max_age
+vacuum_freeze_table_age = 900000000
+```
+
+Note that:
+
+-   autovacuum_freeze_max_age can only be set at server start.
+-   vacuum_freeze_min_age is user-settable, so using a
+    low value will freeze rows early and could result in conflicts being
+    ignored. autovacuum_freeze_min_age and toast.autovacuum_freeze_min_age
+    can also be set for individual tables.
+-   running the CLUSTER or VACUUM FREEZE commands will also
+    freeze rows early and could result in conflicts being ignored.
 
 ### Row Version Conflict Detection
 
@@ -877,13 +919,10 @@ The recognized methods for conflict detection are:
 -   `row_origin` - origin of the previous change made on the tuple (see
     [Origin Conflict Detection] above). This is the only method supported which
     does not require an extra column in the table.
-
 -   `row_version` - row version column (see [Row Version Conflict Detection]
     above).
-
 -   `column_commit_timestamp` - per-column commit timestamps (described in the
     [CLCD](column-level-conflicts) chapter).
-
 -   `column_modify_timestamp` - per-column modification timestamp (described in
     the [CLCD](column-level-conflicts) chapter).
 

product_docs/docs/bdr/3.7/crdt.mdx

@@ -163,6 +163,10 @@ new data types. CRDT types are handled transparently - both `ANALYZE` and
 the optimizer work, so estimation and query planning works fine, without
 having to do anything else.
 
+!!! Note
+    This feature is currently only available on EDB Postgres Extended and
+    EDB Postgres Advanced.
+
 ## State-based and operation-based CRDTs
 
 Following the notation from [1], we do implement both operation-based