From d97e1801525fb350fbb738eb9c0f7fd730c5a0a5 Mon Sep 17 00:00:00 2001 From: Betsy Gitelman <93718720+ebgitelman@users.noreply.github.com> Date: Thu, 16 Jun 2022 14:40:37 -0400 Subject: [PATCH 1/3] edits to three topics --- product_docs/docs/bdr/4/repsets.mdx | 433 +++++++++++++-------------- product_docs/docs/bdr/4/scaling.mdx | 192 ++++++------ product_docs/docs/bdr/4/security.mdx | 202 +++++++------ 3 files changed, 409 insertions(+), 418 deletions(-) diff --git a/product_docs/docs/bdr/4/repsets.mdx b/product_docs/docs/bdr/4/repsets.mdx index d608d80b39a..3e51c74036d 100644 --- a/product_docs/docs/bdr/4/repsets.mdx +++ b/product_docs/docs/bdr/4/repsets.mdx @@ -1,92 +1,92 @@ --- -title: Replication Sets +title: Replication sets --- -A replication set is a group of tables which can be subscribed to by a BDR node. -Replication sets can be used to create more complex replication topologies -than regular symmetric multi-master where each node is exact copy of the other +A replication set is a group of tables that a BDR node can subscribe to. +You can use replication sets to create more complex replication topologies +than regular symmetric multi-master where each node is an exact copy of the other nodes. -Every BDR group automatically creates a replication set with the same name as -the group itself. This replication set is the default replication set, which is -used for all user tables and DDL replication and all nodes are subscribed to it. +Every BDR group creates a replication set with the same name as +the group. This replication set is the default replication set, which is +used for all user tables and DDL replication. All nodes are subscribed to it. In other words, by default all user tables are replicated between all nodes. -## Using Replication Sets +## Using replication sets -Additional replication sets can be created using `create_replication_set()`, -specifying whether to include insert, update, delete or truncate actions. -An option exists to add existing tables to the set automatically, and -a second option defines whether to add tables automatically when they are +You can create replication sets using `create_replication_set()`, +specifying whether to include insert, update, delete, or truncate actions. +One option lets you add existing tables to the set, and +a second option defines whether to add tables when they are created. -You may also define manually which tables are added or removed from a +You can also manually define the tables to add or remove from a replication set. -Tables included in the replication set will be maintained when the node +Tables included in the replication set are maintained when the node joins the cluster and afterwards. -Once the node is joined, you may still remove tables from the replication -set, but adding new tables must be done via a resync operation. +Once the node is joined, you can still remove tables from the replication +set, but you must add new tables using a resync operation. -By default, a newly defined replication set does not replicate DDL or BDR -administration function calls. Use the `replication_set_add_ddl_filter` -to define which commands will be replicated. +By default, a newly defined replication set doesn't replicate DDL or BDR +administration function calls. Use `replication_set_add_ddl_filter` +to define the commands to replicate. BDR creates replication set definitions on all nodes. Each node can then be -defined to publish and/or subscribe to each replication set using +defined to publish or subscribe to each replication set using `alter_node_replication_sets`. -Functions exist to alter these definitions later, or to drop the replication +You can use functions to alter these definitions later or to drop the replication set. !!! Note - Do not use the default replication set for selective replication. - You should not drop or modify the default replication set on any of + Don't use the default replication set for selective replication. + Don't drop or modify the default replication set on any of the BDR nodes in the cluster as it is also used by default for DDL replication and administration function calls. -## Behavior of Partitioned Tables +## Behavior of partitioned tables -BDR supports partitioned tables transparently, meaning that a partitioned -table can be added to a replication set and -changes that involve any of the partitions will be replicated downstream. +BDR supports partitioned tables transparently, meaning that you can add a partitioned +table to a replication set. +Changes that involve any of the partitions are replicated downstream. !!! Note When partitions are replicated through a partitioned table, the statements executed directly on a partition are replicated as they - were executed on the parent table. The exception is the `TRUNCATE` command + were executed on the parent table. The exception is the `TRUNCATE` command, which always replicates with the list of affected tables or partitions. -It's possible to add individual partitions to the replication set, in -which case they will be replicated like regular tables (to the table of the +You can add individual partitions to the replication set, in +which case they are replicated like regular tables (to the table of the same name as the partition on the downstream). This has some performance advantages if the partitioning definition is the same on both provider and -subscriber, as the partitioning logic does not have to be executed. +subscriber, as the partitioning logic doesn't have to be executed. !!! Note If a root partitioned table is part of any replication set, memberships - of individual partitions are ignored, and only the membership of said root - table will be taken into account. + of individual partitions are ignored. only the membership of that root + table is taken into account. -## Behavior with Foreign Keys +## Behavior with foreign keys -A Foreign Key constraint ensures that each row in the referencing +A foreign key constraint ensures that each row in the referencing table matches a row in the referenced table. Therefore, if the referencing table is a member of a replication set, the referenced table must also be a member of the same replication set. -The current version of BDR does not automatically check or enforce -this condition. It is therefore the responsibility of the database -administrator to make sure, when adding a table to a replication set, -that all the tables referenced via foreign keys are also added. +The current version of BDR doesn't automatically check for or enforce +this condition. When adding a table to a replication set, the database administrator must +make sure +that all the tables referenced by foreign keys are also added. -The following query can be used to list all the foreign keys and -replication sets that do not satisfy this requirement, i.e. such that -the referencing table is a member of the replication set, while the -referenced table is not: +You can use the following query to list all the foreign keys and +replication sets that don't satisfy this requirement. +The referencing table is a member of the replication set, while the +referenced table isn't: ```sql SELECT t1.relname, @@ -115,30 +115,29 @@ The output of this query looks like the following: ``` This means that table `t2` is a member of replication set `s2`, but the -table referenced by the foreign key `t2_x_fkey` is not. +table referenced by the foreign key `t2_x_fkey` isn't. -!!! Note - The `TRUNCATE CASCADE` command takes into account the - replication set membership before replicating the command, e.g. +The `TRUNCATE CASCADE` command takes into account the +replication set membership before replicating the command. For example: ```sql TRUNCATE table1 CASCADE; ``` -This will become a `TRUNCATE` without cascade on all the tables that are +This becomes a `TRUNCATE` without cascade on all the tables that are part of the replication set only: ```sql TRUNCATE table1, referencing_table1, referencing_table2 ... ``` -## Replication Set Management +## Replication set management Management of replication sets. -Note that, with the exception of `bdr.alter_node_replication_sets`, the following -functions are considered to be `DDL` so DDL replication and global locking -applies to them, if that is currently active. See [DDL Replication](ddl). +With the exception of `bdr.alter_node_replication_sets`, the following +functions are considered to be `DDL`. DDL replication and global locking +apply to them, if that's currently active. See [DDL replication](ddl). ### bdr.create_replication_set @@ -161,40 +160,40 @@ bdr.create_replication_set(set_name name, #### Parameters -- `set_name` - name of the new replication set; must be unique across the BDR - group -- `replicate_insert` - indicates whether inserts into tables in this - replication set should be replicated -- `replicate_update` - indicates whether updates of tables in this - replication set should be replicated -- `replicate_delete` - indicates whether deletes from tables in this - replication set should be replicated -- `replicate_truncate` - indicates whether truncates of tables in this - replication set should be replicated -- `autoadd_tables` - indicates whether newly created (future) tables should be - added to this replication set -- `autoadd_existing` - indicates whether all existing user tables should be - added to this replication set; this only has effect if `autoadd_tables` is - set to true +- `set_name` — Name of the new replication set. Must be unique across the BDR + group. +- `replicate_insert` — Indicates whether to replicate inserts into tables in this + replication set. +- `replicate_update` — Indicates whether to replicate updates of tables in this + replication set. +- `replicate_delete` — Indicates whether to replicate deletes from tables in this + replication set. +- `replicate_truncate` — Indicates whether to replicate truncates of tables in this + replication set. +- `autoadd_tables` — Indicates whether to replicate newly created (future) tables + to this replication set +- `autoadd_existing` — Indicates whether to add all existing user tables + to this replication set. This parameter has an effect only if `autoadd_tables` is + set to `true`. #### Notes -By default, new replication sets do not replicate DDL or BDR administration -function calls. See [ddl filters](repsets#ddl-replication-filtering) below on how to set -up DDL replication for replication sets. There is a preexisting DDL filter -set up for the default group replication set that replicates all DDL and admin -function calls, which is created when the group is created, but can be dropped -in case it's not desirable for the BDR group default replication set to replicate +By default, new replication sets don't replicate DDL or BDR administration +function calls. See [ddl filters](repsets#ddl-replication-filtering) for how to set +up DDL replication for replication sets. A preexisting DDL filter +is set up for the default group replication set that replicates all DDL and admin +function calls. It's created when the group is created but can be dropped +in case you don't want the BDR group default replication set to replicate DDL or the BDR administration function calls. This function uses the same replication mechanism as `DDL` statements. This means that the replication is affected by the [ddl filters](repsets#ddl-replication-filtering) configuration. -The function will take a `DDL` global lock. +The function takes a `DDL` global lock. -This function is transactional - the effects can be rolled back with the -`ROLLBACK` of the transaction and the changes are visible to the current +This function is transactional. You can roll back the effects with the +`ROLLBACK` of the transaction. The changes are visible to the current transaction. ### bdr.alter_replication_set @@ -217,19 +216,18 @@ bdr.alter_replication_set(set_name name, #### Parameters -- `set_name` - name of an existing replication set -- `replicate_insert` - indicates whether inserts into tables in this - replication set should be replicated -- `replicate_update` - indicates whether updates of tables in this - replication set should be replicated -- `replicate_delete` - indicates whether deletes from tables in this - replication set should be replicated -- `replicate_truncate` - indicates whether truncates of tables in this - replication set should be replicated -- `autoadd_tables` - indicates whether newly created (future) tables should be - added to this replication set - -Any of the options that are set to NULL (the default) will remain the same as +- `set_name` — Name of an existing replication set. +- `replicate_insert` — Indicates whether to replicate inserts into tables in this + replication set. +- `replicate_update` — Indicates whether to replicate updates of tables in this + replication set. +- `replicate_delete` — Indicates whether to replicate deletes from tables in this + replication set. +- `replicate_truncate` — Indicates whether to replicate truncates of tables in this + replication set. +- `autoadd_tables` — Indicates whether to add newly created (future) tables to this replication set. + +Any of the options that are set to NULL (the default) remain the same as before. #### Notes @@ -238,10 +236,10 @@ This function uses the same replication mechanism as `DDL` statements. This mean the replication is affected by the [ddl filters](repsets#ddl-replication-filtering) configuration. -The function will take a `DDL` global lock. +The function takes a `DDL` global lock. -This function is transactional - the effects can be rolled back with the -`ROLLBACK` of the transaction, and the changes are visible to the current +This function is transactional. You can roll back the effects with the +`ROLLBACK` of the transaction. The changes are visible to the current transaction. ### bdr.drop_replication_set @@ -259,7 +257,7 @@ bdr.drop_replication_set(set_name name) #### Parameters -- `set_name` - name of an existing replication set +- `set_name` — Name of an existing replication set. #### Notes @@ -267,31 +265,30 @@ This function uses the same replication mechanism as `DDL` statements. This mean the replication is affected by the [ddl filters](repsets#ddl-replication-filtering) configuration. -The function will take a `DDL` global lock. +The function takes a `DDL` global lock. -This function is transactional - the effects can be rolled back with the -`ROLLBACK` of the transaction, and the changes are visible to the current +This function is transactional. You can roll back the effects with the +`ROLLBACK` of the transaction. The changes are visible to the current transaction. !!! Warning - Do not drop a replication set which is being used by at least - another node, because this will stop replication on that - node. Should this happen, please unsubscribe the affected node + Don't drop a replication set that's being used by at least + another node, because doing so stops replication on that + node. If that happens, unsubscribe the affected node from that replication set. - For the same reason, you should not drop a replication set if - there is a join operation in progress, and the node being joined - is a member of that replication set; replication set membership is - only checked at the beginning of the join. + For the same reason, don't drop a replication set with + a join operation in progress when the node being joined + is a member of that replication set. Replication set membership is + checked only at the beginning of the join. This happens because the information on replication set usage is - local to each node, so that it can be configured on a node before + local to each node, so that you can configure it on a node before it joins the group. -You can manage replication set subscription for a node using `alter_node_replication_sets` -which is mentioned below. +You can manage replication set subscription for a node using `alter_node_replication_sets`. ### bdr.alter_node_replication_sets -This function changes which replication sets a node publishes and is subscribed to. +This function changes the replication sets a node publishes and is subscribed to. #### Synopsis @@ -302,53 +299,53 @@ bdr.alter_node_replication_sets(node_name name, #### Parameters -- `node_name` - which node to modify; currently has to be local node -- `set_names` - array of replication sets to replicate to the specified - node; an empty array will result in the use of the group default replication set +- `node_name` — The node to modify. Currently has to be local node. +- `set_names` — Array of replication sets to replicate to the specified + node. An empty array results in the use of the group default replication set. #### Notes -This function is only executed on the local node and is not replicated in any manner. +This function is executed only on the local node and isn't replicated in any manner. -The replication sets listed are *not* checked for existence, -since this function is designed to be executed before the node joins. Be careful +The replication sets listed aren't checked for existence, +since this function is designed to execute before the node joins. Be careful to specify replication set names correctly to avoid errors. -This allows for calling the function not only on the node that is part of the -BDR group, but also on a node that has not joined any group yet in order to limit -what data is synchronized during the join. However, please note that schema is -*always fully synchronized* without regard to the replication sets setting, -meaning that all tables are copied across, not just the ones specified -in the replication set. Unwanted tables can be dropped by referring to +This allows for calling the function not only on the node that's part of the +BDR group but also on a node that hasn't joined any group yet. This approach limits +the data synchronized during the join. However, the schema is +always fully synchronized without regard to the replication sets setting. +All tables are copied across, not just the ones specified +in the replication set. You can drop unwanted tables by referring to the `bdr.tables` catalog table. These might be removed automatically in later versions of BDR. This is currently true even if the [ddl filters](repsets#ddl-replication-filtering) -configuration would otherwise prevent replication of DDL. +configuration otherwise prevent replication of DDL. -The replication sets that the node subscribes to after this call should be published -by the other node/s for actually replicating the changes from those nodes to +The replication sets that the node subscribes to after this call are published +by the other nodes for actually replicating the changes from those nodes to the node where this function is executed. -## Replication Set Membership +## Replication set membership -Tables can be added and removed to one or multiple replication sets. This only -affects replication of changes (DML) in those tables, schema changes (DDL) are -handled by DDL replication set filters (see [DDL Replication Filtering](#ddl-replication-filtering)). +You can add tables to or remove them from one or more replication sets. This +affects replication only of changes (DML) in those tables. Schema changes (DDL) are +handled by DDL replication set filters (see [DDL replication filtering](#ddl-replication-filtering)). -The replication uses the table membership in replication sets in combination -with the node replication sets configuration to determine which actions should be -replicated to which node. The decision is done using the union of all the -memberships and replication set options. This means that if a table is a member -of replication set A which replicates only INSERTs, and replication set B which -replicates only UPDATEs, both INSERTs and UPDATEs will be replicated if the +The replication uses the table membership in replication sets +with the node replication sets configuration to determine the actions to +replicate to which node. The decision is done using the union of all the +memberships and replication set options. Suppose that a table is a member +of replication set A that replicates only INSERT actions and replication set B that +replicates only UPDATE actions. Both INSERT and UPDATE act8ions are replicated if the target node is also subscribed to both replication set A and B. ### bdr.replication_set_add_table This function adds a table to a replication set. -This will add a table to replication set and start replication of changes +This adds a table to a replication set and starts replicating changes from this moment (or rather transaction commit). Any existing data the table -may have on a node will not be synchronized. +might have on a node isn't synchronized. Replication of this command is affected by DDL replication configuration, including DDL filtering settings. @@ -364,43 +361,42 @@ bdr.replication_set_add_table(relation regclass, #### Parameters -- `relation` - name or Oid of a table -- `set_name` - name of the replication set; if NULL (the default) then the BDR - group default replication set is used -- `columns` - reserved for future use (currently does nothing and must be NULL) -- `row_filter` - SQL expression to be used for filtering the replicated rows; - if this expression is not defined (i.e. NULL - the default) then all rows are sent +- `relation` — Name or Oid of a table. +- `set_name` — Name of the replication set. If NULL (the default), then the BDR + group default replication set is used. +- `columns` — Reserved for future use (currently does nothing and must be NULL). +- `row_filter` — SQL expression to be used for filtering the replicated rows. + If this expression isn't defined (that is, set to NULL, the default) then all rows are sent. The `row_filter` specifies an expression producing a Boolean result, with NULLs. -Expressions evaluating to True or Unknown will replicate the row; a False value -will not replicate the row. Expressions cannot contain subqueries, nor refer to -variables other than columns of the current row being replicated. No system -columns may be referenced. +Expressions evaluating to True or Unknown replicate the row. A False value +doesn't replicate the row. Expressions can't contain subqueries or refer to +variables other than columns of the current row being replicated. You can't reference system +columns. `row_filter` executes on the origin node, not on the target node. This puts an -additional CPU overhead on replication for this specific table, but will -completely avoid sending data for filtered rows, hence reducing network -bandwidth and apply overhead on the target node. +additional CPU overhead on replication for this specific table but +completely avoids sending data for filtered rows. Hence network +bandwidth is reduced and overhead on the target node is applied. -`row_filter` will never remove `TRUNCATE` commands for a specific table. -`TRUNCATE` commands can be filtered away at the replication set level; see -earlier. +`row_filter` never removes `TRUNCATE` commands for a specific table. +You can filter away `TRUNCATE` commands at the replication set level. -It is possible to replicate just some columns of a table, see +You can replicate just some columns of a table. See [Replicating between nodes with differences](appusage). #### Notes -This function uses same replication mechanism as `DDL` statements. This means +This function uses the same replication mechanism as `DDL` statements. This means that the replication is affected by the [ddl filters](repsets#ddl-replication-filtering) configuration. -The function will take a `DML` global lock on the relation that is being -added to the replication set if the `row_filter` is not NULL, otherwise -it will take just a `DDL` global lock. +The function takes a `DML` global lock on the relation that's being +added to the replication set if the `row_filter` isn't NULL. Otherwise +it takes just a `DDL` global lock. -This function is transactional - the effects can be rolled back with the -`ROLLBACK` of the transaction and the changes are visible to the current +This function is transactional. You can roll back the effects with the +`ROLLBACK` of the transaction. The changes are visible to the current transaction. ### bdr.replication_set_remove_table @@ -419,32 +415,32 @@ bdr.replication_set_remove_table(relation regclass, #### Parameters -- `relation` - name or Oid of a table -- `set_name` - name of the replication set; if NULL (the default) then the BDR - group default replication set is used +- `relation` — Name or Oid of a table. +- `set_name` — Name of the replication set. If NULL (the default), then the BDR + group default replication set is used. #### Notes -This function uses same replication mechanism as `DDL` statements. This means +This function uses the same replication mechanism as `DDL` statements. This means the replication is affected by the [ddl filters](repsets#ddl-replication-filtering) configuration. -The function will take a `DDL` global lock. +The function takes a `DDL` global lock. -This function is transactional - the effects can be rolled back with the -`ROLLBACK` of the transaction and the changes are visible to the current +This function is transactional. You can roll back the effects with the +`ROLLBACK` of the transaction. The changes are visible to the current transaction. -## Listing Replication Sets +## Listing replication sets -Existing replication sets can be listed with the following query: +You can list existing replication sets with the following query: ```sql SELECT set_name FROM bdr.replication_sets; ``` -This query can be used to list all the tables in a given replication +You can use this query to list all the tables in a given replication set: ```sql @@ -453,8 +449,8 @@ FROM bdr.tables WHERE set_name = 'myrepset'; ``` -In the section [Behavior with Foreign Keys](#behavior-with-foreign-keys), we report a -query that lists all the foreign keys whose referenced table is not +In [Behavior with foreign keys](#behavior-with-foreign-keys), we show a +query that lists all the foreign keys whose referenced table isn't included in the same replication set as the referencing table. Use the following SQL to show those replication sets that the @@ -472,7 +468,7 @@ current node publishes and subscribes from: FROM bdr.local_node_summary; ``` -This produces output like this: +This code produces output like this: ```sql node_id | node_name | pub_repsets | sub_repsets @@ -481,9 +477,8 @@ This produces output like this: (1 row) ``` -To get the same query executed on against all nodes in the cluster, thus getting -which replication sets are associated to all nodes at the same time, we can use -the following query: +To execute the same query against all nodes in the cluster, you can use the following query. This approach gets +the replication sets associated with all nodes at the same time. ```sql WITH node_repsets AS ( @@ -509,7 +504,7 @@ SELECT j->'response'->'command_tuples'->0->>'node_id' AS node_id, FROM node_repsets; ``` -This will show, for example: +This shows, for example: ```sql node_id | node_name | pub_repsets | sub_repsets @@ -521,17 +516,17 @@ This will show, for example: (4 rows) ``` -## DDL Replication Filtering +## DDL replication filtering -By default, the replication of all supported DDL happens via the default BDR -group replication set. This is achieved by the existence of a DDL filter with -the same name as the BDR group, which is automatically added to the default BDR +By default, the replication of all supported DDL happens by way of the default BDR +group replication set. This is achieved with a DDL filter with +the same name as the BDR group. This filter is added to the default BDR group replication set when the BDR group is created. -The above can be adjusted by changing the DDL replication filters for +You can adjust this by changing the DDL replication filters for all existing replication sets. These filters are independent of table membership in the replication sets. Just like data changes, each DDL statement -will be replicated only once, no matter if it is matched by multiple filters on +is replicated only once, even if it's matched by multiple filters on multiple replication sets. You can list existing DDL filters with the following query, which @@ -542,38 +537,38 @@ tag and to the role name: SELECT * FROM bdr.ddl_replication; ``` -The following functions can be used to manipulate DDL filters. Note -that they are considered to be `DDL`, and therefore subject to DDL +You can use the following functions to manipulate DDL filters. +They are considered to be `DDL` and are therefore subject to DDL replication and global locking. ### bdr.replication_set_add_ddl_filter This function adds a DDL filter to a replication set. -Any DDL that matches the given filter will be replicated to any node which -is subscribed to that set. This also affects replication of BDR admin functions. +Any DDL that matches the given filter is replicated to any node that's +subscribed to that set. This also affects replication of BDR admin functions. -Note that this does not prevent execution of DDL on any node, it only -alters whether DDL is replicated, or not, to other nodes. So if two nodes have -a replication filter between them that excludes all index commands, then -index commands can still be executed freely by directly connecting to +This doesn't prevent execution of DDL on any node. It only +alters whether DDL is replicated to other nodes. Suppose two nodes have +a replication filter between them that excludes all index commands. Index commands can still +be executed freely by directly connecting to each node and executing the desired DDL on that node. The DDL filter can specify a `command_tag` and `role_name` to allow -replication of only some DDL statements. The `command_tag` is same as those +replication of only some DDL statements. The `command_tag` is the same as those used by [EVENT TRIGGERs](https://www.postgresql.org/docs/current/static/event-trigger-matrix.html) for regular PostgreSQL commands. A typical example might be to create a filter that prevents additional index commands on a logical standby from being replicated to all other nodes. -The BDR admin functions use can be filtered using a tagname matching the -qualified function name (for example `bdr.replication_set_add_table` will be the -command tag for the function of the same name). For example, this allows all BDR +You can filter the BDR admin functions used by using a tagname matching the +qualified function name. For example, `bdr.replication_set_add_table` is the +command tag for the function of the same name. In this case, this tag allows all BDR functions to be filtered using `bdr.*`. -The `role_name` is used for matching against the current role which is executing +The `role_name` is used for matching against the current role that is executing the command. Both `command_tag` and `role_name` are evaluated as regular -expressions which are case sensitive. +expressions, which are case sensitive. #### Synopsis @@ -586,29 +581,29 @@ bdr.replication_set_add_ddl_filter(set_name name, #### Parameters -- `set_name` - name of the replication set; if NULL then the BDR - group default replication set is used -- `ddl_filter_name` - name of the DDL filter; this must be unique across the - whole BDR group -- `command_tag` - regular expression for matching command tags; NULL means - match everything -- `role_name` - regular expression for matching role name; NULL means - match all roles +- `set_name` — Name of the replication set. if NULL then the BDR + group default replication set is used. +- `ddl_filter_name` — Name of the DDL filter. This must be unique across the + whole BDR group. +- `command_tag` — Regular expression for matching command tags. NULL means + match everything. +- `role_name` — Regular expression for matching role name. NULL means + match all roles. #### Notes This function uses the same replication mechanism as `DDL` statements. This means that the replication is affected by the [ddl filters](repsets#ddl-replication-filtering) -configuration. Please note - this means that replication of changes to ddl -filter configuration is affected by existing ddl filter configuration! +configuration. This also means that replication of changes to ddl +filter configuration is affected by the existing ddl filter configuration. -The function will take a `DDL` global lock. +The function takes a `DDL` global lock. -This function is transactional - the effects can be rolled back with the -`ROLLBACK` of the transaction, and the changes are visible to the current +This function is transactional. You can roll back the effects with the +`ROLLBACK` of the transaction. The changes are visible to the current transaction. -To view which replication filters are defined, use the view `bdr.ddl_replication`. +To view the defined replication filters, use the view `bdr.ddl_replication`. #### Examples @@ -625,8 +620,8 @@ SELECT bdr.replication_set_add_ddl_filter('mygroup', 'index_filter', '^(?!(CREATE INDEX|DROP INDEX|ALTER INDEX)).*'); ``` -To include all operations on tables and indexes, but exclude all others, add -two filters, one for tables, one for indexes. This illustrates that +To include all operations on tables and indexes but exclude all others, add +two filters: one for tables, one for indexes. This shows that multiple filters provide the union of all allowed DDL commands: ```sql @@ -639,7 +634,7 @@ SELECT bdr.replication_set_add_ddl_filter('bdrgroup','table_filter', '^((?!TABLE This function removes the DDL filter from a replication set. Replication of this command is affected by DDL replication configuration, -including DDL filtering settings themselves! +including DDL filtering settings themselves. #### Synopsis @@ -650,20 +645,20 @@ bdr.replication_set_remove_ddl_filter(set_name name, #### Parameters -- `set_name` - name of the replication set; if NULL then the BDR - group default replication set is used -- `ddl_filter_name` - name of the DDL filter to remove +- `set_name` — Name of the replication set. If NULL then the BDR + group default replication set is used. +- `ddl_filter_name` — Name of the DDL filter to remove. #### Notes This function uses the same replication mechanism as `DDL` statements. This means that the replication is affected by the -[ddl filters](repsets#ddl-replication-filtering) configuration. Please note -that this means that replication of changes to the DDL filter configuration is +[ddl filters](repsets#ddl-replication-filtering) configuration. +This also means that replication of changes to the DDL filter configuration is affected by the existing DDL filter configuration. -The function will take a `DDL` global lock. +The function takes a `DDL` global lock. -This function is transactional - the effects can be rolled back with the -`ROLLBACK` of the transaction, and the changes are visible to the current +This function is transactional. You can roll back the effects with the +`ROLLBACK` of the transaction. The changes are visible to the current transaction. diff --git a/product_docs/docs/bdr/4/scaling.mdx b/product_docs/docs/bdr/4/scaling.mdx index dfa2dd38d97..8e20d943774 100644 --- a/product_docs/docs/bdr/4/scaling.mdx +++ b/product_docs/docs/bdr/4/scaling.mdx @@ -3,113 +3,111 @@ title: AutoPartition --- AutoPartition allows tables to grow easily to large sizes by automatic -partitioning management. This utilizes the additional features of BDR +partitioning management. This capability uses features of BDR such as low-conflict locking of creating and dropping partitions. -New partitions can be created regularly and then dropped when the +You can create new partitions regularly and then drop them when the data retention period expires. -BDR management is primarily accomplished via SQL-callable functions. +BDR management is primarily accomplished by functions that can be called by SQL. All functions in BDR are exposed in the `bdr` schema. Unless you put it into -your `search_path`, you will need to schema-qualify the name of each function. +your search_path, you need to schema-qualify the name of each function. -## Auto Creation of Partitions +## Auto creation of partitions -`bdr.autopartition()` is used to create or alter the definition of automatic -range partitioning for a table. If no definition exists, it will be created, -otherwise later executions will alter the definition. +`bdr.autopartition()` creates or alters the definition of automatic +range partitioning for a table. If no definition exists, it's created. +Otherwise, later executions will alter the definition. -`bdr.autopartition()` does not lock the actual table, it only changes the -definition of when and how new partition maintenance actions will take place. +`bdr.autopartition()` doesn't lock the actual table. It changes the +definition of when and how new partition maintenance actions take place. `bdr.autopartition()` leverages the features that allow a partition to be attached or detached/dropped without locking the rest of the table -(when the underlaying Postgres version supports it). +(when the underlying Postgres version supports it). -An ERROR is raised if the table is not RANGE partitioned or a multi-column +An ERROR is raised if the table isn't RANGE partitioned or a multi-column partition key is used. A new partition is added for every `partition_increment` range of values, with lower and upper bound `partition_increment` apart. For tables with a partition key of type `timestamp` or `date`, the `partition_increment` must be a valid -constant of type `interval`. For example, specifying `1 Day` will cause a new -partition to be added each day, with partition bounds that are 1 day apart. +constant of type `interval`. For example, specifying `1 Day` causes a new +partition to be added each day, with partition bounds that are one day apart. -If the partition column is connected to a `snowflakeid`, `timeshard` or `ksuuid` sequence, -the `partition_increment` must be specified as type `interval`. Otherwise, +If the partition column is connected to a `snowflakeid`, `timeshard`, or `ksuuid` sequence, +you must specify the `partition_increment` as type `interval`. Otherwise, if the partition key is integer or numeric, then the `partition_increment` must be a valid constant of the same datatype. For example, specifying -'1000000' will cause new partitions to be added every 1 million values. +`1000000` causes new partitions to be added every 1 million values. If the table has no existing partition, then the specified `partition_initial_lowerbound` is used as the lower bound for the first -partition. If `partition_initial_lowerbound` is not specified, then the system +partition. If you don't specify `partition_initial_lowerbound`, then the system tries to derive its value from the partition column type and the specified -`partition_increment`. For example, if `partition_increment` is specified as `1 -Day`, then `partition_initial_lowerbound` will be automatically set to CURRENT +`partition_increment`. For example, if `partition_increment` is specified as `1 Day`, +then `partition_initial_lowerbound` is set to CURRENT DATE. If `partition_increment` is specified as `1 Hour`, then -`partition_initial_lowerbound` will be set to the current hour of the current -date. The bounds for the subsequent partitions will be set using the +`partition_initial_lowerbound` is set to the current hour of the current +date. The bounds for the subsequent partitions are set using the `partition_increment` value. The system always tries to have a certain minimum number of advance partitions. -In order to decide whether to create new partitions or not, it uses the -specified `partition_autocreate_expression`. This can be a SQL evaluable -expression, which is evaluated every time a check is performed. For example, +To decide whether to create new partitions, it uses the +specified `partition_autocreate_expression`. This can be an expression that can be evaluated by SQL, +which is evaluated every time a check is performed. For example, for a partitioned table on column type `date`, if -`partition_autocreate_expression` is specified as `DATE_TRUNC('day', -CURRENT_DATE)`, `partition_increment` is specified as `1 Day` and -`minimum_advance_partitions` is specified as 2, then new partitions will be +`partition_autocreate_expression` is specified as `DATE_TRUNC('day',CURRENT_DATE)`, +`partition_increment` is specified as `1 Day` and +`minimum_advance_partitions` is specified as 2, then new partitions are created until the upper bound of the last partition is less than `DATE_TRUNC('day', CURRENT_DATE) + '2 Days'::interval`. The expression is evaluated each time the system checks for new partitions. -For a partitioned table on column type `integer`, the -`partition_autocreate_expression` may be specified as `SELECT max(partcol) FROM +For a partitioned table on column type `integer`, you can specify the +`partition_autocreate_expression` as `SELECT max(partcol) FROM schema.partitioned_table`. The system then regularly checks if the maximum value of -the partitioned column is within the distance of `minimum_advance_partitions * -partition_increment`of the last partition's upper bound. It is expected that -the user creates an index on the `partcol` so that the query runs efficiently. -If the `partition_autocreate_expression` is not specified for a partition table -on column type `integer`, `smallint` or `bigint`, then the system will -automatically set it to `max(partcol)`. - -If the `data_retention_period` is set, partitions will be automatically -dropped after this period. Partitions will be dropped at the same time as new -partitions are added, to minimize locking. If not set, partitions must -be dropped manually. - -The `data_retention_period` parameter is only supported for timestamp (and +the partitioned column is within the distance of `minimum_advance_partitions * partition_increment` +of the last partition's upper bound. Create an index on the `partcol` so that the query runs efficiently. +If the `partition_autocreate_expression` isn't specified for a partition table +on column type `integer`, `smallint`, or `bigint`, then the system +sets it to `max(partcol)`. + +If the `data_retention_period` is set, partitions are +dropped after this period. Partitions are dropped at the same time as new +partitions are added, to minimize locking. If this value isn't set, you must drop the partitions manually. + +The `data_retention_period` parameter is supported only for timestamp (and related) based partitions. The period is calculated by considering the upper -bound of the partition and the partition is either migrated to the secondary +bound of the partition. The partition is either migrated to the secondary tablespace or dropped if either of the given period expires, relative to the upper bound. By default, AutoPartition manages partitions globally. In other words, when a partition is created on one node, the same partition is also created on all other nodes in the cluster. So all partitions are consistent and guaranteed to -be available. For this, AutoPartition makes use of Raft. This behaviour can be -changed by passing `managed_locally` as `true`. In that case, all partitions +be available. For this, AutoPartition makes use of Raft. You can change this behavior +by passing `managed_locally` as `true`. In that case, all partitions are managed locally on each node. This is useful for the case when the -partitioned table is not a replicated table and hence it may not be necessary +partitioned table isn't a replicated table and hence it might not be necessary or even desirable to have all partitions on all nodes. For example, the -built-in `bdr.conflict_history` table is not a replicated table, and is +built-in `bdr.conflict_history` table isn't a replicated table and is managed by AutoPartition locally. Each node creates partitions for this table locally and drops them once they are old enough. -Tables once marked as `managed_locally` cannot be later changed to be managed +You can't later change tables marked as `managed_locally` to be managed globally and vice versa. Activities are performed only when the entry is marked `enabled = on`. -The user is not expected to manually create or drop partitions for tables +You aren't expected to manually create or drop partitions for tables managed by AutoPartition. Doing so can make the AutoPartition metadata -inconsistent and could cause it to fail. +inconsistent and might cause it to fail. ### Configure AutoPartition -The `bdr.autopartition` function configures automatic partinioning of a table. +The `bdr.autopartition` function configures automatic partitioning of a table. #### Synopsis @@ -127,20 +125,20 @@ bdr.autopartition(relation regclass, #### Parameters -- `relation` - name or Oid of a table. -- `partition_increment` - interval or increment to next partition creation. -- `partition_initial_lowerbound` - if the table has no partition, then the +- `relation` — Name or Oid of a table. +- `partition_increment` — Interval or increment to next partition creation. +- `partition_initial_lowerbound` — If the table has no partition, then the first partition with this lower bound and `partition_increment` apart upper - bound will be created. -- `partition_autocreate_expression` - is used to detect if it is time to create new partitions. -- `minimum_advance_partitions` - the system will attempt to always have at + bound is created. +- `partition_autocreate_expression` — Used to detect if it's time to create new partitions. +- `minimum_advance_partitions` — The system attempts to always have at least `minimum_advance_partitions` partitions. -- `maximum_advance_partitions` - number of partitions to be created in a single +- `maximum_advance_partitions` — Number of partitions to be created in a single go once the number of advance partitions falls below `minimum_advance_partitions`. -- `data_retention_period` - interval until older partitions are dropped, if - defined. This must be greater than `migrate_after_period`. -- `managed_locally` - if true then the partitions will be managed locally. -- `enabled` - allows activity to be disabled/paused and later resumed/re-enabled. +- `data_retention_period` — Interval until older partitions are dropped, if + defined. This value must be greater than `migrate_after_period`. +- `managed_locally` — If true, then the partitions are managed locally. +- `enabled` — Allows activity to be disabled or paused and later resumed or reenabled. #### Examples @@ -156,7 +154,7 @@ unitsales int bdr.autopartition('measurement', '1 day', data_retention_period := '30 days'); ``` -Create 5 advance partitions when there are only 2 more partitions remaining (each partition can hold 1 billion orders): +Create five advance partitions when there are only two more partitions remaining (each partition can hold 1 billion orders): ```sql bdr.autopartition('Orders', '1000000000', @@ -166,7 +164,7 @@ bdr.autopartition('Orders', '1000000000', ); ``` -### Create One AutoPartition +### Create one AutoPartition Use `bdr.autopartition_create_partition()` to create a standalone AutoPartition on the parent table. @@ -183,13 +181,13 @@ bdr.autopartition_create_partition(relname regclass, #### Parameters -- `relname` - Name or Oid of the parent table to attach to -- `partname` - Name of the new AutoPartition -- `lowerb` - The lower bound of the partition -- `upperb` - The upper bound of the partition -- `nodes` - List of nodes that the new partition resides on +- `relname` — Name or Oid of the parent table to attach to. +- `partname` — Name of the new AutoPartition. +- `lowerb` — The lower bound of the partition. +- `upperb` — The upper bound of the partition. +- `nodes` — List of nodes that the new partition resides on. -### Stopping Auto-Creation of Partitions +### Stopping automatic creation of partitions Use `bdr.drop_autopartition()` to drop the auto-partitioning rule for the given relation. All pending work items for the relation are deleted and no new @@ -201,13 +199,13 @@ bdr.drop_autopartition(relation regclass); #### Parameters -- `relation` - name or Oid of a table +- `relation` — Name or Oid of a table. ### Drop one AutoPartition Use `bdr.autopartition_drop_partition` once a BDR AutoPartition table has been made, as this function can specify single partitions to drop. If the partitioned -table has successfully been dropped, the function will return true. +table was successfully dropped, the function returns `true`. #### Synopsis @@ -217,24 +215,24 @@ bdr.autopartition_drop_partition(relname regclass) #### Parameters -- `relname` - The name of the partitioned table to be dropped +- `relname` — The name of the partitioned table to drop. ### Notes -This will place a DDL lock on the parent table, before using DROP TABLE on the +This places a DDL lock on the parent table, before using DROP TABLE on the chosen partition table. -### Wait for Partition Creation +### Wait for partition creation Use `bdr.autopartition_wait_for_partitions()` to wait for the creation of partitions on the local node. The function takes the partitioned table name and a partition key column value and waits until the partition that holds that value is created. -The function only waits for the partitions to be created locally. It does not guarantee +The function only waits for the partitions to be created locally. It doesn't guarantee that the partitions also exists on the remote nodes. -In order to wait for the partition to be created on all BDR nodes, use the +To wait for the partition to be created on all BDR nodes, use the `bdr.autopartition_wait_for_partitions_on_all_nodes()` function. This function internally checks local as well as all remote nodes and waits until the partition is created everywhere. @@ -247,8 +245,8 @@ bdr.autopartition_wait_for_partitions(relation regclass, text bound); #### Parameters -- `relation` - name or Oid of a table -- `bound` - partition key column value. +- `relation` — Name or Oid of a table. +- `bound` — Partition key column value. #### Synopsis @@ -258,14 +256,14 @@ bdr.autopartition_wait_for_partitions_on_all_nodes(relation regclass, text bound #### Parameters -- `relation` - name or Oid of a table. -- `bound` - partition key column value. +- `relation` — Name or Oid of a table. +- `bound` — Partition key column value. -### Find Partition +### Find partition Use the `bdr.autopartition_find_partition()` function to find the partition for the -given partition key value. If partition to hold that value does not exist, then -the function returns NULL. Otherwise OID of the partition is returned. +given partition key value. If partition to hold that value doesn't exist, then +the function returns NULL. Otherwise Oid of the partition is returned. #### Synopsis @@ -275,13 +273,13 @@ bdr.autopartition_find_partition(relname regclass, searchkey text); #### Parameters -- `relname` - name of the partitioned table. -- `searchkey` - partition key value to search. +- `relname` — Name of the partitioned table. +- `searchkey` — Partition key value to search. -### Enable/Disable AutoPartitioning +### Enable or disable AutoPartitioning Use `bdr.autopartition_enable()` to enable AutoPartitioning on the given table. -If AutoPartitioning is already enabled, then it will be a no-op. Similarly, use +If AutoPartitioning is already enabled, then no action occurs. Similarly, use `bdr.autopartition_disable()` to disable AutoPartitioning on the given table. #### Synopsis @@ -292,7 +290,7 @@ bdr.autopartition_enable(relname regclass); #### Parameters -- `relname` - name of the relation to enable AutoPartitioning. +- `relname` — Name of the relation to enable AutoPartitioning. #### Synopsis @@ -302,7 +300,7 @@ bdr.autopartition_disable(relname regclass); #### Parameters -- `relname` - name of the relation to disable AutoPartitioning. +- `relname` — Name of the relation to disable AutoPartitioning. #### Synopsis @@ -313,7 +311,7 @@ bdr.autopartition_get_last_completed_workitem(); Return the `id` of the last workitem successfully completed on all nodes in the cluster. -### Check AutoPartition Workers +### Check AutoPartition workers From using the `bdr.autopartition_work_queue_check_status` function, you can see the status of the background workers that are doing their job to maintain @@ -332,11 +330,11 @@ bdr.autopartition_work_queue_check_status(workid bigint #### Parameters -- `workid` - The key of the AutoPartition worker -- `local` - Check the local status only +- `workid` — The key of the AutoPartition worker. +- `local` — Check the local status only. #### Notes -AutoPartition workers are ALWAYS running in the background, even before the -bdr.autopartition function is called for the first time. If an invalid worker ID -is used, the function will return 'unknown'. 'In-progress' is the typical status. +AutoPartition workers are always running in the background, even before the +`bdr.autopartition` function is called for the first time. If an invalid worker ID +is used, the function returns `unknown`. `In-progress` is the typical status. diff --git a/product_docs/docs/bdr/4/security.mdx b/product_docs/docs/bdr/4/security.mdx index d412c66d630..fff196de662 100644 --- a/product_docs/docs/bdr/4/security.mdx +++ b/product_docs/docs/bdr/4/security.mdx @@ -1,77 +1,76 @@ --- -title: Security and Roles +title: Security and roles --- -The BDR extension can be created only by superusers, although if desired, it is possible to set up the `pgextwlist` extension and configure it to allow BDR to be created by a non-superuser. +The BDR extension can be created only by superusers. However, if you want, you can set up the `pgextwlist` extension and configure it to allow BDR to be created by a non-superuser. -Configuring and managing BDR does not require superuser access, nor is that recommended. +Configuring and managing BDR doesn't require superuser access, nor is that recommended. The privileges required by BDR are split across the following default/predefined roles, named similarly to the PostgreSQL default/predefined roles: -- *bdr_superuser* - the highest-privileged role, having access to all BDR tables and functions. -- *bdr_read_all_stats* - the role having read-only access to the tables, views and functions, sufficient to understand the state of BDR. -- *bdr_monitor* - at the moment the same as `bdr_read_all_stats`, to be extended later. -- *bdr_application* - the minimal privileges required by applications running BDR. -- *bdr_read_all_conflicts* - can view *all* conflicts in `bdr.conflict_history`. +- bdr_superuser — The highest-privileged role, having access to all BDR tables and functions. +- bdr_read_all_stats — The role having read-only access to the tables, views, and functions, sufficient to understand the state of BDR. +- bdr_monitor — At the moment, the same as `bdr_read_all_stats`. To be extended later. +- bdr_application — The minimal privileges required by applications running BDR. +- bdr_read_all_conflicts — Can view all conflicts in `bdr.conflict_history`. These BDR roles are created when the BDR extension is -installed. See [BDR Default Roles](#bdr-default-roles) for more details. +installed. See [BDR default roles](#bdr-default-roles) for more details. -Managing BDR does not require that administrators have access to user data. +Managing BDR doesn't require that administrators have access to user data. -Arrangements for securing conflicts are discussed here -[Logging Conflicts to a Table](conflicts). +Arrangements for securing conflicts are discussed in +[Logging conflicts to a table](conflicts). -Conflicts may be monitored using the BDR.conflict_history_summary view. +You can monitor conflicts using the `BDR.conflict_history_summary` view. -## Catalog Tables +## Catalog tables System catalog and Information Schema tables are always excluded from replication by BDR. In addition, tables owned by extensions are excluded from replication. -## BDR Functions & Operators +## BDR functions and operators All BDR functions are exposed in the `bdr` schema. Any calls to these -functions should be schema qualified, rather than putting `bdr` in the -`search_path`. +functions must be schema qualified, rather than putting `bdr` in the +search_path. -All BDR operators are available via `pg_catalog` schema to allow users +All BDR operators are available by way of the `pg_catalog` schema to allow users to exclude the `public` schema from the search_path without problems. ## Granting privileges on catalog objects -Administrators should not grant explicit privileges on catalog -objects such as tables, views and functions; manage access to those objects -by granting one of the roles documented in [BDR -Default Roles](#bdr-default-roles). +Administrators must not grant explicit privileges on catalog +objects such as tables, views, and functions. Manage access to those objects +by granting one of the roles described in [BDR default roles](#bdr-default-roles). This requirement is a consequence of the flexibility that allows -joining a node group even if the nodes on either side of the join do -not have the exact same version of BDR (and therefore of the BDR +joining a node group even if the nodes on either side of the join don't +have the exact same version of BDR (and therefore of the BDR catalog). -More precisely, if privileges on individual catalog objects have been -explicitly granted, then the `bdr.join_node_group()` procedure could +More precisely, if privileges on individual catalog objects were +explicitly granted, then the `bdr.join_node_group()` procedure might fail because the corresponding GRANT statements extracted from the node being joined might not apply to the node that is joining. -## Role Management +## Role management Users are global objects in a PostgreSQL instance. `CREATE USER` and `CREATE ROLE` commands are replicated automatically if they are executed in the database where BDR is running and the `bdr.role_replication` is turned on. However, if these commands are executed -in other databases in the same PostgreSQL instance then they will not be replicated, +in other databases in the same PostgreSQL instance, then they aren't replicated, even if those users have rights on the BDR database. -When a new BDR node joins the BDR group, existing users are not automatically +When a new BDR node joins the BDR group, existing users aren't automatically copied unless the node is added using `bdr_init_physical`. This is intentional and is an important security feature. PostgreSQL allows users to access multiple -databases, with the default being to access any database. BDR does not know -which users access which database and so cannot safely decide +databases, with the default being to access any database. BDR doesn't know +which users access which database and so can't safely decide which users to copy across to the new node. PostgreSQL allows you to dump all users with the command: @@ -81,41 +80,41 @@ pg_dumpall --roles-only > roles.sql ``` The file `roles.sql` can then be edited to remove unwanted users before -re-executing that on the newly created node. +reexecuting that on the newly created node. Other mechanisms are possible, depending on your identity and access -management solution (IAM), but are not automated at this time. +management solution (IAM) but aren't automated at this time. -## Roles and Replication +## Roles and replication DDL changes executed by a user are applied as that same user on each node. DML changes to tables are replicated as the table-owning user on the target node. -It is recommended - but not enforced - that a table is owned by the same user on each node. +We recommend but do not enforce that a table be owned by the same user on each node. -If table A is owned by user X on node1 and owned by user Y on node2, then if user Y -has higher privileges than user X, this could be viewed as a privilege escalation. +Suppose table A is owned by user X on node1 and owned by user Y on node2. If user Y +has higher privileges than user X, this might be viewed as a privilege escalation. Since some nodes have different use cases, we allow this but warn against it to allow the security administrator to plan and audit this situation. -On tables with row level security policies enabled, changes -will be replicated without re-enforcing policies on apply. +On tables with row-level security policies enabled, changes +are replicated without reenforcing policies on apply. This is equivalent to the changes being applied as `NO FORCE ROW LEVEL SECURITY`, even if `FORCE ROW LEVEL SECURITY` is specified. -If this is not desirable, specify a row_filter that avoids -replicating all rows. It is recommended - but not enforced - +If this isn't what you want, specify a row_filter that avoids +replicating all rows. We recommend but don't enforce that the row security policies on all nodes be identical or at least compatible. -Note that bdr_superuser controls replication for BDR and may -add/remove any table from any replication set. bdr_superuser -does not need, nor is it recommended to have, any privileges -over individual tables. If the need exists to restrict access +The user bdr_superuser controls replication for BDR and can +add or remove any table from any replication set. bdr_superuser +doesn't need any privileges +over individual tables, nor is this recommended. If you need to restrict access to replication set functions, restricted versions of these functions can be implemented as `SECURITY DEFINER` functions -and `GRANT`ed to the appropriate users. +and granted to the appropriate users. -## Connection Role +## Connection role When allocating a new BDR node, the user supplied in the DSN for the `local_dsn` argument of `bdr.create_node` and the `join_target_dsn` of @@ -125,50 +124,49 @@ manage database objects. BDR is carefully written to prevent privilege escalation attacks even when using a role with `SUPERUSER` rights in these DSNs. -To further reduce the attack surface, a more restricted user may be -specified in the above DSNs. At a minimum, such a user must be +To further reduce the attack surface, you can specify a more restricted user +in the above DSNs. At a minimum, such a user must be granted permissions on all nodes, such that following stipulations are satisfied: -- the user has the `REPLICATION` attribute -- it is granted the `CREATE` permission on the database -- it inherits the `bdr_superuser` role -- it owns all database objects to replicate, either directly or via - permissions from the owner role(s). +- The user has the `REPLICATION` attribute. +- It is granted the `CREATE` permission on the database. +- It inherits the `bdr_superuser` role. +- It owns all database objects to replicate, either directly or from + permissions from the owner roles. -Once all nodes are joined, the permissions may be further reduced to +Once all nodes are joined, the permissions can be further reduced to just the following to still allow DML and DDL replication: - The user has the `REPLICATION` attribute. - It inherits the `bdr_superuser` role. -## Privilege Restrictions +## Privilege restrictions BDR enforces additional restrictions, effectively preventing the use of DDL that relies solely on TRIGGER or REFERENCES privileges. -The following sub-sections explain these. -`GRANT ALL` will still grant both TRIGGER and REFERENCES privileges, -so it is recommended that you state privileges explicitly, e.g. +`GRANT ALL` still grants both TRIGGER and REFERENCES privileges, +so we recommend that you state privileges explicitly. For example, use `GRANT SELECT, INSERT, UPDATE, DELETE, TRUNCATE` instead of `ALL`. -### Foreign Key Privileges +### Foreign key privileges -`ALTER TABLE ... ADD FOREIGN KEY` is only supported if the user has -SELECT privilege on the referenced table, or if the referenced table -has RLS restrictions enabled which the current user cannot bypass. +`ALTER TABLE ... ADD FOREIGN KEY` is supported only if the user has +SELECT privilege on the referenced table or if the referenced table +has RLS restrictions enabled that the current user can't bypass. -Thus, the REFERENCES privilege is not sufficient to allow creation -of a Foreign Key with BDR. Relying solely on the REFERENCES privilege -is not typically useful since it makes the validation check execute -using triggers rather than a table scan, so is typically too expensive -to used successfully. +Thus, the REFERENCES privilege isn't sufficient to allow creating +a foreign key with BDR. Relying solely on the REFERENCES privilege +isn't typically useful since it makes the validation check execute +using triggers rather than a table scan. It is typically too expensive +to use successfully. ### Triggers -In PostgreSQL, triggers may be created by both the owner of a table and anyone who -has been granted the TRIGGER privilege. Triggers granted by the non-table owner -would execute as the table owner in BDR, which could cause a security issue. +In PostgreSQL, both the owner of a table and anyone who +was granted the TRIGGER privilege can create triggers. Triggers granted by the non-table owner +execute as the table owner in BDR, which might cause a security issue. The TRIGGER privilege is seldom used and PostgreSQL Core Team has said "The separate TRIGGER permission is something we consider obsolescent." @@ -178,47 +176,47 @@ on a table: - superuser - bdr_superuser - Owner of the table can create triggers according to same rules as in PostgreSQL - (must have EXECUTE privilege on function used by the trigger). -- Users who have TRIGGER privilege on the table can only create a trigger if + (must have EXECUTE privilege on the function used by the trigger). +- Users who have TRIGGER privilege on the table can create a trigger only if they create the trigger using a function that is owned by the same owner as the table and they satisfy standard PostgreSQL rules (again must have EXECUTE - privilege on the function). So if both table and function have same owner and the + privilege on the function). So if both table and function have the same owner and the owner decided to give a user both TRIGGER privilege on the table and EXECUTE privilege on the function, it is assumed that it is okay for that user to create a trigger on that table using this function. - Users who have TRIGGER privilege on the table can create triggers using functions that are defined with the SECURITY DEFINER clause if they have EXECUTE privilege on them. This clause makes the function always execute in the context - of the owner of the function itself both in standard PostgreSQL and BDR. + of the owner of the function both in standard PostgreSQL and BDR. -The above logic is built on the fact that in PostgreSQL, the owner of the trigger -is not the user who created it but the owner of the function used by that trigger. +This logic is built on the fact that, in PostgreSQL, the owner of the trigger +isn't the user who created it but the owner of the function used by that trigger. -The same rules apply to existing tables, and if the existing table has triggers which -are not owned by the owner of the table and do not use SECURITY DEFINER functions, -it will not be possible to add it to a replication set. +The same rules apply to existing tables, and if the existing table has triggers that +aren't owned by the owner of the table and don't use SECURITY DEFINER functions, +you can't add it to a replication set. -These checks were added with BDR 3.6.19. An application that +These checks were added with BDR 3.6.19. An application that relies on the behavior of previous versions can set `bdr.backwards_compatibility` to 30618 (or lower) to behave like earlier versions. BDR replication apply uses the system-level default search_path only. -Replica triggers, stream triggers -and index expression functions may assume other search_path settings which will then fail when they -execute on apply. To ensure this does not occur, resolve object references clearly using either the default -search_path only (always use fully qualified references to objects, e.g. schema.objectname), or set the search -path for a function using ALTER FUNCTION ... SET search_path = ... for the functions affected. +Replica triggers, stream triggers, +and index expression functions might assume other search_path settings which then fail when they +execute on apply. To ensure this doesn't occur, resolve object references clearly using either the default +search_path only (always use fully qualified references to objects, e.g., schema.objectname), or set the search +path for a function using `ALTER FUNCTION ... SET search_path = ...` for the functions affected. -## BDR Default/Predefined Roles +## BDR default/predefined roles BDR predefined roles are created when the BDR extension is installed. -Note that after BDR extension is dropped from a database, the roles continue to exist +After BDR extension is dropped from a database, the roles continue to exist and need to be dropped manually if required. This allows BDR to be used in multiple databases on the same PostgreSQL instance without problem. -Remember that the `GRANT ROLE` DDL statement does not participate in BDR replication, -thus you should execute this on each node of a cluster. +The `GRANT ROLE` DDL statement doesn't participate in BDR replication. +Thus, execute this on each node of a cluster. ### bdr_superuser @@ -309,26 +307,26 @@ EXECUTE privilege on - `bdr.wait_for_camo_partner_queue` - `bdr.wait_slot_confirm_lsn` -Note that many of the above functions have additional privileges -required before they can be used, for example, you must be +Many of these functions have additional privileges +required before you can use them. For example, you must be the table owner to successfully execute `bdr.alter_sequence_set_kind`. -These additional rules are documented with each specific function. +These additional rules are described with each specific function. ### bdr_read_all_conflicts BDR logs conflicts into the `bdr.conflict_history` table. Conflicts are -visible to table owners (only), so no extra privileges are required -to read the conflict history. If it is useful to have a user that can -see conflicts for *all* tables, you may optionally grant the role -*bdr_read_all_conflicts* to that user. +visible to table owners only, so no extra privileges are required +to read the conflict history. If it's useful to have a user that can +see conflicts for all tables, you can optionally grant the role +bdr_read_all_conflicts to that user. ## Verification -BDR has been verified using the following tools and approaches. +BDR was verified using the following tools and approaches. ### Coverity -Coverity Scan has been used to verify the BDR stack providing coverage +Coverity Scan was used to verify the BDR stack providing coverage against vulnerabilities using the following rules and coding standards: - MISRA C @@ -341,7 +339,7 @@ against vulnerabilities using the following rules and coding standards: ### CIS Benchmark -CIS PostgreSQL Benchmark v1, 19 Dec 2019 has been used to verify the BDR stack. +CIS PostgreSQL Benchmark v1, 19 Dec 2019 was used to verify the BDR stack. Using the `cis_policy.yml` configuration available as an option with TPAexec gives the following results for the Scored tests: @@ -381,8 +379,8 @@ gives the following results for the Scored tests: | 6.8 | PASS | Ensure SSL is enabled and configured correctly | | 7.3 | PASS | Ensure WAL archiving is configured and functional | -Note that test 5.2 can PASS if audited manually, but does not have an -automatable test. +Test 5.2 can PASS if audited manually, but it doesn't have an +automated test. Test 6.7 succeeds on default deployments using CentOS, but it requires extra packages on Debian variants. From 8bff1abb9a1c63d91f913de44c4dc7837d79a09b Mon Sep 17 00:00:00 2001 From: Florin Irion Date: Tue, 21 Jun 2022 12:45:22 +0200 Subject: [PATCH 2/3] Documentation for BDR 4.1.1 (#2704) Add release notes for PGD 4.1.1 Includes release notes for BDR 4.1.1 and HARP 2.1.1 Also adds clarification for bdr.worker_errors, this includes previous BDR version docs. Co-authored-by: Markus Wanner Co-authored-by: Petr Jelinek --- product_docs/docs/bdr/3.6/index.mdx | 4 +- product_docs/docs/bdr/3.7/backup.mdx | 10 +-- product_docs/docs/bdr/3.7/camo.mdx | 2 +- product_docs/docs/bdr/3.7/camo_clients.mdx | 2 +- product_docs/docs/bdr/3.7/configuration.mdx | 14 ++-- product_docs/docs/bdr/3.7/eager.mdx | 2 +- product_docs/docs/bdr/3.7/index.mdx | 2 +- .../docs/bdr/3.7/isolation_details.mdx | 2 +- product_docs/docs/bdr/3.7/monitoring.mdx | 14 +++- product_docs/docs/bdr/3.7/overview.mdx | 2 +- product_docs/docs/bdr/4/camo.mdx | 5 ++ product_docs/docs/bdr/4/configuration.mdx | 16 ++-- product_docs/docs/bdr/4/eager.mdx | 7 +- product_docs/docs/bdr/4/functions.mdx | 22 ++++++ product_docs/docs/bdr/4/group-commit.mdx | 76 +++++++++++++------ .../4/release_notes/bdr4.1.1_rel_notes.mdx | 29 +++++++ .../01_release_notes/harp2.1.1_rel_notes.mdx | 15 ++++ .../docs/harp/2/01_release_notes/index.mdx | 1 + product_docs/docs/harp/2/03_installation.mdx | 18 ++--- product_docs/docs/pgd/4/backup.mdx | 10 +-- product_docs/docs/pgd/4/monitoring.mdx | 9 ++- .../pgd/4/rel_notes/epd_4_1_1_rel_notes.mdx | 12 +++ .../docs/pgd/4/rel_notes/epd_4_rel_notes.mdx | 10 +-- product_docs/docs/pgd/4/rel_notes/index.mdx | 9 ++- 24 files changed, 208 insertions(+), 85 deletions(-) create mode 100644 product_docs/docs/bdr/4/release_notes/bdr4.1.1_rel_notes.mdx create mode 100644 product_docs/docs/harp/2/01_release_notes/harp2.1.1_rel_notes.mdx create mode 100644 product_docs/docs/pgd/4/rel_notes/epd_4_1_1_rel_notes.mdx diff --git a/product_docs/docs/bdr/3.6/index.mdx b/product_docs/docs/bdr/3.6/index.mdx index e6845be5b39..cd2448a1ea7 100644 --- a/product_docs/docs/bdr/3.6/index.mdx +++ b/product_docs/docs/bdr/3.6/index.mdx @@ -29,7 +29,7 @@ BDR Enterprise requires EDB Postgres Extended v11 (formerly known as 2ndQuadrant !!!note The documentation for the latest stable 3.6 release is available here: - [BDR 3.6 Enterprise Edition](https://documentation.2ndquadrant.com/bdr3-enterprise/release/latest-3.6/) + [BDR 3.6 Enterprise Edition](https://documentation.enterprisedb.com/bdr3-enterprise/release/latest-3.6/) **This is a protected area of our website, if you need access please [contact us](https://www.enterprisedb.com/contact)** !!! @@ -52,7 +52,7 @@ BDR Standard requires PostgreSQL v10 or v11. !!!note The documentation for the latest stable 3.6 release is available here: - [BDR 3.6 Standard Edition](https://documentation.2ndquadrant.com/bdr3/release/latest-3.6/) + [BDR 3.6 Standard Edition](https://documentation.enterprisedb.com/bdr3/release/latest-3.6/) **This is a protected area of our website, if you need access please [contact us](https://www.enterprisedb.com/contact)** !!! diff --git a/product_docs/docs/bdr/3.7/backup.mdx b/product_docs/docs/bdr/3.7/backup.mdx index 963d4b3b609..d94f7b2a449 100644 --- a/product_docs/docs/bdr/3.7/backup.mdx +++ b/product_docs/docs/bdr/3.7/backup.mdx @@ -52,7 +52,7 @@ backup and recovery of BDR. Physical backups of a node in a BDR cluster can be taken using standard PostgreSQL software, such as -[Barman](https://www.2ndquadrant.com/en/resources/barman/). +[Barman](https://www.enterprisedb.com/docs/supported-open-source/barman/). A physical backup of a BDR node can be performed with the same procedure that applies to any PostgreSQL node: a BDR node is just a @@ -86,7 +86,7 @@ entirely *consistent*; a physical backup of a given node will provide Point-In-Time Recovery capabilities limited to the states actually assumed by that node (see the [Example] below). -The following example shows how two nodes in the same BDR cluster might not +The following example shows how two nodes in the same BDR cluster might not (and usually do not) go through the same sequence of states. Consider a cluster with two nodes `N1` and `N2`, which is initially in @@ -106,7 +106,7 @@ node `N1` will go through the following states: That is: node `N1` will *never* assume state `S + W2`, and node `N2` likewise will never assume state `S + W1`, but both nodes will end up -in the same state `S + W1 + W2`. Considering this situation might affect how +in the same state `S + W1 + W2`. Considering this situation might affect how you decide upon your backup strategy. ### Point-In-Time Recovery (PITR) @@ -135,7 +135,7 @@ BDR allows for changes from multiple masters, all recorded within the WAL log for one node, separately identified using replication origin identifiers. -BDR allows PITR of all or some replication origins to a specific point in time, +BDR allows PITR of all or some replication origins to a specific point in time, providing a fully consistent viewpoint across all subsets of nodes. Thus for multi-origins, we view the WAL stream as containing multiple @@ -195,7 +195,7 @@ to the original PostgreSQL PITR behaviour that is designed around the assumption of changes arriving from a single master in COMMIT order. !!! Note - This is feature is only available on EDB Postgres Extended and + This is feature is only available on EDB Postgres Extended and Barman does not currently automatically create a `multi_recovery.conf` file. ## Restore diff --git a/product_docs/docs/bdr/3.7/camo.mdx b/product_docs/docs/bdr/3.7/camo.mdx index 77823dbb597..55f580e0fb5 100644 --- a/product_docs/docs/bdr/3.7/camo.mdx +++ b/product_docs/docs/bdr/3.7/camo.mdx @@ -5,4 +5,4 @@ originalFilePath: camo.md --- - + diff --git a/product_docs/docs/bdr/3.7/camo_clients.mdx b/product_docs/docs/bdr/3.7/camo_clients.mdx index c831143c122..d4065f64d14 100644 --- a/product_docs/docs/bdr/3.7/camo_clients.mdx +++ b/product_docs/docs/bdr/3.7/camo_clients.mdx @@ -5,4 +5,4 @@ originalFilePath: camo_clients.md --- - + diff --git a/product_docs/docs/bdr/3.7/configuration.mdx b/product_docs/docs/bdr/3.7/configuration.mdx index 2770d197839..31ddba8a6f0 100644 --- a/product_docs/docs/bdr/3.7/configuration.mdx +++ b/product_docs/docs/bdr/3.7/configuration.mdx @@ -86,17 +86,17 @@ become relevant with BDR in combination with CAMO. ## pglogical Settings for BDR -BDR is also affected by some of the pglogical settings as it uses +BDR is also affected by some of the pglogical settings as it uses pglogical internally to implement the basic replication. -- `pglogical.track_subscription_apply` - Track apply statistics for +- `pglogical.track_subscription_apply` - Track apply statistics for each subscription. -- `pglogical.track_relation_apply` - Track apply statistics for each +- `pglogical.track_relation_apply` - Track apply statistics for each relation. - `pglogical.track_apply_lock_timing` - Track lock timing when tracking statistics for relations. - `pglogical.standby_slot_names` - When using physical Standby nodes - intended for failover purposes, should be set to the replication + intended for failover purposes, should be set to the replication slot(s) for each intended Standby. - `pglogical.writers_per_subscription` - Default number of writers per subscription (in BDR this can also be changed by @@ -228,7 +228,7 @@ Unless noted otherwise, values may be set by any user at any time. this setting. It is normal for some statements to result in two `WARNING`s, one for skipping the DML lock and one for skipping the DDL lock. -- `bdr.truncate_locking` - False by default, this configuration option sets the +- `bdr.truncate_locking` - False by default, this configuration option sets the TRUNCATE command's locking behavior. Determines whether (when true) TRUNCATE obeys the bdr.ddl_locking setting. @@ -401,7 +401,7 @@ Unless noted otherwise, values may be set by any user at any time. - `bdr.trace_level` - Similar to the above, this defines the log level to use for BDR trace messages. Enabling tracing on all nodes of a - BDR cluster may help 2ndQuadrant Support to diagnose issues. + BDR cluster may help EDB Support to diagnose issues. May only be set at Postgres server start. !!! Warning @@ -432,7 +432,7 @@ Unless noted otherwise, values may be set by any user at any time. - `bdr.raft_group_max_connections` - The maximum number of connections across all BDR groups for a Postgres server. These connections carry bdr consensus requests between the groups' nodes. Default value of this - parameter is 100 connections. May only be set at Postgres server start. + parameter is 100 connections. May only be set at Postgres server start. - `bdr.backwards_compatibility` - Specifies the version to be backwards-compatible to, in the same numerical format as used by `bdr.bdr_version_num`, e.g. `30618`. Enables exact behavior of a diff --git a/product_docs/docs/bdr/3.7/eager.mdx b/product_docs/docs/bdr/3.7/eager.mdx index e1f9583f997..c37b0800794 100644 --- a/product_docs/docs/bdr/3.7/eager.mdx +++ b/product_docs/docs/bdr/3.7/eager.mdx @@ -4,4 +4,4 @@ originalFilePath: eager.md --- - + diff --git a/product_docs/docs/bdr/3.7/index.mdx b/product_docs/docs/bdr/3.7/index.mdx index 10adfb19744..d4539a607d7 100644 --- a/product_docs/docs/bdr/3.7/index.mdx +++ b/product_docs/docs/bdr/3.7/index.mdx @@ -44,7 +44,7 @@ provides a solution for building multi-master clusters with mesh topology. This means that you can write to any server and the changes will be sent row-by-row to all the other servers that are part of the same BDR group. -BDR version 3 ("BDR3") is built on the [pglogical3](https://www.2ndquadrant.com/resources/pglogical/) +BDR version 3 ("BDR3") is built on the [pglogical3](https://www.enterprisedb.com/docs/pglogical/latest/) extension. However, everything you need to know about BDR3 is included here and it is unnecessary, as well as potentially confusing, to refer to pglogical docs. diff --git a/product_docs/docs/bdr/3.7/isolation_details.mdx b/product_docs/docs/bdr/3.7/isolation_details.mdx index e4f94c337d9..cc6145a4733 100644 --- a/product_docs/docs/bdr/3.7/isolation_details.mdx +++ b/product_docs/docs/bdr/3.7/isolation_details.mdx @@ -5,4 +5,4 @@ originalFilePath: isolation/details.md --- - + diff --git a/product_docs/docs/bdr/3.7/monitoring.mdx b/product_docs/docs/bdr/3.7/monitoring.mdx index bb03d9e2e0d..acfeaab8286 100644 --- a/product_docs/docs/bdr/3.7/monitoring.mdx +++ b/product_docs/docs/bdr/3.7/monitoring.mdx @@ -319,10 +319,16 @@ So this view offers these insights into the state of a BDR system: The `bdr.workers` view shows BDR worker specific details, that are not available from `bdr.stat_activity`. -The view `bdr.worker_errors` shows errors (if any) reported by any worker -which has a problem continuing the work. Only active errors are visible in this -view, so if the worker was having transient problems but has recovered, the -view will be empty. +The view `bdr.worker_errors` shows errors (if any) reported by any worker. +BDR 3.7 depended explicitly on pglogical 3.7 as a separate extension. While +pglogical deletes older worker errors, BDR does not aim to, given the additional +complexity of bi-directional replication. A side effect of this dependency is +that in BDR 3.7 some worker errors are deleted over time, while others are retained +indefinitely. Because of this it's important to note the time of the error +and not just the existence of one. +Starting from BDR 4, there is a single extension, and dependency on +pglogical as a separate extension has been removed, meaning that all worker errors +are now retained indefinitely. ## Monitoring Global Locks diff --git a/product_docs/docs/bdr/3.7/overview.mdx b/product_docs/docs/bdr/3.7/overview.mdx index 40ae051b7f9..064afc7a9d2 100644 --- a/product_docs/docs/bdr/3.7/overview.mdx +++ b/product_docs/docs/bdr/3.7/overview.mdx @@ -37,7 +37,7 @@ replicate either from the master or from another standby. You don't have to write to all the masters, all of the time; it's a frequent configuration to direct writes mostly to just one master. However, if you just want one-way replication, the use of -[pglogical](https://2ndquadrant.com/pglogical) may be more appropriate. +[pglogical](https://www.enterprisedb.com/docs/pglogical/latest/) may be more appropriate. ### Asynchronous, by default diff --git a/product_docs/docs/bdr/4/camo.mdx b/product_docs/docs/bdr/4/camo.mdx index 81a0eba8a82..431b1acb174 100644 --- a/product_docs/docs/bdr/4/camo.mdx +++ b/product_docs/docs/bdr/4/camo.mdx @@ -557,6 +557,11 @@ the CAMO transaction uses the timestamp of the prepare on the origin node, which is before the transaction becomes visible on the origin node itself. +- CAMO is not currently compatible with transaction streaming. Please +ensure to disable transaction streaming when planning to use +CAMO. This can be configured globally or in the BDR node group, see +[Transaction Streaming Configuration](transaction-streaming#configuration). + ## Performance implications CAMO extends the Postgres replication protocol by adding a diff --git a/product_docs/docs/bdr/4/configuration.mdx b/product_docs/docs/bdr/4/configuration.mdx index d91a3378034..c996b063bd9 100644 --- a/product_docs/docs/bdr/4/configuration.mdx +++ b/product_docs/docs/bdr/4/configuration.mdx @@ -307,13 +307,15 @@ the failover candidate hadn't received it yet: the change. The subscribers now have inconsistent and irreconcilable states because the subscribers that didn't receive the commit have no way to get it. -Setting `bdr.standby_slot_names` by design causes subscribers to -lag behind the provider if the provider's failover-candidate replicas aren't -keeping up. Monitoring is thus essential. - -Another use case where the `bdr.standby_slot_names` is useful is when using -subscriber-only, to ensure that the subscriber-only node doesn't move ahead -of any of the other BDR nodes. +Setting `bdr.standby_slot_names` by design causes other subscribers +not listed in there to lag behind the provider if the required number +of listed nodes are not keeping up. Monitoring is thus essential. + +Another use case where `bdr.standby_slot_names` is useful is when +using a subscriber-only node, to ensure that it does not move ahead of +any of the regular BDR nodes. This can best be achieved by listing the +logical slots of all regular BDR peer nodes in combination with +setting `bdr.standby_slots_min_confirmed` to at least one. ### `bdr.standby_slots_min_confirmed` diff --git a/product_docs/docs/bdr/4/eager.mdx b/product_docs/docs/bdr/4/eager.mdx index 54ecebd0c42..3331ab398bd 100644 --- a/product_docs/docs/bdr/4/eager.mdx +++ b/product_docs/docs/bdr/4/eager.mdx @@ -114,9 +114,10 @@ You can't combine Eager All-Node Replication with `synchronous_replication_availability = 'async'`. Trying to configure both causes an error. -The Decoding Worker feature isn't currently supported in combination -with Eager All-Node transactions. Installations using Eager must keep -`enable_wal_decoder` disabled for the BDR node group using Eager All-Node transactions. +Eager All-Node transactions are not currently supported in combination +with the Decoding Worker feature nor with transaction streaming. +Installations using Eager must keep `enable_wal_decoder` and `streaming_mode` +disabled for the BDR node group. Synchronous replication uses a mechanism for transaction confirmation different from Eager. The two aren't compatible, and you must not use them diff --git a/product_docs/docs/bdr/4/functions.mdx b/product_docs/docs/bdr/4/functions.mdx index 4ea80d37c1e..f65e020dc06 100644 --- a/product_docs/docs/bdr/4/functions.mdx +++ b/product_docs/docs/bdr/4/functions.mdx @@ -67,6 +67,28 @@ becomes remotely visible. As soon as Postgres assigns a transaction id, if CAMO is enabled, this parameter is updated to show the transaction id just assigned. +### bdr.is_node_connected + +#### Synopsis + +```sql +bdr.is_node_connected(node_name name) +``` + +Returns boolean by checking if the walsender for a given peer is active +on this node. + +### bdr.is_node_ready + +#### Synopsis + +```sql +bdr.is_node_ready(node_name name, span interval DEFAULT NULL) +``` + +Returns boolean by checking if the lag is lower than the given span or +lower than the `bdr.global_commit_timeout` otherwise. + ## Consensus function ### bdr.consensus_disable diff --git a/product_docs/docs/bdr/4/group-commit.mdx b/product_docs/docs/bdr/4/group-commit.mdx index eb52f25a7a8..bb08a81f5b2 100644 --- a/product_docs/docs/bdr/4/group-commit.mdx +++ b/product_docs/docs/bdr/4/group-commit.mdx @@ -10,7 +10,7 @@ confirm a transaction at COMMIT time. ## Requirements During normal operation, Group Commit is completely transparent to the -application. Upon failover, the reconciliation phase must be +application. Upon failover, the reconciliation phase needs to be explicitly triggered or consolidated by either the application or a proxy in between. HARP provides native support for Group Commit and triggers the reconciliation phase, making this equally transparent @@ -70,14 +70,19 @@ remote one. ```sql -- create sub-groups -SELECT bdr.create_node_group(node_group_name := 'left_dc', - parent_group_name := 'top_group', - join_node_group := false); -SELECT bdr.create_node_group(node_group_name := 'right_dc', - parent_group_name := 'top_group', - join_node_group := false); - --- create a commit scope with individual rules for each sub-group +SELECT bdr.create_node_group( + node_group_name := 'left_dc', + parent_group_name := 'top_group', + join_node_group := false +); +SELECT bdr.create_node_group( + node_group_name := 'right_dc', + parent_group_name := 'top_group', + join_node_group := false +); + +-- create a commit scope with individual rules +-- for each sub-group SELECT bdr.add_commit_scope( commit_scope_name := 'example_scope', origin_node_group := 'left_dc', @@ -110,28 +115,40 @@ these are: In rules for commit scopes, you can append these confirmation levels to the node group definition in parenthesis with `ON` as follows: -* `ANY 1 (right_dc) ON remote_write` -* `ALL (left_dc) ON remote_commit_flush` (default and may as well be +* `ANY 2 (right_dc) ON replicated` +* `ALL (left_dc) ON visible` (default and may as well be omitted) -* `ALL (left_dc) ON remote_commit_async AND ANY 1 (right_dc) ON remote_write` +* `ALL (left_dc) ON received AND ANY 1 (right_dc) ON durable` ## Reference ### Commit scope grammar -For reference, the grammar for commit scopes is as follows: +For reference, the grammar for commit scopes is composed as follows: ``` -commit_scope: confirmation - | confirmation AND commit_scope +commit_scope: + confirmation [AND ...] -confirmation: node_def (ON [received|replicated|durable|visible]) +confirmation: + node_def (ON [received|replicated|durable|visible]) -node_def: ANY num (node_group [, ...]) - | MAJORITY (node_group [, ...]) - | ALL (node_group [, ...]) +node_def: + ANY num (node_group [, ...]) + | MAJORITY (node_group [, ...]) + | ALL (node_group [, ...]) ``` +!!! Note + While the grammar for `synchronous_standby_names` and Commit + Scopes looks very similar, it is important to note that the former + does not account for the origin node, but the latter does. + Therefore, for example `synchronous_standby_names = 'ANY 1 (..)'` + is equivalent to a Commit Scope of `ANY 2 (...)`. This choice + makes reasoning about majority easier and reflects that the origin + node also contributes to the durability and visibility of the + transaction. + ### Adding a commit scope rule The function `bdr.add_commit_scope` creates a rule for the given @@ -146,8 +163,10 @@ commit scopes that vary depending on the origin of the transaction. #### Synopsis ```sql -bdr.add_commit_scope(commit_scope_name NAME, origin_node_group NAME, - rule TEXT) +bdr.add_commit_scope( + commit_scope_name NAME, + origin_node_group NAME, + rule TEXT) ``` ### Changing a commit scope rule @@ -158,8 +177,10 @@ commit scope, you can use the function `bdr.alter_commit_scope`. #### Synopsis ```sql -bdr.alter_commit_scope(commit_scope_name NAME, origin_node_group NAME, - rule TEXT) +bdr.alter_commit_scope( + commit_scope_name NAME, + origin_node_group NAME, + rule TEXT) ``` ### Removing a commit scope rule @@ -172,5 +193,12 @@ commit scope. #### Synopsis ```sql -bdr.remove_commit_scope(commit_scope_name NAME, origin_node_group NAME) +bdr.remove_commit_scope( + commit_scope_name NAME, + origin_node_group NAME) ``` + +!!! Note + Removing a commit scope that is still used as default by a node + group is not allowed + diff --git a/product_docs/docs/bdr/4/release_notes/bdr4.1.1_rel_notes.mdx b/product_docs/docs/bdr/4/release_notes/bdr4.1.1_rel_notes.mdx new file mode 100644 index 00000000000..ae14f5ba1cf --- /dev/null +++ b/product_docs/docs/bdr/4/release_notes/bdr4.1.1_rel_notes.mdx @@ -0,0 +1,29 @@ +--- +title: "BDR 4.1.1" +--- + +This is a maintenance release of BDR 4 which includes new features as well +as fixes for issues identified in previous versions. + + +| Type | Category | Description | +| ---- | -------- | ----------- | +| Feature | User Experience | Add generic function bdr.is_node_connected returns true if the walsender for a given peer is active. +| Feature | User Experience | Add generic function bdr.is_node_ready returns boolean if the lag is under a specific span. +| Bug fix | User Experience | Add support for a `--link` argument to bdr_pg_upgrade for using hard-links. +| Bug fix | User Experience | Prevent removing a `bdr.remove_commit_scope` if still referenced by any `bdr.node_group` as the default commit scope. +| Bug fix | Reliability and operability | Correct Raft based switching to Local Mode for CAMO pairs of nodes (RT78928) +| Bug fix | Reliability and operability | Prevent a potential segfault in bdr.drop_node for corner cases (RT81900) +| Bug fix | User Experience | Prevent use of CAMO or Eager All Node transactions in combination with transaction streaming

Transaction streaming turned out to be problematic in combination with CAMO and Eager All Node transactions. Until this is resolved, BDR now prevents its combined use. This may require CAMO deployments to adjust their configuration to disable transaction streaming, see [Transaction Streaming Configuration](transaction-streaming#configuration). | + + +### Upgrades + +This release supports upgrading from the following versions of BDR: + +- 4.0.0 and higher +- 3.7.15 +- 3.7.16 + +Please make sure you read and understand the process and limitations described +in the [Upgrade Guide](/pgd/latest/upgrades/) before upgrading. diff --git a/product_docs/docs/harp/2/01_release_notes/harp2.1.1_rel_notes.mdx b/product_docs/docs/harp/2/01_release_notes/harp2.1.1_rel_notes.mdx new file mode 100644 index 00000000000..d246842271f --- /dev/null +++ b/product_docs/docs/harp/2/01_release_notes/harp2.1.1_rel_notes.mdx @@ -0,0 +1,15 @@ +--- +title: "Version 2.1.1" +--- + +This is a patch release of HARP 2 that includes fixes for issues identified +in previous versions. + +| Type | Description | +| ---- |------------ | +| Enhancement | Log a warning on loss of DCS connection | +| Enhancement | Log a warning when metadata refresh is taking too long - usually due to high latency network | +| Bug Fix | Restart harp_proxy.service on a failure | +| Bug Fix | Fix concurrency issue with connection management in harpctl | +| Bug Fix | Don't try to proxy connections to previous leader on unmanaged cluster | +| Bug Fix | Don't panic in haprctl when location is empty | diff --git a/product_docs/docs/harp/2/01_release_notes/index.mdx b/product_docs/docs/harp/2/01_release_notes/index.mdx index d8e0f48b601..6905f183a81 100644 --- a/product_docs/docs/harp/2/01_release_notes/index.mdx +++ b/product_docs/docs/harp/2/01_release_notes/index.mdx @@ -18,6 +18,7 @@ The release notes in this section provide information on what was new in each re | Version | Release Date | | ----------------------- | ------------ | +| [2.1.1](harp2.1.1_rel_notes) | 2022 June 21 | | [2.1.0](harp2.1.0_rel_notes) | 2022 May 17 | | [2.0.3](harp2.0.3_rel_notes) | 2022 Mar 31 | | [2.0.2](harp2.0.2_rel_notes) | 2022 Feb 24 | diff --git a/product_docs/docs/harp/2/03_installation.mdx b/product_docs/docs/harp/2/03_installation.mdx index a347de1ba26..b30b6d2bdf2 100644 --- a/product_docs/docs/harp/2/03_installation.mdx +++ b/product_docs/docs/harp/2/03_installation.mdx @@ -25,7 +25,7 @@ version as listed here. The easiest way to install and configure HARP is to use the EDB TPAexec utility for cluster deployment and management. For details on this software, see the -[TPAexec product page](https://access.2ndquadrant.com/customer_portal/sw/tpa/). +[TPAexec product page](https://www.enterprisedb.com/docs/pgd/latest/deployments/tpaexec/). !!! Note TPAExec is currently available only through an EULA specifically dedicated @@ -65,13 +65,13 @@ considerations. Currently CentOS/RHEL packages are provided by the EDB packaging infrastructure. For details, see the [HARP product -page](https://access.2ndquadrant.com/customer_portal/sw/harp/). +page](https://www.enterprisedb.com/docs/harp/latest/). ### etcd packages -Currently `etcd` packages for many popular Linux distributions aren't -available by their standard public repositories. EDB has therefore packaged -`etcd` for RHEL and CentOS versions 7 and 8, Debian, and variants such as +Currently `etcd` packages for many popular Linux distributions aren't +available by their standard public repositories. EDB has therefore packaged +`etcd` for RHEL and CentOS versions 7 and 8, Debian, and variants such as Ubuntu LTS. You need access to our HARP package repository to use these libraries. @@ -99,20 +99,20 @@ dcs: - host3:2379 ``` -When using TPAExec, all configured etcd endpoints are entered here +When using TPAExec, all configured etcd endpoints are entered here automatically. ### BDR -The `bdr` native consensus layer is available from BDR 3.6.21 and 3.7.3. This -consensus layer model requires no supplementary software when managing routing +The `bdr` native consensus layer is available from BDR 3.6.21 and 3.7.3. This +consensus layer model requires no supplementary software when managing routing for a BDR cluster. To ensure quorum is possible in the cluster, always use more than two nodes so that BDR's consensus layer remains responsive during node maintenance or outages. -To set BDR as the consensus layer, include this in the `config.yml` +To set BDR as the consensus layer, include this in the `config.yml` configuration file: ```yaml diff --git a/product_docs/docs/pgd/4/backup.mdx b/product_docs/docs/pgd/4/backup.mdx index aeeebcbe36a..434341d9aab 100644 --- a/product_docs/docs/pgd/4/backup.mdx +++ b/product_docs/docs/pgd/4/backup.mdx @@ -52,7 +52,7 @@ backup and recovery of BDR. Physical backups of a node in a BDR cluster can be taken using standard PostgreSQL software, such as -[Barman](https://www.2ndquadrant.com/en/resources/barman/). +[Barman](https://www.enterprisedb.com/docs/supported-open-source/barman/). A physical backup of a BDR node can be performed with the same procedure that applies to any PostgreSQL node: a BDR node is just a @@ -86,7 +86,7 @@ entirely *consistent*; a physical backup of a given node will provide Point-In-Time Recovery capabilities limited to the states actually assumed by that node (see the [Example] below). -The following example shows how two nodes in the same BDR cluster might not +The following example shows how two nodes in the same BDR cluster might not (and usually do not) go through the same sequence of states. Consider a cluster with two nodes `N1` and `N2`, which is initially in @@ -106,7 +106,7 @@ node `N1` will go through the following states: That is: node `N1` will *never* assume state `S + W2`, and node `N2` likewise will never assume state `S + W1`, but both nodes will end up -in the same state `S + W1 + W2`. Considering this situation might affect how +in the same state `S + W1 + W2`. Considering this situation might affect how you decide upon your backup strategy. ### Point-In-Time Recovery (PITR) @@ -135,7 +135,7 @@ BDR allows for changes from multiple masters, all recorded within the WAL log for one node, separately identified using replication origin identifiers. -BDR allows PITR of all or some replication origins to a specific point in time, +BDR allows PITR of all or some replication origins to a specific point in time, providing a fully consistent viewpoint across all subsets of nodes. Thus for multi-origins, we view the WAL stream as containing multiple @@ -195,7 +195,7 @@ to the original PostgreSQL PITR behaviour that is designed around the assumption of changes arriving from a single master in COMMIT order. !!! Note - This is feature is only available on EDB Postgres Extended and + This is feature is only available on EDB Postgres Extended and Barman does not currently automatically create a `multi_recovery.conf` file. ## Restore diff --git a/product_docs/docs/pgd/4/monitoring.mdx b/product_docs/docs/pgd/4/monitoring.mdx index ce2e545c628..870c3057218 100644 --- a/product_docs/docs/pgd/4/monitoring.mdx +++ b/product_docs/docs/pgd/4/monitoring.mdx @@ -319,10 +319,11 @@ So this view offers these insights into the state of a BDR system: The `bdr.workers` view shows BDR worker specific details, that are not available from `bdr.stat_activity`. -The view `bdr.worker_errors` shows errors (if any) reported by any worker -which has a problem continuing the work. Only active errors are visible in this -view, so if the worker was having transient problems but has recovered, the -view will be empty. +The view `bdr.worker_errors` shows last error (if any) reported by any worker +which has a problem continuing the work. This is persistent information, so +it's important to note the time of the error not just the existence of one, +because most errors are transient in their nature and BDR workers will retry +the failed operation. ## Monitoring BDR Writers diff --git a/product_docs/docs/pgd/4/rel_notes/epd_4_1_1_rel_notes.mdx b/product_docs/docs/pgd/4/rel_notes/epd_4_1_1_rel_notes.mdx new file mode 100644 index 00000000000..cb4f7e791f2 --- /dev/null +++ b/product_docs/docs/pgd/4/rel_notes/epd_4_1_1_rel_notes.mdx @@ -0,0 +1,12 @@ +--- +title: "Release notes for EDB Postgres Distributed version 4.1.1" +navTitle: "Version 4.1.1" +--- + +EDB Postgres Distributed version 4.1.1 includes bug fixes for BDR, HARP and +TPAexec. + +## Related release notes + +- [BDR release notes](/bdr/latest/release_notes/bdr4.1.1_rel_notes/) +- [HARP release notes](/harp/latest/01_release_notes/harp2.1.1_rel_notes/) diff --git a/product_docs/docs/pgd/4/rel_notes/epd_4_rel_notes.mdx b/product_docs/docs/pgd/4/rel_notes/epd_4_rel_notes.mdx index 8b164c2ad65..ecd77666c46 100644 --- a/product_docs/docs/pgd/4/rel_notes/epd_4_rel_notes.mdx +++ b/product_docs/docs/pgd/4/rel_notes/epd_4_rel_notes.mdx @@ -1,9 +1,9 @@ --- -title: "Release notes for EDB Postgres Distributed version 4" -navTitle: "Version 4" +title: "Release notes for EDB Postgres Distributed version 4.1" +navTitle: "Version 4.1" --- -Key features of EDB Postgres Distributed version 4 include: +Key features of EDB Postgres Distributed version 4.1 include: | Feature | Description | |--------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| @@ -12,9 +12,9 @@ Key features of EDB Postgres Distributed version 4 include: | Distributed sequences by default | Instead of setting up custom sequence handling configuration on every node, you get distributed sequences configured automatically with the best kind of sequence for the data type used. The new SnowflakeID algorithm replaces Timeshard, which had constraints that prevented us from using it as a default. | | Postgres Distributed Command Line Interface (CLI | You can get information about your Postgres Distributed cluster right from the CLI, including the current state of the replication, consensus, and nodes. You can also query across the nodes of a cluster. Instead of relying on queries, you can use simplified commands in the CLI. See [EDB Postgres Distributed Command Line Interface](/pgd_cli/latest) for more information. | | Simplified synchronous replication configuration | Instead of adjusting multiple different Grand Unified Configurations (GUCs) for each node, you can use a SQL level interface to configure how transactions are replicated and what durability and visibility criteria to use, allowing for all nodes to be configured consistently from a single place. This allows (along with Group Commits) more functionality and flexibility around synchronous replication in the future. | -| Group Commit | You can have synchronous replication to require quorum within a Postgres Distributed group before committing a transaction, making EDB a more attractive alternative to Oracle DataGuard or Postgres Synchronous Replication. This allows more functionality around synchronous replication in the future, like Eager Majority. +| Group Commit | You can have synchronous replication to require quorum within a Postgres Distributed group before committing a transaction, making EDB a more attractive alternative to Oracle DataGuard or Postgres Synchronous Replication. This allows more functionality around synchronous replication in the future, like Eager Majority. ## Related release notes + - [BDR release notes](/bdr/latest/release_notes/bdr4.1.0_rel_notes/) - [HARP release notes](/harp/latest/01_release_notes/harp2.1.0_rel_notes/) - \ No newline at end of file diff --git a/product_docs/docs/pgd/4/rel_notes/index.mdx b/product_docs/docs/pgd/4/rel_notes/index.mdx index e1206d73166..2b371f411fc 100644 --- a/product_docs/docs/pgd/4/rel_notes/index.mdx +++ b/product_docs/docs/pgd/4/rel_notes/index.mdx @@ -2,8 +2,9 @@ title: "Release notes" --- -The EDB Postgres Distributed documentation describes the Always On architectures supported in EDB Postgres Distributed 4. +The EDB Postgres Distributed documentation describes the Always On architectures supported in EDB Postgres Distributed 4. -| Version | Release Date | -| -------------------- | ------------ | -| [4](epd_4_rel_notes) | 2022 May 17 | +| Version | Release Date | +| ------------------------ | ------------ | +| [4.1.1](epd_4_1_1_rel_notes) | 2022 June 21 | +| [4.1](epd_4_rel_notes) | 2022 May 17 | From 80dbdb855157e4267e9eb89193b04999ae31b486 Mon Sep 17 00:00:00 2001 From: drothery-edb Date: Tue, 21 Jun 2022 07:25:16 -0400 Subject: [PATCH 3/3] release note fix and landing page updates --- .../bdr/4/release_notes/bdr4.1.1_rel_notes.mdx | 14 +++++++------- product_docs/docs/bdr/4/release_notes/index.mdx | 2 ++ 2 files changed, 9 insertions(+), 7 deletions(-) diff --git a/product_docs/docs/bdr/4/release_notes/bdr4.1.1_rel_notes.mdx b/product_docs/docs/bdr/4/release_notes/bdr4.1.1_rel_notes.mdx index ae14f5ba1cf..e23b928a0cf 100644 --- a/product_docs/docs/bdr/4/release_notes/bdr4.1.1_rel_notes.mdx +++ b/product_docs/docs/bdr/4/release_notes/bdr4.1.1_rel_notes.mdx @@ -8,13 +8,13 @@ as fixes for issues identified in previous versions. | Type | Category | Description | | ---- | -------- | ----------- | -| Feature | User Experience | Add generic function bdr.is_node_connected returns true if the walsender for a given peer is active. -| Feature | User Experience | Add generic function bdr.is_node_ready returns boolean if the lag is under a specific span. -| Bug fix | User Experience | Add support for a `--link` argument to bdr_pg_upgrade for using hard-links. -| Bug fix | User Experience | Prevent removing a `bdr.remove_commit_scope` if still referenced by any `bdr.node_group` as the default commit scope. -| Bug fix | Reliability and operability | Correct Raft based switching to Local Mode for CAMO pairs of nodes (RT78928) -| Bug fix | Reliability and operability | Prevent a potential segfault in bdr.drop_node for corner cases (RT81900) -| Bug fix | User Experience | Prevent use of CAMO or Eager All Node transactions in combination with transaction streaming

Transaction streaming turned out to be problematic in combination with CAMO and Eager All Node transactions. Until this is resolved, BDR now prevents its combined use. This may require CAMO deployments to adjust their configuration to disable transaction streaming, see [Transaction Streaming Configuration](transaction-streaming#configuration). | +| Feature | User Experience | Add generic function bdr.is_node_connected returns true if the walsender for a given peer is active. | +| Feature | User Experience | Add generic function bdr.is_node_ready returns boolean if the lag is under a specific span. | +| Bug fix | User Experience | Add support for a `--link` argument to bdr_pg_upgrade for using hard-links. | +| Bug fix | User Experience | Prevent removing a `bdr.remove_commit_scope` if still referenced by any `bdr.node_group` as the default commit scope. | +| Bug fix | Reliability and operability | Correct Raft based switching to Local Mode for CAMO pairs of nodes (RT78928) | +| Bug fix | Reliability and operability | Prevent a potential segfault in bdr.drop_node for corner cases (RT81900) | +| Bug fix | User Experience | Prevent use of CAMO or Eager All Node transactions in combination with transaction streaming
Transaction streaming turned out to be problematic in combination with CAMO and Eager All Node transactions. Until this is resolved, BDR now prevents its combined use. This may require CAMO deployments to adjust their configuration to disable transaction streaming, see [Transaction Streaming Configuration](transaction-streaming#configuration). | ### Upgrades diff --git a/product_docs/docs/bdr/4/release_notes/index.mdx b/product_docs/docs/bdr/4/release_notes/index.mdx index 0dfb5faf952..bfac5a52d48 100644 --- a/product_docs/docs/bdr/4/release_notes/index.mdx +++ b/product_docs/docs/bdr/4/release_notes/index.mdx @@ -1,6 +1,7 @@ --- title: Release Notes navigation: +- bdr4.1.1_rel_notes - bdr4.1.0_rel_notes - bdr4.0.2_rel_notes - bdr4.0.1_rel_notes @@ -16,6 +17,7 @@ The release notes in this section provide information on what was new in each re | Version | Release Date | | ----------------------- | ------------ | +| [4.1.1](bdr4.1.1_rel_notes) | 2022 June 21 | | [4.1.0](bdr4.1.0_rel_notes) | 2022 May 17 | | [4.0.2](bdr4.0.2_rel_notes) | 2022 Feb 15 | | [4.0.1](bdr4.0.1_rel_notes) | 2022 Jan 18 |