DOCS-757 - Adding ORIGIN_GROUP

product_docs/docs/pgd/5.6/commit-scopes/commit-scopes.mdx

@@ -1,5 +1,6 @@
 ---
 title: Commit scopes
+deepToC: true
 ---
 
 Commit scopes give applications granular control about durability and consistency of EDB Postgres Distributed.
@@ -69,7 +70,8 @@ The `wait_for_ready` controls whether the `bdr.add_commit_scope()` call blocks u
 
 ## Using a commit scope
 
-To use our example scope, we can set `bdr.commit_scope` within a transaction 
+To use our example scope, we can set `bdr.commit_scope` within a transaction
+
 ```sql
 BEGIN;
 SET LOCAL bdr.commit_scope = 'example_scope';
@@ -117,60 +119,4 @@ Finally, you can set the default commit_scope for a node using:
 SET bdr.commit_scope = 'example_scope';
 ```
 
-Set `bdr.commit_scope` to `local` to  use the PGD default async replication.
-
-
-## Origin groups
-
-Rules for commit scopes can depend on the node the transaction is committed on, that is, the node that acts as the origin for the transaction. To make this transparent for the application, PGD allows a commit scope to define different rules depending on where the transaction originates from.
-
-For example, consider an EDB Postgres Distributed cluster with nodes spread across two data centers: a left and a right one. Assume the top-level PGD node group is called `top_group`. You can use the following commands to set up
-subgroups and create a commit scope requiring all nodes in the local data center to confirm the transaction but only one node from the remote one:
-
-```sql
--- create sub-groups
-SELECT bdr.create_node_group(
-    node_group_name := 'left_dc',
-    parent_group_name := 'top_group',
-    join_node_group := false
-);
-SELECT bdr.create_node_group(
-    node_group_name := 'right_dc',
-    parent_group_name := 'top_group',
-    join_node_group := false
-);
-
--- create a commit scope with individual rules
--- for each sub-group
-SELECT bdr.add_commit_scope(
-    commit_scope_name := 'example_scope',
-    origin_node_group := 'left_dc',
-    rule := 'ALL (left_dc) GROUP COMMIT (commit_decision=raft) AND ANY 1 (right_dc) GROUP COMMIT',
-    wait_for_ready := true
-);
-SELECT bdr.add_commit_scope(
-    commit_scope_name := 'example_scope',
-    origin_node_group := 'right_dc',
-    rule := 'ANY 1 (left_dc) GROUP COMMIT AND ALL (right_dc) GROUP COMMIT (commit_decision=raft)',
-    wait_for_ready := true
-);
-```
-
-Now, using the `example_scope` on any node that's part of `left_dc` uses the first scope. Using the same scope on a node that's part of `right_dc` uses the second scope. By combining the `left_dc` and `right_dc` origin rules under one commit scope name, an application can simply use `example_scope` on either data center and get the appropriate behavior for that data center. 
-
-Each group can also have a default commit scope specified using the `bdr.alter_node_group_option` admin interface.
-
-Making the above scopes the default ones for all transactions originating on nodes in those groups looks like this:
-
-```sql
-SELECT bdr.alter_node_group_option(
-  node_group_name := 'left_dc',
-  config_key := 'default_commit_scope',
-  config_value := 'example_scope'
-);
-SELECT bdr.alter_node_group_option(
-  node_group_name := 'right_dc',
-  config_key := 'default_commit_scope',
-  config_value := 'example_scope'
-);
-``` 
+Set `bdr.commit_scope` to `local` to  use the PGD default async replication.

product_docs/docs/pgd/5.6/commit-scopes/index.mdx

@@ -5,6 +5,7 @@ navigation:
  - overview
  - durabilityterminology
  - commit-scopes
+ - origin_groups
  - commit-scope-rules
  - comparing
  - degrading
@@ -40,7 +41,9 @@ durability options, including how to refer to nodes in replication.
 * [Commit scopes](commit-scopes) is a more in-depth look at the structure
 of commit scopes and how to define them for your needs.
 
-* [Commit scope rules](commit-scope-rules) looks at the syntax of and how to compose
+* [Origin groups](origin_groups) introduces the notion of an origin group, and how to leverage these when defining commit scopes rules.
+
+* [Commit scope rules](commit-scope-rules) looks at the syntax of and how to formulate
 a commit scope rule.
 
 * [Comparing durability options](comparing) compares how commit scope options behave with regard to durability.

product_docs/docs/pgd/5.6/commit-scopes/origin_groups.mdx

@@ -0,0 +1,93 @@
+---
+title: Origin groups
+---
+
+Rules for commit scopes can depend on the node the transaction is committed on, that is, the node that acts as the origin for the transaction. The bottom group of the group tree to which that node belongs is the transaction's *origin group*. To make this transparent for the application, PGD allows a commit scope to define different rules depending on the transaction's origin group.
+
+For example, consider an EDB Postgres Distributed cluster with nodes spread across two data centers: a left (`left_dc`) and a right one (`right_dc`). Assume the top-level PGD node group is called `top_group`. You can use the following commands to set up subgroups and create a commit scope requiring all nodes in the local data center to confirm the transaction but only one node from the remote one:
+
+```sql
+-- create sub-groups
+SELECT bdr.create_node_group(
+    node_group_name := 'left_dc',
+    parent_group_name := 'top_group',
+    join_node_group := false
+);
+SELECT bdr.create_node_group(
+    node_group_name := 'right_dc',
+    parent_group_name := 'top_group',
+    join_node_group := false
+);
+
+-- create a commit scope with individual rules
+-- for each sub-group
+SELECT bdr.add_commit_scope(
+    commit_scope_name := 'example_scope',
+    origin_node_group := 'left_dc',
+    rule := 'ALL (left_dc) GROUP COMMIT (commit_decision=raft) AND ANY 1 (right_dc) GROUP COMMIT',
+    wait_for_ready := true
+);
+SELECT bdr.add_commit_scope(
+    commit_scope_name := 'example_scope',
+    origin_node_group := 'right_dc',
+    rule := 'ANY 1 (left_dc) GROUP COMMIT AND ALL (right_dc) GROUP COMMIT (commit_decision=raft)',
+    wait_for_ready := true
+);
+```
+
+Now, using the `example_scope` on any node that's part of `left_dc` uses the first scope. Using the same scope on a node that's part of `right_dc` uses the second scope. By combining the `left_dc` and `right_dc` origin rules under one commit scope name, an application can simply use `example_scope` on either data center and get the appropriate behavior for that data center. 
+
+Each group can also have a default commit scope specified using the `bdr.alter_node_group_option` admin interface.
+
+Making the above scopes the default ones for all transactions originating on nodes in those groups looks like this:
+
+```sql
+SELECT bdr.alter_node_group_option(
+  node_group_name := 'left_dc',
+  config_key := 'default_commit_scope',
+  config_value := 'example_scope'
+);
+SELECT bdr.alter_node_group_option(
+  node_group_name := 'right_dc',
+  config_key := 'default_commit_scope',
+  config_value := 'example_scope'
+);
+```
+
+## ORIGIN_GROUP
+
+You can also refer to the origin group of a transaction dynamically when creating a commit scope rule by using `ORIGIN_GROUP`. 
+
+This can make certain commit scopes rules like those above in `example_scope`, even easier to specify in that you can simply specify one rule instead of two.
+
+For example, again suppose that for transactions originating from nodes in `right_dc` you want all nodes in `right_dc` to confirm and any 1 from `left_dc` to confirm before the transaction is committed. Also, again suppose that for transactions originating in `left_dc` you want all nodes in `left_dc` and any 1 in `right_dc` to confirm before the transaction is commited. Above we used these two rules for this when defining `example_scope`:
+
+```sql
+SELECT bdr.add_commit_scope(
+    commit_scope_name := 'example_scope',
+    origin_node_group := 'left_dc',
+    rule := 'ALL (left_dc) GROUP COMMIT (commit_decision=raft) AND ANY 1 (right_dc) GROUP COMMIT',
+    wait_for_ready := true
+);
+SELECT bdr.add_commit_scope(
+    commit_scope_name := 'example_scope',
+    origin_node_group := 'right_dc',
+    rule := 'ANY 1 (left_dc) GROUP COMMIT AND ALL (right_dc) GROUP COMMIT (commit_decision=raft)',
+    wait_for_ready := true
+);
+```
+
+However, with `ORIGIN_GROUP`, just adding and using the following single-rule commit scope, `example_scope_2`, will have the same effect as the two individual rules we used above in `example_scope`:
+
+```sql
+SELECT bdr.add_commit_scope(
+    commit_scope_name := 'example_scope_2',
+    origin_node_group := 'top_group',
+    rule := 'ALL ORIGIN_GROUP GROUP COMMIT (commit_decision=raft) AND ANY 1 NOT ORIGIN_GROUP GROUP COMMIT';
+    wait_for_ready := true
+);
+```
+
+Under `example_scope_2`, when a transaction originates from `left_dc`, `ORIGIN_GROUP` maps to `left_dc` and `NOT ORIGIN_GROUP` maps to `right_dc`. Likewise, when a transaction originates from `right_dc`, `ORIGIN_GROUP` maps to `right_dc` and `NOT ORIGIN_GROUP` maps to `left_dc`. So by only specifying one rule, you get the effect of two.
+
+Note that if you added more subgroups, for instance a third child of `top_group`, `middle_dc`, then according to `example_scope_2` above, for transactions originating from `left_dc`, all the nodes in `left_dc` must plus any 1 in `right_dc` and any 1 in `middle_dc` must confirm before the transaction is committed. Of course then for transactions originating in `right_dc` all the nodes in `right_dc` plus any 1 node in `left_dc` and any 1 node in `middle_dc` must confirm before the transaction is committed. Lastly, because `middle_dc` is a child of `top_group`, `example_scope_2` also means that for transactions originating in `middle_dc`, all the nodes in `middle_dc` plus any 1 node in `left_dc` and any 1 node in `right_dc` must confirm before the transaction is committed.

product_docs/docs/pgd/5.6/reference/commit-scopes.mdx

@@ -26,10 +26,14 @@ commit_scope:
 commit_scope_operation:
     commit_scope_group confirmation_level commit_scope_kind
 
+commit_scope_target:
+    { (node_group [, ...])
+      | ORIGIN_GROUP }
+
 commit_scope_group:
-{ ANY num [NOT] (node_group [, ...])
-  | MAJORITY [NOT] (node_group [, ...])
-  | ALL [NOT] (node_group [, ...]) }
+{ ANY num [NOT] commit_scope_target
+  | MAJORITY [NOT] commit_scope_target
+  | ALL [NOT] commit_scope_target }
 
 confirmation_level:
     [ ON { received|replicated|durable|visible } ]
@@ -46,6 +50,23 @@ commit_scope_degrade_operation:
 ```
 
 Where `node_group` is the name of a PGD data node group. 
+
+## Commit scope targets
+
+### ORIGIN_GROUP
+
+Instead of targeting a specific group, you can also use `ORIGIN_GROUP`, which dynamically refers to the bottommost group from which a transaction originates. Therefore, if you have a top level group, `top_group`, and two subgroups as children, `left_dc` and `right_dc`, then adding a commit scope like:
+
+```sql
+SELECT bdr.add_commit_scope(
+    commit_scope_name := 'example_scope',
+    origin_node_group := 'top_level_group',
+    rule := 'MAJORITY ORIGIN_GROUP SYNCHRONOUS_COMMIT',
+    wait_for_ready := true
+);
+```
+
+would mean that for transactions originating on a node in `left_dc`, a majority of the nodes of `left_dc` would need to confirm the transaction synchronously before the transaction is committed. Moreover, the same rule would also mean that for transactions originating from a node in `right_dc`, a majority of nodes from `right_dc` are required to confirm the transaction synchronously before it is committed. This saves the need to add two seperate rules, one for `left_dc` and one for `right_dc`, to the commit scope.
 
 ### commit_scope_degrade_operation
 

product_docs/docs/pgd/5.6/reference/index.json

@@ -126,6 +126,7 @@
   "bdrglobal_lock_timeout": "/pgd/5.6/reference/pgd-settings#bdrglobal_lock_timeout",
   "bdrglobal_lock_statement_timeout": "/pgd/5.6/reference/pgd-settings#bdrglobal_lock_statement_timeout",
   "bdrglobal_lock_idle_timeout": "/pgd/5.6/reference/pgd-settings#bdrglobal_lock_idle_timeout",
+  "bdrlock_table_locking": "/pgd/5.6/reference/pgd-settings#bdrlock_table_locking",
   "bdrpredictive_checks": "/pgd/5.6/reference/pgd-settings#bdrpredictive_checks",
   "bdrreplay_progress_frequency": "/pgd/5.6/reference/pgd-settings#bdrreplay_progress_frequency",
   "bdrstandby_slot_names": "/pgd/5.6/reference/pgd-settings#bdrstandby_slot_names",
@@ -169,7 +170,7 @@
   "bdrglobal_keepalives_count": "/pgd/5.6/reference/pgd-settings#bdrglobal_keepalives_count",
   "bdrglobal_tcp_user_timeout": "/pgd/5.6/reference/pgd-settings#bdrglobal_tcp_user_timeout",
   "bdrraft_global_election_timeout": "/pgd/5.6/reference/pgd-settings#bdrraft_global_election_timeout",
-  "bdrraft_local_election_timeout": "/pgd/5.6/reference/pgd-settings#bdrraft_local_election_timeout",
+  "bdrraft_group_election_timeout": "/pgd/5.6/reference/pgd-settings#bdrraft_group_election_timeout",
   "bdrraft_response_timeout": "/pgd/5.6/reference/pgd-settings#bdrraft_response_timeout",
   "bdrraft_keep_min_entries": "/pgd/5.6/reference/pgd-settings#bdrraft_keep_min_entries",
   "bdrraft_log_min_apply_duration": "/pgd/5.6/reference/pgd-settings#bdrraft_log_min_apply_duration",
@@ -189,6 +190,7 @@
   "bdralter_subscription_disable": "/pgd/5.6/reference/nodes-management-interfaces#bdralter_subscription_disable",
   "bdrcreate_node": "/pgd/5.6/reference/nodes-management-interfaces#bdrcreate_node",
   "bdrcreate_node_group": "/pgd/5.6/reference/nodes-management-interfaces#bdrcreate_node_group",
+  "bdrdrop_node_group": "/pgd/5.6/reference/nodes-management-interfaces#bdrdrop_node_group",
   "bdrjoin_node_group": "/pgd/5.6/reference/nodes-management-interfaces#bdrjoin_node_group",
   "bdrpart_node": "/pgd/5.6/reference/nodes-management-interfaces#bdrpart_node",
   "bdrpromote_node": "/pgd/5.6/reference/nodes-management-interfaces#bdrpromote_node",
@@ -200,6 +202,9 @@
   "bdrdrop_proxy": "/pgd/5.6/reference/routing#bdrdrop_proxy",
   "bdrrouting_leadership_transfer": "/pgd/5.6/reference/routing#bdrrouting_leadership_transfer",
   "cs.commit-scope-syntax": "/pgd/5.6/reference/commit-scopes#commit-scope-syntax",
+  "cs.commit-scope-targets": "/pgd/5.6/reference/commit-scopes#commit-scope-targets",
+  "cs.origin_group": "/pgd/5.6/reference/commit-scopes#origin_group",
+  "cs.commit_scope_degrade_operation": "/pgd/5.6/reference/commit-scopes#commit_scope_degrade_operation",
   "cs.commit-scope-groups": "/pgd/5.6/reference/commit-scopes#commit-scope-groups",
   "cs.any": "/pgd/5.6/reference/commit-scopes#any",
   "cs.any-not": "/pgd/5.6/reference/commit-scopes#any-not",
@@ -213,17 +218,18 @@
   "cs.on-durable": "/pgd/5.6/reference/commit-scopes#on-durable",
   "cs.on-visible": "/pgd/5.6/reference/commit-scopes#on-visible",
   "cs.commit-scope-kinds": "/pgd/5.6/reference/commit-scopes#commit-scope-kinds",
+  "cs.synchronous_commit": "/pgd/5.6/reference/commit-scopes#synchronous_commit",
+  "cs.degrade-on-parameters": "/pgd/5.6/reference/commit-scopes#degrade-on-parameters",
   "cs.group-commit": "/pgd/5.6/reference/commit-scopes#group-commit",
   "cs.group-commit-parameters": "/pgd/5.6/reference/commit-scopes#group-commit-parameters",
   "cs.abort-on-parameters": "/pgd/5.6/reference/commit-scopes#abort-on-parameters",
   "cs.transaction_tracking-settings": "/pgd/5.6/reference/commit-scopes#transaction_tracking-settings",
   "cs.conflict_resolution-settings": "/pgd/5.6/reference/commit-scopes#conflict_resolution-settings",
   "cs.commit_decision-settings": "/pgd/5.6/reference/commit-scopes#commit_decision-settings",
+  "cs.commit_scope_degrade_operation-settings": "/pgd/5.6/reference/commit-scopes#commit_scope_degrade_operation-settings",
   "cs.camo": "/pgd/5.6/reference/commit-scopes#camo",
-  "cs.degrade-on-parameters": "/pgd/5.6/reference/commit-scopes#degrade-on-parameters",
   "cs.lag-control": "/pgd/5.6/reference/commit-scopes#lag-control",
   "cs.lag-control-parameters": "/pgd/5.6/reference/commit-scopes#lag-control-parameters",
-  "cs.synchronous_commit": "/pgd/5.6/reference/commit-scopes#synchronous_commit",
   "conflict-detection": "/pgd/5.6/reference/conflicts#conflict-detection",
   "list-of-conflict-types": "/pgd/5.6/reference/conflicts#list-of-conflict-types",
   "conflict-resolution": "/pgd/5.6/reference/conflicts#conflict-resolution",
@@ -287,6 +293,8 @@
   "tg_table_schema": "/pgd/5.6/reference/streamtriggers/rowvariables#tg_table_schema",
   "tg_nargs": "/pgd/5.6/reference/streamtriggers/rowvariables#tg_nargs",
   "tg_argv": "/pgd/5.6/reference/streamtriggers/rowvariables#tg_argv",
+  "bdrautopartition_partitions": "/pgd/5.6/reference/catalogs-internal#bdrautopartition_partitions",
+  "bdrautopartition_rules": "/pgd/5.6/reference/catalogs-internal#bdrautopartition_rules",
   "bdrddl_epoch": "/pgd/5.6/reference/catalogs-internal#bdrddl_epoch",
   "bdrevent_history": "/pgd/5.6/reference/catalogs-internal#bdrevent_history",
   "bdrevent_summary": "/pgd/5.6/reference/catalogs-internal#bdrevent_summary",
@@ -338,5 +346,18 @@
   "bdrtaskmgr_set_leader": "/pgd/5.6/reference/functions-internal#bdrtaskmgr_set_leader",
   "bdrtaskmgr_get_last_completed_workitem": "/pgd/5.6/reference/functions-internal#bdrtaskmgr_get_last_completed_workitem",
   "bdrtaskmgr_work_queue_check_status": "/pgd/5.6/reference/functions-internal#bdrtaskmgr_work_queue_check_status",
+  "bdrpglogical_proto_version_ranges": "/pgd/5.6/reference/functions-internal#bdrpglogical_proto_version_ranges",
+  "bdrget_min_required_replication_slots": "/pgd/5.6/reference/functions-internal#bdrget_min_required_replication_slots",
+  "bdrget_min_required_worker_processes": "/pgd/5.6/reference/functions-internal#bdrget_min_required_worker_processes",
+  "bdrstat_get_activity": "/pgd/5.6/reference/functions-internal#bdrstat_get_activity",
+  "bdrworker_role_id_name": "/pgd/5.6/reference/functions-internal#bdrworker_role_id_name",
+  "bdrlag_history": "/pgd/5.6/reference/functions-internal#bdrlag_history",
+  "bdrget_raft_instance_by_nodegroup": "/pgd/5.6/reference/functions-internal#bdrget_raft_instance_by_nodegroup",
+  "bdrmonitor_camo_on_all_nodes": "/pgd/5.6/reference/functions-internal#bdrmonitor_camo_on_all_nodes",
+  "bdrmonitor_raft_details_on_all_nodes": "/pgd/5.6/reference/functions-internal#bdrmonitor_raft_details_on_all_nodes",
+  "bdrmonitor_replslots_details_on_all_nodes": "/pgd/5.6/reference/functions-internal#bdrmonitor_replslots_details_on_all_nodes",
+  "bdrmonitor_subscription_details_on_all_nodes": "/pgd/5.6/reference/functions-internal#bdrmonitor_subscription_details_on_all_nodes",
+  "bdrmonitor_version_details_on_all_nodes": "/pgd/5.6/reference/functions-internal#bdrmonitor_version_details_on_all_nodes",
+  "bdrnode_group_member_info": "/pgd/5.6/reference/functions-internal#bdrnode_group_member_info",
   "bdrcolumn_timestamps_create": "/pgd/5.6/reference/clcd#bdrcolumn_timestamps_create"
 }

product_docs/docs/pgd/5.6/reference/index.mdx

@@ -284,6 +284,8 @@ The reference section is a definitive listing of all functions, views, and comma
 
 ## [Commit scopes](commit-scopes)
  * [Commit scope syntax](commit-scopes#commit-scope-syntax)
+ * [Commit scope targets](commit-scopes#commit-scope-targets)
+   * [ORIGIN_GROUP](commit-scopes#origin_group)
    * [commit_scope_degrade_operation](commit-scopes#commit_scope_degrade_operation)
  * [Commit scope groups](commit-scopes#commit-scope-groups)
    * [ANY](commit-scopes#any)