Skip to content

Commit

Permalink
Merge pull request #6245 from EnterpriseDB/docs/edits_to_pgd_pr6188
Browse files Browse the repository at this point in the history
Edits to Fix default setting and type for force_full_mesh. #6188
  • Loading branch information
ebgitelman authored Nov 21, 2024
2 parents 08c99d9 + 26a46fb commit 6ecf6f1
Showing 1 changed file with 22 additions and 20 deletions.
42 changes: 22 additions & 20 deletions product_docs/docs/pgd/5.6/nodes/subscriber_only/optimizing-so.mdx
Original file line number Diff line number Diff line change
@@ -1,73 +1,75 @@
---
title: Optimizing subscriber-only groups
navTitle: Optimizing
description: 'Optimizing subscriber-only groups in EDB Postgres Distributed'
description: 'Optimizing subscriber-only groups in EDB Postgres Distributed.'
---

With PGD 5.6 and later, it's possible to optimize the topology of [subscriber-only groups](overview).

In this optimized topology, a small number of fully active nodes, the write leaders of the data groups, replicate changes to the group leaders of subscriber-only groups, these group leaders then replicate changes to the other members of its subscriber-only group.
In this optimized topology, a small number of fully active nodes—the write leaders of the data groups—replicate changes to the group leaders of subscriber-only groups. These group leaders then replicate changes to the other members of its subscriber-only group.

## Requirements for the optimized topology

You can't enable this model if a cluster has any of the following:

* Data-nodes which are directly members of the top-level group.
* No data-node sub-groups.
* No data-node sub-groups with proxy routing enabled.
* Data nodes that are directly members of the top-level group
* No data-node subgroups
* No data-node subgroups with proxy routing enabled

If this is the case, the nodes in a subscriber only groups automatically revert to the full mesh topology.
If any of these are the case, the nodes in subscriber-only groups revert to the full mesh topology.

To get the benefit of the new SO group and node replication, you must have your data-nodes in subgroups, with proxy routing enabled on the sub groups.
To get the benefit of the new SO group and node replication, you must have your data nodes in subgroups, with proxy routing enabled on the subgroups.

## How the optimized topology works

For clusters using groups for their data nodes, subscriber only groups can use a more efficient model which uses a subscriber-only group leader, similar to a write leader in PGD Proxies.
For clusters using groups for their data nodes, subscriber-only groups can use a more efficient model. This model uses subscriber-only group leaders, similar to write leaders in PGD proxies.

Each subscriber-only group uses that group leader to replicate changes to other subscriber-only nodes in its group. The group leader acts as a replication proxy for incoming changes.

The write leader nodes in other non-subscriber-only groups replicate changes to the group leader of the subscriber-only group. Other nodes only replicate within their data groups, both within their own group and in other data groups.
The write leader nodes in data groups replicate changes to the group leaders of the subscriber-only groups. Other nodes in the data groups only replicate with nodes in their data group and with data nodes in other data groups. They do not directly replicate their changes to the subscriber-only groups.

## Subscriber-only group leaders

With PGD 5.6 and later, each subscriber-only group gets assigned a group leader of its own.
This is because subscriber-only groups don't have a group Raft consensus mechanism of their own.
Instead, the cluster’s top level group uses its Raft consensus mechanism to handle selection of each subscriber-only group’s group leader.
This group leader selection is on by default on PGD 5.6, regardless of the topology optimization settings.
Instead, the cluster’s top-level group uses its Raft consensus mechanism to handle selecting each subscriber-only group’s group leader.
This group leader selection is on by default in PGD 5.6, regardless of the topology optimization settings.

Group leaders in subscriber-only groups are regularly tested for connectivity and if unavailable, the voting nodes of top-level group select a new subscriber-only node from the subscriber-only group to become group leader.
The new group leader is then selected.
Group leaders in subscriber-only groups are regularly tested for connectivity and, if unavailable, the voting nodes of top-level group select a new subscriber-only node from the subscriber-only group to become group leader.
The new group leader is then selected.

With optimized technology turned off, this election has no effect on the replication topology. Without the optimized topology, all data nodes replicate changes to all other nodes in the cluster.

## Group leaders in the optimized topology

With the optimized topology enabled, only the subscriber-only group's group leader receives changes from other data group's write leaders in the cluster and it takes on the responsibility of replicating those changes to the other nodes in the subscriber-only group.
With the optimized topology enabled, only the subscriber-only group's group leader receives changes from other data groups' write leaders in the cluster. The group leader takes on the responsibility of replicating those changes to the other nodes in the subscriber-only group.

The other voting nodes choose the group leader from a subscriber-only group's nodes.
Once selected, the whole cluster becomes aware of the change and any data group's write leaders then replicate data only to this newly selected group leader node.
Once the group leader is selected, the whole cluster becomes aware of the change, and any data group's write leaders then replicate data only to this newly selected group leader node.
Other data nodes in the data groups don't replicate data to the subscriber-only group's nodes.

This approach avoids the explosion of active connections that can happen when there are large numbers of SO nodes and reduces the amount of replication traffic.

The Subscriber-Only node and group form the building block for PGD tree topologies.
The subscriber-only node and group form the building block for PGD tree topologies.

## Enabling the optimized model

By default, PGD 5.6 forces the full mesh topology. This means the optimization described here is off.
To enable the optimized topology, you must have your data-nodes in subgroups, with proxy routing enabled on the sub groups.
To enable the optimized topology, you must have your data nodes in subgroups, with proxy routing enabled on the subgroups.
You can then set the GUC [`bdr.force_full_mesh`](/pgd/5.6/reference/pgd-settings#bdrforce_full_mesh) to `off` to allow the optimization to be activated.
Note: This GUC needs to be set in the postgresql.conf file on each data node and each node restarted for the change to take effect.

If any requirements of the optimized topology aren't met, the nodes in a subscriber-only group automatically revert to the full mesh topology. When this happens, you'll find in the logs of the nodes in the cluster messages that explain why the optimization was not possible, such as:
!!! Note
This GUC needs to be set in the `postgresql.conf` file on each data node and each node restarted for the change to take effect.

If any requirements of the optimized topology aren't met, the nodes in a subscriber-only group revert to the full mesh topology. When this happens, you'll find in the logs of the nodes in the cluster messages why the optimization wasn't possible, such as:

When a data node is part of the top-level node group:

```plaintext
node: <nodename> is part of top-level nodegroup: <toplevelgroupname>: changing to full mesh".
```

and when a data group does not have proxy routing enabled:
When a data group doesn't have proxy routing enabled:

```plaintext
node: <nodename> is in nodegroup: <nodegroupname> that does not have proxy routing: changing to full mesh.
Expand Down

1 comment on commit 6ecf6f1

@github-actions
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please sign in to comment.