Skip to content

Commit

Permalink
Edits to Fix default setting and type for force_full_mesh. #6188
Browse files Browse the repository at this point in the history
  • Loading branch information
ebgitelman committed Nov 15, 2024
1 parent add7b08 commit 7cd95cf
Showing 1 changed file with 21 additions and 19 deletions.
40 changes: 21 additions & 19 deletions product_docs/docs/pgd/5.6/nodes/subscriber_only/optimizing-so.mdx
Original file line number Diff line number Diff line change
@@ -1,28 +1,28 @@
---
title: Optimizing subscriber-only groups
navTitle: Optimizing
description: 'Optimizing subscriber-only groups in EDB Postgres Distributed'
description: 'Optimizing subscriber-only groups in EDB Postgres Distributed.'
---

With PGD 5.6 and later, it's possible to optimize the topology of [subscriber-only groups](overview).

In this optimized topology, a small number of fully active nodes, the write leaders of the data groups, replicate changes to the group leaders of subscriber-only groups, these group leaders then replicate changes to the other members of its subscriber-only group.
In this optimized topology, a small number of fully active nodes—the write leaders of the data groups—replicate changes to the group leaders of subscriber-only groups. These group leaders then replicate changes to the other members of its subscriber-only group.

## Requirements for the optimized topology

You can't enable this model if a cluster has any of the following:

* Data-nodes which are directly members of the top-level group.
* No data-node sub-groups.
* No data-node sub-groups with proxy routing enabled.
* Data nodes that are directly members of the top-level group
* No data-node subgroups
* No data-node subgroups with proxy routing enabled

If this is the case, the nodes in a subscriber only groups automatically revert to the full mesh topology.
If any of these are the case, the nodes in subscriber-only groups revert to the full mesh topology.

To get the benefit of the new SO group and node replication, you must have your data-nodes in subgroups, with proxy routing enabled on the sub groups.
To get the benefit of the new SO group and node replication, you must have your data nodes in subgroups, with proxy routing enabled on the subgroups.

## How the optimized topology works

For clusters using groups for their data nodes, subscriber only groups can use a more efficient model which uses a subscriber-only group leader, similar to a write leader in PGD Proxies.
For clusters using groups for their data nodes, subscriber-only groups can use a more efficient model. This model uses a subscriber-only group leader, similar to a write leader in PGD proxies.

Each subscriber-only group uses that group leader to replicate changes to other subscriber-only nodes in its group. The group leader acts as a replication proxy for incoming changes.

Expand All @@ -32,42 +32,44 @@ The write leader nodes in other non-subscriber-only groups replicate changes to

With PGD 5.6 and later, each subscriber-only group gets assigned a group leader of its own.
This is because subscriber-only groups don't have a group Raft consensus mechanism of their own.
Instead, the cluster’s top level group uses its Raft consensus mechanism to handle selection of each subscriber-only group’s group leader.
This group leader selection is on by default on PGD 5.6, regardless of the topology optimization settings.
Instead, the cluster’s top-level group uses its Raft consensus mechanism to handle selecting each subscriber-only group’s group leader.
This group leader selection is on by default in PGD 5.6, regardless of the topology optimization settings.

Group leaders in subscriber-only groups are regularly tested for connectivity and if unavailable, the voting nodes of top-level group select a new subscriber-only node from the subscriber-only group to become group leader.
The new group leader is then selected.
Group leaders in subscriber-only groups are regularly tested for connectivity and, if unavailable, the voting nodes of top-level group select a new subscriber-only node from the subscriber-only group to become group leader.
The new group leader is then selected.

With optimized technology turned off, this election has no effect on the replication topology. Without the optimized topology, all data nodes replicate changes to all other nodes in the cluster.

## Group leaders in the optimized topology

With the optimized topology enabled, only the subscriber-only group's group leader receives changes from other data group's write leaders in the cluster and it takes on the responsibility of replicating those changes to the other nodes in the subscriber-only group.
With the optimized topology enabled, only the subscriber-only group's group leader receives changes from other data groups' write leaders in the cluster. The group leader takes on the responsibility of replicating those changes to the other nodes in the subscriber-only group.

The other voting nodes choose the group leader from a subscriber-only group's nodes.
Once selected, the whole cluster becomes aware of the change and any data group's write leaders then replicate data only to this newly selected group leader node.
Once the group leader is selected, the whole cluster becomes aware of the change, and any data group's write leaders then replicate data only to this newly selected group leader node.
Other data nodes in the data groups don't replicate data to the subscriber-only group's nodes.

This approach avoids the explosion of active connections that can happen when there are large numbers of SO nodes and reduces the amount of replication traffic.

The Subscriber-Only node and group form the building block for PGD tree topologies.
The subscriber-only node and group form the building block for PGD tree topologies.

## Enabling the optimized model

By default, PGD 5.6 forces the full mesh topology. This means the optimization described here is off.
To enable the optimized topology, you must have your data-nodes in subgroups, with proxy routing enabled on the sub groups.
To enable the optimized topology, you must have your data nodes in subgroups, with proxy routing enabled on the subgroups.
You can then set the GUC [`bdr.force_full_mesh`](/pgd/5.6/reference/pgd-settings#bdrforce_full_mesh) to `off` to allow the optimization to be activated.
Note: This GUC needs to be set in the postgresql.conf file on each data node and each node restarted for the change to take effect.

If any requirements of the optimized topology aren't met, the nodes in a subscriber-only group automatically revert to the full mesh topology. When this happens, you'll find in the logs of the nodes in the cluster messages that explain why the optimization was not possible, such as:
!!! Note
This GUC needs to be set in the `postgresql.conf` file on each data node and each node restarted for the change to take effect.

If any requirements of the optimized topology aren't met, the nodes in a subscriber-only group revert to the full mesh topology. When this happens, you'll find in the logs of the nodes in the cluster messages why the optimization wasn't possible, such as:

When a data node is part of the top-level node group:

```plaintext
node: <nodename> is part of top-level nodegroup: <toplevelgroupname>: changing to full mesh".
```

and when a data group does not have proxy routing enabled:
When a data group doesn't have proxy routing enabled:

```plaintext
node: <nodename> is in nodegroup: <nodegroupname> that does not have proxy routing: changing to full mesh.
Expand Down

0 comments on commit 7cd95cf

Please sign in to comment.