Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release/2023-12-12 #5084

Merged
merged 74 commits into from
Dec 12, 2023
Merged
Changes from 1 commit
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
c320cb8
Edits to postgresql uninstall topic pr5055
ebgitelman Nov 30, 2023
76a1239
Update advocacy_docs/supported-open-source/postgresql/uninstalling.mdx
ebgitelman Dec 5, 2023
fed4edf
Update advocacy_docs/supported-open-source/postgresql/uninstalling.mdx
ebgitelman Dec 5, 2023
4620455
Update advocacy_docs/supported-open-source/postgresql/uninstalling.mdx
ebgitelman Dec 5, 2023
9a4f294
Update advocacy_docs/supported-open-source/postgresql/uninstalling.mdx
ebgitelman Dec 5, 2023
4c5266e
Update advocacy_docs/supported-open-source/postgresql/uninstalling.mdx
ebgitelman Dec 5, 2023
3c33d0b
Update advocacy_docs/supported-open-source/postgresql/uninstalling.mdx
ebgitelman Dec 5, 2023
d22e460
Update advocacy_docs/supported-open-source/postgresql/uninstalling.mdx
ebgitelman Dec 5, 2023
c3aecf8
Update advocacy_docs/supported-open-source/postgresql/uninstalling.mdx
ebgitelman Dec 5, 2023
cf29a50
Update advocacy_docs/supported-open-source/postgresql/uninstalling.mdx
ebgitelman Dec 5, 2023
5c2ef98
Added PGE 15.5
djw-m Dec 5, 2023
48ef797
Merge pull request #5069 from EnterpriseDB/doc/pge/fix/releasenotes_15.5
djw-m Dec 6, 2023
fdd4646
Update postgres-configuration.mdx to address invalid ordering
djw-m Dec 6, 2023
4ed0a3c
Merge pull request #5075 from EnterpriseDB/docs/pgd/fix/fixbdrordering
djw-m Dec 6, 2023
93570ea
Added API compat table
djw-m Dec 6, 2023
986e8f4
Adjusted table
djw-m Dec 6, 2023
a1c296d
Fixed step numbering issue
ebgitelman Dec 7, 2023
7343c86
Merge pull request #5074 from EnterpriseDB/docs/pgd/add/apibreakcompa…
djw-m Dec 7, 2023
109ebce
update old efm version reference
EFM-Bobby Dec 7, 2023
a5da9f8
Update product_docs/docs/efm/4/04_configuring_efm/04_extending_efm_pe…
EFM-Bobby Dec 7, 2023
29c877b
Merge pull request #5078 from EnterpriseDB/efm/old_efm42_reference
djw-m Dec 7, 2023
19bae6c
Merge pull request #5061 from EnterpriseDB/docs/edits_to_postgresql_p…
nidhibhammar Dec 7, 2023
87cd821
First draft pass on referencizing durability
djw-m Jul 3, 2023
29e1090
Text tweaks
djw-m Jul 3, 2023
af19270
Fixed wrong link
djw-m Jul 4, 2023
659fce0
Case fixes
djw-m Jul 4, 2023
0af4064
Extracted Legacy Sync options into their own page
djw-m Jul 4, 2023
94cfff0
Resolving comments
djw-m Jul 5, 2023
8a6dbb2
Typos and formatting
djw-m Jul 10, 2023
f6b7875
More frontmatter dupes
djw-m Jul 10, 2023
604d9d8
Move tables into legacy sync
djw-m Jul 11, 2023
e62a95e
First Split
djw-m Aug 23, 2023
105ba4d
Brought out limitations
djw-m Aug 29, 2023
6551676
Update product_docs/docs/pgd/5/durability/administering.mdx
djw-m Sep 8, 2023
8c61af7
Update product_docs/docs/pgd/5/durability/commit-scopes.mdx
djw-m Oct 9, 2023
b04ea19
Update product_docs/docs/pgd/5/durability/comparing.mdx
djw-m Oct 9, 2023
df585b9
Index and admin tweaks
djw-m Oct 9, 2023
968bb26
WIP Checkin
djw-m Oct 11, 2023
29de4eb
Fix defaults and some text.
djw-m Oct 23, 2023
b5ac766
Added notes on 0 being not set
djw-m Oct 23, 2023
d8e3fbd
Start of making kind sections similar
djw-m Nov 7, 2023
528788d
Split nav
djw-m Nov 8, 2023
06a86b4
Update overview format
djw-m Nov 14, 2023
8d236ab
More small changes
djw-m Nov 14, 2023
0809132
Reference updates
djw-m Nov 14, 2023
09fffd1
Updates for reference - content fill out and checks
djw-m Nov 21, 2023
8b679d8
Added timing section (was adrift in rework)
djw-m Nov 21, 2023
7b5f731
Various edits reworking the commit scopes
djw-m Nov 23, 2023
f2c6cb0
Actual commit (as missed file previously)
djw-m Nov 23, 2023
6c1c6d0
Update product_docs/docs/pgd/5/durability/index.mdx
djw-m Nov 27, 2023
c78bdd1
Update product_docs/docs/pgd/5/durability/index.mdx
djw-m Nov 27, 2023
39ea3c9
Review changes
djw-m Nov 27, 2023
1dfeb39
Remove terms from overview, update reference
djw-m Nov 27, 2023
d503952
Update product_docs/docs/pgd/5/durability/lag-control.mdx
djw-m Nov 28, 2023
b0e8edc
WIP Commit
djw-m Nov 27, 2023
7a3ce6d
WIP Commit
djw-m Nov 28, 2023
2ccccd8
Added commit scope kinds
djw-m Nov 28, 2023
329cffa
Update index for rules
djw-m Nov 28, 2023
9b79c30
Review edits
djw-m Nov 29, 2023
50ae642
Review edits cont.
djw-m Nov 29, 2023
07052f4
Shorter version focussing on GC
djw-m Nov 30, 2023
7a54ea7
Format and typo fix
djw-m Nov 30, 2023
9e685fd
Add deepToC
djw-m Nov 30, 2023
168ce4e
wrapped sync commit wordage - verification needed
djw-m Nov 30, 2023
e434a67
Clear end of doc
djw-m Nov 30, 2023
f91d961
Fix reference commit scope entry
djw-m Nov 30, 2023
add4876
Link fixes
djw-m Nov 30, 2023
56810c3
Review Edits
djw-m Dec 4, 2023
f3e17cf
GH review comments resolved
djw-m Dec 4, 2023
7f5a3ae
Fixes for review comments from pjmodos
djw-m Dec 5, 2023
3ab277a
Edits to comparing
djw-m Dec 8, 2023
98b365e
Merge pull request #4677 from EnterpriseDB/doc/fix/durabilityref3
djw-m Dec 11, 2023
976d86a
fix synchronous_commit link
djw-m Dec 12, 2023
12377a1
Fix reference link to como_partner_queue
djw-m Dec 12, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Review edits cont.
Signed-off-by: Dj Walker-Morgan <[email protected]>
djw-m committed Dec 8, 2023

Verified

This commit was signed with the committer’s verified signature.
edubart Eduardo Bart
commit 50ae64210ed951575e30d3c16b8b2d2234c2ff46
15 changes: 11 additions & 4 deletions product_docs/docs/pgd/5/durability/commit-scope-rules.mdx
Original file line number Diff line number Diff line change
@@ -6,7 +6,15 @@ Commit scope rules are at the core of the commit scope mechanism. They define wh

Commit scope rules are composed of one or more operations combined with an `AND`.

Each operation is made up of two (or three) parts, the commit scope group, an optional confirmation level, and the kind of commit scope (which may have it's own parameters).
Each operation is made up of two (or three) parts, the commit scope group, an optional confirmation level, and the kind of commit scope (which may have it's own parameters).

```
commit_scope_group [ confimation_level ] commit_scope_kind
```

A full formal syntax diagram is available in the [commit scope reference section.](/pgd/latest/reference/commit-scopes/#commit-scope-syntax).

If we look at a typical commit scope rule, we can now break it down into its components:

```
ANY 2 (group) GROUP COMMIT
@@ -32,7 +40,7 @@ There are three kinds of commit scope group, `ANY`, `ALL` and `MAJORITY`. They a
- `ALL` &mdash is followed by the groups and translates to all nodes in the listed groups noes.
- `MAJORITY` &mdsh is followed by the groups and translates to requiring a half, plus one, of the listed groups nodes to confirm to give a majority.

## The Confirmation Level
## The confirmation Level

PGD nodes can send confirmations for a transaction at different times. In increasing levels of protection, from the perspective of the confirming node, these are:

@@ -69,8 +77,7 @@ For further details see [`CAMO`](camo).

### `LAG CONTROL`

With Lag control, when the system's replication performance exceeds specified limits, a delay can be automatically injected into client interaction with the database, providing a back pressure on clients.

With Lag control, when the system's replication performance exceeds specified limits, a commit delay can be automatically injected into client interaction with the database, providing a back pressure on clients. Lag control has parameters to set the maximum commit delay that can be exerted, and limits in terms of time to process or queue size which will trigger increases in that commit delay.

### `SYNCHRONOUS_COMMIT`

47 changes: 27 additions & 20 deletions product_docs/docs/pgd/5/durability/group-commit.mdx
Original file line number Diff line number Diff line change
@@ -11,7 +11,9 @@ Commit scope kind: `GROUP COMMIT`

The goal of Group Commit is to protect against data loss in case of single node
failures or temporary outages. You achieve this by requiring more than one PGD
node to successfully receive and confirm a transaction at COMMIT time.
node to successfully confirm a transaction at COMMIT time. Confirmation can be sent
at a number of points in the transaction processing, but defaults to "visible" when
the transaction has been flushed to disk and is visible to all other transactions.


## Example
@@ -37,9 +39,10 @@ activity currently happens only when either the origin node recovers or when
it's parted from the cluster. This behavior is the same as with Postgres legacy
built-in synchronous replication.

Transactions committed with Group Commit use two-phase commit underneath.
Therefore, configure `max_prepared_transactions` high enough to handle all such
transactions originating per node.
Transactions committed with Group Commit use [two-phase
commit](../terminology/#two-phase-commit-2pc) underneath. Therefore, configure
`max_prepared_transactions` high enough to handle all such transactions
originating per node.

## Limitations

@@ -61,10 +64,10 @@ scope.
You can configure Group Commit to decide commits in three different ways: `group`,
`partner`, and `raft`.

The `group` decision is done through the consensus specified using the same
commit scope group settings used for the durability purposes. The difference
is that the commit decision is made based on PREPARE replication while the
durability checks COMMIT (PREPARED) replication.
The `group` decision is the default. It specifies that the commit is confirmed
by the origin node upon it recieving as many confirmations as required by the
commit scope group. The difference is that the commit decision is made based on
PREPARE replication while the durability checks COMMIT (PREPARED) replication.

The `partner` decision is what [Commit At Most Once](camo) uses. This approach
works only when there are two data nodes in the node group. These two nodes are
@@ -73,23 +76,27 @@ to commit something. This approach requires application changes to use
the CAMO transaction protocol to work correctly, as the application is in some way
part of the consensus. For more on this approach, see [CAMO](camo).

The `raft` option uses the built-in Raft consensus to decide whether commit can
happen. Currently, the global Raft is used. For this to work, the majority of
nodes across the whole cluster must work.
The `raft` option uses the PGD's built-in Raft consensus to decide whether
commit can happen. Currently, the global Raft is used. For this to work, the
majority of nodes across the whole cluster must work. This option means that all
commits are effectively ordered and every node moves forward in step with those
commits at the same rate.

### Conflict resolution

Conflict resolution can be `async` or `eager`.

Async means that PGD does optimistic conflict resolution during replication using the row-level resolution as configured for given node. This happens regardless of whether the origin transaction committed or is still in progress. See
[Conflicts](../consistency/conflicts) for details about
how the asynchronous conflict resolution works.

Eager means that conflicts are resolved eagerly (as part of agreement on COMMIT),
and conflicting transactions get aborted with a serialization error. This approach
provides greater isolation than the asynchronous resolution at the price of
performance. For the details about how Eager conflict resolution works, see
[Eager conflict resolution](../consistency/eager).
Async means that PGD does optimistic conflict resolution during replication
using the row-level resolution as configured for given node. This happens
regardless of whether the origin transaction committed or is still in progress.
See [Conflicts](../consistency/conflicts) for details about how the asynchronous
conflict resolution works.

Eager means that conflicts are resolved eagerly (as part of agreement on
COMMIT), and conflicting transactions get aborted with a serialization error.
This approach provides greater isolation than the asynchronous resolution at the
price of performance. For the details about how Eager conflict resolution works,
see [Eager conflict resolution](../consistency/eager).

### Aborts

12 changes: 6 additions & 6 deletions product_docs/docs/pgd/5/durability/lag-control.mdx
Original file line number Diff line number Diff line change
@@ -33,11 +33,10 @@ such as RPO, RCO, and GEO at risk.
- **GEO** (Group elasticity objective) ensures that any node isn't originating new
data at a rate that can't be saved to its peer nodes.

To allow organizations to achieve their objectives, PGD offers Lag Control,
also known as Replication Lag Control. This feature provides a means to
precisely regulate the potential imbalance without intruding on applications, by
transparently introducing a delay to READ WRITE transactions that modify data.
This delay, the PGD Commit Delay, starts at 0ms.
To allow organizations to achieve their objectives, PGD offers Lag Control. This
feature provides a means to precisely regulate the potential imbalance without
intruding on applications, by transparently introducing a delay to READ WRITE
transactions that modify data. This delay, the PGD Commit Delay, starts at 0ms.

Using the LAG CONTROL commit scope kind, you can set a maximum time that commits
commits can be delayed between nodes in a group, maximum lag time or maximum lag
@@ -220,4 +219,5 @@ transactions from observing or modifying its values or acquiring its resources.
The same guarantee can't be made for external resources managed by Postgres
extensions. Regardless of extension dependencies, the same guarantee can be made
if the PGD extension is listed before extension-based resource managers in
postgresql.conf.
postgresql.conf.

26 changes: 13 additions & 13 deletions product_docs/docs/pgd/5/durability/timing.mdx
Original file line number Diff line number Diff line change
@@ -8,33 +8,33 @@ different ways.

With Legacy PSR, the order of operations is:

- Origin flushes a commit record to WAL, making the transaction
1. Origin flushes a commit record to WAL, making the transaction
visible locally.
- Peer node receives changes and issues a write.
- Peer flushes the received changes to disk.
- Peer applies changes, making the transaction visible on the peer.
1. Peer node receives changes and issues a write.
1. Peer flushes the received changes to disk.
1. Peer applies changes, making the transaction visible on the peer.

Note that the change is written to the disk before applying the chnages.

With PGD, by default and with Lag Control, the order of operations is different.
In these cases, the change becomes visible on the peer before the transaction is
flushed to the peer's disk:

- Origin flushes a commit record to WAL, making the transaction
1. Origin flushes a commit record to WAL, making the transaction
visible locally.
- Peer node receives changes into its apply queue in memory.
- Peer applies changes, making the transaction visible on the peer.
- Peer persists the transaction by flushing to disk.
1. Peer node receives changes into its apply queue in memory.
1. Peer applies changes, making the transaction visible on the peer.
1. Peer persists the transaction by flushing to disk.

For PGD's Group Commit and CAMO, the origin node waits for a certain number of
confirmations prior to making the transaction visible locally. The order of
operations is:

- Origin flushes a prepare or precommit record to WAL.
- Peer node receives changes into its apply queue in memory.
- Peer applies changes, making the transaction visible on the peer.
- Peer persists the transaction by flushing to disk.
- Origin commits and makes the transaction visible locally.
1. Origin flushes a prepare or precommit record to WAL.
1. Peer node receives changes into its apply queue in memory.
1. Peer applies changes, making the transaction visible on the peer.
1. Peer persists the transaction by flushing to disk.
1. Origin commits and makes the transaction visible locally.

The following table summarizes the differences.